4ff3998c2c
docs(cli): Table create should have help text match arg for database
2023-08-31 16:47:11 +01:00
00873fac09
test(cli): Add e2e test for `table list` command
2023-08-31 16:47:10 +01:00
2a396c1b73
feat(cli): Add `table list <DATABASE>` command
...
This commit hooks the influxdb_iox_client and CLI up to the new
`GetTables` gRPC endpoint on the `TableService`, allowing users to
query a list of tables within a database and consequently see any
custom partitioning schemes.
2023-08-31 16:47:09 +01:00
0b2c883440
fix: clippy ( #8625 )
...
I think by default we don't check all tests, but this one annoyed me
because my IDE complained about it.
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-08-31 12:32:27 +00:00
c0fdd97212
test(e2e): Add minimal coverage that CLI reports custom partition templates
2023-08-30 16:27:03 +01:00
12b8095c46
feat: Upgrade to Rust 1.72.0 ( #8589 )
...
* feat: Upgrade to Rust 1.72.0
* fix: Allow a warning about an error we're intentionally creating
This is a test for an error. This lint warns that this code will cause
an error. Thanks lint, that's what we wanted!
* chore: rustfmt 1.72
* fix: Remove unnecessary hashes in raw string literals
Thanks Clippy!
https://rust-lang.github.io/rust-clippy/master/index.html#/needless_raw_string_hashes
Note that there are a number of false negatives with this lint; see
https://github.com/rust-lang/rust-clippy/issues/11420
* fix: Remove unnecessary explicit iteration
Looks like clippy::explicit_iter_loop was improved.
https://rust-lang.github.io/rust-clippy/master/index.html#/explicit_iter_loop
* fix: Allow clippy::manual_try_fold in a few places
Some of these might not be possible to rewrite with try_fold, or at
least not trivially. I don't feel confident enough to change these, in
any case. I think the lint is good to have on for future code though, so
that new code can be written with try_fold.
* fix: Remove useless creation of vectors when an array will do
Mostly in tests. Also fix some long lines.
Thanks Clippy!
https://rust-lang.github.io/rust-clippy/master/index.html#/useless_vec
* fix: Allow a single range in a vec init, which is actually what we want
Looks like Clippy's trying to catch a common mistake here, but for realz
we actually want `Vec<Range<usize>>` not `Vec<usize>`
https://rust-lang.github.io/rust-clippy/master/index.html#/single_range_in_vec_init
* fix: Remove a useless conversion
This looks like removing explicit iteration, but it's actually caught by
useless_conversion.
https://rust-lang.github.io/rust-clippy/master/index.html#/useless_conversion
* fix: Remove redundant pattern matching
Thanks Clippy!
https://rust-lang.github.io/rust-clippy/master/index.html#/redundant_pat
* fix: Allow an unwrap on a literal None in a test
This matches with the other tests better, and also when I tried to
remove the `unwrap_or_default` it changed the JSON sent from something
with an empty value to `null`, so I think the `or_default` part is
actually changing from one `None` to another `None`.
https://rust-lang.github.io/rust-clippy/master/index.html#/unnecessary_literal_unwrap
2023-08-29 05:57:38 +00:00
8739f30451
chore(deps): Bump serde from 1.0.186 to 1.0.188 ( #8583 )
...
Bumps [serde](https://github.com/serde-rs/serde ) from 1.0.186 to 1.0.188.
- [Release notes](https://github.com/serde-rs/serde/releases )
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.186...v1.0.188 )
---
updated-dependencies:
- dependency-name: serde
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Dom <dom@itsallbroken.com>
2023-08-28 09:25:33 +00:00
7ca074a00a
chore(deps): Bump serde from 1.0.185 to 1.0.186 ( #8563 )
...
Bumps [serde](https://github.com/serde-rs/serde ) from 1.0.185 to 1.0.186.
- [Release notes](https://github.com/serde-rs/serde/releases )
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.185...v1.0.186 )
---
updated-dependencies:
- dependency-name: serde
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Dom <dom@itsallbroken.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-08-24 09:02:20 +00:00
694756ef6d
chore(deps): Bump serde from 1.0.183 to 1.0.185 ( #8535 )
...
Bumps [serde](https://github.com/serde-rs/serde ) from 1.0.183 to 1.0.185.
- [Release notes](https://github.com/serde-rs/serde/releases )
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.183...v1.0.185 )
---
updated-dependencies:
- dependency-name: serde
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-08-22 09:06:06 +00:00
18783f9462
test: harden `end_to_end_cases::debug::build_catalog` ( #8537 )
...
This seems to fail a lot in CI, try to work around it. A proper fix is
tracked under #8287 .
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-08-22 08:56:19 +00:00
967aef0e9d
chore: Update datafusion ( #8515 )
...
* chore: Update datafusion
* fix: update for API
* fix: Verify unsupported statements, with tests
* fix: update tests
---------
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-08-21 17:49:21 +00:00
158e2a108d
chore(deps): Bump tempfile from 3.7.1 to 3.8.0 ( #8527 )
...
Bumps [tempfile](https://github.com/Stebalien/tempfile ) from 3.7.1 to 3.8.0.
- [Changelog](https://github.com/Stebalien/tempfile/blob/master/CHANGELOG.md )
- [Commits](https://github.com/Stebalien/tempfile/compare/v3.7.1...v3.8.0 )
---
updated-dependencies:
- dependency-name: tempfile
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-08-21 08:52:19 +00:00
0fd5af64bc
refactor: Extract a builder of namespace commands
2023-08-18 15:59:30 -04:00
64eb3be4c2
refactor: Split long strings for readability and rustfmt
...
When there are long strings in a file, rustfmt just gives up and
ignores the whole file.
2023-08-18 15:59:06 -04:00
defd20c7ed
refactor: Reduce duplication and increase consistency in namespace names in tests
...
Create consts outside the test steps when possible to share values that
need to be the same. Call all of these namespace_name to distinguish
from a namespace object or command.
2023-08-18 15:52:38 -04:00
d575873ceb
refactor: Extract CLI test helper functions into their own module
2023-08-18 15:52:37 -04:00
412df8dc4e
fix: Remove debugging printlns from tests
2023-08-18 15:52:37 -04:00
65207b2f9c
fix: Remove unneeded use of lazy_static
...
This can just be a static slice rather than a vec.
2023-08-18 15:52:37 -04:00
3d1e49e57a
refactor: Extract table CLI tests to their own module
2023-08-18 15:52:37 -04:00
22f1d6f469
refactor: Extract namespace CLI tests to their own module
2023-08-18 15:52:37 -04:00
5d17a99dbb
feat: read null sort_key_ids ( #8489 )
...
* feat: read null sort_key_ids
* chore: clearer explanation about test strategy
* chore: Apply suggestions from code review
Co-authored-by: Marco Neumann <marco@crepererum.net>
* test: tests that add partition with NULL sort_key_ids
* chore: address review comments
* chore: remove unecessary comments and tests
* fix: typos
* chore: remove unecessary tests
* fix: chec duplicates for SortedColumnSet
---------
Co-authored-by: Marco Neumann <marco@crepererum.net>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-08-18 14:15:27 +00:00
d2c71bfe67
chore(deps): Bump thiserror from 1.0.46 to 1.0.47 ( #8519 )
...
Bumps [thiserror](https://github.com/dtolnay/thiserror ) from 1.0.46 to 1.0.47.
- [Release notes](https://github.com/dtolnay/thiserror/releases )
- [Commits](https://github.com/dtolnay/thiserror/compare/1.0.46...1.0.47 )
---
updated-dependencies:
- dependency-name: thiserror
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-08-18 09:02:48 +00:00
af8967f9e1
chore: Update DataFusion to get fix for string functions on tags ( #8479 )
...
* chore: Update DataFusion pin
* test: add test
* fix: Update test with correct query
2023-08-17 17:00:04 +00:00
7e2f85a24e
feat: accept relative memory size for memory pools ( #8503 )
...
Closes https://github.com/influxdata/idpe/issues/18006 .
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-08-17 11:53:27 +00:00
7094189004
chore(deps): Bump tokio from 1.31.0 to 1.32.0 ( #8507 )
...
Bumps [tokio](https://github.com/tokio-rs/tokio ) from 1.31.0 to 1.32.0.
- [Release notes](https://github.com/tokio-rs/tokio/releases )
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.31.0...tokio-1.32.0 )
---
updated-dependencies:
- dependency-name: tokio
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-08-17 08:06:29 +00:00
b6a4213b4f
Merge branch 'main' into savage/remove-router-health-check-error-window-configuration
2023-08-16 11:33:59 +00:00
34f63a67dc
Merge branch 'main' into savage/remove-router-health-check-error-window-configuration
2023-08-16 11:33:32 +00:00
33900936da
chore(deps): Bump serde_json from 1.0.104 to 1.0.105
...
Bumps [serde_json](https://github.com/serde-rs/json ) from 1.0.104 to 1.0.105.
- [Release notes](https://github.com/serde-rs/json/releases )
- [Commits](https://github.com/serde-rs/json/compare/v1.0.104...v1.0.105 )
---
updated-dependencies:
- dependency-name: serde_json
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-08-16 11:00:44 +00:00
fff313b80c
chore(deps): Bump thiserror from 1.0.44 to 1.0.46 ( #8496 )
...
Bumps [thiserror](https://github.com/dtolnay/thiserror ) from 1.0.44 to 1.0.46.
- [Release notes](https://github.com/dtolnay/thiserror/releases )
- [Commits](https://github.com/dtolnay/thiserror/compare/1.0.44...1.0.46 )
---
updated-dependencies:
- dependency-name: thiserror
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-08-16 10:54:47 +00:00
34b8585931
chore(deps): Bump tokio from 1.30.0 to 1.31.0 ( #8482 )
...
Bumps [tokio](https://github.com/tokio-rs/tokio ) from 1.30.0 to 1.31.0.
- [Release notes](https://github.com/tokio-rs/tokio/releases )
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.30.0...tokio-1.31.0 )
---
updated-dependencies:
- dependency-name: tokio
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-08-14 06:32:34 +00:00
3675043585
chore(deps): Bump tokio from 1.29.1 to 1.30.0 ( #8464 )
...
Bumps [tokio](https://github.com/tokio-rs/tokio ) from 1.29.1 to 1.30.0.
- [Release notes](https://github.com/tokio-rs/tokio/releases )
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.29.1...tokio-1.30.0 )
---
updated-dependencies:
- dependency-name: tokio
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-08-10 07:50:18 +00:00
25ab230898
refactor(flightsql): Use upstream implementation of XdbcTypeInfo builder, support filter by datatype ( #8455 )
...
* refactor(flightsql): Use upstream implementation of XdbyTypeInfo builder
* fix: Update test
---------
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-08-08 20:16:39 +00:00
a3ab4d33da
refactor(router): Revert configurable health-check `ERROR_WINDOW`
...
Configuring the `ERROR_WINDOW` of the router's on-path health check
did not provide a consistent improvement for low write volume clusters.
Now that the `NUM_PROBES` parameter is configurable, this can be
un-exposed to simplify configuration options and clean up boiler plate.
2023-08-08 17:49:14 +01:00
8e2662befb
feat(influxql): minimal SHOW RETENTION POLICIES implementation ( #8433 )
...
For backwards compatibility with version 1 clients support a minimal
implementation of SHOW RETENTION POLICIES. This advertises a single,
default, retention policy for any database. This is suffiecient for
compatibility with the grafana plugin and avoids the need for
cross-database catalog queries in the querier.
The values used for the "duration", "shardGroupDuration", and
"resourceN" are the same canned values used in the cloud 2
implementation.
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-08-07 13:39:53 +00:00
db1429b37d
chore(deps): Bump serde from 1.0.181 to 1.0.183 ( #8431 )
...
Bumps [serde](https://github.com/serde-rs/serde ) from 1.0.181 to 1.0.183.
- [Release notes](https://github.com/serde-rs/serde/releases )
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.181...v1.0.183 )
---
updated-dependencies:
- dependency-name: serde
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-08-07 10:24:11 +00:00
0e478af470
chore(deps): Bump tempfile from 3.7.0 to 3.7.1 ( #8429 )
...
Bumps [tempfile](https://github.com/Stebalien/tempfile ) from 3.7.0 to 3.7.1.
- [Changelog](https://github.com/Stebalien/tempfile/blob/master/CHANGELOG.md )
- [Commits](https://github.com/Stebalien/tempfile/compare/v3.7.0...v3.7.1 )
---
updated-dependencies:
- dependency-name: tempfile
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-08-07 09:21:11 +00:00
5530d90074
chore(deps): Bump serde from 1.0.180 to 1.0.181 ( #8413 )
...
Bumps [serde](https://github.com/serde-rs/serde ) from 1.0.180 to 1.0.181.
- [Release notes](https://github.com/serde-rs/serde/releases )
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.180...v1.0.181 )
---
updated-dependencies:
- dependency-name: serde
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-08-04 08:11:16 +00:00
81b5d80a91
feat(idpe-17935): move filtering of skipped partitions to the scheduler ( #8358 )
...
* catalog.get_in_skipped_compaction() should handle for multiple partitions
* add the ability to perform transformation on sets of partitions (rather than filtering one by one). Start with the transformation to remove skipped partitions, in the scheduler.
* move the env var and cli flag setting, for when to ignore skipped partitions, to the scheduler config.
2023-08-03 11:43:09 -07:00
6e13ff8cb8
chore: Update DataFusion pin ( #8390 )
...
* chore: Update DataFusion pin
* chore: Update for API
* fix: update plans
---------
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-08-02 14:58:16 +00:00
c8242c7469
chore(cli): add `--partition-template` to `namespace create` ( #8365 )
...
* chore(cli): add `--partition-template` to namespace create
* chore: fix typo in doc for `PartitionTemplateConfig`
chore: add max limit 8 for partition template in doc
* chore: add e2e tests
* chore: fmt
* chore: add more e2e tests for namespace create with partition template
* chore: show doc comments in cli help interface
---------
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-08-01 14:37:00 +00:00
e4a5d2efaa
feat(router): Expose `num_probes` request count used to health-check ingesters as config option
...
This allows routers to be configured to mark downstreams as healthy/
unhealthy with a requirement for the number of probe requests
which can/must be collected to transition the health checkers circuit
state to healthy/unhealthy.
2023-08-01 14:21:56 +01:00
72feefc3cc
chore(deps): Bump serde from 1.0.179 to 1.0.180 ( #8376 )
...
Bumps [serde](https://github.com/serde-rs/serde ) from 1.0.179 to 1.0.180.
- [Release notes](https://github.com/serde-rs/serde/releases )
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.179...v1.0.180 )
---
updated-dependencies:
- dependency-name: serde
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-08-01 08:24:30 +00:00
de79619e71
chore: Update datafusion ( #8355 )
...
* chore: Update datafusion pin
* fix: Update for change in API
* chore: Update plan
---------
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-07-31 15:41:00 +00:00
e98188b181
Merge branch 'main' into dom/enable-gossip
2023-07-31 15:27:44 +01:00
4a9e76b8b7
feat: Make parquet_file.partition_id optional in the catalog ( #8339 )
...
* feat: Make parquet_file.partition_id optional in the catalog
This will acquire a short lock on the table in postgres, per:
<https://stackoverflow.com/questions/52760971/will-making-column-nullable-lock-the-table-for-reads >
This allows us to persist data for new partitions and associate the
Parquet file catalog records with the partition records using only the
partition hash ID, rather than both that are used now.
* fix: Support transition partition ID in the catalog service
* fix: Use transition partition ID in import/export
This commit also removes support for the `--partition-id` flag of the
`influxdb_iox remote store get-table` command, which Andrew approved.
The `--partition-id` filter was getting the results of the catalog gRPC
service's query for Parquet files of a table and then keeping only the
files whose partition IDs matched. The gRPC query is no longer returning
the partition ID from the Parquet file table, and really, this command
should instead be using `GetParquetFilesByPartitionId` to only request
what's needed rather than filtering.
* feat: Support looking up Parquet files by either kind of Partition id
Regardless of which is actually stored on the Parquet file record.
That is, say there's a Partition in the catalog with:
Partition {
id: 3,
hash_id: abcdefg,
}
and a Parquet file that has:
ParquetFile {
partition_hash_id: abcdefg,
}
calling `list_by_partition_not_to_delete(PartitionId(3))` should still
return this Parquet file because it is associated with the partition
that has ID 3.
This is important for the compactor, which is currently only dealing in
PartitionIds, and I'd like to keep it that way for now to avoid having
to change Even More in this PR.
* fix: Use and set new partition ID fields everywhere they want to be
---------
Co-authored-by: Dom <dom@itsallbroken.com>
2023-07-31 12:40:56 +00:00
8da08fa574
feat(router): optionally enable gossip subsystem
...
Allows the router to optionally enable and start the gossip subsystem
(disabled by default).
No code uses the gossip system, so no application-level messages are
exchanged, but this allows the gossip subsystem to run and exchange
control frames / perform discovery / etc.
2023-07-31 11:01:30 +02:00
1ec1b9155a
feat(ingester): optional gossip configuration
...
Exposes configuration parameters (on the ingester only) for
configuration of the gossip sub-system.
2023-07-31 11:01:25 +02:00
4fa6ead27d
chore(deps): Bump serde from 1.0.177 to 1.0.179
...
Bumps [serde](https://github.com/serde-rs/serde ) from 1.0.177 to 1.0.179.
- [Release notes](https://github.com/serde-rs/serde/releases )
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.177...v1.0.179 )
---
updated-dependencies:
- dependency-name: serde
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-07-31 08:38:48 +00:00
940f441259
chore(deps): Bump tikv-jemalloc-ctl from 0.5.0 to 0.5.4 ( #8359 )
...
Bumps [tikv-jemalloc-ctl](https://github.com/tikv/jemallocator ) from 0.5.0 to 0.5.4.
- [Release notes](https://github.com/tikv/jemallocator/releases )
- [Changelog](https://github.com/tikv/jemallocator/blob/main/CHANGELOG.md )
- [Commits](https://github.com/tikv/jemallocator/compare/0.5.0...0.5.4 )
---
updated-dependencies:
- dependency-name: tikv-jemalloc-ctl
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Dom <dom@itsallbroken.com>
2023-07-28 12:45:43 +00:00
29d66444e5
chore(deps): Bump serde from 1.0.176 to 1.0.177
...
Bumps [serde](https://github.com/serde-rs/serde ) from 1.0.176 to 1.0.177.
- [Release notes](https://github.com/serde-rs/serde/releases )
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.176...v1.0.177 )
---
updated-dependencies:
- dependency-name: serde
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-07-28 10:40:15 +00:00
Fraser Savage
Marco Neumann
Carol (Nichols || Goulding)
dependabot[bot]
Andrew Lamb
Carol (Nichols || Goulding)
Nga Tran
kodiakhq[bot]
Martin Hilton
wiedld
Chunchun Ye
Dom