Returns a `NotImplemented` error when attempting to execute a
selector query, which projects a single selector function and additional
tags or fields until #7533 is implemented.
Introduced `error` module to simplify error handling and ensure
consistency of error messages.
This will be used to complete queries that have selector semantics,
meaning they project a single selector function and therefore
use the timestamp for the time column.
This will be used to complete queries that have selector semantics,
meaning they project a single selector function and therefore
use the timestamp for the time column.
This PR makes 3 improvements.
* It adds the configured sleep interval at the start of the object store
checker to avoid issues with making a remote list immediately at
startup. We see issues with the s3 api.
* the --dry-run flag was stopping deletes of objects from object store,
but the retention flagger was still making updates to the catalog.
These writes to the catalog are surprising when the --dry-run flag is
provided. Now, with --dry-run the catalog is not updated. The logging
instead says how many records would be updated because of retention.
* It decreases logging in should_delete of the checker as it will be
extremely noisey when reporting files it skips. An internal
environment has 3.8 million parquet files, most of which would be
skipped.
* related to #7363
* fixesinfluxdata/idpe#17451
It's a little weird to have such similar types and have to convert them,
but doing this prevents too many crates from having to depend on/know
about each other.
This creates a separate option for the number of minutes *without* a
write that a partition must have before being considered for cold
compaction.
This is a new CLI flag so that it can have a different default from hot
compaction's compaction_partition_minute_threshold.
I didn't add "hot" to compaction_partition_minute_threshold's name so
that k8s-idpe doesn't have to change to continue running hot compaction
as it is today.
Then use the relevant threshold earlier, when creating the
PartitionsSourceConfig, to make it clearer which threshold is used
where.
Right now, this will silently ignore any CLI flag specified that isn't
relevant to the current compaction mode. We might want to change that to
warn or error to save debugging time in the future.
Reduce N concurrent partition fetch requests for the same partition into
a single catalog query. This prevents multiple queries executing when
all but one result is thrown away.
This removes a potential request amplification when the catalog is
unavailable, where a number of queries for the same partition execute,
see an error, and retry forever (with backoff) until the catalog
recovers, while more catalog queries are started.
Changes the PartitionResolver trait to return a ref-counted
PartitionData instance, instead of a plain PartitionData (which is then
wrapped in an Arc anyway).
This allows resolver implementations to return multiple references to
the same physical instance.
* chore: Update datafusion and arrow/parquet to 37, tonic to 0.9.1
* refactor: Update for FieldRef and other API changes
* fix: Update field size calculation
* fix: Use `NullBuffer` directly
* fix: remove outdated comment
* chore: Update test for tonic
* chore: Run cargo hakari tasks
* chore: cargo update
---------
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
* feat: support CommandGetCrossReference metadata endpoint with tests
* chore: create two tables in the test for GetCrossReference endpoint
---------
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
* chore: support returning the database name if all the keys refer to the same database
* test: add test cases to check for same, different, and no database in request header
* chore: lint
* chore: more lint
* refactor: replace empty string with None for database_name
* refactor: simplify logic for NoFlightSQLDatabase error
---------
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
* feat: support CommandGetImportedKeys metadata endpoint with tests
* chore: remove comments that is no longer valid
---------
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
Stuart Carnie
Phil Bracikowski
Carol (Nichols || Goulding)
Andrew Lamb
kodiakhq[bot]
Dom
wiedld
dependabot[bot]
wiedld
Chunchun Ye