167cde1838
fix(iox): Add transition shard to catalog setup ( #6799 )
2023-02-01 17:04:03 +00:00
7590fa5aaa
fix: Change test data and predicates to get better coverage
...
These two tests weren't actually ensuring that the combination of these
predicates worked, because the tests would still pass if some of the
predicate parts were removed.
2023-02-01 11:09:10 -05:00
c77e1214ff
Merge pull request #6788 from influxdata/dom/fatal-panics
...
feat: fatal panics for ingester2 and router2
2023-02-01 15:55:20 +00:00
cffc4d7233
Merge branch 'main' into dom/fatal-panics
2023-02-01 15:43:52 +00:00
d16eaac64b
refactor: ensure that we only have a partition ID ONCE within the pipeline ( #6774 )
...
* refactor: `MockPartitionDoneSink::{errors => results}`
* feat: ensure we don't concurrently process the same partition
Helps w/ #6750 .
* test: extend `test_unique`
* docs: explain `unique_partitions`
---------
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-02-01 15:42:40 +00:00
7dace5a50c
Merge branch 'main' into dom/fatal-panics
2023-02-01 15:34:10 +00:00
0a20bd404e
feat: allow selection of different compactor algo versions ( #6800 )
...
Required to lift&shift to hot-cold compaction w/ keeping the codebase
maintainable.
2023-02-01 15:33:41 +00:00
a0b0bb0a93
refactor: fatal panics for ingester2, not ingester
...
I put the calls in the wrong "ingester".
2023-02-01 16:26:24 +01:00
45c08d58b1
feat(router): fatal panics
...
Panics are exceptional, fatal events.
2023-02-01 16:26:24 +01:00
246e3a4684
feat(ingester): fatal panics
...
Panics are exceptional, fatal events.
2023-02-01 16:26:23 +01:00
f2ecfa2adb
feat(iox): fatal panics
...
Allows code to opt-in to fatal panic semantics instead of just logging
them using our normal handler.
2023-02-01 16:26:23 +01:00
d49dd48900
fix: Clean up remaining query_tests influx RPC helper code
2023-02-01 10:26:14 -05:00
072e228d35
test: Port tag_values query_tests to end-to-end tests
2023-02-01 10:26:03 -05:00
629867c6ac
refactor: Start a new file for tag values tests; move the one existing test
2023-02-01 10:25:48 -05:00
967450f0de
Merge pull request #6798 from influxdata/dom/doc-retention-0
...
docs: retention period value mappings
2023-02-01 14:11:57 +00:00
77ff4dc4b8
docs: retention period values in Namespace client
...
Document the 0/null and negative value behaviours for retention periods
in the namespace RPC client.
2023-02-01 14:41:10 +01:00
52ac1b97a9
docs: namespace retention protobuf mappings
...
Document that the caller can specify 0 or NULL for an infinite retention
period, and that IOx will respond with NULL.
Document that negative retention periods are rejected.
2023-02-01 14:37:21 +01:00
a8cb3aeba8
Merge pull request #6795 from influxdata/dom/0-retention
...
fix(router): 0 value retention periods
2023-02-01 11:44:07 +00:00
ab32781cf1
Merge branch 'main' into dom/0-retention
2023-02-01 11:36:34 +00:00
b8b59bf73b
feat: allow partition filters to error + flag partitions ( #6796 )
...
* refactor: error handling clean-up and testing
* feat: allow partition filters to error + flag partitions
Closes #6748 .
2023-02-01 11:34:53 +00:00
9ec92eedbf
Merge branch 'main' into dom/0-retention
2023-02-01 11:10:44 +00:00
c8da5a8e07
refactor: extract early ID-based partition filters into new components ( #6794 )
...
This should greatly improve efficiency of the two filters ("by ID" -
used for mocking / tests / dev, "shard"). This also changes the metrics
slightly since the partitions filtered there no longer count into the
overall "backlog" (which makes sense for the two filters).
I've left the "never skipped" filter where it is, because it needs to
perform IO (i.e. shouldn't be done for all partitions upfront) and we
shall only have a few skipped partitions, so it's not such a big if
needlessly fetch the parquet files for them.
Closes #6783 .
2023-02-01 11:08:47 +00:00
7f363b55df
test(router): e2e namespace retention coverage
...
Assert the correct handling of 0 and negative retention periods when
interacting with the namespace create & update gRPC handlers.
2023-02-01 11:49:53 +01:00
ec5b35e3cc
fix: reject negative retention periods
...
Do not allow a negative retention period to be specified.
2023-02-01 11:29:56 +01:00
bf6ae3dc4d
fix: map retention period 0 values to NULL
...
Treat a retention period of 0 as "infinite", and not "none".
2023-02-01 11:29:13 +01:00
bfbd939ef3
Merge pull request #6793 from influxdata/dependabot/cargo/uuid-1.3.0
...
chore(deps): Bump uuid from 1.2.2 to 1.3.0
2023-02-01 09:47:11 +00:00
1a9c27cd9a
chore(deps): Bump uuid from 1.2.2 to 1.3.0
...
Bumps [uuid](https://github.com/uuid-rs/uuid ) from 1.2.2 to 1.3.0.
- [Release notes](https://github.com/uuid-rs/uuid/releases )
- [Commits](https://github.com/uuid-rs/uuid/compare/1.2.2...1.3.0 )
---
updated-dependencies:
- dependency-name: uuid
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-02-01 09:32:39 +00:00
27eb2f9096
Merge pull request #6791 from influxdata/dependabot/cargo/bytes-1.4.0
...
chore(deps): Bump bytes from 1.3.0 to 1.4.0
2023-02-01 09:28:57 +00:00
d0e6b16450
chore(deps): Bump bytes from 1.3.0 to 1.4.0
...
Bumps [bytes](https://github.com/tokio-rs/bytes ) from 1.3.0 to 1.4.0.
- [Release notes](https://github.com/tokio-rs/bytes/releases )
- [Changelog](https://github.com/tokio-rs/bytes/blob/master/CHANGELOG.md )
- [Commits](https://github.com/tokio-rs/bytes/compare/v1.3.0...v1.4.0 )
---
updated-dependencies:
- dependency-name: bytes
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-02-01 00:30:56 +00:00
57f55e14c8
feat: IOx InfluxQL planner learns how to process time range expressions ( #6772 )
...
* feat: IOx learns InfluxQL time-range expression → DF logical Expr
IOx now understand the how to evaluate an InfluxQL time-range filter
expression and transform that to a DataFusion logical expression.
* chore: move time range expression to independent functions
There is no need for these to be part of the `InfluxQLToLogicalPlan`
struct and makes them easier to test.
* chore: support scalar now on either side of binary expression
* chore: improve error messages
* chore: address clippy concerns
* chore: add tests for time ranges
* chore: add a test where time appears on the right-hand side
Ensure time is correctly identified on the right-hand side of a
conditional expression.
* chore: add tests that specify a timezone
* chore: Run cargo hakari tasks
* chore: fix linting issues
* chore: Remove unnecessary line
* chore: Feedback: Add API to parse a conditional expression
Based on feedback from @alamb, we don't want to hide the error from
parsing a `ConditionalExpression`. To do this, we use the
public API, `parse_statements` as a model and provide a new API,
`parse_conditional_expression`, which returns a `Result` with the error
being a `ParseError`. Additionally, `ConditionalExpression` implements
the `FromStr` API using the `parse_conditional_expression` API.
* chore: PR feedback reverting this change
I believe my intention was to update all instances in the match, but
never completed the change. Will leave for another day.
* chore: PR feedback add additional comments
* chore: rustfmt
---------
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
2023-02-01 00:27:17 +00:00
38e128fad8
chore(deps): Bump zstd from 0.12.2+zstd.1.5.2 to 0.12.3+zstd.1.5.2 ( #6785 )
...
Bumps [zstd](https://github.com/gyscos/zstd-rs ) from 0.12.2+zstd.1.5.2 to 0.12.3+zstd.1.5.2.
- [Release notes](https://github.com/gyscos/zstd-rs/releases )
- [Commits](https://github.com/gyscos/zstd-rs/commits )
---
updated-dependencies:
- dependency-name: zstd
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-31 16:47:39 +00:00
5035017a58
chore(deps): Bump zstd-sys from 2.0.5+zstd.1.5.2 to 2.0.6+zstd.1.5.2 ( #6784 )
...
Bumps [zstd-sys](https://github.com/gyscos/zstd-rs ) from 2.0.5+zstd.1.5.2 to 2.0.6+zstd.1.5.2.
- [Release notes](https://github.com/gyscos/zstd-rs/releases )
- [Commits](https://github.com/gyscos/zstd-rs/commits )
---
updated-dependencies:
- dependency-name: zstd-sys
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-31 15:42:02 +00:00
ad390131c4
chore(deps): Bump zstd-safe from 6.0.2+zstd.1.5.2 to 6.0.3+zstd.1.5.2 ( #6786 )
...
Bumps [zstd-safe](https://github.com/gyscos/zstd-rs ) from 6.0.2+zstd.1.5.2 to 6.0.3+zstd.1.5.2.
- [Release notes](https://github.com/gyscos/zstd-rs/releases )
- [Commits](https://github.com/gyscos/zstd-rs/commits )
---
updated-dependencies:
- dependency-name: zstd-safe
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-01-31 15:19:36 +00:00
d7d8c84203
chore: preserve `query_tests2` output in CI ( #6782 )
...
This shall help to debug flaky tests.
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-01-31 15:11:15 +00:00
fa15a48c31
refactor: use types rather than `Any` in `flightsql` interface ( #6705 )
...
* refactor: use types rather than Any in flightsql interface
* fix: Update service_grpc_flight/src/lib.rs
Co-authored-by: Raphael Taylor-Davies <1781103+tustvold@users.noreply.github.com>
* chore: review feedback
* fix: fmt
* fix: error conversion
---------
Co-authored-by: Raphael Taylor-Davies <1781103+tustvold@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-01-31 14:45:46 +00:00
62697265c1
feat: compactor sharding ( #6729 )
...
I'm not saying we have to use this, but this is a demonstration how easy
it would be to add sharding to the compaction tier and also acts as a
"backup / insurance" if we ever need it.
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-01-31 14:37:06 +00:00
80f0125940
feat: Add number of rows to explain of RecordBatchesExec ( #6781 )
...
* feat: Add number of rows to explain of RecordBatchesExec
* fix: Update test output
2023-01-31 14:26:20 +00:00
ff32a042b0
test: Port tag_keys query_tests to end-to-end tests ( #6760 )
...
* refactor: Start a new file for tag keys tests; move the one existing test
* test: Port tag_keys query_tests to end-to-end tests
---------
Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
2023-01-31 14:05:22 +00:00
9434a8f5d3
Merge pull request #6779 from influxdata/dom/disable-ns-delete
...
refactor: disable ns delete
2023-01-31 13:44:44 +00:00
363ce1629c
refactor: log namespace RPC actions
...
Adds info logs for namespace CRUD actions.
2023-01-31 14:36:15 +01:00
c70ccee4a1
fix: disable namespace delete gRPC endpoint
...
Do not allow this endpoint to be called - the logic is incorrect.
2023-01-31 14:34:44 +01:00
d18d2c34e4
test: Port measurement_names/table_names query_tests to end-to-end tests ( #6757 )
...
* refactor: Start a new file for measurement names tests; move the one existing test
* fix: Pass on predicate when sending a measurement names request with GrpcRequestBuilder
* feat: Support literal integer queries too
* test: Port measurement_names/table_names query_tests to end-to-end tests
* fix: merge conflict error
---------
Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-01-31 12:55:30 +00:00
e8e50df692
chore: Enable last SQL test, retention.sql ( #6721 )
2023-01-31 12:46:50 +00:00
cff422b795
test: Port read_window_aggregate query_tests to end-to-end tests ( #6755 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-01-31 12:15:50 +00:00
7cadd38a3c
fix: do not panic when partition was removed from catalog ( #6773 )
...
Fixes https://github.com/influxdata/idpe/issues/17040 .
2023-01-31 11:54:34 +00:00
b682ffcb36
refactor: add partition ID to commit ( #6775 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-01-31 11:41:25 +00:00
875b6a3e99
chore(deps): Bump futures from 0.3.25 to 0.3.26 ( #6766 )
...
Bumps [futures](https://github.com/rust-lang/futures-rs ) from 0.3.25 to 0.3.26.
- [Release notes](https://github.com/rust-lang/futures-rs/releases )
- [Changelog](https://github.com/rust-lang/futures-rs/blob/master/CHANGELOG.md )
- [Commits](https://github.com/rust-lang/futures-rs/compare/0.3.25...0.3.26 )
---
updated-dependencies:
- dependency-name: futures
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-01-31 11:33:50 +00:00
cb8254e2eb
Merge pull request #6776 from influxdata/dom/log-ids
...
refactor: log IDs in panic messages
2023-01-31 11:26:19 +00:00
93ad02dc2e
Merge branch 'main' into dom/log-ids
2023-01-31 11:19:04 +00:00
a90ca290b2
chore(deps): Bump futures-util from 0.3.25 to 0.3.26 ( #6767 )
...
Bumps [futures-util](https://github.com/rust-lang/futures-rs ) from 0.3.25 to 0.3.26.
- [Release notes](https://github.com/rust-lang/futures-rs/releases )
- [Changelog](https://github.com/rust-lang/futures-rs/blob/master/CHANGELOG.md )
- [Commits](https://github.com/rust-lang/futures-rs/compare/0.3.25...0.3.26 )
---
updated-dependencies:
- dependency-name: futures-util
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-31 11:17:32 +00:00
Marko Mikulicic
Carol (Nichols || Goulding)
Dom
Marco Neumann
kodiakhq[bot]
dependabot[bot]
Stuart Carnie
Andrew Lamb
Carol (Nichols || Goulding)