57f55e14c8
feat: IOx InfluxQL planner learns how to process time range expressions ( #6772 )
...
* feat: IOx learns InfluxQL time-range expression → DF logical Expr
IOx now understand the how to evaluate an InfluxQL time-range filter
expression and transform that to a DataFusion logical expression.
* chore: move time range expression to independent functions
There is no need for these to be part of the `InfluxQLToLogicalPlan`
struct and makes them easier to test.
* chore: support scalar now on either side of binary expression
* chore: improve error messages
* chore: address clippy concerns
* chore: add tests for time ranges
* chore: add a test where time appears on the right-hand side
Ensure time is correctly identified on the right-hand side of a
conditional expression.
* chore: add tests that specify a timezone
* chore: Run cargo hakari tasks
* chore: fix linting issues
* chore: Remove unnecessary line
* chore: Feedback: Add API to parse a conditional expression
Based on feedback from @alamb, we don't want to hide the error from
parsing a `ConditionalExpression`. To do this, we use the
public API, `parse_statements` as a model and provide a new API,
`parse_conditional_expression`, which returns a `Result` with the error
being a `ParseError`. Additionally, `ConditionalExpression` implements
the `FromStr` API using the `parse_conditional_expression` API.
* chore: PR feedback reverting this change
I believe my intention was to update all instances in the match, but
never completed the change. Will leave for another day.
* chore: PR feedback add additional comments
* chore: rustfmt
---------
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
2023-02-01 00:27:17 +00:00
38e128fad8
chore(deps): Bump zstd from 0.12.2+zstd.1.5.2 to 0.12.3+zstd.1.5.2 ( #6785 )
...
Bumps [zstd](https://github.com/gyscos/zstd-rs ) from 0.12.2+zstd.1.5.2 to 0.12.3+zstd.1.5.2.
- [Release notes](https://github.com/gyscos/zstd-rs/releases )
- [Commits](https://github.com/gyscos/zstd-rs/commits )
---
updated-dependencies:
- dependency-name: zstd
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-31 16:47:39 +00:00
5035017a58
chore(deps): Bump zstd-sys from 2.0.5+zstd.1.5.2 to 2.0.6+zstd.1.5.2 ( #6784 )
...
Bumps [zstd-sys](https://github.com/gyscos/zstd-rs ) from 2.0.5+zstd.1.5.2 to 2.0.6+zstd.1.5.2.
- [Release notes](https://github.com/gyscos/zstd-rs/releases )
- [Commits](https://github.com/gyscos/zstd-rs/commits )
---
updated-dependencies:
- dependency-name: zstd-sys
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-31 15:42:02 +00:00
ad390131c4
chore(deps): Bump zstd-safe from 6.0.2+zstd.1.5.2 to 6.0.3+zstd.1.5.2 ( #6786 )
...
Bumps [zstd-safe](https://github.com/gyscos/zstd-rs ) from 6.0.2+zstd.1.5.2 to 6.0.3+zstd.1.5.2.
- [Release notes](https://github.com/gyscos/zstd-rs/releases )
- [Commits](https://github.com/gyscos/zstd-rs/commits )
---
updated-dependencies:
- dependency-name: zstd-safe
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-01-31 15:19:36 +00:00
d7d8c84203
chore: preserve `query_tests2` output in CI ( #6782 )
...
This shall help to debug flaky tests.
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-01-31 15:11:15 +00:00
fa15a48c31
refactor: use types rather than `Any` in `flightsql` interface ( #6705 )
...
* refactor: use types rather than Any in flightsql interface
* fix: Update service_grpc_flight/src/lib.rs
Co-authored-by: Raphael Taylor-Davies <1781103+tustvold@users.noreply.github.com>
* chore: review feedback
* fix: fmt
* fix: error conversion
---------
Co-authored-by: Raphael Taylor-Davies <1781103+tustvold@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-01-31 14:45:46 +00:00
62697265c1
feat: compactor sharding ( #6729 )
...
I'm not saying we have to use this, but this is a demonstration how easy
it would be to add sharding to the compaction tier and also acts as a
"backup / insurance" if we ever need it.
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-01-31 14:37:06 +00:00
80f0125940
feat: Add number of rows to explain of RecordBatchesExec ( #6781 )
...
* feat: Add number of rows to explain of RecordBatchesExec
* fix: Update test output
2023-01-31 14:26:20 +00:00
ff32a042b0
test: Port tag_keys query_tests to end-to-end tests ( #6760 )
...
* refactor: Start a new file for tag keys tests; move the one existing test
* test: Port tag_keys query_tests to end-to-end tests
---------
Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
2023-01-31 14:05:22 +00:00
9434a8f5d3
Merge pull request #6779 from influxdata/dom/disable-ns-delete
...
refactor: disable ns delete
2023-01-31 13:44:44 +00:00
363ce1629c
refactor: log namespace RPC actions
...
Adds info logs for namespace CRUD actions.
2023-01-31 14:36:15 +01:00
c70ccee4a1
fix: disable namespace delete gRPC endpoint
...
Do not allow this endpoint to be called - the logic is incorrect.
2023-01-31 14:34:44 +01:00
d18d2c34e4
test: Port measurement_names/table_names query_tests to end-to-end tests ( #6757 )
...
* refactor: Start a new file for measurement names tests; move the one existing test
* fix: Pass on predicate when sending a measurement names request with GrpcRequestBuilder
* feat: Support literal integer queries too
* test: Port measurement_names/table_names query_tests to end-to-end tests
* fix: merge conflict error
---------
Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-01-31 12:55:30 +00:00
e8e50df692
chore: Enable last SQL test, retention.sql ( #6721 )
2023-01-31 12:46:50 +00:00
cff422b795
test: Port read_window_aggregate query_tests to end-to-end tests ( #6755 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-01-31 12:15:50 +00:00
7cadd38a3c
fix: do not panic when partition was removed from catalog ( #6773 )
...
Fixes https://github.com/influxdata/idpe/issues/17040 .
2023-01-31 11:54:34 +00:00
b682ffcb36
refactor: add partition ID to commit ( #6775 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-01-31 11:41:25 +00:00
875b6a3e99
chore(deps): Bump futures from 0.3.25 to 0.3.26 ( #6766 )
...
Bumps [futures](https://github.com/rust-lang/futures-rs ) from 0.3.25 to 0.3.26.
- [Release notes](https://github.com/rust-lang/futures-rs/releases )
- [Changelog](https://github.com/rust-lang/futures-rs/blob/master/CHANGELOG.md )
- [Commits](https://github.com/rust-lang/futures-rs/compare/0.3.25...0.3.26 )
---
updated-dependencies:
- dependency-name: futures
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-01-31 11:33:50 +00:00
cb8254e2eb
Merge pull request #6776 from influxdata/dom/log-ids
...
refactor: log IDs in panic messages
2023-01-31 11:26:19 +00:00
93ad02dc2e
Merge branch 'main' into dom/log-ids
2023-01-31 11:19:04 +00:00
a90ca290b2
chore(deps): Bump futures-util from 0.3.25 to 0.3.26 ( #6767 )
...
Bumps [futures-util](https://github.com/rust-lang/futures-rs ) from 0.3.25 to 0.3.26.
- [Release notes](https://github.com/rust-lang/futures-rs/releases )
- [Changelog](https://github.com/rust-lang/futures-rs/blob/master/CHANGELOG.md )
- [Commits](https://github.com/rust-lang/futures-rs/compare/0.3.25...0.3.26 )
---
updated-dependencies:
- dependency-name: futures-util
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-31 11:17:32 +00:00
97342fcdc7
chore(deps): Bump futures-io from 0.3.25 to 0.3.26 ( #6765 )
...
Bumps [futures-io](https://github.com/rust-lang/futures-rs ) from 0.3.25 to 0.3.26.
- [Release notes](https://github.com/rust-lang/futures-rs/releases )
- [Changelog](https://github.com/rust-lang/futures-rs/blob/master/CHANGELOG.md )
- [Commits](https://github.com/rust-lang/futures-rs/compare/0.3.25...0.3.26 )
---
updated-dependencies:
- dependency-name: futures-io
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-31 11:15:41 +00:00
efe684e8f0
Merge branch 'main' into dom/log-ids
2023-01-31 11:00:22 +00:00
51b0483699
chore(deps): Bump futures-task from 0.3.25 to 0.3.26 ( #6768 )
...
Bumps [futures-task](https://github.com/rust-lang/futures-rs ) from 0.3.25 to 0.3.26.
- [Release notes](https://github.com/rust-lang/futures-rs/releases )
- [Changelog](https://github.com/rust-lang/futures-rs/blob/master/CHANGELOG.md )
- [Commits](https://github.com/rust-lang/futures-rs/compare/0.3.25...0.3.26 )
---
updated-dependencies:
- dependency-name: futures-task
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-31 10:58:40 +00:00
a029de95f4
refactor: panic with unresolvable table IDs
...
Include the table ID in the panic message.
2023-01-31 11:47:33 +01:00
6e540bc8d6
refactor: panic with unresolvable namespace IDs
...
Include the namespace ID in the panic message.
2023-01-31 11:46:40 +01:00
0d9b773693
refactor: panic with unresolvable partition IDs
...
Include the partition ID in the panic message.
2023-01-31 11:43:32 +01:00
6f032b1d57
chore(deps): Bump async-trait from 0.1.63 to 0.1.64 ( #6769 )
...
Bumps [async-trait](https://github.com/dtolnay/async-trait ) from 0.1.63 to 0.1.64.
- [Release notes](https://github.com/dtolnay/async-trait/releases )
- [Commits](https://github.com/dtolnay/async-trait/compare/0.1.63...0.1.64 )
---
updated-dependencies:
- dependency-name: async-trait
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-31 10:18:27 +00:00
17e118d8c3
chore(deps): Bump futures-channel from 0.3.25 to 0.3.26 ( #6770 )
...
Bumps [futures-channel](https://github.com/rust-lang/futures-rs ) from 0.3.25 to 0.3.26.
- [Release notes](https://github.com/rust-lang/futures-rs/releases )
- [Changelog](https://github.com/rust-lang/futures-rs/blob/master/CHANGELOG.md )
- [Commits](https://github.com/rust-lang/futures-rs/compare/0.3.25...0.3.26 )
---
updated-dependencies:
- dependency-name: futures-channel
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-31 10:04:22 +00:00
370a0ef7dd
chore(deps): Bump toml from 0.7.0 to 0.7.1 ( #6771 )
...
Bumps [toml](https://github.com/toml-rs/toml ) from 0.7.0 to 0.7.1.
- [Release notes](https://github.com/toml-rs/toml/releases )
- [Commits](https://github.com/toml-rs/toml/compare/toml-v0.7.0...toml-v0.7.1 )
---
updated-dependencies:
- dependency-name: toml
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-31 09:53:48 +00:00
9279570299
fix: PG `flag_for_delete_by_retention` ( #6762 )
...
* test: failing test
* fix: PG `flag_for_delete_by_retention`
2023-01-30 21:27:12 +00:00
5b14caa780
chore: Update DataFusion ( #6753 )
...
* chore: Update datafusion
* fix: Update for changes
* chore: Run cargo hakari tasks
---------
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-01-30 14:48:52 +00:00
51e324378c
chore: Add test for `SHOW TABLES` ( #6754 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-01-30 14:05:26 +00:00
2254ee06b7
test: Port read_group query_tests to end-to-end tests ( #6743 )
...
* fix: Delete the read filter *file*; last PR only deleted the *contents*
* test: Port read_group query_tests to end-to-end tests
---------
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-01-30 12:10:39 +00:00
b635dda85c
feat: throttle when there are not partitions to compact ( #6752 )
...
* test: `MockPartitionsSource::set`
* test: `AssertFutureExt`
* feat: throttle when there are not partitions to compact
Fixes #6727 .
---------
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-01-30 12:00:03 +00:00
f133cda00c
docs: Add docstring to Scratchpad ( #6742 )
...
* docs: Add docstring to Scratchpad
* fix: put derive macro after docstring
* fix: Update compactor2/src/components/scratchpad/mod.rs
Co-authored-by: Marco Neumann <marco@crepererum.net>
* fix: Update other italic markdown
---------
Co-authored-by: Marco Neumann <marco@crepererum.net>
2023-01-30 11:52:13 +00:00
dc37163a45
Merge pull request #6751 from influxdata/dom/envoy-hax
...
fix(router): envoy network error translation
2023-01-30 11:35:48 +00:00
048f7c455a
Merge branch 'main' into dom/envoy-hax
2023-01-30 11:23:32 +00:00
0ddef54b09
fix(router): envoy network error translation
...
Envoy will connect to an endpoint on demand, and return an
application-level error if it fails with a gRPC status code of
"Unavailable".
It also embeds a metadata entry of {"server": "envoy"} - this commit
uses the two signals (error status code + metadata entry) to drive an
immediate reconnection when observed, assuming the connection is bad.
2023-01-30 12:15:38 +01:00
d707709cce
fix: invalidate querier->ingester conn on error ( #6747 )
...
It seems that tonic is caching DNS results for too long and clings to an
old ingester that no longer exists.
See https://github.com/influxdata/idpe/issues/17022 (not sure though if
this fix is sufficient, let's see).
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-01-30 11:06:12 +00:00
515f0eef64
feat: simple compactor2 self-protection ( #6728 )
...
Add some rough "partition is too big" filter for now until we can deal
with them (the framework allows that but we need to set up the proper
divide-and-conquer components).
This will hopefully prevent our prod compactor from dying that often.
Note that this is also duct-tape around two issues:
- DataFusion not accounting in-flight data all the time
- Our wide fan-out query plans (see https://github.com/influxdata/idpe/issues/16768#issuecomment-1387056833 )
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-01-30 10:57:47 +00:00
7bf609ce7b
Merge pull request #6730 from influxdata/dom/reduce-write-timeout
...
feat: configurable RPC write request timeout
2023-01-30 10:12:20 +00:00
a7770f0f7a
Merge branch 'main' into dom/reduce-write-timeout
2023-01-30 09:59:37 +00:00
ca98cb4d0c
Merge pull request #6745 from influxdata/dependabot/cargo/cc-1.0.79
...
chore(deps): Bump cc from 1.0.78 to 1.0.79
2023-01-30 09:56:14 +00:00
1d46fa1a6f
chore(deps): Bump cc from 1.0.78 to 1.0.79
...
Bumps [cc](https://github.com/rust-lang/cc-rs ) from 1.0.78 to 1.0.79.
- [Release notes](https://github.com/rust-lang/cc-rs/releases )
- [Commits](https://github.com/rust-lang/cc-rs/compare/1.0.78...1.0.79 )
---
updated-dependencies:
- dependency-name: cc
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-01-30 09:45:58 +00:00
f1f5144f8b
Merge pull request #6746 from influxdata/dependabot/cargo/tokio-1.25.0
...
chore(deps): Bump tokio from 1.24.2 to 1.25.0
2023-01-30 09:43:33 +00:00
ed7d02a225
chore(deps): Bump tokio from 1.24.2 to 1.25.0
...
Bumps [tokio](https://github.com/tokio-rs/tokio ) from 1.24.2 to 1.25.0.
- [Release notes](https://github.com/tokio-rs/tokio/releases )
- [Commits](https://github.com/tokio-rs/tokio/commits/tokio-1.25.0 )
---
updated-dependencies:
- dependency-name: tokio
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-01-30 01:57:27 +00:00
55257b46c9
chore: validate ingester URIs on querier CLI ( #6740 )
...
* chore: add validate for ingesters on querier CLI
* chore: fix typo and tests
* chore: clippy
* chore: review feedback
---------
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-01-27 21:13:52 +00:00
4a6b4f78d2
refactor: Make some nicer methods on the GrpcRequestBuilder ( #6741 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-01-27 21:06:22 +00:00
56825f119e
chore(deps): Bump toml from 0.6.0 to 0.7.0 ( #6739 )
...
Bumps [toml](https://github.com/toml-rs/toml ) from 0.6.0 to 0.7.0.
- [Release notes](https://github.com/toml-rs/toml/releases )
- [Commits](https://github.com/toml-rs/toml/compare/toml-v0.6.0...toml-v0.7.0 )
---
updated-dependencies:
- dependency-name: toml
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-01-27 20:44:53 +00:00
Stuart Carnie
dependabot[bot]
Marco Neumann
Andrew Lamb
Carol (Nichols || Goulding)
kodiakhq[bot]
Dom Dwyer
Christopher M. Wolff