1ad0160d30
chore(deps): Bump serde_json from 1.0.94 to 1.0.95 ( #7348 )
...
Bumps [serde_json](https://github.com/serde-rs/json ) from 1.0.94 to 1.0.95.
- [Release notes](https://github.com/serde-rs/json/releases )
- [Commits](https://github.com/serde-rs/json/compare/v1.0.94...v1.0.95 )
---
updated-dependencies:
- dependency-name: serde_json
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-03-28 11:09:59 +00:00
44551e7519
chore(deps): Bump regex from 1.7.2 to 1.7.3 ( #7338 )
...
Bumps [regex](https://github.com/rust-lang/regex ) from 1.7.2 to 1.7.3.
- [Release notes](https://github.com/rust-lang/regex/releases )
- [Changelog](https://github.com/rust-lang/regex/blob/master/CHANGELOG.md )
- [Commits](https://github.com/rust-lang/regex/compare/1.7.2...1.7.3 )
---
updated-dependencies:
- dependency-name: regex
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-03-27 09:20:35 +00:00
6172e6c513
chore(deps): Bump regex from 1.7.1 to 1.7.2 ( #7299 )
...
Bumps [regex](https://github.com/rust-lang/regex ) from 1.7.1 to 1.7.2.
- [Release notes](https://github.com/rust-lang/regex/releases )
- [Changelog](https://github.com/rust-lang/regex/blob/master/CHANGELOG.md )
- [Commits](https://github.com/rust-lang/regex/compare/1.7.1...1.7.2 )
---
updated-dependencies:
- dependency-name: regex
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-03-22 14:45:20 +00:00
535f3d92a2
chore(deps): Bump serde_json from 1.0.93 to 1.0.94 ( #7132 )
...
Bumps [serde_json](https://github.com/serde-rs/json ) from 1.0.93 to 1.0.94.
- [Release notes](https://github.com/serde-rs/json/releases )
- [Commits](https://github.com/serde-rs/json/compare/v1.0.93...v1.0.94 )
---
updated-dependencies:
- dependency-name: serde_json
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-03-06 12:12:23 +00:00
c538cac4ef
chore(deps): Bump tokio from 1.25.0 to 1.26.0 ( #7107 )
...
* chore(deps): Bump tokio from 1.25.0 to 1.26.0
Bumps [tokio](https://github.com/tokio-rs/tokio ) from 1.25.0 to 1.26.0.
- [Release notes](https://github.com/tokio-rs/tokio/releases )
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.25.0...tokio-1.26.0 )
---
updated-dependencies:
- dependency-name: tokio
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* chore: Run cargo hakari tasks
---------
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: Dom <dom@itsallbroken.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-03-02 09:50:39 +00:00
faae5eb438
chore: Rerun cargo hakari manage-deps
2023-02-27 11:56:15 +01:00
b76b75e911
fix: do not panic for unsupported expressions ( #7052 )
...
We see this in one of our prod clusters ATM.
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-02-23 17:03:43 +00:00
6327e3d9c0
chore(deps): Bump serde_json from 1.0.92 to 1.0.93 ( #6918 )
...
Bumps [serde_json](https://github.com/serde-rs/json ) from 1.0.92 to 1.0.93.
- [Release notes](https://github.com/serde-rs/json/releases )
- [Commits](https://github.com/serde-rs/json/compare/v1.0.92...v1.0.93 )
---
updated-dependencies:
- dependency-name: serde_json
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-02-09 10:39:33 +00:00
6f4e287a3a
chore(deps): Bump serde_json from 1.0.91 to 1.0.92 ( #6860 )
...
Bumps [serde_json](https://github.com/serde-rs/json ) from 1.0.91 to 1.0.92.
- [Release notes](https://github.com/serde-rs/json/releases )
- [Commits](https://github.com/serde-rs/json/compare/v1.0.91...v1.0.92 )
---
updated-dependencies:
- dependency-name: serde_json
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-02-06 08:27:41 +00:00
2cdfaba4e2
fix: Remove unnecessary lifetime annotations. Thanks clippy!
...
These lifetime annotations are the same as what the lifetime elision
rules would produce, so they're not needed.
See <https://rust-lang.github.io/rust-clippy/master/index.html#needless_lifetimes >
2023-02-03 13:06:19 -05:00
30fea67701
fix: Move variables within format strings. Thanks clippy!
...
Changes made automatically using `cargo clippy --fix`.
2023-02-03 13:06:17 -05:00
ed7d02a225
chore(deps): Bump tokio from 1.24.2 to 1.25.0
...
Bumps [tokio](https://github.com/tokio-rs/tokio ) from 1.24.2 to 1.25.0.
- [Release notes](https://github.com/tokio-rs/tokio/releases )
- [Commits](https://github.com/tokio-rs/tokio/commits/tokio-1.25.0 )
---
updated-dependencies:
- dependency-name: tokio
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-01-30 01:57:27 +00:00
c68049c37a
chore(deps): Bump regex from 1.7.0 to 1.7.1 ( #6546 )
...
Bumps [regex](https://github.com/rust-lang/regex ) from 1.7.0 to 1.7.1.
- [Release notes](https://github.com/rust-lang/regex/releases )
- [Changelog](https://github.com/rust-lang/regex/blob/master/CHANGELOG.md )
- [Commits](https://github.com/rust-lang/regex/compare/1.7.0...1.7.1 )
---
updated-dependencies:
- dependency-name: regex
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-01-10 09:55:41 +00:00
b49cc2e35e
chore(deps): Bump tokio from 1.24.0 to 1.24.1 ( #6545 )
...
Bumps [tokio](https://github.com/tokio-rs/tokio ) from 1.24.0 to 1.24.1.
- [Release notes](https://github.com/tokio-rs/tokio/releases )
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.24.0...tokio-1.24.1 )
---
updated-dependencies:
- dependency-name: tokio
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-01-10 09:48:44 +00:00
e1036a0c63
refactor: cleanup schema boxing ( #6511 )
...
* refactor: cleanup Schema boxing
* chore: clippy
2023-01-06 10:57:39 +00:00
46ff8854ec
fix: Use code backticks around invalid HTML tags in doc strings
2022-12-21 16:36:17 -05:00
d0d5906476
chore: Update datafusion pin ( #6442 )
...
* chore: Update datafusion pin
* refactor: Update iox_query for new apis
* chore: Update some more apis
* chore: Run cargo hakari tasks
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
2022-12-19 20:02:42 +00:00
7f2aa8b10c
chore(deps): Bump serde_json from 1.0.89 to 1.0.91
...
Bumps [serde_json](https://github.com/serde-rs/json ) from 1.0.89 to 1.0.91.
- [Release notes](https://github.com/serde-rs/json/releases )
- [Commits](https://github.com/serde-rs/json/compare/v1.0.89...v1.0.91 )
---
updated-dependencies:
- dependency-name: serde_json
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2022-12-19 01:44:18 +00:00
080aff8f71
fix: account for memory allocations in InfluxRPC group outputs ( #6345 )
...
* fix: account for memory allocations in InfluxRPC group outputs
This should prevent the querier from OOMing.
See https://github.com/influxdata/idpe/issues/16614 .
* docs: improve
Co-authored-by: Andrew Lamb <alamb@influxdata.com>
* refactor: pull out constant
Co-authored-by: Andrew Lamb <alamb@influxdata.com>
2022-12-08 09:55:31 +00:00
7e43fe57fa
fix: support InfluxRPC OR-chains w/ arbitrary child nodes ( #6343 )
...
* fix: support InfluxRPC OR-chains w/ arbitrary child nodes
Also convert another assertion regarding child nodes of Eq-nodes into a
proper error.
See https://github.com/influxdata/idpe/issues/16582 .
* test: more tests
2022-12-06 18:04:27 +00:00
6cecc439d4
refactor: revert "simplify `SeriesSet` ( #6277 )" ( #6298 )
...
This reverts commit c41200536e
2022-12-01 13:30:19 +00:00
c41200536e
refactor: simplify `SeriesSet` ( #6277 )
...
`RecordBatch` offers zero-copy slicing, so there is no need to store the
row range manually. This makes #6216 simpler.
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-11-30 16:48:31 +00:00
514aa60f91
refactor: stream-based(TM) `to_series_and_groups`, part 1 ( #6261 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-11-29 14:16:22 +00:00
a216c4d0f5
refactor: stream-based series-to-frame conversion ( #6260 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-11-29 12:42:28 +00:00
7b6ce7da5d
refactor: clean-up and stream-based `QueryCompletedToken` handling ( #6244 )
...
* refactor: avoid channels to to create a one-element stream
* refactor: move `StreamWithPermit` into its own module
* refactor: make `QueryCompletedToken` handling stream-based
For #6216 .
* docs: improve
Co-authored-by: Andrew Lamb <alamb@influxdata.com>
Co-authored-by: Andrew Lamb <alamb@influxdata.com>
2022-11-29 09:46:52 +00:00
5e64c2e4b7
refactor: make `ReadResponse` chunking stream-based ( #6239 )
...
* refactor: make `ReadResponse` chunking stream-based
* docs: improve
Co-authored-by: Andrew Lamb <alamb@influxdata.com>
* refactor: error out on oversized frames
Co-authored-by: Andrew Lamb <alamb@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-11-28 17:50:09 +00:00
aa7a3a7721
fix: ignore fields when considering tag predicates ( #6212 )
...
* fix: ignore fields when considering tag predicates
* chore: update test to not use time column in predicate
* chore: update with review feedback
* chore: update tests to avoid fields refs in RPC preds
This is more like what would be coming off the wire from
Influx RPC.
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-11-28 15:16:55 +00:00
caa595a6fc
chore(deps): Bump serde_json from 1.0.88 to 1.0.89 ( #6203 )
...
Bumps [serde_json](https://github.com/serde-rs/json ) from 1.0.88 to 1.0.89.
- [Release notes](https://github.com/serde-rs/json/releases )
- [Commits](https://github.com/serde-rs/json/compare/v1.0.88...v1.0.89 )
---
updated-dependencies:
- dependency-name: serde_json
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-11-22 09:28:31 +00:00
a9db7581cd
chore(deps): Bump tokio from 1.21.2 to 1.22.0 ( #6183 )
...
Bumps [tokio](https://github.com/tokio-rs/tokio ) from 1.21.2 to 1.22.0.
- [Release notes](https://github.com/tokio-rs/tokio/releases )
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.21.2...tokio-1.22.0 )
---
updated-dependencies:
- dependency-name: tokio
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-11-21 10:21:24 +00:00
52c50c16e1
chore(deps): Bump serde_json from 1.0.87 to 1.0.88
...
Bumps [serde_json](https://github.com/serde-rs/json ) from 1.0.87 to 1.0.88.
- [Release notes](https://github.com/serde-rs/json/releases )
- [Commits](https://github.com/serde-rs/json/compare/v1.0.87...v1.0.88 )
---
updated-dependencies:
- dependency-name: serde_json
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2022-11-21 01:52:18 +00:00
6d3dfa781e
chore: marshal InfluxDbError into status details ( #6161 )
...
* chore: marshal InfluxDbError into status details
* chore: address feedback and CI issues
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-11-17 19:51:01 +00:00
3dde82b3b9
fix: Rename QueryDatabaseProvider to QueryNamespaceProvider
2022-11-11 16:14:12 -05:00
0657ad9600
fix: Rename QueryDatabase to QueryNamespace
2022-11-11 16:14:12 -05:00
621560a0dc
fix: Rename QueryDatabaseMeta to QueryNamespaceMeta
2022-11-11 16:14:12 -05:00
694443bb87
chore: Rename DatabaseName to NamespaceName ( #6100 )
...
* chore: Rename DatabaseName to NamespaceName
* fix: fmt
* chore: Updates some more references
* chore: more cleanup
* fix: adjust test
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-11-10 14:13:59 +00:00
5f7a6e696f
refactor: chunk InfluxRPC `ReadResponses` ( #6094 )
...
Currently we see some prod panics:
```
'assertion failed: len <= std::u32::MAX as usize', tonic/src/codec/encode.rs:127:5
```
This is due to an upstream bug in tonic:
https://github.com/hyperium/tonic/issues/1141
However the fix will only turn this into an error instead of panicking.
We should instead NOT return such overlarge results, esp. because
InfluxRPC supports streaming.
While we currently don't perform streaming conversion (like streaming
the data out of the query stack into the gRPC layer), the 4GB size limit
can easily be triggered (in prod) w/ enough RAM. So let's re-chunk our
in-memory responses so that they stream nicely to the client.
We may later implement proper streaming conversion, see #4445 and #503 .
2022-11-10 08:13:22 +00:00
ce39b2a385
chore(deps): Bump regex from 1.6.0 to 1.7.0 ( #6078 )
...
Bumps [regex](https://github.com/rust-lang/regex ) from 1.6.0 to 1.7.0.
- [Release notes](https://github.com/rust-lang/regex/releases )
- [Changelog](https://github.com/rust-lang/regex/blob/master/CHANGELOG.md )
- [Commits](https://github.com/rust-lang/regex/compare/1.6.0...1.7.0 )
---
updated-dependencies:
- dependency-name: regex
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-11-08 09:57:49 +00:00
034d9b371d
chore: Update datafusion and arrow/arrow-flight/parquet to `26.0.0` ( #6061 )
...
* chore: Update datafusion and arrow/arrow-flight/parquet to `26.0.0`
* fix: Update query_functions
* fix: update for TimestampNanosecondArray API changes
* fix: update for TimestampNanosecondArray API changes
* chore: Update flatbuffers and remove rustsec warning
* chore: Update text
* fix: update more test
* fix: Lock ahash to exactly 0.8.0
* fix: Update datafusion pin
* chore: Run cargo hakari tasks
Co-authored-by: Carol (Nichols || Goulding) <carol.nichols@gmail.com>
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-11-07 11:01:58 +00:00
f511db380c
refactor: remove table name from chunks ( #6063 )
...
It should be always clear from the context to which table a chunk
belongs.
I think having a table name bound to a chunk goes back to a time where
chunks had multiple tables.
Helps with #6049 .
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-11-07 10:42:57 +00:00
3145e2c05b
feat: Use workspace dep inheritance for the arrow crate
2022-10-26 10:34:29 -04:00
44936f661a
feat: Use workspace dep inheritance for datafusion instead of shim crate
2022-10-26 10:33:56 -04:00
2e83e04eab
feat: Use workspace package metadata to reduce differences and repetition
2022-10-24 13:04:09 -04:00
3e4db81bc6
refactor: make `SchemaBuilder::field` fallible
...
It would be nice if the IOx data type would not be optional and this is
a prep clean-up to achieve that.
2022-10-24 18:12:42 +02:00
c8f2a5772e
fix: Add comments about rationale and context for headers ( #5926 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-10-20 11:08:10 +00:00
d76d787630
fix: use structured errors for logging influxrpc errors ( #5912 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-10-20 09:38:30 +00:00
487f4a420f
fix: add `storage-type=iox` header to influxrpc responses ( #5917 )
...
* refactor: Move response creation into a single location
* fix: add storage-type=iox header to influxrpc responses
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-10-20 09:24:55 +00:00
bebb15d30f
chore(deps): Bump serde_json from 1.0.86 to 1.0.87
...
Bumps [serde_json](https://github.com/serde-rs/json ) from 1.0.86 to 1.0.87.
- [Release notes](https://github.com/serde-rs/json/releases )
- [Commits](https://github.com/serde-rs/json/compare/v1.0.86...v1.0.87 )
---
updated-dependencies:
- dependency-name: serde_json
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2022-10-20 07:52:33 +00:00
82d6fc3bda
feat: support queries via influxrpc with periods in field names ( #5919 )
...
* feat: support queries via influxrpc with periods in field names
* fix: update comments
* fix: more tests
* fix: more tests
2022-10-19 20:09:55 +00:00
d706f8221d
chore: Update datafusion and arrow / parquet / arrow-flight 25.0.0 ( #5900 )
...
* chore: Update datafusion and `arrow` / `parquet` / `arrow-flight` 25.0.0
* chore: Update for structure changes
* chore: Update for new projection pushdown
* chore: Run cargo hakari tasks
* fix: fmt
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-10-18 20:58:47 +00:00
e16b04891e
fix: improve tonic status codes returned for query errors ( #5864 )
...
* fix: improve tonic status codes returned for query errors
* fix: update tests
* fix: Add rationale for categorization and avoid recurson handling DataFusionError::Context
* fix: Apply suggestions from code review
Thanks @carols10cents!
Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
* fix: Update tests
* fix: add future TOOD
Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-10-18 10:47:52 +00:00
9134ccd6c3
chore: Update datafusion again ( #5855 )
...
* chore: Update datafusion
* chore: Updates for changes in datafusion
* chore: more updates
* fix: update doc example
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-10-13 19:18:57 +00:00
d57c99638c
chore: Update datafusion + `arrow`, `arrow-flight`, and `parquet` to 24.0.0.0 ( #5792 )
...
* chore: Update datafusion + `arrow`, `arrow-flight`, and `parquet` to 24.0.0.0
* fix: Update for coercion, fix explain plans for change in column name display
* chore: Update datafusion lock
* fix: Update for other API changes
* chore: Update to latest datafusion pin
* chore: Run cargo hakari tasks
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-10-12 16:19:14 +00:00
2277fcf08a
chore(deps): Bump serde_json from 1.0.85 to 1.0.86
...
Bumps [serde_json](https://github.com/serde-rs/json ) from 1.0.85 to 1.0.86.
- [Release notes](https://github.com/serde-rs/json/releases )
- [Commits](https://github.com/serde-rs/json/compare/v1.0.85...v1.0.86 )
---
updated-dependencies:
- dependency-name: serde_json
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2022-10-10 01:42:37 +00:00
8013781ac2
feat: rewrite missing column references to NULL ( #5818 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-10-07 18:05:54 +00:00
c4c83e0840
fix: query error propagation ( #5801 )
...
- treat OOM protection as "resource exhausted"
- use `DataFusionError` in more places instead of opaque `Box<dyn Error>`
- improve conversion from/into `DataFusionError` to preserve more
semantics
Overall, this improves our error handling. DF can now return errors like
"resource exhausted" and gRPC should now automatically generate a
sensible status code for it.
Fixes #5799 .
2022-10-06 08:54:01 +00:00
56a1c579a1
refactor: Change influxdb_iox client to use http rather than grpc for write ( #5756 )
...
* refactor: Change influxdb_iox client to use http rather than grpc for write
* refactor: remove custom variants
* refactor: consolidate more
2022-09-29 11:12:51 +00:00
66dbb9541f
chore: Update datafusion and `arrow`/`parquet`/`arrow-flight` to 23.0.0, `thrift` to 0.16.0 ( #5694 )
...
* chore: Update datafusion and `arrow`/`parquet`/`arrow-flight` to 23.0.0
* chore: Update thrift / remove parquet_format
* fix: Update APIs
* chore: Update lock + Run cargo hakari tasks
* fix: use patched version of arrow-rs to work around https://github.com/apache/arrow-rs/issues/2779
* chore: Run cargo hakari tasks
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-09-27 12:50:54 +00:00
7e00426d49
refactor: concurrent table scan for "tag values" ( #5671 )
...
Ref #5668 .
2022-09-19 14:11:51 +00:00
1fd31ee3bf
chore: Update datafusion / `arrow` / `arrow-flight` / `parquet` to version 22.0.0 ( #5591 )
...
* chore: Update datafusion / `arrow` / `arrow-flight` / `parquet` to version 22.0.0
* fix: enable dynamic comparison flag
* chore: derive Eq for clippy
* chore: update explain plans
* chore: Update sizes for ReadBuffer encoding
* chore: update more tests
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-09-12 17:45:03 +00:00
8933f47ec1
refactor: make `QueryChunk::partition_id` non-optional ( #5614 )
...
In our data model, a chunk always belongs to a partition[^1], so let's
not make this attribute optional. The optional value only leads to
-- mostly surprising -- conditional behavior, ranging from "do not equalize
the partition sort key" (querier) to "always consider the chunk overlapping"
(iox_query when dealing with ingester chunks).
[^1]: This is even true when the chunk belongs to a parquet file that is not
yet added to the catalog, contrary to what a comment in the ingester
stated. The catalog and data model used by the querier are two totally
different things.
2022-09-12 13:52:51 +00:00
b676049358
fix: apply selection in `TestChunk::read_filter` ( #5613 )
...
* fix: apply selection in `TestChunk::read_filter`
TBH I have no idea how this worked so well before, but the chunks are
expected to apply the given selection. This is because
`IOxReadFilterNode::execute` will wrap the `QueryChunk::read_filter`
output into a `SchemaAdapterStream` and this one expects that there are
no input columns that are absent in the output schema (i.e. it will only
add null columns, it won't remove any). Funnily the `SchemaAdapterStream`
error will blame DataFusion for the mess.
* test: make `test_storage_rpc_tag_values_grouped_by_measurement_and_tag_key` a bit harder
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-09-12 13:10:37 +00:00
adeacf416c
ci: fix ( #5569 )
...
* ci: use same feature set in `build_dev` and `build_release`
* ci: also enable unstable tokio for `build_dev`
* chore: update tokio to 1.21 (to fix console-subscriber 0.1.8
* fix: "must use"
2022-09-06 14:13:28 +00:00
6669d85fb4
chore: Update datafusion + arrow/parquet to `21.0.0` ( #5519 )
...
* chore: Update arrow/arrow-flight/parquet to 21.0.0
* chore: Update datafusion pin
* chore: Fix arrow update script
* chore: Update Cargo.lock
* chore: Update for new API
2022-08-31 13:30:47 +00:00
b982bdaf2f
fix: Derive Eq when we derive PartialEq and members can derive Eq
...
Allow this in generated code that we don't control, though.
Recommended by clippy now. https://rust-lang.github.io/rust-clippy/master/index.html#derive_partial_eq_without_eq
2022-08-11 15:04:06 -04:00
16ddc5efc6
chore: Update datafusion / arrow/parquet/arrow-flight and prost/tonic ecosystem ( #5360 )
...
* chore: Update datafusion and arrow
* chore: Update Cargo.lock
* chore: update to Decimal128
* chore: Update tonic/prost/pbjson/etc
* chore: Run cargo hakari tasks
* fix: doctest in generated types
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
2022-08-09 17:30:44 +00:00
e0ea335b70
fix: Support RegExMatch and RegExNotMatch predicates on `_field` ( #5301 )
...
* test: add tests for regex_match_on_field
* feat: more general `_field` predicate handling
* fix: remove old comment
* fix: update tests
* fix: improve test a little more
* fix: fmt
* fix: Update predicate/src/rpc_predicate/field_rewrite.rs
Co-authored-by: Raphael Taylor-Davies <1781103+tustvold@users.noreply.github.com>
* fix: Handle predicates that can not be evaluated
Co-authored-by: Raphael Taylor-Davies <1781103+tustvold@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-08-04 19:42:16 +00:00
e8231b2986
chore(deps): Bump serde_json from 1.0.82 to 1.0.83 ( #5297 )
...
* chore(deps): Bump serde_json from 1.0.82 to 1.0.83
Bumps [serde_json](https://github.com/serde-rs/json ) from 1.0.82 to 1.0.83.
- [Release notes](https://github.com/serde-rs/json/releases )
- [Commits](https://github.com/serde-rs/json/compare/v1.0.82...v1.0.83 )
---
updated-dependencies:
- dependency-name: serde_json
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* chore: Run cargo hakari tasks
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-08-04 14:28:29 +00:00
87bdabb38a
feat: log external span for query gRPC requests ( #5187 )
...
* feat: log external span for query gRPC requests
This should simplify the correlation with our binlog data.
* refactor: address review comments
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-07-28 12:53:12 +00:00
3fbe860bb9
fix: interpret [MIN_NANO_TIME, MAX_NANO_TIME) range as all time for optimization ( #5231 )
...
InfluxQL queries can send (technically incorrect) ranges like this, meaning all time
but excluding the max nanosecond time.
Since this is an important case, we should handle it specially and use the optimized
'all time' handling for meta queries even though this is technically wrong in that
it does not filter out column names / measurement names at MAX_NANO_TIME exactly.
Closes: https://github.com/influxdata/conductor/issues/1072
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-07-28 12:24:26 +00:00
9215a534d0
chore: Update datafusion and `arrow`/`parquet`/`arrow-flight` to `19.0.0` ( #5229 )
...
* chore: Update datafusion and `arrow`/`parquet`/`arrow-flight` to `19.0.0`
* chore: Run cargo hakari tasks
* fix: Update for API changes
* fix: clippy
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-07-28 08:10:47 +00:00
fbf672015e
refactor: Reduce ceremony requried to create a `Span` from `SpanContext` ( #5181 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-07-22 11:19:38 +00:00
5a0af921c8
chore: Roll forward: Sync ReadWindowAggregate API: TagKeyMetaNames ( #5186 )
...
This reverts commit 5d02c755687ef041f5f45dbfc3e633a833284edb.
2022-07-22 10:44:06 +00:00
baee020efe
refactor: improve query gRPC logging ( #5185 )
...
- ensure that logging is done BEFORE the DB/namespace is requested (i.e.
any actual work is done) but AFTER the query semaphore is acquired
- simplify tag key decoding (so that the logging statements are simpler
to write)
2022-07-22 10:30:45 +00:00
07cdb99192
chore: Revert "Sync ReadWindowAggregate API: TagKeyMetaNames" ( #5184 )
...
We're noticing a possible regression (OOMs) in our testing cluster that roughly correlates with this.
2022-07-22 09:26:42 +00:00
0f54281d24
feat: trace namespace cache
...
For #5129 .
2022-07-21 16:10:06 +02:00
21d033eafd
fix: Sync ReadWindowAggregate API: TagKeyMetaNames
...
The storage API has been updated in https://github.com/influxdata/idpe/pull/12868
in January, but since we forked the `.proto` files we never noticed.
2022-07-21 15:07:04 +02:00
c20288f60e
fix: Add TagKeyMetaNamesCapability capability ( #5160 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-07-20 10:52:40 +00:00
b8236e2b9d
fix: Fix SeriesKey sort order for special _measurement and _field ( #5150 )
...
* fix: Fix SeriesKey sort order for special _measurement and _field
* fix: Update expected test output
* fix: Update more tests
* fix: Re-sort tag key when using binary encoding
Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
2022-07-20 08:45:17 +00:00
e2d871b00b
chore: Update datafusion and arrow/parquet/arrow-flight to `18.0.0` ( #5079 )
...
* chore: Update datafusion to 10.0.0, arrow/parquet/arrow-flight to 18
* chore: Run cargo hakari tasks
* fix: update cargo pin
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-07-18 15:01:03 +00:00
f0bd278652
feat: add tracing to instrumented semaphores ( #5130 )
...
This will allow us to easily see how much time we spend during query
processing waiting for the query semaphore.
Ref #5129 .
2022-07-15 07:50:28 +00:00
9b67de2f43
chore(deps): Bump tokio from 1.19.2 to 1.20.0
...
Bumps [tokio](https://github.com/tokio-rs/tokio ) from 1.19.2 to 1.20.0.
- [Release notes](https://github.com/tokio-rs/tokio/releases )
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.19.2...tokio-1.20.0 )
---
updated-dependencies:
- dependency-name: tokio
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2022-07-14 01:21:43 +00:00
c46e1c6347
chore: Update datafusion + arrow/parquet/arrow-flight to `17.0.0` ( #5021 )
...
* fix: correct nullability declaration of system tables
* chore: Update datafusion and arrow/parquet/arrow-flight
* chore: Run cargo hakari tasks
* fix: Update tests
* fix: Update tests
* fix: predicate pruning
* fix: add some tests
* fix: query_functions
* fix: fix read_buffer test
* fix: fix clippy
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-07-07 19:22:15 +00:00
e193913ed3
fix: optimize field columns for all-time predicates ( #5046 )
...
* fix: optimize field columns for all-time predicates
Also fix timestamp range to allow selecting points at MAX_NANO_TIME
* fix: clamp end to MIN_NANO_TIME for safety
* refactor: add contains_all method to TimestampRange
2022-07-06 12:01:28 +00:00
2b527bbf64
chore(deps): Bump regex from 1.5.6 to 1.6.0 ( #5048 )
...
Bumps [regex](https://github.com/rust-lang/regex ) from 1.5.6 to 1.6.0.
- [Release notes](https://github.com/rust-lang/regex/releases )
- [Changelog](https://github.com/rust-lang/regex/blob/master/CHANGELOG.md )
- [Commits](https://github.com/rust-lang/regex/compare/1.5.6...1.6.0 )
---
updated-dependencies:
- dependency-name: regex
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-07-06 10:25:28 +00:00
03f456d8fd
fix: optimize tag_keys to go only to schema when predicate is empty ( #4985 )
...
* docs: fix comment
* test: add test for delete behaviour
* fix: tag_keys optimization for empty predicate
Also need to eliminate 'true' predicates from simplified predicate so
is_empty works correctly.
* refactor: use lit instead of spelling out literal true
Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-07-05 12:45:25 +00:00
40a8525520
chore(deps): Bump serde_json from 1.0.81 to 1.0.82 ( #4992 )
...
Bumps [serde_json](https://github.com/serde-rs/json ) from 1.0.81 to 1.0.82.
- [Release notes](https://github.com/serde-rs/json/releases )
- [Commits](https://github.com/serde-rs/json/compare/v1.0.81...v1.0.82 )
---
updated-dependencies:
- dependency-name: serde_json
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-06-30 09:54:08 +00:00
e91d00b10c
chore: Update datafusion + `arrow`/`parquet`/`arrow-flight` to `16.0.0 ( #4851 )
...
* chore: TEMP Update DataFusion to pre-release
* chore: update arrow et al to 16.0.0
* chore: Run cargo hakari tasks
* fix: update reader read_dictionary API
* chore: Update to real Datafusion release
* fix: Update parquet API
* fix: update test
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
2022-06-14 16:31:40 +00:00
66623fe0cd
feat: expose query semaphore metrics ( #4836 )
...
The groundwork for that was already done, just needed a bit of wiring.
This might help us to judge timeouts.
2022-06-13 09:36:50 +00:00
2ec7764fdd
refactor: rename builder like predicate methods to be `with_` ( #4808 )
...
* refactor: rename builder like predicate methods to be `with_`
* fix: merge conflict
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-06-09 11:26:03 +00:00
afc1c12062
refactor: consolidate `PredicateBuilder` into `Predicate` ( #4799 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-06-08 12:21:24 +00:00
8e96a2721d
chore: Update datafusion (again) ( #4788 )
...
* chore: Update datafusion
* chore: Update imports
* refactor: update API usage
* refactor: clean up some uses of binary_expr
* fix: remove unused export
* fix: update explain output
* chore: update more explain tests
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-06-07 08:17:56 +00:00
e03bf94420
chore(deps): Bump tokio from 1.18.2 to 1.19.1 ( #4783 )
...
Bumps [tokio](https://github.com/tokio-rs/tokio ) from 1.18.2 to 1.19.1.
- [Release notes](https://github.com/tokio-rs/tokio/releases )
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.18.2...tokio-1.19.1 )
---
updated-dependencies:
- dependency-name: tokio
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-06-06 14:15:12 +00:00
3592aa52d8
chore: Update datafusion + `arrow`/`parquet`/`arrow-flight` to `15.0.0` ( #4743 )
...
* chore: Update datafusion + `arrow`/`parquet`/`arrow-flight` to `15.0.0`
* chore: Update APIs
* chore: Run cargo hakari tasks
* feat: normalize parquet file metadata
* chore: update size tests
* chore: add docs on metadata stripping
* chore: TEMP UPDATE TO DF BRANCH
* chore: Update for new API
* fix: Update to latest DF
* fix: cargo hakari
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: Raphael Taylor-Davies <r.taylordavies@googlemail.com>
2022-06-03 10:32:26 +00:00
f7cbd5d490
test: query limits ( #4769 )
...
* test: query limits
This was left out of #4760 .
* test: additional debugging
Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
2022-06-03 07:30:30 +00:00
9e30a3eb29
refactor: rework querier concurrency limiting ( #4760 )
...
* refactor: rework querier concurrency limiting
With #4752 we introduced a concurrency limit into the querier. It works
by drawing permits from a central semaphore whenever we create a
`QuerierNamespace`. This however only limits concurrency during query
planning and not query execution, because the objects contained within
the plan (chunks and some metadata) neither reference the permit nor the
`QuerierNamespace`.
Now one approach to fix that would be to wire up the permit all the down
into all the query-related data structures. This however is very fiddly
and potentially will get lost at some point, because as soon as we
transform these data structures -- e.g. into streams -- the permit might
get lost again. This will be potentially query-dependent and very hard
to debug.
So instead we reverse the approach and track the permits at the upper
layer of the stack: the gRPC service entry points. There we also need to
be careful -- e.g. when we return streams to tonic -- but it's way
easier to review that then the deeply nested object hierarchy that is
involved with queries. Also the separation of concerns is a bit clearer,
because why would a "chunk" care about the "query concurrency" as a
whole.
* refactor: improve gRPC permit keeping and prepare tests
2022-06-02 09:49:58 +00:00
257aaa7e7b
fix: Support `_field != <name>` predicates ( #4721 )
...
* fix: Support `_field != <name>` predicates
* fix: update test
* fix: add negative test
* fix: improve comments
* refactor: make `add_include` and `add_exclude` infallible
* chore: add type annotations
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-06-01 16:04:53 +00:00
b52a3586a7
fix: Turn cargo doc warnings into errors ( #4710 )
...
* fix: Correct intra-doc links
* fix: Turn cargo doc warnings into errors
Co-authored-by: Jake Goulding <jake.goulding@integer32.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-05-28 11:24:22 +00:00
5c033b462e
chore(deps): Bump regex from 1.5.5 to 1.5.6 ( #4655 )
...
Bumps [regex](https://github.com/rust-lang/regex ) from 1.5.5 to 1.5.6.
- [Release notes](https://github.com/rust-lang/regex/releases )
- [Changelog](https://github.com/rust-lang/regex/blob/master/CHANGELOG.md )
- [Commits](https://github.com/rust-lang/regex/compare/1.5.5...1.5.6 )
---
updated-dependencies:
- dependency-name: regex
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-05-23 08:39:01 +00:00
52346642a0
ci: fix cargo deny ( #4629 )
...
* ci: fix cargo deny
* chore: downgrade `socket2`, version 0.4.5 was yanked
* chore: rename `query` to `iox_query`
`query` is already taken on crates.io and yanked and I am getting tired
of working around that.
2022-05-18 09:38:35 +00:00
3a33e806c7
chore: Update datafusion + `arrow`/`parquet`/`arrow-flight` to `14.0.0` ( #4619 )
...
* chore: Update datafusion deps
* chore: update arrow/parquet/arrow flight deps
* chore: Run cargo hakari tasks
* chore: Update location of utils
* chore: Update some more APIs
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
2022-05-17 14:13:03 +00:00
dependabot[bot]
Carol (Nichols || Goulding)
Marco Neumann
Raphael Taylor-Davies
Andrew Lamb
Christopher M. Wolff
Andrew Lamb
Sam Arnold
Marko Mikulicic
Sam Arnold
Carol (Nichols || Goulding)