e49bee0c26
test: Make test ingester2 instances either persist very quickly or not at all
2023-01-04 10:06:57 -05:00
afd09d96be
test: Create a Querier2 test server type
2023-01-04 10:06:56 -05:00
96029654ab
test: Add a shared MiniCluster for version 2 services
2023-01-04 10:06:56 -05:00
0aacef3c59
chore(deps): Bump once_cell from 1.16.0 to 1.17.0 ( #6473 )
...
* chore(deps): Bump once_cell from 1.16.0 to 1.17.0
Bumps [once_cell](https://github.com/matklad/once_cell ) from 1.16.0 to 1.17.0.
- [Release notes](https://github.com/matklad/once_cell/releases )
- [Changelog](https://github.com/matklad/once_cell/blob/master/CHANGELOG.md )
- [Commits](https://github.com/matklad/once_cell/compare/v1.16.0...v1.17.0 )
---
updated-dependencies:
- dependency-name: once_cell
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* chore: Change once_cell version specifier to major.minor for less churn
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Carol (Nichols || Goulding) <carol.nichols@gmail.com>
2023-01-02 17:07:15 +00:00
46ff8854ec
fix: Use code backticks around invalid HTML tags in doc strings
2022-12-21 16:36:17 -05:00
72aab99951
fix: Remove needless borrow. Thanks clippy!
2022-12-21 14:32:34 -05:00
9b22ede3f0
refactor: Make arrow flight client return `futures::Streams` ( #6438 )
...
* refactor: Make arrow flight client use futures::Streams
* refactor: concision
2022-12-19 17:09:26 +00:00
94c2f94ea1
refactor: Extract common ArrowFlight client into iox_arrow_flight ( #6427 )
...
* refactor: Extract common ArrowFlight client into iox_arrow_flight
* chore: Run cargo hakari tasks
* fix: clarify intent of iox_arrow_flight crate
* refactor: Apply suggestions from code review
Co-authored-by: Raphael Taylor-Davies <1781103+tustvold@users.noreply.github.com>
* fix: loop --> while let
* fix: REmove make_tonic_error in favor of From impl
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: Raphael Taylor-Davies <1781103+tustvold@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-12-19 11:35:20 +00:00
2a1e540ee3
fix: Rename INFLUXDB_IOX_MODE to INFLUXDB_IOX_RPC_MODE
2022-12-15 14:13:01 -05:00
aec98015d7
fix: Remove the rpc_write feature flag and use INFLUXDB_IOX_MODE env var instead
...
And standardize on ingester2 and router2 for consistency.
Connects to #6402 .
2022-12-15 14:06:59 -05:00
47cd6821e1
feat: Document IOx Flight API and add convenience methods ( #6392 )
...
* feat: Document IOx Flight API and add convenience methods
* fix: InfluxQL handling
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-12-13 17:32:37 +00:00
c3a7575d46
feat: Enable rpc_write on the inner command if it's enabled for tests
...
And only run rpc_write specific tests if the feature is enabled when
running the tests.
2022-12-08 17:45:30 -05:00
6014c10866
test: Enable running ingester2/router RPC write servers in e2e tests
...
Add configuration and server types to be able to create server fixtures
for them.
2022-12-08 17:22:52 -05:00
62db312a8f
feat: Switch to escargot to get more control over running Cargo bins
2022-12-08 15:29:44 -05:00
4ded68de62
test: "not found" end2end tests for querier ( #6352 )
...
I couldn't find any end2end tests for these cases and I was kinda
worried that our error codes were wrong. Turns out they are correct, but
let's have some nice tests for this behavior.
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-12-08 18:17:53 +00:00
cd6a8a1a82
refactor: DF-driven on-demand mem limit instead of ahead-of-time heuristics ( #6313 )
...
* refactor: DF-driven on-demand mem limit instead of ahead-of-time heuristics
Closes #6310 .
* refactor: rename and tune default exec mem limits
* fix: ingester2 bits after rebase
2022-12-05 12:38:28 +00:00
fc9431f6ae
chore(deps): Bump assert_cmd from 2.0.6 to 2.0.7 ( #6330 )
...
Bumps [assert_cmd](https://github.com/assert-rs/assert_cmd ) from 2.0.6 to 2.0.7.
- [Release notes](https://github.com/assert-rs/assert_cmd/releases )
- [Changelog](https://github.com/assert-rs/assert_cmd/blob/master/CHANGELOG.md )
- [Commits](https://github.com/assert-rs/assert_cmd/compare/v2.0.6...v2.0.7 )
---
updated-dependencies:
- dependency-name: assert_cmd
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-12-05 09:05:56 +00:00
9356868562
chore(deps): Bump nix from 0.25.0 to 0.26.1 ( #6273 )
...
Bumps [nix](https://github.com/nix-rust/nix ) from 0.25.0 to 0.26.1.
- [Release notes](https://github.com/nix-rust/nix/releases )
- [Changelog](https://github.com/nix-rust/nix/blob/master/CHANGELOG.md )
- [Commits](https://github.com/nix-rust/nix/compare/v0.25.0...v0.26.1 )
---
updated-dependencies:
- dependency-name: nix
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-11-30 10:31:46 +00:00
2306c383f3
feat: Introduce InfluxQL to Flight ( #6166 )
...
* feat: Introduce InfluxQL to Flight
All InfluxQL queries will fail with an error
* chore: Temper protobuf lint
* chore: Finalize flight.proto changes; fix tests
* chore: Add tests for InfluxQL planner
* chore: Update docs
* chore: Update docs
* chore: Rename back to original
* chore: Use .into() rather than cast
* chore: Use function rather than field
* chore: Improved InfluxQL planner name
* chore: Restore `impl Into<String>` argument
* chore: Add a comment that Go clients are unable to execute InfluxQL
* chore: Add a test for the `--lang` argument and InfluxQL
2022-11-23 00:33:49 +00:00
04c00bbb62
chore(deps): Bump bytes from 1.2.1 to 1.3.0 ( #6199 )
...
Bumps [bytes](https://github.com/tokio-rs/bytes ) from 1.2.1 to 1.3.0.
- [Release notes](https://github.com/tokio-rs/bytes/releases )
- [Changelog](https://github.com/tokio-rs/bytes/blob/master/CHANGELOG.md )
- [Commits](https://github.com/tokio-rs/bytes/commits )
---
updated-dependencies:
- dependency-name: bytes
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-11-22 08:23:24 +00:00
a9db7581cd
chore(deps): Bump tokio from 1.21.2 to 1.22.0 ( #6183 )
...
Bumps [tokio](https://github.com/tokio-rs/tokio ) from 1.21.2 to 1.22.0.
- [Release notes](https://github.com/tokio-rs/tokio/releases )
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.21.2...tokio-1.22.0 )
---
updated-dependencies:
- dependency-name: tokio
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-11-21 10:21:24 +00:00
62851afc27
feat: add querier->ingester circuit breaker ( #6147 )
...
* feat: add log ingester memory pressure persist
* feat: add querier->ingester circuit breaker
Closes #4608 .
* docs: explain high-level circuit breaker
* docs: improve
Co-authored-by: Andrew Lamb <alamb@influxdata.com>
* test: add additional test assertion
* refactor: upgrade info to warning log
Co-authored-by: Andrew Lamb <alamb@influxdata.com>
2022-11-16 10:50:33 +00:00
dcc7b10bcf
feat: define router -> ingester write protocol
...
Specify a gRPC service and request/response message formats to push
writes directly from a router to an ingester.
2022-11-15 15:00:38 +01:00
bdff4e8848
fix: Consistently use 'namespace' instead of 'database' in comments and other internal text
2022-11-11 15:46:04 -05:00
d9c97795fc
feat: use IDs in ingester query API ( #6093 )
...
* refactor: NS+table ID (instead of name) in querier<>ingester
* feat(ingester): use IDs for query API
Changes the ingester to utilise the ID fields (instead of names) sent
over the query wire message wrapped within the Flight API.
BREAKING: this changes the "query-ingester" CLI command arguments which
now expects the namespace & table IDs, rather than their names.
* refactor(ingester): add more query logging context
Updates the log messages during query execution to include more context
fields.
* style: remove unused import
Co-authored-by: Marco Neumann <marco@crepererum.net>
2022-11-09 11:25:13 +00:00
2e74727baf
fix: handle recursing limit in querier<>ingester comm ( #6020 )
...
* test: check server exit status on `TestServer` drop
* fix: handle recursing limit in querier<>ingester comm
Fixes #5974 .
* docs: improve
Co-authored-by: Andrew Lamb <alamb@influxdata.com>
* test: simplify
Co-authored-by: Andrew Lamb <alamb@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-11-02 09:16:34 +00:00
b1572c50a6
chore(deps): Bump once_cell from 1.15.0 to 1.16.0 ( #6009 )
...
Bumps [once_cell](https://github.com/matklad/once_cell ) from 1.15.0 to 1.16.0.
- [Release notes](https://github.com/matklad/once_cell/releases )
- [Changelog](https://github.com/matklad/once_cell/blob/master/CHANGELOG.md )
- [Commits](https://github.com/matklad/once_cell/compare/v1.15.0...v1.16.0 )
---
updated-dependencies:
- dependency-name: once_cell
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-10-31 16:23:40 +00:00
3145e2c05b
feat: Use workspace dep inheritance for the arrow crate
2022-10-26 10:34:29 -04:00
2e83e04eab
feat: Use workspace package metadata to reduce differences and repetition
2022-10-24 13:04:09 -04:00
6e6e180aad
chore(deps): Bump assert_cmd from 2.0.4 to 2.0.5 ( #5937 )
...
Bumps [assert_cmd](https://github.com/assert-rs/assert_cmd ) from 2.0.4 to 2.0.5.
- [Release notes](https://github.com/assert-rs/assert_cmd/releases )
- [Changelog](https://github.com/assert-rs/assert_cmd/blob/master/CHANGELOG.md )
- [Commits](https://github.com/assert-rs/assert_cmd/compare/v2.0.4...v2.0.5 )
---
updated-dependencies:
- dependency-name: assert_cmd
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-10-21 10:50:06 +00:00
d706f8221d
chore: Update datafusion and arrow / parquet / arrow-flight 25.0.0 ( #5900 )
...
* chore: Update datafusion and `arrow` / `parquet` / `arrow-flight` 25.0.0
* chore: Update for structure changes
* chore: Update for new projection pushdown
* chore: Run cargo hakari tasks
* fix: fmt
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-10-18 20:58:47 +00:00
c28ac4a3c3
fix: Return an error for unsupported SQL queries ( #5876 )
...
* test: Failing tests for unsupported queries
* fix: Catch unsupported SQL operations and error rather than return nothing
* test: Document a few more error messages that come through DataFusion
* refactor: Extract a Step to make query error tests nicer to read and write
* fix: update tests for new error codes
Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-10-18 19:27:29 +00:00
d57c99638c
chore: Update datafusion + `arrow`, `arrow-flight`, and `parquet` to 24.0.0.0 ( #5792 )
...
* chore: Update datafusion + `arrow`, `arrow-flight`, and `parquet` to 24.0.0.0
* fix: Update for coercion, fix explain plans for change in column name display
* chore: Update datafusion lock
* fix: Update for other API changes
* chore: Update to latest datafusion pin
* chore: Run cargo hakari tasks
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-10-12 16:19:14 +00:00
c4c83e0840
fix: query error propagation ( #5801 )
...
- treat OOM protection as "resource exhausted"
- use `DataFusionError` in more places instead of opaque `Box<dyn Error>`
- improve conversion from/into `DataFusionError` to preserve more
semantics
Overall, this improves our error handling. DF can now return errors like
"resource exhausted" and gRPC should now automatically generate a
sensible status code for it.
Fixes #5799 .
2022-10-06 08:54:01 +00:00
82d5c7f336
feat: support parallel, chunked upload via `influxdb_iox write` of line protocol, gzip'd line protocol, and parquet ( #5757 )
...
* feat: Upload in small chunks and in parallel
* fix: doclink
* fix: Apply suggestions from code review
Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
* fix: Update influxdb_iox_client/src/client/write.rs
* fix: fixup error handling and fmt
* fix: Make default chunk sizes the same and add docs
* fix: clippy
Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
2022-09-30 21:02:38 +00:00
04ae0aee80
refactor: Remove protobuf based write service ( #5750 )
...
* refactor: Remove grpc WriteService
* fix: update end to end test
* fix: Update generated_types/protos/influxdata/pbdata/v1/influxdb_pb_data_protocol.proto
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-09-30 10:55:03 +00:00
56a1c579a1
refactor: Change influxdb_iox client to use http rather than grpc for write ( #5756 )
...
* refactor: Change influxdb_iox client to use http rather than grpc for write
* refactor: remove custom variants
* refactor: consolidate more
2022-09-29 11:12:51 +00:00
66dbb9541f
chore: Update datafusion and `arrow`/`parquet`/`arrow-flight` to 23.0.0, `thrift` to 0.16.0 ( #5694 )
...
* chore: Update datafusion and `arrow`/`parquet`/`arrow-flight` to 23.0.0
* chore: Update thrift / remove parquet_format
* fix: Update APIs
* chore: Update lock + Run cargo hakari tasks
* fix: use patched version of arrow-rs to work around https://github.com/apache/arrow-rs/issues/2779
* chore: Run cargo hakari tasks
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-09-27 12:50:54 +00:00
0d18943ad2
chore(deps): Bump once_cell from 1.14.0 to 1.15.0 ( #5701 )
...
Bumps [once_cell](https://github.com/matklad/once_cell ) from 1.14.0 to 1.15.0.
- [Release notes](https://github.com/matklad/once_cell/releases )
- [Changelog](https://github.com/matklad/once_cell/blob/master/CHANGELOG.md )
- [Commits](https://github.com/matklad/once_cell/compare/v1.14.0...v1.15.0 )
---
updated-dependencies:
- dependency-name: once_cell
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-09-21 05:20:55 +00:00
1fd31ee3bf
chore: Update datafusion / `arrow` / `arrow-flight` / `parquet` to version 22.0.0 ( #5591 )
...
* chore: Update datafusion / `arrow` / `arrow-flight` / `parquet` to version 22.0.0
* fix: enable dynamic comparison flag
* chore: derive Eq for clippy
* chore: update explain plans
* chore: Update sizes for ReadBuffer encoding
* chore: update more tests
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-09-12 17:45:03 +00:00
267a53a9e8
chore: update `tracing-subscriber`, fix trogging, fix CLI test port allocation ( #5581 )
...
* test: use dedicated ports for CLI tests
* chore: update `tracing-subscriber`
* fix: work around tracing-subscriber weirdness
It seems that trogging with tracing-subscriber >= 0.3.14 does not
produce any output at all. I suspect we are hitting
<https://github.com/tokio-rs/tracing/issues/2265 >. Let's change the
construct to not use multiple optional layers but a single dyn-dispatch
layer. Logging shouldn't have such a high throughput that his makes any
difference, esp. because the dyn-dispatch happens AFTER the filter.
2022-09-08 09:37:37 +00:00
52ae60bf2e
chore: fix comment typo ( #5551 )
...
Co-authored-by: Dom <dom@itsallbroken.com>
2022-09-07 08:49:29 +00:00
adeacf416c
ci: fix ( #5569 )
...
* ci: use same feature set in `build_dev` and `build_release`
* ci: also enable unstable tokio for `build_dev`
* chore: update tokio to 1.21 (to fix console-subscriber 0.1.8
* fix: "must use"
2022-09-06 14:13:28 +00:00
366c4d9965
chore(deps): Bump once_cell from 1.13.1 to 1.14.0 ( #5555 )
...
Bumps [once_cell](https://github.com/matklad/once_cell ) from 1.13.1 to 1.14.0.
- [Release notes](https://github.com/matklad/once_cell/releases )
- [Changelog](https://github.com/matklad/once_cell/blob/master/CHANGELOG.md )
- [Commits](https://github.com/matklad/once_cell/compare/v1.13.1...v1.14.0 )
---
updated-dependencies:
- dependency-name: once_cell
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-09-06 09:02:28 +00:00
6669d85fb4
chore: Update datafusion + arrow/parquet to `21.0.0` ( #5519 )
...
* chore: Update arrow/arrow-flight/parquet to 21.0.0
* chore: Update datafusion pin
* chore: Fix arrow update script
* chore: Update Cargo.lock
* chore: Update for new API
2022-08-31 13:30:47 +00:00
dbd27f648f
refactor: Rename more mentions of Kafka to their other name where appropriate
2022-08-29 14:27:02 -04:00
74c9529062
fix: Rename KafkaPartition to ShardIndex
2022-08-29 14:07:18 -04:00
78665d3092
chore(deps): Bump once_cell from 1.13.0 to 1.13.1 ( #5413 )
...
Bumps [once_cell](https://github.com/matklad/once_cell ) from 1.13.0 to 1.13.1.
- [Release notes](https://github.com/matklad/once_cell/releases )
- [Changelog](https://github.com/matklad/once_cell/blob/master/CHANGELOG.md )
- [Commits](https://github.com/matklad/once_cell/compare/v1.13.0...v1.13.1 )
---
updated-dependencies:
- dependency-name: once_cell
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-08-17 08:31:46 +00:00
38277cf0fb
chore(deps): Bump nix from 0.24.2 to 0.25.0 ( #5391 )
...
* chore(deps): Bump nix from 0.24.2 to 0.25.0
Bumps [nix](https://github.com/nix-rust/nix ) from 0.24.2 to 0.25.0.
- [Release notes](https://github.com/nix-rust/nix/releases )
- [Changelog](https://github.com/nix-rust/nix/blob/master/CHANGELOG.md )
- [Commits](https://github.com/nix-rust/nix/compare/v0.24.2...v0.25.0 )
---
updated-dependencies:
- dependency-name: nix
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* chore: Run cargo hakari tasks
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-08-15 08:10:09 +00:00
d8d29bc4b4
fix: Reduce type complexity
...
As now caught by clippy. https://rust-lang.github.io/rust-clippy/master/index.html#type_complexity
2022-08-11 15:04:15 -04:00
9321c96aaf
test: Add a step for running compaction in e2e tests
2022-08-10 11:30:22 -04:00
8ec9117836
test: Log a command run by e2e tests in a convenient way
...
So that you can copy-paste it if you want to run it again! 🎉
2022-08-10 11:30:22 -04:00
9981d1636b
refactor: Move dump_log_to_stdout so it can be used in more than server fixtures
2022-08-10 11:30:22 -04:00
463a13b814
test: Remove the compactor from the test MiniCluster
2022-08-10 11:28:51 -04:00
16ddc5efc6
chore: Update datafusion / arrow/parquet/arrow-flight and prost/tonic ecosystem ( #5360 )
...
* chore: Update datafusion and arrow
* chore: Update Cargo.lock
* chore: update to Decimal128
* chore: Update tonic/prost/pbjson/etc
* chore: Run cargo hakari tasks
* fix: doctest in generated types
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
2022-08-09 17:30:44 +00:00
273b3cc165
chore: replace `dotenv` with `dotenvy` ( #5285 )
...
The latter one is a maintained fork. This avoids having both crates
after #5282 .
2022-08-03 12:41:38 +00:00
9215a534d0
chore: Update datafusion and `arrow`/`parquet`/`arrow-flight` to `19.0.0` ( #5229 )
...
* chore: Update datafusion and `arrow`/`parquet`/`arrow-flight` to `19.0.0`
* chore: Run cargo hakari tasks
* fix: Update for API changes
* fix: clippy
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-07-28 08:10:47 +00:00
5a0af921c8
chore: Roll forward: Sync ReadWindowAggregate API: TagKeyMetaNames ( #5186 )
...
This reverts commit 5d02c755687ef041f5f45dbfc3e633a833284edb.
2022-07-22 10:44:06 +00:00
07cdb99192
chore: Revert "Sync ReadWindowAggregate API: TagKeyMetaNames" ( #5184 )
...
We're noticing a possible regression (OOMs) in our testing cluster that roughly correlates with this.
2022-07-22 09:26:42 +00:00
21d033eafd
fix: Sync ReadWindowAggregate API: TagKeyMetaNames
...
The storage API has been updated in https://github.com/influxdata/idpe/pull/12868
in January, but since we forked the `.proto` files we never noticed.
2022-07-21 15:07:04 +02:00
278a7f91af
chore(deps): Bump bytes from 1.1.0 to 1.2.0 ( #5156 )
...
Bumps [bytes](https://github.com/tokio-rs/bytes ) from 1.1.0 to 1.2.0.
- [Release notes](https://github.com/tokio-rs/bytes/releases )
- [Changelog](https://github.com/tokio-rs/bytes/blob/master/CHANGELOG.md )
- [Commits](https://github.com/tokio-rs/bytes/compare/v1.1.0...v1.2.0 )
---
updated-dependencies:
- dependency-name: bytes
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-07-20 10:00:08 +00:00
e2d871b00b
chore: Update datafusion and arrow/parquet/arrow-flight to `18.0.0` ( #5079 )
...
* chore: Update datafusion to 10.0.0, arrow/parquet/arrow-flight to 18
* chore: Run cargo hakari tasks
* fix: update cargo pin
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-07-18 15:01:03 +00:00
9b67de2f43
chore(deps): Bump tokio from 1.19.2 to 1.20.0
...
Bumps [tokio](https://github.com/tokio-rs/tokio ) from 1.19.2 to 1.20.0.
- [Release notes](https://github.com/tokio-rs/tokio/releases )
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.19.2...tokio-1.20.0 )
---
updated-dependencies:
- dependency-name: tokio
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2022-07-14 01:21:43 +00:00
c46e1c6347
chore: Update datafusion + arrow/parquet/arrow-flight to `17.0.0` ( #5021 )
...
* fix: correct nullability declaration of system tables
* chore: Update datafusion and arrow/parquet/arrow-flight
* chore: Run cargo hakari tasks
* fix: Update tests
* fix: Update tests
* fix: predicate pruning
* fix: add some tests
* fix: query_functions
* fix: fix read_buffer test
* fix: fix clippy
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-07-07 19:22:15 +00:00
89f5091546
refactor: Don't require DSN for test config
...
This enables writing a test for all-in-one's ephemeral mode, which
currently isn't working
2022-07-06 11:00:29 -04:00
68eff79594
chore(deps): Bump once_cell from 1.12.0 to 1.13.0 ( #5033 )
...
Bumps [once_cell](https://github.com/matklad/once_cell ) from 1.12.0 to 1.13.0.
- [Release notes](https://github.com/matklad/once_cell/releases )
- [Changelog](https://github.com/matklad/once_cell/blob/master/CHANGELOG.md )
- [Commits](https://github.com/matklad/once_cell/compare/v1.12.0...v1.13.0 )
---
updated-dependencies:
- dependency-name: once_cell
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-07-05 08:54:51 +00:00
3049479b78
feat: Implement new querier to ingester config design
2022-06-30 08:26:50 -04:00
59da2dccb8
feat: Assert if no ingester addresses are found
...
Temporarily support `--ingester-addresses` (and always return all
ingesters) so that this PR can be deployed during the switchover.
2022-06-30 08:22:47 -04:00
edf3f08e81
refactor: Replace all uses of lazy_static with once_cell
...
Went through and remove all lazy_static uses with once_cell (while waiting for the project to compile). There are still dependencies using lazy_static so it is still in the crate graph but at least there isn't an explicit dependency on it (and it is easier to update to `std::lazy::Lazy` once that is stable).
2022-06-29 16:22:02 +02:00
c1f7154031
feat: propagate partition key through kafka
...
Changes the kafka message wire format to include the partition key for
serialised DML writes on the wire.
After this commit, the kafka messages will contain the partition key for
each op, but this information will go unused in the ingester - this
enables us to roll out the producer side, before making the value's
presence necessary on the consumer side.
A follow-up PR will change the ingester to utilise this embedded
partition key.
This has the unfortunate side effect of making the partition key part of
the public gRPC write API:
https://github.com/influxdata/influxdb_iox/issues/4866
2022-06-20 13:42:51 +01:00
0fbff981ec
chore(deps): Bump sqlx to 0.6.0 and uuid to 1 ( #4894 )
...
Closes #4889 .
Closes #4890 .
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-06-17 10:28:28 +00:00
e91d00b10c
chore: Update datafusion + `arrow`/`parquet`/`arrow-flight` to `16.0.0 ( #4851 )
...
* chore: TEMP Update DataFusion to pre-release
* chore: update arrow et al to 16.0.0
* chore: Run cargo hakari tasks
* fix: update reader read_dictionary API
* chore: Update to real Datafusion release
* fix: Update parquet API
* fix: update test
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
2022-06-14 16:31:40 +00:00
3ecb1ee056
chore(deps): Bump http from 0.2.7 to 0.2.8 ( #4796 )
...
Bumps [http](https://github.com/hyperium/http ) from 0.2.7 to 0.2.8.
- [Release notes](https://github.com/hyperium/http/releases )
- [Changelog](https://github.com/hyperium/http/blob/master/CHANGELOG.md )
- [Commits](https://github.com/hyperium/http/compare/v0.2.7...v0.2.8 )
---
updated-dependencies:
- dependency-name: http
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-06-07 13:35:01 +00:00
e03bf94420
chore(deps): Bump tokio from 1.18.2 to 1.19.1 ( #4783 )
...
Bumps [tokio](https://github.com/tokio-rs/tokio ) from 1.18.2 to 1.19.1.
- [Release notes](https://github.com/tokio-rs/tokio/releases )
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.18.2...tokio-1.19.1 )
---
updated-dependencies:
- dependency-name: tokio
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-06-06 14:15:12 +00:00
3592aa52d8
chore: Update datafusion + `arrow`/`parquet`/`arrow-flight` to `15.0.0` ( #4743 )
...
* chore: Update datafusion + `arrow`/`parquet`/`arrow-flight` to `15.0.0`
* chore: Update APIs
* chore: Run cargo hakari tasks
* feat: normalize parquet file metadata
* chore: update size tests
* chore: add docs on metadata stripping
* chore: TEMP UPDATE TO DF BRANCH
* chore: Update for new API
* fix: Update to latest DF
* fix: cargo hakari
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: Raphael Taylor-Davies <r.taylordavies@googlemail.com>
2022-06-03 10:32:26 +00:00
b52a3586a7
fix: Turn cargo doc warnings into errors ( #4710 )
...
* fix: Correct intra-doc links
* fix: Turn cargo doc warnings into errors
Co-authored-by: Jake Goulding <jake.goulding@integer32.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-05-28 11:24:22 +00:00
a08a91c5ba
fix: ensure querier cache is refreshed for partition sort key ( #4660 )
...
* test: call `maybe_start_logging` in auto-generated cases
* fix: ensure querier cache is refreshed for partition sort key
Fixes #4631 .
* docs: explain querier sort key handling and test
* test: test another version of issue 4631
* fix: correctly invalidate partition sort keys
* fix: fix `table_not_found_on_ingester`
2022-05-25 10:44:42 +00:00
9c1ffc2b0d
test: panic handling, add compactor to end to end test harness ( #4677 )
...
* feat: add test gRPC client
* test: start compactor in mini cluster
* test: assert panic handling
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-05-24 14:55:26 +00:00
76f7043417
chore(deps): Bump once_cell from 1.11.0 to 1.12.0 ( #4666 )
...
Bumps [once_cell](https://github.com/matklad/once_cell ) from 1.11.0 to 1.12.0.
- [Release notes](https://github.com/matklad/once_cell/releases )
- [Changelog](https://github.com/matklad/once_cell/blob/master/CHANGELOG.md )
- [Commits](https://github.com/matklad/once_cell/compare/v1.11.0...v1.12.0 )
---
updated-dependencies:
- dependency-name: once_cell
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-05-24 08:14:03 +00:00
2029bd16ba
feat: enable debugging of failed querier->ingester requests ( #4659 )
...
* feat: enable debugging of failed querier->ingester requests
- extend `query-ingester` CLI to allow usage of predicates
- on failed requests: log all information that required for the CLI
- test the "ingester fails" scenario
* test: explain
Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
* docs: improve
Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
* refactor: move b64 pred. serde into a single crate
Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
2022-05-23 15:37:31 +00:00
6bc0c74c7d
chore(deps): Bump once_cell from 1.10.0 to 1.11.0 ( #4646 )
...
* chore(deps): Bump once_cell from 1.10.0 to 1.11.0
Bumps [once_cell](https://github.com/matklad/once_cell ) from 1.10.0 to 1.11.0.
- [Release notes](https://github.com/matklad/once_cell/releases )
- [Changelog](https://github.com/matklad/once_cell/blob/master/CHANGELOG.md )
- [Commits](https://github.com/matklad/once_cell/compare/v1.10.0...v1.11.0 )
---
updated-dependencies:
- dependency-name: once_cell
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* chore: Run cargo hakari tasks
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-05-20 07:40:38 +00:00
3a33e806c7
chore: Update datafusion + `arrow`/`parquet`/`arrow-flight` to `14.0.0` ( #4619 )
...
* chore: Update datafusion deps
* chore: update arrow/parquet/arrow flight deps
* chore: Run cargo hakari tasks
* chore: Update location of utils
* chore: Update some more APIs
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
2022-05-17 14:13:03 +00:00
55313d290a
fix: Update or remove comments that mention NG or OG
...
Connects to #4450 .
2022-05-12 16:09:08 -04:00
48e6e5713d
fix: Rename test_helpers_end_to_end_ng to test_helpers_end_to_end
...
Connects to #4450 .
2022-05-12 16:09:07 -04:00
Carol (Nichols || Goulding)
dependabot[bot]
Andrew Lamb
Marco Neumann
Stuart Carnie
Dom Dwyer
Carol (Nichols || Goulding)
Andrew Lamb
YIXIAO SHI
Marko Mikulicic
Markus Westerlind