56a1c579a1
refactor: Change influxdb_iox client to use http rather than grpc for write ( #5756 )
...
* refactor: Change influxdb_iox client to use http rather than grpc for write
* refactor: remove custom variants
* refactor: consolidate more
2022-09-29 11:12:51 +00:00
66dbb9541f
chore: Update datafusion and `arrow`/`parquet`/`arrow-flight` to 23.0.0, `thrift` to 0.16.0 ( #5694 )
...
* chore: Update datafusion and `arrow`/`parquet`/`arrow-flight` to 23.0.0
* chore: Update thrift / remove parquet_format
* fix: Update APIs
* chore: Update lock + Run cargo hakari tasks
* fix: use patched version of arrow-rs to work around https://github.com/apache/arrow-rs/issues/2779
* chore: Run cargo hakari tasks
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-09-27 12:50:54 +00:00
0d18943ad2
chore(deps): Bump once_cell from 1.14.0 to 1.15.0 ( #5701 )
...
Bumps [once_cell](https://github.com/matklad/once_cell ) from 1.14.0 to 1.15.0.
- [Release notes](https://github.com/matklad/once_cell/releases )
- [Changelog](https://github.com/matklad/once_cell/blob/master/CHANGELOG.md )
- [Commits](https://github.com/matklad/once_cell/compare/v1.14.0...v1.15.0 )
---
updated-dependencies:
- dependency-name: once_cell
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-09-21 05:20:55 +00:00
1fd31ee3bf
chore: Update datafusion / `arrow` / `arrow-flight` / `parquet` to version 22.0.0 ( #5591 )
...
* chore: Update datafusion / `arrow` / `arrow-flight` / `parquet` to version 22.0.0
* fix: enable dynamic comparison flag
* chore: derive Eq for clippy
* chore: update explain plans
* chore: Update sizes for ReadBuffer encoding
* chore: update more tests
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-09-12 17:45:03 +00:00
267a53a9e8
chore: update `tracing-subscriber`, fix trogging, fix CLI test port allocation ( #5581 )
...
* test: use dedicated ports for CLI tests
* chore: update `tracing-subscriber`
* fix: work around tracing-subscriber weirdness
It seems that trogging with tracing-subscriber >= 0.3.14 does not
produce any output at all. I suspect we are hitting
<https://github.com/tokio-rs/tracing/issues/2265 >. Let's change the
construct to not use multiple optional layers but a single dyn-dispatch
layer. Logging shouldn't have such a high throughput that his makes any
difference, esp. because the dyn-dispatch happens AFTER the filter.
2022-09-08 09:37:37 +00:00
52ae60bf2e
chore: fix comment typo ( #5551 )
...
Co-authored-by: Dom <dom@itsallbroken.com>
2022-09-07 08:49:29 +00:00
adeacf416c
ci: fix ( #5569 )
...
* ci: use same feature set in `build_dev` and `build_release`
* ci: also enable unstable tokio for `build_dev`
* chore: update tokio to 1.21 (to fix console-subscriber 0.1.8
* fix: "must use"
2022-09-06 14:13:28 +00:00
366c4d9965
chore(deps): Bump once_cell from 1.13.1 to 1.14.0 ( #5555 )
...
Bumps [once_cell](https://github.com/matklad/once_cell ) from 1.13.1 to 1.14.0.
- [Release notes](https://github.com/matklad/once_cell/releases )
- [Changelog](https://github.com/matklad/once_cell/blob/master/CHANGELOG.md )
- [Commits](https://github.com/matklad/once_cell/compare/v1.13.1...v1.14.0 )
---
updated-dependencies:
- dependency-name: once_cell
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-09-06 09:02:28 +00:00
6669d85fb4
chore: Update datafusion + arrow/parquet to `21.0.0` ( #5519 )
...
* chore: Update arrow/arrow-flight/parquet to 21.0.0
* chore: Update datafusion pin
* chore: Fix arrow update script
* chore: Update Cargo.lock
* chore: Update for new API
2022-08-31 13:30:47 +00:00
dbd27f648f
refactor: Rename more mentions of Kafka to their other name where appropriate
2022-08-29 14:27:02 -04:00
74c9529062
fix: Rename KafkaPartition to ShardIndex
2022-08-29 14:07:18 -04:00
78665d3092
chore(deps): Bump once_cell from 1.13.0 to 1.13.1 ( #5413 )
...
Bumps [once_cell](https://github.com/matklad/once_cell ) from 1.13.0 to 1.13.1.
- [Release notes](https://github.com/matklad/once_cell/releases )
- [Changelog](https://github.com/matklad/once_cell/blob/master/CHANGELOG.md )
- [Commits](https://github.com/matklad/once_cell/compare/v1.13.0...v1.13.1 )
---
updated-dependencies:
- dependency-name: once_cell
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-08-17 08:31:46 +00:00
38277cf0fb
chore(deps): Bump nix from 0.24.2 to 0.25.0 ( #5391 )
...
* chore(deps): Bump nix from 0.24.2 to 0.25.0
Bumps [nix](https://github.com/nix-rust/nix ) from 0.24.2 to 0.25.0.
- [Release notes](https://github.com/nix-rust/nix/releases )
- [Changelog](https://github.com/nix-rust/nix/blob/master/CHANGELOG.md )
- [Commits](https://github.com/nix-rust/nix/compare/v0.24.2...v0.25.0 )
---
updated-dependencies:
- dependency-name: nix
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* chore: Run cargo hakari tasks
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-08-15 08:10:09 +00:00
d8d29bc4b4
fix: Reduce type complexity
...
As now caught by clippy. https://rust-lang.github.io/rust-clippy/master/index.html#type_complexity
2022-08-11 15:04:15 -04:00
9321c96aaf
test: Add a step for running compaction in e2e tests
2022-08-10 11:30:22 -04:00
8ec9117836
test: Log a command run by e2e tests in a convenient way
...
So that you can copy-paste it if you want to run it again! 🎉
2022-08-10 11:30:22 -04:00
9981d1636b
refactor: Move dump_log_to_stdout so it can be used in more than server fixtures
2022-08-10 11:30:22 -04:00
463a13b814
test: Remove the compactor from the test MiniCluster
2022-08-10 11:28:51 -04:00
16ddc5efc6
chore: Update datafusion / arrow/parquet/arrow-flight and prost/tonic ecosystem ( #5360 )
...
* chore: Update datafusion and arrow
* chore: Update Cargo.lock
* chore: update to Decimal128
* chore: Update tonic/prost/pbjson/etc
* chore: Run cargo hakari tasks
* fix: doctest in generated types
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
2022-08-09 17:30:44 +00:00
273b3cc165
chore: replace `dotenv` with `dotenvy` ( #5285 )
...
The latter one is a maintained fork. This avoids having both crates
after #5282 .
2022-08-03 12:41:38 +00:00
9215a534d0
chore: Update datafusion and `arrow`/`parquet`/`arrow-flight` to `19.0.0` ( #5229 )
...
* chore: Update datafusion and `arrow`/`parquet`/`arrow-flight` to `19.0.0`
* chore: Run cargo hakari tasks
* fix: Update for API changes
* fix: clippy
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-07-28 08:10:47 +00:00
5a0af921c8
chore: Roll forward: Sync ReadWindowAggregate API: TagKeyMetaNames ( #5186 )
...
This reverts commit 5d02c755687ef041f5f45dbfc3e633a833284edb.
2022-07-22 10:44:06 +00:00
07cdb99192
chore: Revert "Sync ReadWindowAggregate API: TagKeyMetaNames" ( #5184 )
...
We're noticing a possible regression (OOMs) in our testing cluster that roughly correlates with this.
2022-07-22 09:26:42 +00:00
21d033eafd
fix: Sync ReadWindowAggregate API: TagKeyMetaNames
...
The storage API has been updated in https://github.com/influxdata/idpe/pull/12868
in January, but since we forked the `.proto` files we never noticed.
2022-07-21 15:07:04 +02:00
278a7f91af
chore(deps): Bump bytes from 1.1.0 to 1.2.0 ( #5156 )
...
Bumps [bytes](https://github.com/tokio-rs/bytes ) from 1.1.0 to 1.2.0.
- [Release notes](https://github.com/tokio-rs/bytes/releases )
- [Changelog](https://github.com/tokio-rs/bytes/blob/master/CHANGELOG.md )
- [Commits](https://github.com/tokio-rs/bytes/compare/v1.1.0...v1.2.0 )
---
updated-dependencies:
- dependency-name: bytes
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-07-20 10:00:08 +00:00
e2d871b00b
chore: Update datafusion and arrow/parquet/arrow-flight to `18.0.0` ( #5079 )
...
* chore: Update datafusion to 10.0.0, arrow/parquet/arrow-flight to 18
* chore: Run cargo hakari tasks
* fix: update cargo pin
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-07-18 15:01:03 +00:00
9b67de2f43
chore(deps): Bump tokio from 1.19.2 to 1.20.0
...
Bumps [tokio](https://github.com/tokio-rs/tokio ) from 1.19.2 to 1.20.0.
- [Release notes](https://github.com/tokio-rs/tokio/releases )
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.19.2...tokio-1.20.0 )
---
updated-dependencies:
- dependency-name: tokio
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2022-07-14 01:21:43 +00:00
c46e1c6347
chore: Update datafusion + arrow/parquet/arrow-flight to `17.0.0` ( #5021 )
...
* fix: correct nullability declaration of system tables
* chore: Update datafusion and arrow/parquet/arrow-flight
* chore: Run cargo hakari tasks
* fix: Update tests
* fix: Update tests
* fix: predicate pruning
* fix: add some tests
* fix: query_functions
* fix: fix read_buffer test
* fix: fix clippy
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-07-07 19:22:15 +00:00
89f5091546
refactor: Don't require DSN for test config
...
This enables writing a test for all-in-one's ephemeral mode, which
currently isn't working
2022-07-06 11:00:29 -04:00
68eff79594
chore(deps): Bump once_cell from 1.12.0 to 1.13.0 ( #5033 )
...
Bumps [once_cell](https://github.com/matklad/once_cell ) from 1.12.0 to 1.13.0.
- [Release notes](https://github.com/matklad/once_cell/releases )
- [Changelog](https://github.com/matklad/once_cell/blob/master/CHANGELOG.md )
- [Commits](https://github.com/matklad/once_cell/compare/v1.12.0...v1.13.0 )
---
updated-dependencies:
- dependency-name: once_cell
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-07-05 08:54:51 +00:00
3049479b78
feat: Implement new querier to ingester config design
2022-06-30 08:26:50 -04:00
59da2dccb8
feat: Assert if no ingester addresses are found
...
Temporarily support `--ingester-addresses` (and always return all
ingesters) so that this PR can be deployed during the switchover.
2022-06-30 08:22:47 -04:00
edf3f08e81
refactor: Replace all uses of lazy_static with once_cell
...
Went through and remove all lazy_static uses with once_cell (while waiting for the project to compile). There are still dependencies using lazy_static so it is still in the crate graph but at least there isn't an explicit dependency on it (and it is easier to update to `std::lazy::Lazy` once that is stable).
2022-06-29 16:22:02 +02:00
c1f7154031
feat: propagate partition key through kafka
...
Changes the kafka message wire format to include the partition key for
serialised DML writes on the wire.
After this commit, the kafka messages will contain the partition key for
each op, but this information will go unused in the ingester - this
enables us to roll out the producer side, before making the value's
presence necessary on the consumer side.
A follow-up PR will change the ingester to utilise this embedded
partition key.
This has the unfortunate side effect of making the partition key part of
the public gRPC write API:
https://github.com/influxdata/influxdb_iox/issues/4866
2022-06-20 13:42:51 +01:00
0fbff981ec
chore(deps): Bump sqlx to 0.6.0 and uuid to 1 ( #4894 )
...
Closes #4889 .
Closes #4890 .
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-06-17 10:28:28 +00:00
e91d00b10c
chore: Update datafusion + `arrow`/`parquet`/`arrow-flight` to `16.0.0 ( #4851 )
...
* chore: TEMP Update DataFusion to pre-release
* chore: update arrow et al to 16.0.0
* chore: Run cargo hakari tasks
* fix: update reader read_dictionary API
* chore: Update to real Datafusion release
* fix: Update parquet API
* fix: update test
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
2022-06-14 16:31:40 +00:00
3ecb1ee056
chore(deps): Bump http from 0.2.7 to 0.2.8 ( #4796 )
...
Bumps [http](https://github.com/hyperium/http ) from 0.2.7 to 0.2.8.
- [Release notes](https://github.com/hyperium/http/releases )
- [Changelog](https://github.com/hyperium/http/blob/master/CHANGELOG.md )
- [Commits](https://github.com/hyperium/http/compare/v0.2.7...v0.2.8 )
---
updated-dependencies:
- dependency-name: http
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-06-07 13:35:01 +00:00
e03bf94420
chore(deps): Bump tokio from 1.18.2 to 1.19.1 ( #4783 )
...
Bumps [tokio](https://github.com/tokio-rs/tokio ) from 1.18.2 to 1.19.1.
- [Release notes](https://github.com/tokio-rs/tokio/releases )
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.18.2...tokio-1.19.1 )
---
updated-dependencies:
- dependency-name: tokio
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-06-06 14:15:12 +00:00
3592aa52d8
chore: Update datafusion + `arrow`/`parquet`/`arrow-flight` to `15.0.0` ( #4743 )
...
* chore: Update datafusion + `arrow`/`parquet`/`arrow-flight` to `15.0.0`
* chore: Update APIs
* chore: Run cargo hakari tasks
* feat: normalize parquet file metadata
* chore: update size tests
* chore: add docs on metadata stripping
* chore: TEMP UPDATE TO DF BRANCH
* chore: Update for new API
* fix: Update to latest DF
* fix: cargo hakari
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: Raphael Taylor-Davies <r.taylordavies@googlemail.com>
2022-06-03 10:32:26 +00:00
b52a3586a7
fix: Turn cargo doc warnings into errors ( #4710 )
...
* fix: Correct intra-doc links
* fix: Turn cargo doc warnings into errors
Co-authored-by: Jake Goulding <jake.goulding@integer32.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-05-28 11:24:22 +00:00
a08a91c5ba
fix: ensure querier cache is refreshed for partition sort key ( #4660 )
...
* test: call `maybe_start_logging` in auto-generated cases
* fix: ensure querier cache is refreshed for partition sort key
Fixes #4631 .
* docs: explain querier sort key handling and test
* test: test another version of issue 4631
* fix: correctly invalidate partition sort keys
* fix: fix `table_not_found_on_ingester`
2022-05-25 10:44:42 +00:00
9c1ffc2b0d
test: panic handling, add compactor to end to end test harness ( #4677 )
...
* feat: add test gRPC client
* test: start compactor in mini cluster
* test: assert panic handling
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-05-24 14:55:26 +00:00
76f7043417
chore(deps): Bump once_cell from 1.11.0 to 1.12.0 ( #4666 )
...
Bumps [once_cell](https://github.com/matklad/once_cell ) from 1.11.0 to 1.12.0.
- [Release notes](https://github.com/matklad/once_cell/releases )
- [Changelog](https://github.com/matklad/once_cell/blob/master/CHANGELOG.md )
- [Commits](https://github.com/matklad/once_cell/compare/v1.11.0...v1.12.0 )
---
updated-dependencies:
- dependency-name: once_cell
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-05-24 08:14:03 +00:00
2029bd16ba
feat: enable debugging of failed querier->ingester requests ( #4659 )
...
* feat: enable debugging of failed querier->ingester requests
- extend `query-ingester` CLI to allow usage of predicates
- on failed requests: log all information that required for the CLI
- test the "ingester fails" scenario
* test: explain
Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
* docs: improve
Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
* refactor: move b64 pred. serde into a single crate
Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
2022-05-23 15:37:31 +00:00
6bc0c74c7d
chore(deps): Bump once_cell from 1.10.0 to 1.11.0 ( #4646 )
...
* chore(deps): Bump once_cell from 1.10.0 to 1.11.0
Bumps [once_cell](https://github.com/matklad/once_cell ) from 1.10.0 to 1.11.0.
- [Release notes](https://github.com/matklad/once_cell/releases )
- [Changelog](https://github.com/matklad/once_cell/blob/master/CHANGELOG.md )
- [Commits](https://github.com/matklad/once_cell/compare/v1.10.0...v1.11.0 )
---
updated-dependencies:
- dependency-name: once_cell
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* chore: Run cargo hakari tasks
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-05-20 07:40:38 +00:00
3a33e806c7
chore: Update datafusion + `arrow`/`parquet`/`arrow-flight` to `14.0.0` ( #4619 )
...
* chore: Update datafusion deps
* chore: update arrow/parquet/arrow flight deps
* chore: Run cargo hakari tasks
* chore: Update location of utils
* chore: Update some more APIs
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
2022-05-17 14:13:03 +00:00
55313d290a
fix: Update or remove comments that mention NG or OG
...
Connects to #4450 .
2022-05-12 16:09:08 -04:00
48e6e5713d
fix: Rename test_helpers_end_to_end_ng to test_helpers_end_to_end
...
Connects to #4450 .
2022-05-12 16:09:07 -04:00
Andrew Lamb
dependabot[bot]
Marco Neumann
YIXIAO SHI
Carol (Nichols || Goulding)
Marko Mikulicic
Markus Westerlind
Dom Dwyer
Carol (Nichols || Goulding)