6f4c5506d8
chore(deps): Bump tracing-log from 0.1.2 to 0.1.3 ( #4388 )
...
Bumps [tracing-log](https://github.com/tokio-rs/tracing ) from 0.1.2 to 0.1.3.
- [Release notes](https://github.com/tokio-rs/tracing/releases )
- [Commits](https://github.com/tokio-rs/tracing/compare/tracing-log-0.1.2...tracing-log-0.1.3 )
---
updated-dependencies:
- dependency-name: tracing-log
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-04-21 21:25:48 +00:00
a6e3da4dc3
Merge pull request #4386 from influxdata/cn/port-tracing-test
...
feat: Port end-to-end tracing tests to NG end-to-end tests
2022-04-21 21:16:14 +00:00
8250c77253
Merge branch 'main' into cn/port-tracing-test
2022-04-21 21:07:18 +00:00
e67cc9dbce
chore: Update datafusion again ( #4385 )
...
* chore: Update datafusion
* fix: Update imports
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-04-21 21:05:16 +00:00
2087dad1f7
feat: Port tracing tests to NG end-to-end tests
2022-04-21 16:22:43 -04:00
02692751c7
feat: Copy end-to-end tracing tests to NG end-to-end tests
...
This doesn't compile at all, but this is a straight cp so should make it
a bit easier to see what has changed in the port in the next commit.
2022-04-21 16:20:34 -04:00
d88cf324fa
feat: Add a method to create an all-in-one MiniCluster from one TestConfig
2022-04-21 16:18:55 -04:00
bf8603cc31
feat: Add test config methods to configure tracing
2022-04-21 16:18:43 -04:00
3499207911
feat: Make the end-to-end UDP listener available in NG end-to-end tests
2022-04-21 15:16:57 -04:00
c7a1c496cf
fix: incorrect overlapped grouping ( #4082 )
...
* test: Failing test for finding overlapped groups
* test: Failing test for query overlap too :(
* fix: Group parquet files overlapped by time correctly
Inspired by https://towardsdatascience.com/overlapping-time-period-problem-b7f1719347db
Not sure what the real name for this algorithm is
* refactor: Group items without an intermediate hashmap needed
* chore: cleanup
Co-authored-by: NGA-TRAN <nga-tran@live.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-04-21 18:51:30 +00:00
d41086ac7f
refactor: Move Connection in ServerFixture to make TestServer shareable ( #4381 )
...
* refactor: Move Connection in ServerFixture to make TestServer shareable
* fix: Update docstrings
* fix: restore panic on previous error
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-04-21 18:15:53 +00:00
abd005e0c2
refactor: Consolidate cluster creation in end to end tests ( #4373 )
2022-04-21 18:03:17 +00:00
cc9205024c
test: Begin porting influxrpc tests to ng end to end tests ( #4372 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-04-21 16:24:45 +00:00
f302ce370b
chore(deps): Bump prost-types from 0.10.0 to 0.10.1 ( #4377 )
...
Bumps [prost-types](https://github.com/tokio-rs/prost ) from 0.10.0 to 0.10.1.
- [Release notes](https://github.com/tokio-rs/prost/releases )
- [Commits](https://github.com/tokio-rs/prost/compare/v0.10.0...v0.10.1 )
---
updated-dependencies:
- dependency-name: prost-types
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-04-21 13:22:53 +00:00
f444e63960
test: include materialized delete predicates in NG query tests ( #4371 )
...
* refactor: move `batch_filter` to `datafusion_util`
* fix: outdated docstring
* feat: allow passing record batches to `iox_tests` parquet files
* test: include materialized delete predicates in NG query tests
* docs: improve wording
Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-04-21 13:00:13 +00:00
b88900b08c
chore(deps): Bump nix from 0.23.1 to 0.24.0 ( #4378 )
...
Bumps [nix](https://github.com/nix-rust/nix ) from 0.23.1 to 0.24.0.
- [Release notes](https://github.com/nix-rust/nix/releases )
- [Changelog](https://github.com/nix-rust/nix/blob/master/CHANGELOG.md )
- [Commits](https://github.com/nix-rust/nix/commits )
---
updated-dependencies:
- dependency-name: nix
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-04-21 12:51:31 +00:00
2984578aac
chore: Update pprof, remove old versions of prost ( #4374 )
...
* chore: Upgrade pprof to 0.8
* chore: Update heappy
2022-04-21 10:52:23 +00:00
200a066d46
chore(deps): Bump prost from 0.10.0 to 0.10.1 ( #4375 )
...
Bumps [prost](https://github.com/tokio-rs/prost ) from 0.10.0 to 0.10.1.
- [Release notes](https://github.com/tokio-rs/prost/releases )
- [Commits](https://github.com/tokio-rs/prost/compare/v0.10.0...v0.10.1 )
---
updated-dependencies:
- dependency-name: prost
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-04-21 10:25:10 +00:00
c084785bc3
feat: fuse ingester and catalog states in querier ( #4355 )
...
* feat: fuse ingester and catalog states in querier
This now correctly combines the data we get from the ingester w/ the
data we get from the catalog. Right now it bails out if during the very
small time windows between asking the ingester and querying the catalog
the compactor combines the newest files w/ "too new" files (see tests).
* fix: improve error wording
Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
* fix: improve doc comment
Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
* fix: explain tests
Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
* refactor: improve tests, method naming and docs
Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-04-20 14:37:19 +00:00
3961568fff
fix: Add protobuf compiler to docker build image ( #4369 )
2022-04-20 13:33:06 +00:00
c05612935b
test: add end to end querier test with multiple ingesters ( #4309 )
...
* test: add a test with multiple ingesters
* docs: improve doc test
* refactor: Print out step number in StepTest
* fix: make timeout more explicit
* fix: add tests for merge_info
2022-04-20 11:56:41 +00:00
fcb82b2c99
Merge pull request #4367 from influxdata/dom/panic-metrics
...
feat: panic metrics
2022-04-20 11:43:20 +00:00
c363242902
refactor: emit panic metrics for server types
...
Configures the long-running / server modes to emit panic metrics.
2022-04-20 12:30:02 +01:00
49c38c2976
feat: emit thread panic count metric
...
Allows the panic handler to be optionally configured to emit a thread
panic metric, increasing by 1 each time a panic is observed.
2022-04-20 12:30:02 +01:00
73bed810da
chore: Update arrow, arrow-flight, parquet, tonic, prost, etc ( #4357 )
...
* chore: Update datafusion
* chore: Update arrow/arrow-flight/parquet to 12
* chore: update datafusion correctly
* chore: Update prost, tonic, and dependents
* fix: Fixup some api changes
* fix: Update test output in db
* fix: Update test output in parquet_file
* fix: remove old pbjson types
* fix: Add "--experimental_allow_proto3_optional" flag
* chore: Run cargo hakari tasks
* fix: compile error
* chore: Update heappy
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-04-20 11:12:17 +00:00
383c0b328d
feat: Issue queries from querier to ingester in parallel ( #4359 )
...
* feat: Issue queries from querier to ingester in parallel
* refactor: complete Arc-ification
* refactor: use a named struct to pass the state
2022-04-20 10:55:14 +00:00
9cc7d73671
chore: Install custom protoc in perf-builder image ( #4358 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-04-20 10:35:05 +00:00
0642ec0b82
docs: add note about write_info API being internal ( #4356 )
...
* docs: add note about write_info API being internal
* fix: update doc urls
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-04-20 09:25:14 +00:00
ca464b5ab6
chore(deps): Bump clap from 3.1.9 to 3.1.10 ( #4361 )
...
Bumps [clap](https://github.com/clap-rs/clap ) from 3.1.9 to 3.1.10.
- [Release notes](https://github.com/clap-rs/clap/releases )
- [Changelog](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md )
- [Commits](https://github.com/clap-rs/clap/compare/v3.1.9...v3.1.10 )
---
updated-dependencies:
- dependency-name: clap
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-04-20 08:44:17 +00:00
dbddffee6f
chore: add semantic PR and commit message checks ( #4344 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-04-19 17:29:03 +00:00
5ea676d3f7
feat: add per kafka partition durability reporting to write info response ( #4341 )
...
* feat: add per kafka partition durability reporting to write info response
* fix: buf lint + test cleanup
* fix: clean up protobuf
* refactor: pull out conversion of KafkaPartitionStatus into a function
* fix: fmt
* fix: typo
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-04-19 16:46:20 +00:00
7ac7fdcd5b
Merge pull request #4351 from influxdata/dom/revert-service-limits
...
fix: revert column service limits (#4179 )
2022-04-19 16:38:19 +00:00
752490fe34
Merge branch 'main' into dom/revert-service-limits
2022-04-19 16:30:09 +00:00
6088c1a588
refactor: rewrite schema command end to end tests to use new StepTest framework ( #4337 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-04-19 16:29:50 +00:00
103629b01d
feat: add client and CLI to get file from object store ( #4343 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-04-19 16:16:27 +00:00
f55319cdc2
Merge pull request #4353 from influxdata/dom/logfmt-quoting
...
fix(logfmt): quote strings containing '='
2022-04-19 16:08:32 +00:00
3788a6105e
Merge branch 'main' into dom/logfmt-quoting
2022-04-19 16:00:02 +00:00
d711816548
feat: add sequencer ID and correct partition key to `IngesterPartition` ( #4348 )
...
* feat: impl `Debug` for `TestCatalog`
* feat: add sequencer ID and correct partition key to `IngesterPartition`
- simplifies debugging (parquet chunks and ingester chunks now use the
same partition key naming)
- the sequencer ID is required to correctly reason about tombstones (to
be implemented in a later PR)
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-04-19 15:59:49 +00:00
8ec9d0eac4
fix(logfmt): quote strings containing '='
...
Logfmt uses '=' as a key/value separator, and prior to this commit the
formatter did not escape strings containing '='.
2022-04-19 16:47:18 +01:00
320f1073e0
fix: revert column service limits ( #4179 )
...
This reverts commit ea865b63f4
2022-04-19 16:08:56 +01:00
e3d83fe757
chore: update datafusion ( #4342 )
...
* chore: update datafusion
* fix: Update imports for change in datafusion organization
2022-04-19 13:38:12 +00:00
3588a06647
feat: correctly dismantle ingester gRPC reponse in querier ( #4323 )
...
This now correctly processes record batches for the different
partitions. The actual code change is rather small but it requires some
substantial test infrastructure.
2022-04-19 11:09:40 +00:00
5b48675435
fix: actually transmit record-batch metadata from querier ( #4347 )
...
Attaching the "batch => partition" mapping via per-batch schema KV
metadata does NOT work because flight will transmit the schema once for
all batches (even though on the Rust side we have a schema ref attached
to every batch, probably for convenience). Instead we now use the same
global protobuf metadata that we also use for the "partition => max
sequence number" information. This somewhat limits our ability to create
record batches lazily on the ingester side (since the global metadata is
sent before any actual payload) but I think we should not modify the
usage of the flight protocol too much right now (e.g. by sending more
schema messages). If this becomes an issue, we can always find a more
complex solution in the future.
2022-04-19 10:54:23 +00:00
cfac332000
chore(deps): Bump backtrace from 0.3.64 to 0.3.65 ( #4346 )
...
Bumps [backtrace](https://github.com/rust-lang/backtrace-rs ) from 0.3.64 to 0.3.65.
- [Release notes](https://github.com/rust-lang/backtrace-rs/releases )
- [Commits](https://github.com/rust-lang/backtrace-rs/compare/0.3.64...0.3.65 )
---
updated-dependencies:
- dependency-name: backtrace
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-04-19 10:12:06 +00:00
3d5896d18d
chore(deps): Bump libc from 0.2.123 to 0.2.124 ( #4345 )
...
Bumps [libc](https://github.com/rust-lang/libc ) from 0.2.123 to 0.2.124.
- [Release notes](https://github.com/rust-lang/libc/releases )
- [Commits](https://github.com/rust-lang/libc/compare/0.2.123...0.2.124 )
---
updated-dependencies:
- dependency-name: libc
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-04-19 08:47:40 +00:00
9c3b145714
chore(deps): Bump tracing-core from 0.1.25 to 0.1.26 ( #4331 )
...
Bumps [tracing-core](https://github.com/tokio-rs/tracing ) from 0.1.25 to 0.1.26.
- [Release notes](https://github.com/tokio-rs/tracing/releases )
- [Commits](https://github.com/tokio-rs/tracing/compare/tracing-core-0.1.25...tracing-core-0.1.26 )
---
updated-dependencies:
- dependency-name: tracing-core
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-04-18 17:19:58 +00:00
2a601c3099
fix: Revert "chore: Revert "fx: Revert "fix: Revert "feat: Use the sort key stored in the catalog during compaction" ( #4299 )" ( #4303 )" ( #4327 )" ( #4328 )
...
* fix: Revert "chore: Revert "fx: Revert "fix: Revert "feat: Use the sort key stored in the catalog during compaction" (#4299 )" (#4303 )" (#4327 )"
This reverts commit 7e5d719027
2022-04-18 15:27:39 +00:00
b37a92307f
chore(deps): Bump clap from 3.1.8 to 3.1.9 ( #4340 )
...
Bumps [clap](https://github.com/clap-rs/clap ) from 3.1.8 to 3.1.9.
- [Release notes](https://github.com/clap-rs/clap/releases )
- [Changelog](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md )
- [Commits](https://github.com/clap-rs/clap/compare/v3.1.8...v3.1.9 )
---
updated-dependencies:
- dependency-name: clap
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-04-18 10:56:44 +00:00
053bcefaee
chore(deps): Bump sqlx from 0.5.12 to 0.5.13 ( #4339 )
...
Bumps [sqlx](https://github.com/launchbadge/sqlx ) from 0.5.12 to 0.5.13.
- [Release notes](https://github.com/launchbadge/sqlx/releases )
- [Changelog](https://github.com/launchbadge/sqlx/blob/master/CHANGELOG.md )
- [Commits](https://github.com/launchbadge/sqlx/compare/v0.5.12...v0.5.13 )
---
updated-dependencies:
- dependency-name: sqlx
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-04-18 10:48:17 +00:00
52ffa4a482
chore(deps): Bump sqlx-core from 0.5.11 to 0.5.12 ( #4335 )
...
* chore(deps): Bump sqlx-core from 0.5.11 to 0.5.12
Bumps [sqlx-core](https://github.com/launchbadge/sqlx ) from 0.5.11 to 0.5.12.
- [Release notes](https://github.com/launchbadge/sqlx/releases )
- [Changelog](https://github.com/launchbadge/sqlx/blob/master/CHANGELOG.md )
- [Commits](https://github.com/launchbadge/sqlx/compare/v0.5.11...v0.5.12 )
---
updated-dependencies:
- dependency-name: sqlx-core
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* chore: Run cargo hakari tasks
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-04-18 10:48:02 +00:00
dependabot[bot]
kodiakhq[bot]
Andrew Lamb
Carol (Nichols || Goulding)
Carol (Nichols || Goulding)
Marco Neumann
Dom Dwyer
Jacob Marble
Paul Dix
Nga Tran