3a33e806c7
chore: Update datafusion + `arrow`/`parquet`/`arrow-flight` to `14.0.0` ( #4619 )
...
* chore: Update datafusion deps
* chore: update arrow/parquet/arrow flight deps
* chore: Run cargo hakari tasks
* chore: Update location of utils
* chore: Update some more APIs
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
2022-05-17 14:13:03 +00:00
779f0e9cdf
feat: querier RAM pool ( #4593 )
...
* feat: `SortKey::size`
* feat: `FunctionEstimator`
* feat: querier RAM pool
Let's put all the caches into a single RAM pool, so we can at least
somewhat control RAM usage. Note that this does NOT limit the peak
memory during query execution though, but should at least stop unlimited
cache growth. A follow-up PR will add metrics.
* refactor: improve some size calculations
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-05-17 13:11:20 +00:00
87d8a8b684
chore(deps): Bump schemars from 0.8.8 to 0.8.9 ( #4614 )
...
Bumps [schemars](https://github.com/GREsau/schemars ) from 0.8.8 to 0.8.9.
- [Release notes](https://github.com/GREsau/schemars/releases )
- [Changelog](https://github.com/GREsau/schemars/blob/master/CHANGELOG.md )
- [Commits](https://github.com/GREsau/schemars/compare/v0.8.8...v0.8.9 )
---
updated-dependencies:
- dependency-name: schemars
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-05-17 09:49:27 +00:00
5a0006e40b
chore(deps): Bump syn from 1.0.94 to 1.0.95 ( #4610 )
...
Bumps [syn](https://github.com/dtolnay/syn ) from 1.0.94 to 1.0.95.
- [Release notes](https://github.com/dtolnay/syn/releases )
- [Commits](https://github.com/dtolnay/syn/compare/1.0.94...1.0.95 )
---
updated-dependencies:
- dependency-name: syn
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-05-17 09:40:36 +00:00
524c77de5d
Merge pull request #4607 from influxdata/cn/kube-update
...
chore(deps): Merge all the kube deps together
2022-05-17 09:32:22 +00:00
3ef01032ed
Merge branch 'main' into cn/kube-update
2022-05-17 09:26:43 +00:00
d243ada6db
Merge pull request #4609 from influxdata/cn/clearer-logging
...
feat: Add more logging in particular situations to debug flaky test
2022-05-17 00:47:01 +00:00
9eb21095e7
feat: Add more logging in particular situations to debug flaky test
2022-05-16 16:46:29 -04:00
a91a9d5789
Merge remote-tracking branch 'origin/dependabot/cargo/kube-runtime-0.72.0' into cn/kube-update
2022-05-16 10:50:51 -04:00
42b52da0d4
Merge remote-tracking branch 'origin/dependabot/cargo/kube-derive-0.72.0' into cn/kube-update
2022-05-16 10:49:51 -04:00
edd3cce73c
Merge remote-tracking branch 'origin/dependabot/cargo/kube-0.72.0' into cn/kube-update
2022-05-16 10:26:06 -04:00
93d9db0ac8
chore(deps): Bump kube-derive from 0.71.0 to 0.72.0
...
Bumps [kube-derive](https://github.com/kube-rs/kube-rs ) from 0.71.0 to 0.72.0.
- [Release notes](https://github.com/kube-rs/kube-rs/releases )
- [Changelog](https://github.com/kube-rs/kube-rs/blob/master/CHANGELOG.md )
- [Commits](https://github.com/kube-rs/kube-rs/compare/0.71.0...0.72.0 )
---
updated-dependencies:
- dependency-name: kube-derive
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2022-05-16 11:44:54 +00:00
259d2486c1
chore(deps): Bump tokio-util from 0.7.1 to 0.7.2 ( #4605 )
...
Bumps [tokio-util](https://github.com/tokio-rs/tokio ) from 0.7.1 to 0.7.2.
- [Release notes](https://github.com/tokio-rs/tokio/releases )
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-util-0.7.1...tokio-util-0.7.2 )
---
updated-dependencies:
- dependency-name: tokio-util
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-05-16 11:42:31 +00:00
f668a11f3e
chore(deps): Bump syn from 1.0.93 to 1.0.94 ( #4606 )
...
Bumps [syn](https://github.com/dtolnay/syn ) from 1.0.93 to 1.0.94.
- [Release notes](https://github.com/dtolnay/syn/releases )
- [Commits](https://github.com/dtolnay/syn/compare/1.0.93...1.0.94 )
---
updated-dependencies:
- dependency-name: syn
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-05-16 11:21:37 +00:00
59a671125e
chore(deps): Bump rustls from 0.20.4 to 0.20.5 ( #4602 )
...
Bumps [rustls](https://github.com/rustls/rustls ) from 0.20.4 to 0.20.5.
- [Release notes](https://github.com/rustls/rustls/releases )
- [Changelog](https://github.com/rustls/rustls/blob/main/RELEASE_NOTES.md )
- [Commits](https://github.com/rustls/rustls/compare/v/0.20.4...v/0.20.5 )
---
updated-dependencies:
- dependency-name: rustls
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-05-16 11:13:40 +00:00
ca19567710
chore: Run cargo hakari tasks
2022-05-16 01:38:59 +00:00
ca1edc3b4d
chore: Run cargo hakari tasks
2022-05-16 01:38:11 +00:00
6ba1a5f253
chore(deps): Bump kube-runtime from 0.71.0 to 0.72.0
...
Bumps [kube-runtime](https://github.com/kube-rs/kube-rs ) from 0.71.0 to 0.72.0.
- [Release notes](https://github.com/kube-rs/kube-rs/releases )
- [Changelog](https://github.com/kube-rs/kube-rs/blob/master/CHANGELOG.md )
- [Commits](https://github.com/kube-rs/kube-rs/compare/0.71.0...0.72.0 )
---
updated-dependencies:
- dependency-name: kube-runtime
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2022-05-16 01:37:57 +00:00
d7a0ca7710
chore(deps): Bump kube from 0.71.0 to 0.72.0
...
Bumps [kube](https://github.com/kube-rs/kube-rs ) from 0.71.0 to 0.72.0.
- [Release notes](https://github.com/kube-rs/kube-rs/releases )
- [Changelog](https://github.com/kube-rs/kube-rs/blob/master/CHANGELOG.md )
- [Commits](https://github.com/kube-rs/kube-rs/compare/0.71.0...0.72.0 )
---
updated-dependencies:
- dependency-name: kube
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2022-05-16 01:37:13 +00:00
9530e73925
chore: move noisy debug to trace and fix some comments ( #4598 )
...
* chore: move noisy debug to trace and fix some comments
* chore: Apply suggestions from code review
Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
* chore: fix format
Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-05-13 19:18:15 +00:00
22c1853f3e
chore: Update datafusion dependency ( #4599 )
...
* chore: Update datafusion dependency
* chore: Update for new API
2022-05-13 19:10:22 +00:00
f2bb0fdf77
feat: update to crates.io object_store version ( #4595 )
...
* feat: update to crates.io object_store version
* chore: Run cargo hakari tasks
* fix: tests
* chore: remove object store integration test plumbing
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
2022-05-13 16:26:07 +00:00
35d80fc512
refactor: rename `iox_catalog_service` to `service_grpc_catalog` for consistency ( #4581 )
...
* refactor: rename `iox_catalog_service` to `service_grpc_catalog`
* fix: fmt
* fix: alphabetize crate
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-05-13 14:07:58 +00:00
04160d29e6
Merge pull request #4586 from influxdata/cn/remove-chunk-addr
...
fix: Remove ChunkAddr and old_gen_partition_key
2022-05-13 14:01:21 +00:00
0f8f294319
Merge branch 'main' into cn/remove-chunk-addr
2022-05-13 13:54:44 +00:00
720220c618
Merge pull request #4584 from influxdata/cn/rename-no-ng
...
fix: Remove mentions of "NG", "OG"
2022-05-13 13:54:23 +00:00
542ec97b66
Merge branch 'main' into cn/rename-no-ng
2022-05-13 13:47:48 +00:00
cb0a4176fd
refactor: move `querier::cache_system` into its own crate ( #4592 )
2022-05-13 13:12:07 +00:00
ff8241ea57
fix: Use one (shared) http2 connection per querier and ingester pair ( #4583 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-05-13 11:15:10 +00:00
31f3d988ae
feat: LRU cache infrastructure ( #4189 )
...
* feat: `AddressableHeap::is_empty`
* feat: add type-safe size trait
* feat: LRU cache infrastructure
* fix: typos
Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
* fix: update after rebase
* docs: explain test code
* test: ensure that values are dropped from LRU cache
* test: ensure that backends are dropped from LRU cache
* docs: explain where LRU state is stored
* docs: explain high-level LRU usage
* refactor: "memory (size)" => "resource (consumption)"
This should make the reasoning more generic and easier to understand.
Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-05-13 07:34:59 +00:00
d15abc5e2d
chore(deps): Bump zstd-safe from 5.0.1+zstd.1.5.2 to 5.0.2+zstd.1.5.2 ( #4591 )
...
Bumps [zstd-safe](https://github.com/gyscos/zstd-rs ) from 5.0.1+zstd.1.5.2 to 5.0.2+zstd.1.5.2.
- [Release notes](https://github.com/gyscos/zstd-rs/releases )
- [Commits](https://github.com/gyscos/zstd-rs/commits )
---
updated-dependencies:
- dependency-name: zstd-safe
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-05-13 07:27:24 +00:00
335374fed5
chore: Enable pprof feature back ( #4587 ) ( #4590 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-05-12 23:33:28 +00:00
4434ec6836
chore: convert some debug to trace to reduce noises ( #4589 )
2022-05-12 22:40:29 +00:00
24020403ba
fix: Remove pprof feature; it fails to build ( #4588 )
2022-05-12 21:45:13 +00:00
d807941804
chore: Enable pprof feature back ( #4587 )
2022-05-12 21:25:22 +00:00
55313d290a
fix: Update or remove comments that mention NG or OG
...
Connects to #4450 .
2022-05-12 16:09:08 -04:00
07c7c75067
fix: Remove ng_chunk method
...
Connects to #4450 .
2022-05-12 16:09:08 -04:00
a38a25437b
fix: Remove split_overlapped_chunks_og and group_potential_duplicates_og
...
Connects to #4450 .
2022-05-12 16:09:07 -04:00
025a775cc6
fix: Rename make_ng_chunk to make_chunk.
...
Connects to #4450 .
2022-05-12 16:09:07 -04:00
b581a42fde
fix: Rename new_id_for_ng to new_id
...
Connects to #4450 .
2022-05-12 16:09:07 -04:00
30e53fd09c
fix: Rename end-to-end NG tests to not contain NG
...
Connects to #4450 .
2022-05-12 16:09:07 -04:00
48e6e5713d
fix: Rename test_helpers_end_to_end_ng to test_helpers_end_to_end
...
Connects to #4450 .
2022-05-12 16:09:07 -04:00
78bbe629b2
feat: Add more logging to understand the flaky multi ingester test better ( #4580 )
...
* feat: Increase logging to investigate multi ingester flaky test
* feat: Temporarily disable a test while logging is increased in CI
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-05-12 20:05:05 +00:00
71d1f2c9bd
fix: Remove old_gen_partition_key
2022-05-12 16:05:03 -04:00
faba90d992
fix: Remove ChunkAddr
2022-05-12 15:50:41 -04:00
975dd288d4
refactor: Rearrange delegation to remove one usage of generated types ( #4579 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-05-12 16:10:38 +00:00
7eff52d04e
refactor: rename `iox_object_store_service` to `service_grpc_object_store` for consisntency ( #4572 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-05-12 14:22:18 +00:00
66fe4c54ec
fix: make test output deterministic ( #4578 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-05-12 14:15:51 +00:00
1bf519c2f6
Merge pull request #4573 from influxdata/cn/write-info-grpc-more
...
feat: Querier can get write status from ingesters
2022-05-12 13:59:41 +00:00
2eb2db5286
Merge branch 'main' into cn/write-info-grpc-more
2022-05-12 13:53:19 +00:00
Andrew Lamb
Marco Neumann
dependabot[bot]
kodiakhq[bot]
Carol (Nichols || Goulding)
CircleCI[bot]
Nga Tran
Raphael Taylor-Davies
Marko Mikulicic
Carol (Nichols || Goulding)