1d4f8d2c8d
test: Ingester integration tests that can have a little a internal state
...
As a treat.
2023-02-16 11:06:44 -05:00
2fe9d9647f
refactor: Change the types returned from the IngesterRpcInterface
2023-02-16 10:02:17 -05:00
b840ed0ad9
fix: Use `as_expr` vs `col` to avoid splitting identifiers with periods ( #7011 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-02-16 11:03:06 +00:00
b11228c72e
test: Write an e2e test of namespace soft-deletion ( #7002 )
...
Co-authored-by: Dom <dom@itsallbroken.com>
2023-02-16 09:59:14 +00:00
95736ddd1a
chore(deps): Bump clap from 4.1.4 to 4.1.6 ( #7010 )
...
Bumps [clap](https://github.com/clap-rs/clap ) from 4.1.4 to 4.1.6.
- [Release notes](https://github.com/clap-rs/clap/releases )
- [Changelog](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md )
- [Commits](https://github.com/clap-rs/clap/compare/v4.1.4...v4.1.6 )
---
updated-dependencies:
- dependency-name: clap
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-02-16 08:26:22 +00:00
822063b7f2
feat: remember `QueryChunk` for every parquet file ( #7000 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-02-16 08:02:13 +00:00
e41cf080b4
feat: `RecordBatchesExec` remembers chunks ( #6999 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-02-16 07:55:35 +00:00
67794bccdb
refactor: `group_potential_duplicates` cannot fail ( #6998 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-02-15 19:28:02 +00:00
9e7153daee
Merge pull request #7006 from influxdata/alamb/target_level_choice
...
chore(compactor): Move target level choice into round info
2023-02-15 14:19:27 -05:00
63877ab314
chore: Move target level choice into round info
2023-02-15 14:00:35 -05:00
063ac9f2cb
test: Add another layout test to compactor ( #7005 )
2023-02-15 18:57:36 +00:00
c8d4471be7
Merge pull request #7003 from influxdata/alamb/round-info
...
feat(compactor): Add RoundInfo structure
2023-02-15 13:18:54 -05:00
999aba9d56
feat(compactor): Add RoundInfo structure
2023-02-15 13:00:58 -05:00
4adbc9f764
Merge pull request #6990 from influxdata/jrb_5_add_shutdown_log
...
chore: add shutdown logging
2023-02-15 08:26:47 -06:00
391e64772c
Merge branch 'main' into jrb_5_add_shutdown_log
2023-02-15 08:17:59 -06:00
ff6499cf9f
Merge pull request #6992 from influxdata/dom/rename
...
refactor: SequenceNumberSet::as_bytes -> to_bytes
2023-02-15 11:15:16 +00:00
1805f7fbe3
refactor: SequenceNumberSet::as_bytes -> to_bytes
...
The conversion is not (always) a cheap cast/conversion, and therefore
should be prefixed with "to_" and not "as_".
2023-02-15 11:03:15 +01:00
ac1502aeba
Merge pull request #6984 from influxdata/dom/soft-deleted-namespaces
...
feat(router): soft-delete namespace RPC handler
2023-02-15 09:57:41 +00:00
5ca165b76e
docs: fix two typos
...
Comments in test code.
2023-02-15 10:49:39 +01:00
dbe9219a8b
test: assert correct retention period
...
Use the correct constant in the retention period assert.
2023-02-15 10:49:38 +01:00
61fb92b85c
feat(router): soft-delete RPC handler
...
This implements the RPC "delete_namespace" handler, allowing a namespace
to be soft-deleted.
Adds unit coverage for all handlers & e2e test coverage for the new
handler (the rest were already covered).
The tests also highlight the caching issue documented here:
https://github.com/influxdata/influxdb_iox/issues/6175
2023-02-15 10:49:38 +01:00
4a30df0e07
chore(deps): Bump once_cell from 1.17.0 to 1.17.1 ( #6991 )
...
Bumps [once_cell](https://github.com/matklad/once_cell ) from 1.17.0 to 1.17.1.
- [Release notes](https://github.com/matklad/once_cell/releases )
- [Changelog](https://github.com/matklad/once_cell/blob/master/CHANGELOG.md )
- [Commits](https://github.com/matklad/once_cell/compare/v1.17.0...v1.17.1 )
---
updated-dependencies:
- dependency-name: once_cell
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-02-15 09:37:50 +00:00
f499022511
feat: add compaction level to commit metrics ( #6985 )
...
* feat: add compaction level to commit metrics
* test: more realism
2023-02-15 09:28:19 +00:00
e9f4b8f769
chore: add shutdown logging
2023-02-14 17:37:33 -06:00
7fb052208f
feat: allow gap filling to produce multiple batches ( #6986 )
...
* feat: allow gap filling to produce multiple batches
* chore: code review feedback
---------
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-02-14 22:48:29 +00:00
4db9df7197
feat: Add logging of commands sent via the `influxdb storage` command ( #6987 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-02-14 22:41:54 +00:00
0ffb211c54
test: more compactor layout tests ( #6988 )
...
* test: more compactor layout tests
* chore: address review comments
2023-02-14 22:14:06 +00:00
04bd47e64a
feat(compactor): Add more tests, improve sizes to simulator run display more ( #6981 )
...
* refactor: Split layout tests into their own module
* feat: Add more tests, improve sizes to simulator run display more
* fix: Apply suggestions from code review
Co-authored-by: Nga Tran <nga-tran@live.com>
* fix: fix comment wording
* fix: reporting order of skipped compactions
* chore: Run cargo hakari tasks
* fix: revert changes to Cargo.lock
* fix: revert workspace hack change
---------
Co-authored-by: Nga Tran <nga-tran@live.com>
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
2023-02-14 19:34:19 +00:00
ed007cb71f
refactor: replace IF-statement w/ optimizer rule ( #6982 )
...
* refactor: replace IF-statement w/ optimizer rule
This replaces a single IF-statement within the physical plan
construction with a physical optimizer rule. While on its own this seems
kinda pointless, it sets the foundation for #6098 . W/o the optimizer
some EXPLAIN query tests would fail.
* test: use insta snapshots
* fix: update test snapshots
---------
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-02-14 17:10:41 +00:00
5c506058da
feat: skip partitions of wide tables ( #6978 )
...
* feat: skip partitions of wide tables
* test: one more test
* refactor: address review comments
2023-02-14 16:42:13 +00:00
3a1b16e890
chore: ignore `RUSTSEC-2022-0090` ( #6983 )
2023-02-14 15:12:11 +00:00
363afd7271
Merge pull request #6979 from influxdata/cn/ie2e
...
test: Add an e2e test of just the ingester's API
2023-02-14 11:40:58 +00:00
2934e15f2f
Merge branch 'main' into cn/ie2e
2023-02-14 11:34:32 +00:00
a3b662f30b
docs: explain new physical plan construction ( #6970 )
...
* docs: explain new physical plan construction
There is no code yet, but this is the rough plan.
Ref #6098 .
* docs: clarify wording
Co-authored-by: Andrew Lamb <alamb@influxdata.com>
* docs: typos
Co-authored-by: Andrew Lamb <alamb@influxdata.com>
* docs: add links
* docs: extend reasoning
---------
Co-authored-by: Andrew Lamb <alamb@influxdata.com>
2023-02-14 11:13:22 +00:00
123c36e5b4
Merge branch 'main' into cn/ie2e
2023-02-14 11:12:38 +00:00
ce31ad7607
chore: `cargo update` ( #6980 )
...
dependabot doesn't seem to keep up at the moment, so let's do a global
manual update:
```console
❯ cargo update
Updating crates.io index
Updating git repository `https://github.com/apache/arrow-datafusion.git `
Updating git repository `https://github.com/influxdata/rskafka.git `
Updating git repository `https://github.com/mkmik/heappy `
Updating anyhow v1.0.68 -> v1.0.69
Updating axum v0.6.2 -> v0.6.6
Updating axum-core v0.3.1 -> v0.3.2
Removing bstr v0.2.17
Removing bstr v1.1.0
Adding bstr v1.2.0
Updating bytemuck v1.12.3 -> v1.13.0
Updating crc v3.0.0 -> v3.0.1
Updating csv v1.1.6 -> v1.2.0
Updating cxx v1.0.86 -> v1.0.90
Updating cxx-build v1.0.86 -> v1.0.90
Updating cxxbridge-flags v1.0.86 -> v1.0.90
Updating cxxbridge-macro v1.0.86 -> v1.0.90
Updating encoding_rs v0.8.31 -> v0.8.32
Updating fastrand v1.8.0 -> v1.9.0
Updating fd-lock v3.0.8 -> v3.0.10
Updating gimli v0.27.0 -> v0.27.1
Updating hermit-abi v0.3.0 -> v0.3.1
Updating inferno v0.11.13 -> v0.11.15
Removing itoa v0.4.8
Updating js-sys v0.3.60 -> v0.3.61
Updating num-complex v0.4.2 -> v0.4.3
Updating object v0.30.2 -> v0.30.3
Updating parking_lot_core v0.9.6 -> v0.9.7
Updating pest v2.5.3 -> v2.5.5
Updating pest_derive v2.5.3 -> v2.5.5
Updating pest_generator v2.5.3 -> v2.5.5
Updating pest_meta v2.5.3 -> v2.5.5
Updating petgraph v0.6.2 -> v0.6.3
Updating rayon-core v1.10.1 -> v1.10.2
Updating rgb v0.8.34 -> v0.8.35
Updating signal-hook-registry v1.4.0 -> v1.4.1
Updating spin v0.9.4 -> v0.9.5
Updating sync_wrapper v0.1.1 -> v0.1.2
Updating thread_local v1.1.4 -> v1.1.7
Updating tinyvec_macros v0.1.0 -> v0.1.1
Updating unicode-bidi v0.3.8 -> v0.3.10
Updating unicode-segmentation v1.10.0 -> v1.10.1
Updating wasm-bindgen v0.2.83 -> v0.2.84
Updating wasm-bindgen-backend v0.2.83 -> v0.2.84
Updating wasm-bindgen-futures v0.4.33 -> v0.4.34
Updating wasm-bindgen-macro v0.2.83 -> v0.2.84
Updating wasm-bindgen-macro-support v0.2.83 -> v0.2.84
Updating wasm-bindgen-shared v0.2.83 -> v0.2.84
Updating web-sys v0.3.60 -> v0.3.61
Updating which v4.3.0 -> v4.4.0
❯ cargo hakari generate
info: contents updated
```
2023-02-14 10:34:48 +00:00
969319dfd3
fix: Allow all valid characters following a keyword ( #6959 )
...
* fix: Allow all valid characters following a keyword
Closes #6382
* chore: Identified additional test cases
2023-02-13 22:21:11 +00:00
263d8fe21f
chore: Layout tests with `TargetLevel` algorithm + update display ( #6977 )
...
* refactor: move ParquetFileSimulator to compactor2_test_utils
* chore: Test with new algorithm + update display
* chore: Updates
* chore: Update setting to match prod
2023-02-13 22:12:55 +00:00
9e27709c84
test: Add an e2e test of just the ingester's API
2023-02-13 16:36:59 -05:00
adddd71c4d
chore: Add some comments about fatal panic rationale ( #6955 )
...
* chore: Add some comments and log messages about fatal panics
* fix: Update panic_logging/src/lib.rs
---------
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-02-13 18:54:57 +00:00
1424d4a0b7
chore: remove dead code ( #6969 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-02-13 17:14:02 +00:00
13ce6da3df
refactor: extract `FileClassifer` component ( #6946 )
...
* refactor: extract `FileClassifer` component
Make the driver slightly smaller. Also makes the "all-in-one" mode
easier to understand.
* docs: add some
---------
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-02-13 16:25:05 +00:00
9c18a3e3ef
Merge pull request #6971 from influxdata/dom/router-e2e-restart
...
test(router): router restarts + caching effects
2023-02-13 16:12:33 +00:00
4a0149a418
test(router): retention change after cache reset
...
Assert that the new retention period value is used once a router is
restarted and the cache converged.
2023-02-13 14:40:14 +01:00
ce12daa96e
refactor(e2e): simulate router restart
...
Allow a router to be "restarted" within an e2e test.
2023-02-13 14:40:11 +01:00
0e10561ce0
fix: Port router unit tests to the RPC write path ( #6956 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-02-13 12:28:06 +00:00
2277cd80f5
Merge pull request #6952 from influxdata/dom/namespace-soft-delete-catalog
...
feat: namespace soft-delete support
2023-02-13 12:21:24 +00:00
4f330b1935
Merge branch 'main' into dom/namespace-soft-delete-catalog
2023-02-13 12:14:55 +00:00
2751c77de6
chore: add extra debugging to help track down jdbc_client failures ( #6968 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-02-13 12:14:39 +00:00
9d20f26006
Merge branch 'main' into dom/namespace-soft-delete-catalog
2023-02-13 12:07:03 +00:00
Carol (Nichols || Goulding)
Stuart Carnie
Carol (Nichols || Goulding)
dependabot[bot]
Marco Neumann
Nga Tran
Andrew Lamb
Andrew Lamb
Joe-Blount
Dom
Joe-Blount
Christopher M. Wolff
kodiakhq[bot]