de732b4273
Merge branch 'main' into crepererum/parquet_file_wo_query
2021-09-15 07:15:19 +00:00
63cc7b3fb0
test: more tests to discover what still need to be done
2021-09-14 17:57:30 -04:00
f4f140d3b7
chore: merge main to branch
2021-09-14 13:25:32 -04:00
939c9ca038
fix: reset Database background worker on deletion ( #2530 )
...
* fix: reset Database background worker on deletion
* chore: update server/src/database.rs
Co-authored-by: Andrew Lamb <alamb@influxdata.com>
* chore: add database delete restore test
* chore: fix logical conflicts
Co-authored-by: Andrew Lamb <alamb@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-09-14 16:57:33 +00:00
aad3c04bc4
Merge pull request #2531 from influxdata/crepererum/chunk_order_type
...
refactor: introduce `ChunkOrder` type
2021-09-14 16:36:34 +00:00
509c07330d
refactor: decouple `parquet_file` from `query`
2021-09-14 18:26:16 +02:00
d60aa5940b
Merge branch 'main' into crepererum/chunk_order_type
2021-09-14 16:25:17 +00:00
a372973b67
fix: typos and wording
...
Co-authored-by: Andrew Lamb <alamb@influxdata.com>
2021-09-14 16:24:45 +00:00
c33e5c22e6
feat: pull WriteBuffer consumer out of Db and onto Database ( #2243 ) ( #2525 )
...
* feat: pull WriteBuffer consumer out of Db and onto Database (#2243 )
* chore: restore WritingOnlyAllowedThroughWriteBuffer error
* refactor: remove WriteBufferConfig
* chore: fix docs
* chore: move WriteBufferConsumer tests out of db.rs
* chore: document WriteBufferFactory member functions
* chore: fmt
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-09-14 16:04:58 +00:00
9514a566f7
Merge pull request #2527 from influxdata/crepererum/issue2493b
...
refactor: move `predicate` into its own crate
2021-09-14 15:23:27 +00:00
bfaba78dc3
refactor: move `predicate` into its own crate
...
Two reasons:
1. I wanna decouple `parquet_file` from `query` (nearly done, needs a
small follow-up PR).
2. `predicate` will have more and more features (like serialization)
which justifies a new home
2021-09-14 17:13:02 +02:00
becef1c75f
refactor: introduce `ChunkOrder` type
2021-09-14 17:10:23 +02:00
a499f3ba4b
Merge pull request #2523 from influxdata/jgm-perf-tracing
...
fix: emit traces to Jaeger in perf.py
2021-09-14 14:23:18 +00:00
03d9a28933
Merge branch 'main' into jgm-perf-tracing
2021-09-14 14:13:38 +00:00
a98ea6fe5e
Merge pull request #2480 from influxdata/crepererum/issue1963
...
fix: order chunks correctly during query processing
2021-09-14 11:47:57 +00:00
9ea61cd434
Merge branch 'main' into crepererum/issue1963
2021-09-14 11:38:59 +00:00
a109c3be71
Merge pull request #2526 from influxdata/crepererum/issue2493a
...
refactor: stricter delete predicate TS parsing
2021-09-14 11:37:47 +00:00
4795bd5c9c
refactor: stricter delete predicate TS parsing
...
As a a nice side effect, the parser no longer depends on the line
protocol parser.
2021-09-14 13:14:19 +02:00
1d8edd4683
fix: metadata size increased
2021-09-14 13:03:26 +02:00
0dc5bd0921
fix: remove overstrict check for chunk orders in lifecycle policy
2021-09-14 13:00:55 +02:00
2591bcac13
test: ensure chunk IDs are as documented
2021-09-14 13:00:55 +02:00
8e360f0c5a
refactor: inline `sort_chunks` since it is only used once
2021-09-14 13:00:55 +02:00
aaeb67ae5d
refactor: make chunk iterations sorted by `order, ID`
2021-09-14 13:00:55 +02:00
804790711b
refactor: isolate `sort_chunks`
2021-09-14 13:00:55 +02:00
9a60af7fa3
docs: explain `ChunkOrder` query test scenario
2021-09-14 13:00:55 +02:00
c28f38309a
docs: improve chunk ordering docs
2021-09-14 13:00:55 +02:00
96618af6a2
fix: respect chunk order when invoking lifecycle actions
2021-09-14 13:00:55 +02:00
1b788732da
fix: order chunks correctly during query processing
...
The query processing was implicitly relying on the order provided by the
catalog. This had two issues:
- this ordering was not defined in the API contract (neither via docs
nor via typing)
- the order was based on chunk IDs which is not adequate in some cases
(e.g. when chunks are created while a persistence operations is in
progress)
Now we explicitly sort chunks by `(order, ID)`.
Fixes #1963 .
2021-09-14 13:00:55 +02:00
8a531be05b
feat: expose chunk order via API and in system table
2021-09-14 13:00:55 +02:00
45cb00d8c0
refactor: track chunk order in chunks
2021-09-14 13:00:55 +02:00
d9da764153
Merge pull request #2524 from influxdata/crepererum/prune_transactions
...
feat: prune old transactions from preserved catalog
2021-09-14 10:57:04 +00:00
3f2e46c397
feat: prune old transactions from preserved catalog
2021-09-14 12:08:17 +02:00
4769b67d14
feat: API-level code to prune old transaction from catalog
2021-09-14 10:26:38 +02:00
59d7996a3e
Merge pull request #2517 from influxdata/crepererum/show_metadata_bytes_in_catalog_dump
...
feat: show number of stripped bytes in catalog dump
2021-09-14 07:52:14 +00:00
f93984cd94
refactor: clarify wording
...
Co-authored-by: Andrew Lamb <alamb@influxdata.com>
2021-09-14 09:43:55 +02:00
e7edb65b1d
feat: show number of stripped bytes in catalog dump
2021-09-14 09:43:55 +02:00
6d762d2ba0
fix: emit traces to Jaeger in perf.py
2021-09-13 16:11:59 -07:00
042a78e5a7
feat: apply delete predicate during query to emilimate deleted data
2021-09-13 18:02:55 -04:00
21d68ac2bf
Merge pull request #2499 from influxdata/pd/data-generator-values
...
feat: Add pre-generated values and tag sets to data generator
2021-09-13 21:57:34 +00:00
6d3ac4db46
chore: pr cleanup on data generator
2021-09-13 17:45:53 -04:00
32f2410597
feat: Add print to stdout to data generator ( #2512 )
...
This adds a flag to the data generator to print samples to standard out. It disables logging output so that only the line protocol is output.
2021-09-13 17:45:53 -04:00
914c6e712b
chore: remove rogue println in data generator
2021-09-13 17:45:53 -04:00
a94854d4d0
fix: build error in data generator bench
2021-09-13 17:45:53 -04:00
5f0b3b336e
refactor: optimizations to make tag set generation significantly faster
2021-09-13 17:45:53 -04:00
7f915ba9d4
feat: Add pre-generated values and tag sets to data generator
...
This adds the ability to pre-generate values and tag sets in the data generator. This makes it easy to have tags that depend on other tag values (like buckets in an org) and have tag values that have one associated tag (like if something is in production or staging environment). Follow on work will add the actual generation to the agent spec. An added bonus of these pre-generated values is that generating samples won't require any sort of template evaluation for all of the tags in the tag sets. Only unique values (like trace_id or span_id) would need to be generated during sampling generation.
2021-09-13 17:45:53 -04:00
5eef76c868
chore: Update dependencies (including datafusion) ( #2521 )
...
* chore: Update datafusion deps to pre-release
* refactor: Update IOx to use new datafusion Statistics
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-09-13 21:30:44 +00:00
7e434d16d2
fix: MockBufferForReading waker registration ( #2520 )
...
* fix: MockBufferForReading waker registration
* chore: remove unnecessary sleep
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-09-13 21:21:21 +00:00
f3bcafcfea
feat: migrate http metrics to metric crate ( #2508 )
...
* feat: migrate http metrics
* chore: review feedback
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-09-13 18:56:20 +00:00
525e99610e
Merge pull request #2465 from influxdata/jpg/delete-e2e-test
...
test: Ensure that a deleted and recreated database returns new data
2021-09-13 18:26:42 +00:00
c06943e355
Merge branch 'main' into jpg/delete-e2e-test
2021-09-13 18:17:52 +00:00
kodiakhq[bot]
Nga Tran
Raphael Taylor-Davies
Marco Neumann
Jacob Marble
Paul Dix
Paul Dix
Andrew Lamb