f7228ddd60
test: add test for byte trimmed floats
2021-09-16 10:01:44 +01:00
d387108dab
fix: float byte trimmer filter range
2021-09-16 10:01:44 +01:00
0250bd1337
fix: ensure range filter works with null
2021-09-16 10:01:44 +01:00
1a70865a03
fix: ensure float byte trimmed predicate pushdown works for unencodable values
2021-09-16 10:01:44 +01:00
483508e3c6
feat: add rle method for identifying all non-null row IDs
2021-09-16 10:01:44 +01:00
d04a0d1137
feat: add method for identifying all non-null row IDs
2021-09-16 10:01:44 +01:00
70b0ba44b3
test: failing filter test
2021-09-16 10:01:44 +01:00
ca1973fca7
Merge pull request #2556 from influxdata/crepererum/preserved_catalog_api_split
...
refactor: split preserved catalog `api` into `core` and `interface`
2021-09-16 08:38:37 +00:00
2820db5583
refactor: split preserved catalog `api` into `core` and `interface`
...
This makes it clearer which traits and functions users of the preserved
catalog must implement. This also splits the error types into smaller
enums that are easier to understand.
This change should make it easier to implement new functionality (like
capturing delete predicates).
2021-09-16 10:30:11 +02:00
1d55d9a1b5
feat: add pbjson support ( #2468 )
...
* feat: add pbjson support
* chore: fix test
2021-09-16 07:33:27 +00:00
91fd32d506
fix: Reset restored db's state instead of restarting background worker
2021-09-15 19:04:05 -04:00
61e1eac135
fix: fix the cases of multi[le expressions in delete predicate
2021-09-15 17:00:21 -04:00
d70d94100e
refactor: Extract a type alias
2021-09-15 17:00:09 -04:00
81feced9d6
fix: Restart database background worker when it's restored
2021-09-15 16:59:49 -04:00
7c81c280cf
fix: Shut down a database when it's deleted
2021-09-15 16:59:49 -04:00
7b6d8f9327
feat: Add an API for restoring a database that was marked deleted
2021-09-15 16:59:37 -04:00
185f45f56b
chore: Update datafusion + other deps ( #2545 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-09-15 20:56:44 +00:00
74f1e0377c
Merge pull request #2547 from influxdata/cn/more-database-states
...
test: Check for other error Database States
2021-09-15 19:52:51 +00:00
957df68da3
Merge branch 'main' into cn/more-database-states
2021-09-15 19:44:11 +00:00
c66095cad1
feat: remove metrics crate ( #2552 )
2021-09-15 19:43:33 +00:00
6e7fa3e574
feat: migrate http ingest metrics ( #2542 )
...
* feat: migrate http ingest metrics
* chore: review feedback
* refactor: RAII entry ingest recorder
2021-09-15 19:01:10 +00:00
7175488133
chore: add some comments
2021-09-15 14:45:04 -04:00
8f54dbcb58
test: Check for other error Database States
...
Connects to #2539 .
2021-09-15 14:44:14 -04:00
3486cc8b38
fix: should not send an empty delete predicate predicate which means delete everything (no time range)
2021-09-15 14:14:26 -04:00
74d3c2e6d2
feat: Translate DataFusion execution metrics to IOx Spans ( #2529 )
...
* feat: Translate DataFusion execution metrics to IOx Spans
* fix: add end to end test to ensure plumbing is hookedup
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-09-15 18:14:23 +00:00
1ea4335ff3
fix: report correct DatabaseStateCode ( #2543 )
...
* fix: report correct DatabaseStateCode
* chore: fix lint
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-09-15 18:02:54 +00:00
6f2301e16c
feat: migrate write buffer metrics ( #2536 )
...
* feat: migrate write buffer metrics
* feat: update server/src/write_buffer.rs
Co-authored-by: Andrew Lamb <alamb@influxdata.com>
Co-authored-by: Andrew Lamb <alamb@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-09-15 17:41:55 +00:00
c4d39f328f
Merge pull request #2537 from influxdata/crepererum/issue2493c
...
feat: `Predicate` serialization
2021-09-15 16:27:02 +00:00
72021e010f
test: test unsupported cases of predicate serialization
2021-09-15 18:18:05 +02:00
c5ebf4a2e6
refactor: remove unused predicate serialization variants
2021-09-15 18:05:11 +02:00
44eb3b994d
feat: `Predicate` serialization
...
Closes #2493 .
2021-09-15 16:37:25 +02:00
27b0c46f79
feat: shutdown WriteBufferConsumer on drop ( #2533 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-09-15 11:48:56 +00:00
c9321f4991
Merge pull request #2535 from influxdata/crepererum/time_of_write
...
refactor: make `time_of_write` a parameter
2021-09-15 10:05:52 +00:00
86031b03dc
refactor: make time_of_write a parameter
...
Instead of depending on the `chrono` clock implicitly via `Utc::now()`
we should make it an explicit parameter. This is essential for testing:
1. We have many tests guessing around this value by taking `Utc::now()`
directly before or after the write (this commit doesn't fix that, but
allows us to fix it in a follow-up).
2. We have some tests that ignore the `time_of_write` values in some
comparisons because they cannot control that value (fix not included
here but left as a follow-up).
3. The upcoming compression (#2528 ) needs to control timestamps within
the compressed payload (and `time_of_write` is embedded in the
parquet metadata) because the compressed size depends on it (even if
the uncompressed size is stable).
In general I argue that a "clock" is always data and should be passed
(either as a value or as a "now"-function) from the API layer. Hidden
clock checks just make mocking and tests a nightmare (we've seen this w/
replay tests as well).
2021-09-15 11:39:15 +02:00
764cf1fdf0
Merge pull request #2532 from influxdata/crepererum/parquet_file_wo_query
...
refactor: decouple `parquet_file` from `query`
2021-09-15 08:35:41 +00:00
de732b4273
Merge branch 'main' into crepererum/parquet_file_wo_query
2021-09-15 07:15:19 +00:00
63cc7b3fb0
test: more tests to discover what still need to be done
2021-09-14 17:57:30 -04:00
f4f140d3b7
chore: merge main to branch
2021-09-14 13:25:32 -04:00
939c9ca038
fix: reset Database background worker on deletion ( #2530 )
...
* fix: reset Database background worker on deletion
* chore: update server/src/database.rs
Co-authored-by: Andrew Lamb <alamb@influxdata.com>
* chore: add database delete restore test
* chore: fix logical conflicts
Co-authored-by: Andrew Lamb <alamb@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-09-14 16:57:33 +00:00
aad3c04bc4
Merge pull request #2531 from influxdata/crepererum/chunk_order_type
...
refactor: introduce `ChunkOrder` type
2021-09-14 16:36:34 +00:00
509c07330d
refactor: decouple `parquet_file` from `query`
2021-09-14 18:26:16 +02:00
d60aa5940b
Merge branch 'main' into crepererum/chunk_order_type
2021-09-14 16:25:17 +00:00
a372973b67
fix: typos and wording
...
Co-authored-by: Andrew Lamb <alamb@influxdata.com>
2021-09-14 16:24:45 +00:00
c33e5c22e6
feat: pull WriteBuffer consumer out of Db and onto Database ( #2243 ) ( #2525 )
...
* feat: pull WriteBuffer consumer out of Db and onto Database (#2243 )
* chore: restore WritingOnlyAllowedThroughWriteBuffer error
* refactor: remove WriteBufferConfig
* chore: fix docs
* chore: move WriteBufferConsumer tests out of db.rs
* chore: document WriteBufferFactory member functions
* chore: fmt
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-09-14 16:04:58 +00:00
9514a566f7
Merge pull request #2527 from influxdata/crepererum/issue2493b
...
refactor: move `predicate` into its own crate
2021-09-14 15:23:27 +00:00
bfaba78dc3
refactor: move `predicate` into its own crate
...
Two reasons:
1. I wanna decouple `parquet_file` from `query` (nearly done, needs a
small follow-up PR).
2. `predicate` will have more and more features (like serialization)
which justifies a new home
2021-09-14 17:13:02 +02:00
becef1c75f
refactor: introduce `ChunkOrder` type
2021-09-14 17:10:23 +02:00
a499f3ba4b
Merge pull request #2523 from influxdata/jgm-perf-tracing
...
fix: emit traces to Jaeger in perf.py
2021-09-14 14:23:18 +00:00
03d9a28933
Merge branch 'main' into jgm-perf-tracing
2021-09-14 14:13:38 +00:00
a98ea6fe5e
Merge pull request #2480 from influxdata/crepererum/issue1963
...
fix: order chunks correctly during query processing
2021-09-14 11:47:57 +00:00
Edd Robinson
kodiakhq[bot]
Marco Neumann
Raphael Taylor-Davies
Carol (Nichols || Goulding)
Nga Tran
Andrew Lamb