22778a3a80
chore: upgrade rskafka and parking_lot ( #3592 )
2022-02-01 11:50:42 +00:00
4101d16f71
chore: feature flag consistency ( #3574 )
...
* chore: feature flag consistency
* chore: add aarch64-apple-darwin to hakari
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-01-28 16:38:59 +00:00
2062267d0f
chore: Update hashbrown ( #3551 )
...
* chore: Update hashbrown
* fix: hakari
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-01-27 15:34:10 +00:00
21c1824a7a
refactor: remove table_names from Predicate ( #3545 )
...
* refactor: remove table_names from Predicate
* chore: fix benchmarks
* chore: review feedback
Co-authored-by: Edd Robinson <me@edd.io>
* chore: review feedback
* chore: replace Default::default with InfluxRpcPredicate::default()
Co-authored-by: Edd Robinson <me@edd.io>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-01-27 14:44:49 +00:00
16d584b2ff
feat: Add db_name/namespace to DmlWrite and DmlDelete ( #3531 )
...
* feat: Add db_name/namespace to DmlWrite and DmlDelete
This is required for the new ingester to be able to work with the write buffer. The protobuf that gets serialized over Kafka already includes the database name, it just wasn't getting carried through to the marshaled Dml operation.
* fix: database != namespace, propagation through write buffer
Co-authored-by: Marco Neumann <marco@crepererum.net>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-01-27 14:12:20 +00:00
9c19cd6cc4
fix: clamp start/end of TimestampRange to min/max valid timestamp values ( #3487 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-01-20 16:08:00 +00:00
f3f6f335a9
chore: upgrade to snafu 0.7 ( #3440 )
2022-01-11 19:22:36 +00:00
6d20ce1f9e
feat: Allow wipe catalog in `ReplayError` and `WriteBufferCreationError` states ( #3425 )
...
* feat: feat: Allow wipe catalog in ReplayError
* fix: comments
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-01-07 17:07:44 +00:00
336ffd1966
refactor: Remove `Result` in QueryDatabase trait (none of the functions can fail) ( #3422 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-01-06 22:03:08 +00:00
8d732ad78c
refactor: Split database.rs into smaller modules ( #3424 )
...
* refactor: Split database.rs into smaller modules
* fix: clippy
2022-01-06 21:42:36 +00:00
f9174c483b
refactor: Extract server::db into its own crate ( #3417 )
...
* refactor: Extract JobRegistry from the server crate
Both the server crate and a db crate that I'm about to extract depend on
JobRegistry, so to avoid making circular dependencies, extract the
JobRegistry to its own crate.
* refactor: Move db out of server into its own crate
Fixes #2821 .
2021-12-23 22:01:17 +00:00
ccba68fe3e
chore: Update datafusion and sqlparser, and arrow ( #3385 )
...
* chore: Update datafusion and sqlparser, and arrow
* fix: cargo hakari
* test: fix metadata size
* fix: update some metadata sizes
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-12-16 12:13:40 +00:00
499265b01e
chore: add db_name to background worker log messages ( #3386 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-12-16 11:13:13 +00:00
290f8c7630
Merge branch 'main' into crepererum/issue3335
2021-12-15 16:50:57 +00:00
758b65dd29
feat: Add database initialization state and errors to CLI and remove list_databases_detailed gRPC ( #3377 )
...
* feat: Add database initialization state and errors to CLI:
* fix: do not use optional in protobuf
* fix: clippy
* fix: correct check I broke appeasing clippy
2021-12-15 12:18:41 +00:00
e9041e6da0
refactor: improve database init routine
...
1. Simpler locking/freezing
2. Don't commit the rollback from `ReplayError` to `RulesLoaded` because
it will confuse the heck out of our users/admins.
2021-12-15 12:43:33 +01:00
3841598296
fix: typo
...
Co-authored-by: Raphael Taylor-Davies <1781103+tustvold@users.noreply.github.com>
2021-12-15 12:41:30 +01:00
8520b9b9db
fix: improve docs and naming
2021-12-15 12:41:30 +01:00
0682fc208f
feat: also recover from `NoActiveDatabase`
2021-12-15 12:41:30 +01:00
0f3cfb024b
test: add database startup recovery test
2021-12-15 12:41:30 +01:00
e80ac94af5
fix: database state diagram
2021-12-15 12:41:30 +01:00
7a1f70f970
fix: move warning to better place
2021-12-15 12:41:30 +01:00
11df3091a1
fix: misleading comment
2021-12-15 12:41:30 +01:00
496043a767
fix: PR feedback
...
Co-authored-by: Andrew Lamb <alamb@influxdata.com>
2021-12-15 12:41:30 +01:00
10a51bac61
refactor: allow database init to recover from errors
...
Closes #3335 .
2021-12-15 12:41:30 +01:00
7783149af1
fix: typos
2021-12-13 10:06:17 -05:00
e76157f0e4
chore: cleanup
2021-12-10 12:08:04 -05:00
5253c71e03
test: replay test for compact OS chunks
2021-12-10 12:04:00 -05:00
2e3aa2e1ab
Merge branch 'main' into ntran/test_compact_os_delete_propagation
2021-12-10 15:11:11 +00:00
dc152abb55
Merge branch 'main' into alamb/update_state_machine
2021-12-10 09:51:40 +00:00
7d53fad427
Merge branch 'main' into ntran/test_compact_os_delete_propagation
2021-12-09 17:27:22 -05:00
0d6fefdd09
test: test concurrent deletes and OS compaction
2021-12-09 17:13:18 -05:00
c0ba69f09e
chore: marge main to branch and resolve conflict
2021-12-09 15:40:33 -05:00
191e743ce0
fix: Update server/src/database.rs
...
Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
2021-12-09 15:04:39 -05:00
b2f7306d5a
docs: Update database startup machine diagram
2021-12-09 14:22:29 -05:00
471c3181bb
feat: Make a top-level influxdb_iox feature for kafka functionality
...
Thread the feature through router and server to the write buffer crate.
Move an end-to-end test that uses Kafka behind the feature flag.
2021-12-09 08:49:34 -05:00
4e86d0ef30
test: propogate delete tests for compact OS chunks
2021-12-08 17:50:51 -05:00
efbfbb1a0b
feat: compact all object store chunks of a given partition
2021-12-08 16:06:03 -05:00
a8bc5b3b6d
fix: allow setting the same server ID twice
...
This is important for idempotence and simplifies clients and helper
scripts a lot.
2021-12-08 19:03:30 +01:00
43e3e76de2
chore: Merge branch 'main' into ntran/compact_os_sql_tests
2021-12-08 12:07:02 -05:00
428851b467
chore: Apply suggestions from code review
...
Co-authored-by: Andrew Lamb <alamb@influxdata.com>
2021-12-08 11:42:25 -05:00
218042784f
feat: add `system.queries `system table ( #3328 )
...
* feat: Add query log and system table
* docs: Add docstrings for test normalization
2021-12-08 16:26:24 +00:00
c6a3765d76
feat: Add force flag to RebuildCatalog ( #3292 )
...
* feat: Add force flag to RebuildCatalog
* fix: small cleanups
* docs: Update comments and add WARNING
2021-12-08 15:36:07 +00:00
2a2c5889c3
perf: loading chunks doesn't require sort
2021-12-08 12:03:56 +00:00
a13c3120a6
test: sql tests and checkpoint test for compact OS chunks
2021-12-07 17:32:48 -05:00
c992c82582
chore: Merge branch 'main' into ntran/compact_os_tests
2021-12-07 11:08:12 -05:00
d9e8bafeb3
fix: improve state machine for write buffer creation retries
2021-12-07 10:29:08 +01:00
e558f4c708
fix: address review comments
2021-12-07 09:43:47 +01:00
ef9c49087e
fix: retry write buffer creation during database init
2021-12-07 09:23:03 +01:00
75cd1d24f4
Merge branch 'main' into ntran/grpc_compact_os_chunks
2021-12-07 03:26:06 +00:00
Marco Neumann
Raphael Taylor-Davies
Andrew Lamb
Paul Dix
Carol (Nichols || Goulding)
kodiakhq[bot]
Nga Tran
Andrew Lamb
Carol (Nichols || Goulding)
Edd Robinson