Commit Graph

1383 Commits (5ad6ca1fdac7b25acdef6772fdca903d8a1d1e78)

Author SHA1 Message Date
Andrew Lamb 9c19cd6cc4
fix: clamp start/end of TimestampRange to min/max valid timestamp values (#3487)
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-01-20 16:08:00 +00:00
Marco Neumann f3f6f335a9
chore: upgrade to snafu 0.7 (#3440) 2022-01-11 19:22:36 +00:00
Andrew Lamb 6d20ce1f9e
feat: Allow wipe catalog in `ReplayError` and `WriteBufferCreationError` states (#3425)
* feat: feat: Allow wipe catalog in ReplayError

* fix: comments

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-01-07 17:07:44 +00:00
Andrew Lamb 336ffd1966
refactor: Remove `Result` in QueryDatabase trait (none of the functions can fail) (#3422)
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-01-06 22:03:08 +00:00
Andrew Lamb 8d732ad78c
refactor: Split database.rs into smaller modules (#3424)
* refactor: Split database.rs into smaller modules

* fix: clippy
2022-01-06 21:42:36 +00:00
Carol (Nichols || Goulding) f9174c483b
refactor: Extract server::db into its own crate (#3417)
* refactor: Extract JobRegistry from the server crate

Both the server crate and a db crate that I'm about to extract depend on
JobRegistry, so to avoid making circular dependencies, extract the
JobRegistry to its own crate.

* refactor: Move db out of server into its own crate

Fixes #2821.
2021-12-23 22:01:17 +00:00
Andrew Lamb ccba68fe3e
chore: Update datafusion and sqlparser, and arrow (#3385)
* chore: Update datafusion and sqlparser, and arrow

* fix: cargo hakari

* test: fix metadata size

* fix: update some metadata sizes

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-12-16 12:13:40 +00:00
Andrew Lamb 499265b01e
chore: add db_name to background worker log messages (#3386)
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-12-16 11:13:13 +00:00
kodiakhq[bot] 290f8c7630
Merge branch 'main' into crepererum/issue3335 2021-12-15 16:50:57 +00:00
Andrew Lamb 758b65dd29
feat: Add database initialization state and errors to CLI and remove list_databases_detailed gRPC (#3377)
* feat: Add database initialization state and errors to CLI:

* fix: do not use optional in protobuf

* fix: clippy

* fix: correct check I broke appeasing clippy
2021-12-15 12:18:41 +00:00
Marco Neumann e9041e6da0 refactor: improve database init routine
1. Simpler locking/freezing
2. Don't commit the rollback from `ReplayError` to `RulesLoaded` because
   it will confuse the heck out of our users/admins.
2021-12-15 12:43:33 +01:00
Marco Neumann 3841598296 fix: typo
Co-authored-by: Raphael Taylor-Davies <1781103+tustvold@users.noreply.github.com>
2021-12-15 12:41:30 +01:00
Marco Neumann 8520b9b9db fix: improve docs and naming 2021-12-15 12:41:30 +01:00
Marco Neumann 0682fc208f feat: also recover from `NoActiveDatabase` 2021-12-15 12:41:30 +01:00
Marco Neumann 0f3cfb024b test: add database startup recovery test 2021-12-15 12:41:30 +01:00
Marco Neumann e80ac94af5 fix: database state diagram 2021-12-15 12:41:30 +01:00
Marco Neumann 7a1f70f970 fix: move warning to better place 2021-12-15 12:41:30 +01:00
Marco Neumann 11df3091a1 fix: misleading comment 2021-12-15 12:41:30 +01:00
Marco Neumann 496043a767 fix: PR feedback
Co-authored-by: Andrew Lamb <alamb@influxdata.com>
2021-12-15 12:41:30 +01:00
Marco Neumann 10a51bac61 refactor: allow database init to recover from errors
Closes #3335.
2021-12-15 12:41:30 +01:00
Nga Tran 7783149af1 fix: typos 2021-12-13 10:06:17 -05:00
Nga Tran e76157f0e4 chore: cleanup 2021-12-10 12:08:04 -05:00
Nga Tran 5253c71e03 test: replay test for compact OS chunks 2021-12-10 12:04:00 -05:00
kodiakhq[bot] 2e3aa2e1ab
Merge branch 'main' into ntran/test_compact_os_delete_propagation 2021-12-10 15:11:11 +00:00
kodiakhq[bot] dc152abb55
Merge branch 'main' into alamb/update_state_machine 2021-12-10 09:51:40 +00:00
Nga Tran 7d53fad427
Merge branch 'main' into ntran/test_compact_os_delete_propagation 2021-12-09 17:27:22 -05:00
Nga Tran 0d6fefdd09 test: test concurrent deletes and OS compaction 2021-12-09 17:13:18 -05:00
Nga Tran c0ba69f09e chore: marge main to branch and resolve conflict 2021-12-09 15:40:33 -05:00
Andrew Lamb 191e743ce0
fix: Update server/src/database.rs
Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
2021-12-09 15:04:39 -05:00
Andrew Lamb b2f7306d5a docs: Update database startup machine diagram 2021-12-09 14:22:29 -05:00
Carol (Nichols || Goulding) 471c3181bb
feat: Make a top-level influxdb_iox feature for kafka functionality
Thread the feature through router and server to the write buffer crate.
Move an end-to-end test that uses Kafka behind the feature flag.
2021-12-09 08:49:34 -05:00
Nga Tran 4e86d0ef30 test: propogate delete tests for compact OS chunks 2021-12-08 17:50:51 -05:00
Nga Tran efbfbb1a0b feat: compact all object store chunks of a given partition 2021-12-08 16:06:03 -05:00
Marco Neumann a8bc5b3b6d fix: allow setting the same server ID twice
This is important for idempotence and simplifies clients and helper
scripts a lot.
2021-12-08 19:03:30 +01:00
Nga Tran 43e3e76de2 chore: Merge branch 'main' into ntran/compact_os_sql_tests 2021-12-08 12:07:02 -05:00
Nga Tran 428851b467
chore: Apply suggestions from code review
Co-authored-by: Andrew Lamb <alamb@influxdata.com>
2021-12-08 11:42:25 -05:00
Andrew Lamb 218042784f
feat: add `system.queries `system table (#3328)
* feat: Add query log and system table

* docs: Add docstrings for test normalization
2021-12-08 16:26:24 +00:00
Andrew Lamb c6a3765d76
feat: Add force flag to RebuildCatalog (#3292)
* feat: Add force flag to RebuildCatalog

* fix: small cleanups

* docs: Update comments and add WARNING
2021-12-08 15:36:07 +00:00
Edd Robinson 2a2c5889c3 perf: loading chunks doesn't require sort 2021-12-08 12:03:56 +00:00
Nga Tran a13c3120a6 test: sql tests and checkpoint test for compact OS chunks 2021-12-07 17:32:48 -05:00
Nga Tran c992c82582 chore: Merge branch 'main' into ntran/compact_os_tests 2021-12-07 11:08:12 -05:00
Marco Neumann d9e8bafeb3 fix: improve state machine for write buffer creation retries 2021-12-07 10:29:08 +01:00
Marco Neumann e558f4c708 fix: address review comments 2021-12-07 09:43:47 +01:00
Marco Neumann ef9c49087e fix: retry write buffer creation during database init 2021-12-07 09:23:03 +01:00
kodiakhq[bot] 75cd1d24f4
Merge branch 'main' into ntran/grpc_compact_os_chunks 2021-12-07 03:26:06 +00:00
Nga Tran 3834435620 chore: Merge branch 'ntran/grpc_compact_os_chunks' of https://github.com/influxdata/influxdb_iox into ntran/grpc_compact_os_chunks 2021-12-06 12:06:22 -05:00
Nga Tran d8ed8a3bf7 refactor: address review comments 2021-12-06 12:06:07 -05:00
Carol (Nichols || Goulding) 7499eac067
fix: Disable uuid serde feature; we're not actually serializing any UUIDs
Connects to #3117.
2021-12-06 09:37:31 -05:00
Carol (Nichols || Goulding) 16d8ae5e04
fix: Match tokio features to what's actually in use in each crate
Some crates listed features they don't use; other crates ware relying on
feature flags enabled by something else. I tested these changes by
disabling the workspace hack crate and testing each crate.
2021-12-06 09:37:16 -05:00
Carol (Nichols || Goulding) 02c297e850
fix: Always specify the parking_lot feature of tokio to get potential perf boost 2021-12-06 09:37:15 -05:00