Commit Graph

260 Commits (788fda351f42fda815a583154bce941b00a978d1)

Author SHA1 Message Date
Marco Neumann 11413eae8d docs: mention that `Chunk{Id,Order}::next` might panic 2021-09-20 15:32:44 +02:00
Marco Neumann cef5aeee52 refactor: introduce `ChunkId` type 2021-09-20 13:10:41 +02:00
Marco Neumann acf698c366 fix: delete predicate sorting 2021-09-20 10:48:32 +02:00
Carol (Nichols || Goulding) 44a89cdf75 refactor: Change DeletedDatabase to DetailedDatabase
So this info can be reused for active databases in detailed database
lists.
2021-09-17 15:27:22 -04:00
Raphael Taylor-Davies 37b615f301
feat: migrate operations CLI to use pbjson (#2562)
* feat: migrate operations CLI to use pbjson

* fix: reserve removed field

* chore: review feedback

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-09-16 19:15:24 +00:00
Raphael Taylor-Davies 6e7fa3e574
feat: migrate http ingest metrics (#2542)
* feat: migrate http ingest metrics

* chore: review feedback

* refactor: RAII entry ingest recorder
2021-09-15 19:01:10 +00:00
Raphael Taylor-Davies 1ea4335ff3
fix: report correct DatabaseStateCode (#2543)
* fix: report correct DatabaseStateCode

* chore: fix lint

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-09-15 18:02:54 +00:00
Marco Neumann a372973b67
fix: typos and wording
Co-authored-by: Andrew Lamb <alamb@influxdata.com>
2021-09-14 16:24:45 +00:00
Marco Neumann becef1c75f refactor: introduce `ChunkOrder` type 2021-09-14 17:10:23 +02:00
Marco Neumann 8a531be05b feat: expose chunk order via API and in system table 2021-09-14 13:00:55 +02:00
Marco Neumann 3f2e46c397 feat: prune old transactions from preserved catalog 2021-09-14 12:08:17 +02:00
Raphael Taylor-Davies b8f7319704
feat: migrate read buffer metrics to metric crate (#2510)
* feat: migrate read buffer metrics to metric crate

* chore: review feedback

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-09-10 19:51:43 +00:00
Marco Neumann bbb8898d36 refactor: make writer buffer auto-creation types nicer to read 2021-09-08 11:13:48 +02:00
Marco Neumann b4fe829da0 fix: docs 2021-09-07 18:24:59 +02:00
Marco Neumann e7cf91bfc4 refactor: more structures write buffer auto-creation config 2021-09-07 18:18:20 +02:00
Marco Neumann d5662328b0 refactor: `n_sequencers` should be non-zero 2021-09-07 18:18:20 +02:00
Nga Tran dbe4bcff22 chore: merge main to branch 2021-09-07 10:54:59 -04:00
Marco Neumann 04a301cc64 refactor: make job descriptions a static string 2021-09-06 13:39:22 +02:00
Nga Tran de0bd80c3d refactor: cleanup 2021-09-06 01:07:07 -04:00
Carol (Nichols || Goulding) ce6030a3cb feat: Wire list deleted databases through gRPC and CLI APIs 2021-09-02 15:48:07 -04:00
Marco Neumann 81e672db14 feat: add `auto_create_sequencers` write buffer flag 2021-09-02 16:41:35 +02:00
Marco Neumann ecf1f99ddb refactor: more flexible writer buffer config
This allows:

- different types (instead of guessing through the connection URL)
- sequencer counts (not used yet but will be by #2455)
- extensible configs (e.g. to configure Kafka in a more granular way,
  not wired up yet)
- future extensions (since we use a message now instead of a single
  string)

**BREAKING: This requires changes for deployed systems / existing DBs!**
2021-09-02 16:41:35 +02:00
Andrew Lamb 779b027271
feat: Store only the database rules sent by the client (do not store default values) (#2430)
* feat: add omit_default to protobuf definition

* feat: Persist only the client provided rules

* fix: Remove race conditions

* fix: merge confit

* refactor: do not use macro

* refactor: restore use of glob import

* fix: review comments

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-08-28 10:26:32 +00:00
Nga Tran 499af57299 chore: merge mian to branch and resolve conflicts 2021-08-27 17:51:07 -04:00
Nga Tran b79eaa34d1 refactor: address review comments 2021-08-27 15:53:27 -04:00
Nga Tran bcd39e225c feat: Management API for delete 2021-08-26 17:31:21 -04:00
Carol (Nichols || Goulding) 7cf7fb02ed refactor: Rename database ObjectStore state types to DatabaseObjectStore 2021-08-26 09:14:23 -04:00
Carol (Nichols || Goulding) c7eceac8a3 refactor: Have server determine database generation from object store 2021-08-26 09:14:23 -04:00
Marco Neumann 60ea637c38 refactor: remove `drop_non_persisted`
This flag should always be `!persist`.

Closes #2212.
2021-08-17 16:53:45 +02:00
Marco Neumann 77892a0998 feat: add API to drop entire partitions 2021-08-17 09:44:35 +02:00
Nga Tran fd4ea89454 chore: revert panic commit 2021-08-13 12:51:37 -04:00
Andrew Lamb db5fd935dd
fix: revert "Convert panic --> `warn` on statistics mismatch (#2125)" (#2250)
This reverts commit 7415fa0d05.

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-08-13 11:39:12 +00:00
Edd Robinson f82646e7f8 test: ignore failing tests on aarch64 2021-08-12 11:46:23 +01:00
Dom 3de6b44e23
build: use new rustdoc lint name (#2261)
* fix: nocache feature code rot

The MBChunk::snapshot code when using the "nocache" option no longer
compiles - this commit updates it to match the not(nocache) code.

* build: use updated broken_intra_doc_links name

The broken_intra_doc_links lint was renamed
rustdoc::broken_intra_doc_links

https://doc.rust-lang.org/rustdoc/lints.html
2021-08-11 19:48:51 +00:00
Andrew Lamb 0c300a4abb
fix: Handle all null columns in ColumnSummary::update_from (#2251) 2021-08-10 20:37:00 +00:00
Raphael Taylor-Davies 2344c28f4e
feat: drain database jobs on shutdown (#2239)
* feat: drain database jobs on shutdown

* chore: fmt

* chore: review feedback

* chore: use join() not member directly

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-08-10 16:47:37 +00:00
Raphael Taylor-Davies 1f450ef371
feat: add Database abstraction (#2186) (#2203)
* feat: add Database abstraction

* chore: minor tweaks

* chore: remove redundant test fixture restart

* chore: review feedback

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-08-08 17:14:23 +00:00
Andrew Lamb 096281bbc4
fix: set distinct count for all entirely null columns to 1 (#2160)
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-07-30 17:17:10 +00:00
Carol (Nichols || Goulding) 9d15798288 fix: Address or allow Clippy warnings new with Rust 1.54 2021-07-30 09:59:59 -04:00
kodiakhq[bot] 545222303f
Merge branch 'main' into cn/cc-only 2021-07-29 17:18:16 +00:00
Carol (Nichols || Goulding) 8ee02bf5ce docs: Fix copypasta in doc comment 2021-07-28 15:27:23 -04:00
Carol (Nichols || Goulding) a1e2ce77f0 refactor: Remove TableSummaryAndTimes type 2021-07-28 14:12:58 -04:00
Marko Mikulicic fe7f65bfa7
feat(iox): Implement max_active_compactions_cpu_fraction 2021-07-28 17:31:17 +02:00
Carol (Nichols || Goulding) 8add00e761 feat: Make CatalogChunk first/last write times required
Connects to #1927.
2021-07-28 09:22:06 -04:00
Carol (Nichols || Goulding) 7c9a21632b refactor: Organize uses 2021-07-28 09:22:04 -04:00
Dom 7d3b6bf80d feat: DatabaseRules (de)serialisation support
Derives serde::Serialise & Deserialise for the DatabaseRules struct and
children.
2021-07-27 14:29:26 +01:00
kodiakhq[bot] 5551dd3a87
Merge branch 'main' into devnull 2021-07-27 09:57:16 +00:00
Andrew Lamb 7415fa0d05
fix: Convert panic --> `warn` on statistics mismatch (#2125) 2021-07-26 22:11:10 +00:00
Andrew Lamb 5fb3e00f2a
fix: Properly record total_count and null_count in statistics (#2103)
* fix: Properly record total_count and null_count in statistics

* fix: fix statistics calculation in mutable_buffer

* refactor: expose null counts in read_buffer

* refactor: expose null_count in parquet_file

* fix: update server crate tests

* fix: update query_tests tests

* docs: tweak comments

* refactor: Use storage_stats rather than adding `null_count`

* refactor: rename test data field for clarity

* fix: fixup merge conflicts

* refactor: rename initial_non_null_count to initial_total_count

* refactor: caculate null_count as row_count - to_add
2021-07-26 18:13:36 +00:00
Marko Mikulicic 094945a72d
feat: Add '/dev/null' sink 2021-07-26 19:19:11 +02:00