3dfe400e6b
feat: migrate write path to TimeProvider ( #2722 ) ( #2807 )
2021-10-12 12:09:08 +00:00
b39e01f7ba
feat: migrate PersistenceWindows to TimeProvider ( #2722 ) ( #2798 )
2021-10-11 20:40:00 +00:00
c4a2641764
refactor: remove `time_closed`
...
The "time closed" is a leftover from an old lifecycle system, where
chunks moved through the system (open=>closed=>persisted) without being
merged. Now we have the compaction as well as the split query for
persistence that can merge chunks, so a single "time closed" doesn't
make sense any longer. So in fact it is `None` for many chunks and is
also not persisted. Also the current lifecycle policy doesn't use this
value. So let's just remove it.
Closes #1846 .
2021-10-11 15:49:34 +02:00
f35a49edd0
refactor: move Sequence to data_types ( #2780 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-10-11 09:23:00 +00:00
64bda1fc08
feat: improve `Debug`/`Display` for test `ChunkId`s
2021-10-08 13:55:56 +02:00
ae7d893199
feat: Add UUID to databases
...
Connects to #2675 .
When a database is created, assign it a UUID and serialize the UUID to
object storage by wrapping the database rules in a new
`PersistedDatabaseRules` type that also contains the UUID.
All APIs to the end user involving rules should continue using only
`DatabaseRules` so the UUID is an internal implementation detail.
2021-10-07 10:19:14 -04:00
27e7a1f925
refactor: Organize use statements
2021-10-07 10:17:19 -04:00
fb9119fa4c
feat: remove serde from data types ( #2652 )
2021-10-07 08:58:06 +00:00
8b06d72a58
fix: address review comments
2021-10-07 10:24:19 +02:00
63d74be490
refactor: make `ChunkId` a UUID
2021-10-07 10:23:27 +02:00
39157828b1
feat: remove remaining usages of Instant ( #2722 ) ( #2749 )
...
* feat: remove remaining usages of Instant (#2722 )
* chore: review feedback
Co-authored-by: Andrew Lamb <alamb@influxdata.com>
Co-authored-by: Andrew Lamb <alamb@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-10-06 16:44:02 +00:00
bb7a27e5ed
refactor: use proper sets during delete predicate collection
...
We no longer need hacky pointer tricks to de-duplicate delete predicates
when collecting them for catalog checkpoints. This was once required
when the delete predicates didn't implement `Eq` and `Hash` but now it's
all way easier.
2021-10-05 10:37:34 +02:00
97881079e8
refactor: make `ChunkOrder` non-zero
...
This will make it easier to handle missing values.
Helps with #2633 .
2021-10-04 17:49:12 +02:00
b402423e9e
feat: remove move lifecycle action ( #2674 )
...
* feat: remove move_chunk lifecycle action
* chore: review feedback
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-09-30 16:58:05 +00:00
11413eae8d
docs: mention that `Chunk{Id,Order}::next` might panic
2021-09-20 15:32:44 +02:00
cef5aeee52
refactor: introduce `ChunkId` type
2021-09-20 13:10:41 +02:00
acf698c366
fix: delete predicate sorting
2021-09-20 10:48:32 +02:00
44a89cdf75
refactor: Change DeletedDatabase to DetailedDatabase
...
So this info can be reused for active databases in detailed database
lists.
2021-09-17 15:27:22 -04:00
37b615f301
feat: migrate operations CLI to use pbjson ( #2562 )
...
* feat: migrate operations CLI to use pbjson
* fix: reserve removed field
* chore: review feedback
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-09-16 19:15:24 +00:00
6e7fa3e574
feat: migrate http ingest metrics ( #2542 )
...
* feat: migrate http ingest metrics
* chore: review feedback
* refactor: RAII entry ingest recorder
2021-09-15 19:01:10 +00:00
1ea4335ff3
fix: report correct DatabaseStateCode ( #2543 )
...
* fix: report correct DatabaseStateCode
* chore: fix lint
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-09-15 18:02:54 +00:00
a372973b67
fix: typos and wording
...
Co-authored-by: Andrew Lamb <alamb@influxdata.com>
2021-09-14 16:24:45 +00:00
becef1c75f
refactor: introduce `ChunkOrder` type
2021-09-14 17:10:23 +02:00
8a531be05b
feat: expose chunk order via API and in system table
2021-09-14 13:00:55 +02:00
3f2e46c397
feat: prune old transactions from preserved catalog
2021-09-14 12:08:17 +02:00
b8f7319704
feat: migrate read buffer metrics to metric crate ( #2510 )
...
* feat: migrate read buffer metrics to metric crate
* chore: review feedback
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-09-10 19:51:43 +00:00
bbb8898d36
refactor: make writer buffer auto-creation types nicer to read
2021-09-08 11:13:48 +02:00
b4fe829da0
fix: docs
2021-09-07 18:24:59 +02:00
e7cf91bfc4
refactor: more structures write buffer auto-creation config
2021-09-07 18:18:20 +02:00
d5662328b0
refactor: `n_sequencers` should be non-zero
2021-09-07 18:18:20 +02:00
dbe4bcff22
chore: merge main to branch
2021-09-07 10:54:59 -04:00
04a301cc64
refactor: make job descriptions a static string
2021-09-06 13:39:22 +02:00
de0bd80c3d
refactor: cleanup
2021-09-06 01:07:07 -04:00
ce6030a3cb
feat: Wire list deleted databases through gRPC and CLI APIs
2021-09-02 15:48:07 -04:00
81e672db14
feat: add `auto_create_sequencers` write buffer flag
2021-09-02 16:41:35 +02:00
ecf1f99ddb
refactor: more flexible writer buffer config
...
This allows:
- different types (instead of guessing through the connection URL)
- sequencer counts (not used yet but will be by #2455 )
- extensible configs (e.g. to configure Kafka in a more granular way,
not wired up yet)
- future extensions (since we use a message now instead of a single
string)
**BREAKING: This requires changes for deployed systems / existing DBs!**
2021-09-02 16:41:35 +02:00
779b027271
feat: Store only the database rules sent by the client (do not store default values) ( #2430 )
...
* feat: add omit_default to protobuf definition
* feat: Persist only the client provided rules
* fix: Remove race conditions
* fix: merge confit
* refactor: do not use macro
* refactor: restore use of glob import
* fix: review comments
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-08-28 10:26:32 +00:00
499af57299
chore: merge mian to branch and resolve conflicts
2021-08-27 17:51:07 -04:00
b79eaa34d1
refactor: address review comments
2021-08-27 15:53:27 -04:00
bcd39e225c
feat: Management API for delete
2021-08-26 17:31:21 -04:00
7cf7fb02ed
refactor: Rename database ObjectStore state types to DatabaseObjectStore
2021-08-26 09:14:23 -04:00
c7eceac8a3
refactor: Have server determine database generation from object store
2021-08-26 09:14:23 -04:00
60ea637c38
refactor: remove `drop_non_persisted`
...
This flag should always be `!persist`.
Closes #2212 .
2021-08-17 16:53:45 +02:00
77892a0998
feat: add API to drop entire partitions
2021-08-17 09:44:35 +02:00
fd4ea89454
chore: revert panic commit
2021-08-13 12:51:37 -04:00
db5fd935dd
fix: revert "Convert panic --> `warn` on statistics mismatch ( #2125 )" ( #2250 )
...
This reverts commit 7415fa0d05
2021-08-13 11:39:12 +00:00
f82646e7f8
test: ignore failing tests on aarch64
2021-08-12 11:46:23 +01:00
3de6b44e23
build: use new rustdoc lint name ( #2261 )
...
* fix: nocache feature code rot
The MBChunk::snapshot code when using the "nocache" option no longer
compiles - this commit updates it to match the not(nocache) code.
* build: use updated broken_intra_doc_links name
The broken_intra_doc_links lint was renamed
rustdoc::broken_intra_doc_links
https://doc.rust-lang.org/rustdoc/lints.html
2021-08-11 19:48:51 +00:00
0c300a4abb
fix: Handle all null columns in ColumnSummary::update_from ( #2251 )
2021-08-10 20:37:00 +00:00
2344c28f4e
feat: drain database jobs on shutdown ( #2239 )
...
* feat: drain database jobs on shutdown
* chore: fmt
* chore: review feedback
* chore: use join() not member directly
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-08-10 16:47:37 +00:00
Raphael Taylor-Davies
Marco Neumann
Carol (Nichols || Goulding)
Nga Tran
Andrew Lamb
Edd Robinson
Dom