5b01f9b3a5
docs: explain btree map decision
2021-11-08 11:11:52 +01:00
11f4a1dee8
feat: add connection management for router
2021-11-08 11:11:52 +01:00
06dfdc4af8
feat: simplify sharding ( #3031 )
...
* feat: simplify sharding
* chore: fix lint
2021-11-04 16:57:19 +00:00
4840dc11cb
feat: add basic CRUD operations for router configs
2021-11-04 12:03:14 +01:00
0d0c0cb42b
refactor: move write buffer configs to new home
...
Write buffer configs will partially be shared by database and router
nodes, so lets move them into a shared home.
2021-11-02 10:17:01 +01:00
f1a6468e7b
feat: migrate write buffer consumer to use DbWrite ( #2724 ) ( #3003 )
...
* feat: migrate write buffer consumer to use DbWrite (#2724 )
* fix: doc
* chore: fmt
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-11-01 16:38:48 +00:00
0bccd35f8c
fix: List details about databases only prints active dbs and their UUIDs
...
We can no longer list deleted databases because the server no longer
knows about them, and we now have UUIDs that are useful to know about,
so change the detailed listing of databases to return UUID instead of
possible deleted at time.
2021-10-28 13:20:29 -04:00
f160712f5e
fix: Remove the concept of generations
...
From iox_object_store and everywhere that fails to compile as a result.
Also make path handling in iox_object_store a bit more consistent.
2021-10-28 13:20:28 -04:00
ec07c11d36
feat: add PartitionWrite ( #2724 ) ( #2974 )
...
* feat: add PartitionWrite (#2724 )
* chore: review feedback
2021-10-28 06:18:10 +00:00
bc7244c48e
chore: use Rust edition 2021
2021-10-25 10:58:20 +02:00
cfd980d1bc
fix: only string columns have distinct counts
2021-10-22 13:04:10 +01:00
d5d885aac6
feat: add MutableBatch::extend_from_range ( #2917 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-10-21 11:58:11 +00:00
ce0127a1f7
feat: MutableBatch write API ( #2090 ) ( #2724 ) ( #2882 )
...
* feat: MutableBatch write API (#2090 ) (#2724 )
* chore: fix lint
* fix: handle dictionaries with unused mappings
* chore: review feedback
* chore: further review feedback
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-10-20 08:44:14 +00:00
3c953cdf59
feat: improve legibility of task tracker logs ( #2826 )
...
* feat: improve legibility of task tracker logs
* feat: include start time
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-10-13 14:32:47 +00:00
3dfe400e6b
feat: migrate write path to TimeProvider ( #2722 ) ( #2807 )
2021-10-12 12:09:08 +00:00
b39e01f7ba
feat: migrate PersistenceWindows to TimeProvider ( #2722 ) ( #2798 )
2021-10-11 20:40:00 +00:00
c4a2641764
refactor: remove `time_closed`
...
The "time closed" is a leftover from an old lifecycle system, where
chunks moved through the system (open=>closed=>persisted) without being
merged. Now we have the compaction as well as the split query for
persistence that can merge chunks, so a single "time closed" doesn't
make sense any longer. So in fact it is `None` for many chunks and is
also not persisted. Also the current lifecycle policy doesn't use this
value. So let's just remove it.
Closes #1846 .
2021-10-11 15:49:34 +02:00
f35a49edd0
refactor: move Sequence to data_types ( #2780 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-10-11 09:23:00 +00:00
64bda1fc08
feat: improve `Debug`/`Display` for test `ChunkId`s
2021-10-08 13:55:56 +02:00
ae7d893199
feat: Add UUID to databases
...
Connects to #2675 .
When a database is created, assign it a UUID and serialize the UUID to
object storage by wrapping the database rules in a new
`PersistedDatabaseRules` type that also contains the UUID.
All APIs to the end user involving rules should continue using only
`DatabaseRules` so the UUID is an internal implementation detail.
2021-10-07 10:19:14 -04:00
27e7a1f925
refactor: Organize use statements
2021-10-07 10:17:19 -04:00
fb9119fa4c
feat: remove serde from data types ( #2652 )
2021-10-07 08:58:06 +00:00
8b06d72a58
fix: address review comments
2021-10-07 10:24:19 +02:00
63d74be490
refactor: make `ChunkId` a UUID
2021-10-07 10:23:27 +02:00
39157828b1
feat: remove remaining usages of Instant ( #2722 ) ( #2749 )
...
* feat: remove remaining usages of Instant (#2722 )
* chore: review feedback
Co-authored-by: Andrew Lamb <alamb@influxdata.com>
Co-authored-by: Andrew Lamb <alamb@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-10-06 16:44:02 +00:00
bb7a27e5ed
refactor: use proper sets during delete predicate collection
...
We no longer need hacky pointer tricks to de-duplicate delete predicates
when collecting them for catalog checkpoints. This was once required
when the delete predicates didn't implement `Eq` and `Hash` but now it's
all way easier.
2021-10-05 10:37:34 +02:00
97881079e8
refactor: make `ChunkOrder` non-zero
...
This will make it easier to handle missing values.
Helps with #2633 .
2021-10-04 17:49:12 +02:00
b402423e9e
feat: remove move lifecycle action ( #2674 )
...
* feat: remove move_chunk lifecycle action
* chore: review feedback
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-09-30 16:58:05 +00:00
11413eae8d
docs: mention that `Chunk{Id,Order}::next` might panic
2021-09-20 15:32:44 +02:00
cef5aeee52
refactor: introduce `ChunkId` type
2021-09-20 13:10:41 +02:00
acf698c366
fix: delete predicate sorting
2021-09-20 10:48:32 +02:00
44a89cdf75
refactor: Change DeletedDatabase to DetailedDatabase
...
So this info can be reused for active databases in detailed database
lists.
2021-09-17 15:27:22 -04:00
37b615f301
feat: migrate operations CLI to use pbjson ( #2562 )
...
* feat: migrate operations CLI to use pbjson
* fix: reserve removed field
* chore: review feedback
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-09-16 19:15:24 +00:00
6e7fa3e574
feat: migrate http ingest metrics ( #2542 )
...
* feat: migrate http ingest metrics
* chore: review feedback
* refactor: RAII entry ingest recorder
2021-09-15 19:01:10 +00:00
1ea4335ff3
fix: report correct DatabaseStateCode ( #2543 )
...
* fix: report correct DatabaseStateCode
* chore: fix lint
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-09-15 18:02:54 +00:00
a372973b67
fix: typos and wording
...
Co-authored-by: Andrew Lamb <alamb@influxdata.com>
2021-09-14 16:24:45 +00:00
becef1c75f
refactor: introduce `ChunkOrder` type
2021-09-14 17:10:23 +02:00
8a531be05b
feat: expose chunk order via API and in system table
2021-09-14 13:00:55 +02:00
3f2e46c397
feat: prune old transactions from preserved catalog
2021-09-14 12:08:17 +02:00
b8f7319704
feat: migrate read buffer metrics to metric crate ( #2510 )
...
* feat: migrate read buffer metrics to metric crate
* chore: review feedback
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-09-10 19:51:43 +00:00
bbb8898d36
refactor: make writer buffer auto-creation types nicer to read
2021-09-08 11:13:48 +02:00
b4fe829da0
fix: docs
2021-09-07 18:24:59 +02:00
e7cf91bfc4
refactor: more structures write buffer auto-creation config
2021-09-07 18:18:20 +02:00
d5662328b0
refactor: `n_sequencers` should be non-zero
2021-09-07 18:18:20 +02:00
dbe4bcff22
chore: merge main to branch
2021-09-07 10:54:59 -04:00
04a301cc64
refactor: make job descriptions a static string
2021-09-06 13:39:22 +02:00
de0bd80c3d
refactor: cleanup
2021-09-06 01:07:07 -04:00
ce6030a3cb
feat: Wire list deleted databases through gRPC and CLI APIs
2021-09-02 15:48:07 -04:00
81e672db14
feat: add `auto_create_sequencers` write buffer flag
2021-09-02 16:41:35 +02:00
ecf1f99ddb
refactor: more flexible writer buffer config
...
This allows:
- different types (instead of guessing through the connection URL)
- sequencer counts (not used yet but will be by #2455 )
- extensible configs (e.g. to configure Kafka in a more granular way,
not wired up yet)
- future extensions (since we use a message now instead of a single
string)
**BREAKING: This requires changes for deployed systems / existing DBs!**
2021-09-02 16:41:35 +02:00
Marco Neumann
Raphael Taylor-Davies
Carol (Nichols || Goulding)
Edd Robinson
Nga Tran