a9ec0720b2
feat: allow to format delete predicates as SQL strings
...
This is required to feed them back into the gRPC delete API.
2021-11-23 15:39:53 +01:00
d16a7759ca
Merge branch 'main' into cn/workspace-hack
2021-11-22 17:05:31 +00:00
3b2e5c445b
chore: Merge branch 'main' into ntran/compact_os_framework
2021-11-19 16:18:24 -05:00
9fd4a560f5
feat: Results of running cargo hakari manage-deps
2021-11-19 09:21:57 -05:00
e32d367e85
feat: flush delete mailbox on persist ( #3126 ) ( #3147 )
...
* feat: flush delete mailbox on persist (#3126 )
* chore: review feedback
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-11-19 09:45:29 +00:00
a5c04e5fe4
feat: framework for compact os chunks
2021-11-17 18:12:51 -05:00
8155747735
feat: add write buffer delete encoding ( #2731 ) ( #3127 )
...
* feat: add write buffer delete encoding (#2731 )
* chore: fix doc
* chore: review feedback
* chore: review feedback
* chore: fmt
* chore: review feedback
2021-11-17 16:12:19 +00:00
4e71de508e
fix: ensure `ConsistenHasher` is consistent
...
The std `DefaultHasher` is NOT guaranteed to stay the same, so let's
directly use the `SipHasher13` which at the moment (2021-11-15) is used
by the standard lib.
Fixes #3063 .
2021-11-15 17:39:17 +01:00
3d091208af
refactor: move delete predicate into data_types ( #2731 ) ( #3094 )
...
* refactor: move delete predicate into dml (#2731 )
* refactor: move DeletePredicate to data_types
* chore: fix doc
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-11-15 10:28:58 +00:00
da7ba511b2
feat: remove legacy per-line partitioning logic ( #3087 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-11-11 16:50:42 +00:00
c1fa41196d
feat: impl `Display` for `ShardId`
2021-11-08 14:09:33 +01:00
5b01f9b3a5
docs: explain btree map decision
2021-11-08 11:11:52 +01:00
11f4a1dee8
feat: add connection management for router
2021-11-08 11:11:52 +01:00
06dfdc4af8
feat: simplify sharding ( #3031 )
...
* feat: simplify sharding
* chore: fix lint
2021-11-04 16:57:19 +00:00
4840dc11cb
feat: add basic CRUD operations for router configs
2021-11-04 12:03:14 +01:00
0d0c0cb42b
refactor: move write buffer configs to new home
...
Write buffer configs will partially be shared by database and router
nodes, so lets move them into a shared home.
2021-11-02 10:17:01 +01:00
f1a6468e7b
feat: migrate write buffer consumer to use DbWrite ( #2724 ) ( #3003 )
...
* feat: migrate write buffer consumer to use DbWrite (#2724 )
* fix: doc
* chore: fmt
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-11-01 16:38:48 +00:00
0bccd35f8c
fix: List details about databases only prints active dbs and their UUIDs
...
We can no longer list deleted databases because the server no longer
knows about them, and we now have UUIDs that are useful to know about,
so change the detailed listing of databases to return UUID instead of
possible deleted at time.
2021-10-28 13:20:29 -04:00
f160712f5e
fix: Remove the concept of generations
...
From iox_object_store and everywhere that fails to compile as a result.
Also make path handling in iox_object_store a bit more consistent.
2021-10-28 13:20:28 -04:00
ec07c11d36
feat: add PartitionWrite ( #2724 ) ( #2974 )
...
* feat: add PartitionWrite (#2724 )
* chore: review feedback
2021-10-28 06:18:10 +00:00
bc7244c48e
chore: use Rust edition 2021
2021-10-25 10:58:20 +02:00
cfd980d1bc
fix: only string columns have distinct counts
2021-10-22 13:04:10 +01:00
d5d885aac6
feat: add MutableBatch::extend_from_range ( #2917 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-10-21 11:58:11 +00:00
ce0127a1f7
feat: MutableBatch write API ( #2090 ) ( #2724 ) ( #2882 )
...
* feat: MutableBatch write API (#2090 ) (#2724 )
* chore: fix lint
* fix: handle dictionaries with unused mappings
* chore: review feedback
* chore: further review feedback
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-10-20 08:44:14 +00:00
3c953cdf59
feat: improve legibility of task tracker logs ( #2826 )
...
* feat: improve legibility of task tracker logs
* feat: include start time
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-10-13 14:32:47 +00:00
3dfe400e6b
feat: migrate write path to TimeProvider ( #2722 ) ( #2807 )
2021-10-12 12:09:08 +00:00
b39e01f7ba
feat: migrate PersistenceWindows to TimeProvider ( #2722 ) ( #2798 )
2021-10-11 20:40:00 +00:00
c4a2641764
refactor: remove `time_closed`
...
The "time closed" is a leftover from an old lifecycle system, where
chunks moved through the system (open=>closed=>persisted) without being
merged. Now we have the compaction as well as the split query for
persistence that can merge chunks, so a single "time closed" doesn't
make sense any longer. So in fact it is `None` for many chunks and is
also not persisted. Also the current lifecycle policy doesn't use this
value. So let's just remove it.
Closes #1846 .
2021-10-11 15:49:34 +02:00
f35a49edd0
refactor: move Sequence to data_types ( #2780 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-10-11 09:23:00 +00:00
64bda1fc08
feat: improve `Debug`/`Display` for test `ChunkId`s
2021-10-08 13:55:56 +02:00
ae7d893199
feat: Add UUID to databases
...
Connects to #2675 .
When a database is created, assign it a UUID and serialize the UUID to
object storage by wrapping the database rules in a new
`PersistedDatabaseRules` type that also contains the UUID.
All APIs to the end user involving rules should continue using only
`DatabaseRules` so the UUID is an internal implementation detail.
2021-10-07 10:19:14 -04:00
27e7a1f925
refactor: Organize use statements
2021-10-07 10:17:19 -04:00
fb9119fa4c
feat: remove serde from data types ( #2652 )
2021-10-07 08:58:06 +00:00
8b06d72a58
fix: address review comments
2021-10-07 10:24:19 +02:00
63d74be490
refactor: make `ChunkId` a UUID
2021-10-07 10:23:27 +02:00
39157828b1
feat: remove remaining usages of Instant ( #2722 ) ( #2749 )
...
* feat: remove remaining usages of Instant (#2722 )
* chore: review feedback
Co-authored-by: Andrew Lamb <alamb@influxdata.com>
Co-authored-by: Andrew Lamb <alamb@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-10-06 16:44:02 +00:00
bb7a27e5ed
refactor: use proper sets during delete predicate collection
...
We no longer need hacky pointer tricks to de-duplicate delete predicates
when collecting them for catalog checkpoints. This was once required
when the delete predicates didn't implement `Eq` and `Hash` but now it's
all way easier.
2021-10-05 10:37:34 +02:00
97881079e8
refactor: make `ChunkOrder` non-zero
...
This will make it easier to handle missing values.
Helps with #2633 .
2021-10-04 17:49:12 +02:00
b402423e9e
feat: remove move lifecycle action ( #2674 )
...
* feat: remove move_chunk lifecycle action
* chore: review feedback
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-09-30 16:58:05 +00:00
11413eae8d
docs: mention that `Chunk{Id,Order}::next` might panic
2021-09-20 15:32:44 +02:00
cef5aeee52
refactor: introduce `ChunkId` type
2021-09-20 13:10:41 +02:00
acf698c366
fix: delete predicate sorting
2021-09-20 10:48:32 +02:00
44a89cdf75
refactor: Change DeletedDatabase to DetailedDatabase
...
So this info can be reused for active databases in detailed database
lists.
2021-09-17 15:27:22 -04:00
37b615f301
feat: migrate operations CLI to use pbjson ( #2562 )
...
* feat: migrate operations CLI to use pbjson
* fix: reserve removed field
* chore: review feedback
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-09-16 19:15:24 +00:00
6e7fa3e574
feat: migrate http ingest metrics ( #2542 )
...
* feat: migrate http ingest metrics
* chore: review feedback
* refactor: RAII entry ingest recorder
2021-09-15 19:01:10 +00:00
1ea4335ff3
fix: report correct DatabaseStateCode ( #2543 )
...
* fix: report correct DatabaseStateCode
* chore: fix lint
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-09-15 18:02:54 +00:00
a372973b67
fix: typos and wording
...
Co-authored-by: Andrew Lamb <alamb@influxdata.com>
2021-09-14 16:24:45 +00:00
becef1c75f
refactor: introduce `ChunkOrder` type
2021-09-14 17:10:23 +02:00
8a531be05b
feat: expose chunk order via API and in system table
2021-09-14 13:00:55 +02:00
3f2e46c397
feat: prune old transactions from preserved catalog
2021-09-14 12:08:17 +02:00
Marco Neumann
kodiakhq[bot]
Nga Tran
Carol (Nichols || Goulding)
Raphael Taylor-Davies
Carol (Nichols || Goulding)
Edd Robinson