80fb75d90b
feat: add a flag to enable per-partition tracing ( #3928 )
...
* feat: add a flag to enable per-partition tracing
* chore: rename constant
* feat: use BooleanFlag and cache result
2022-03-07 13:49:23 +00:00
8d00aaba90
feat: sync chunks in querier ( #3911 )
...
* feat: `ParquetFileRepo::list_by_namespace_not_to_delete`
* feat: `ChunkAddr: Clone`
* test: ensure that querier keeps same partition objects
* test: improve `create_parquet_file` flexibility
* feat: sync chunks in querier
* test: improve `test_parquet_file`
2022-03-04 08:53:39 +00:00
33851be3a5
chore: upgrade Rust to 1.59 ( #3875 )
...
Mostly a few new clippy crates around `flat_map`, `and_then`, and
"underscore locks" (!!!):
https://rust-lang.github.io/rust-clippy/master/index.html#let_underscore_lock
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-02-28 15:14:19 +00:00
442d63e65b
feat: catalog timestamp pruning ( #3571 )
...
* feat: catalog timestamp pruning
* chore: test
2022-01-28 13:45:13 +00:00
32d7c4cbfe
refactor: remove InfluxColumnType::IOx ( #3565 )
...
* refactor: remove InfluxColumnType::IOx
Remove unused column variant - see #3554 for context.
* refactor: reserve SEMANTIC_TYPE_IOX name in proto
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-01-27 21:15:36 +00:00
21c1824a7a
refactor: remove table_names from Predicate ( #3545 )
...
* refactor: remove table_names from Predicate
* chore: fix benchmarks
* chore: review feedback
Co-authored-by: Edd Robinson <me@edd.io>
* chore: review feedback
* chore: replace Default::default with InfluxRpcPredicate::default()
Co-authored-by: Edd Robinson <me@edd.io>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-01-27 14:44:49 +00:00
9c19cd6cc4
fix: clamp start/end of TimestampRange to min/max valid timestamp values ( #3487 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-01-20 16:08:00 +00:00
f0d50f447a
fix: Special case tag_keys with max timestamp range ( #3485 )
...
* fix: Special case tag_keys with max timestamp range
* docs: comment
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-01-20 14:14:34 +00:00
168afb63ad
feat: add `size` methods to DML-related types
...
This will be helpful when we want to batch DML operations in memory
(e.g. when using RSKafka).
This also ensures that `MBChunk` accounts for the column names that
are stored within `MutableBatch`.
2022-01-18 13:52:31 +01:00
5cce69b481
fix: reject empty org & bucket
...
Previously a request that specified an empty org & bucket value would be
mapped to a database named "_".
This commit changes the org/bucket mapping fn to return an error if
either org or bucket is empty.
2022-01-14 12:17:26 +00:00
d1816b662f
chore: update rust to 1.58 ( #3461 )
...
* chore: update rust to 1.58
* fix: clippy
Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
2022-01-13 21:12:46 +00:00
cdf5c21cd4
fix: Fix max timestamp value comparison in chunk metadata ( #3453 )
...
* fix: Fix max timestamp value comparison in chunk metadata
* refactor: rename contains to overlaps
Co-authored-by: Edd Robinson <me@edd.io>
2022-01-13 16:58:30 +00:00
f3f6f335a9
chore: upgrade to snafu 0.7 ( #3440 )
2022-01-11 19:22:36 +00:00
a93ae739a9
feat: Add table_name to Partition API ( #3421 )
2022-01-06 16:38:39 +00:00
758b65dd29
feat: Add database initialization state and errors to CLI and remove list_databases_detailed gRPC ( #3377 )
...
* feat: Add database initialization state and errors to CLI:
* fix: do not use optional in protobuf
* fix: clippy
* fix: correct check I broke appeasing clippy
2021-12-15 12:18:41 +00:00
30c4da7ca7
fix: Be consistent with regex version range specification
2021-12-06 09:37:15 -05:00
0dffcad109
Merge branch 'main' into ntran/chunkid_catalogchunk
2021-12-03 14:21:04 +00:00
10b1598e68
refactor: remove comments
2021-12-03 09:20:12 -05:00
2857b6a990
Merge branch 'main' into er/feat/load_chunk_cli
2021-12-02 20:20:56 +00:00
b4ea9887ba
refactor: error name
2021-12-02 20:14:02 +00:00
ffc970a60f
feat: add to-be-create-chunk-id for compacting OS chunks in CatalogChunk
2021-12-02 14:49:06 -05:00
1a899b939e
fix: Remove redundant closures identified by clippy
...
https://rust-lang.github.io/rust-clippy/master/index.html#redundant_closure
2021-12-02 11:52:02 -05:00
88aedc556e
feat: add FromStr implementation
2021-12-02 12:59:52 +00:00
e8edab0b43
Merge branch 'main' into ntran/compact_os_preserved_catalog
2021-12-01 18:26:33 +00:00
332485d2c9
fix: use correct `bytes_read` in `DmlMeta`
...
- for file-based write buffers: Use headers + payload
- for Kafka-based write buffers: Use the estimation that we also use for
other metrics
- as a side effect we can now just use `PartialEq` for more types
Fixes #3186 .
2021-12-01 15:57:21 +01:00
6bb7a796e7
chore: Merge branch 'main' into ntran/compact_os_preserved_catalog
2021-11-30 16:55:38 -05:00
e8034cdc53
refactor: handle delete predicates for compacting os
2021-11-30 16:45:04 -05:00
88acf3788e
feat: rebuild catalog ( #3225 ) ( #3253 )
...
* feat: rebuild catalog (#3225 )
* chore: fix doc
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-11-30 12:17:27 +00:00
23dffefcf8
Merge branch 'main' into crepererum/remove_routing_from_database_mode_7
2021-11-29 11:38:12 +00:00
197634ed50
feat: reload chunk back into read buffer ( #3209 ) ( #3216 )
...
* feat: reload chunk back into read buffer (#3209 )
* chore: fix logical conflict
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-11-29 11:34:55 +00:00
4e043ecb55
refactor: remove old routing / sharding config
...
This is superseded by the new router subsystem.
2021-11-29 12:33:48 +01:00
7f2e4f4342
refactor: remove write buffer direction
...
The direction was required when a database could read or write from/to a
write buffer. Now it is clear from the usage context of a write buffer
context which of the two applications is meant (databases read, routers
write) so the direction flag is no longer required.
2021-11-26 12:38:40 +01:00
a9ec0720b2
feat: allow to format delete predicates as SQL strings
...
This is required to feed them back into the gRPC delete API.
2021-11-23 15:39:53 +01:00
d16a7759ca
Merge branch 'main' into cn/workspace-hack
2021-11-22 17:05:31 +00:00
3b2e5c445b
chore: Merge branch 'main' into ntran/compact_os_framework
2021-11-19 16:18:24 -05:00
9fd4a560f5
feat: Results of running cargo hakari manage-deps
2021-11-19 09:21:57 -05:00
e32d367e85
feat: flush delete mailbox on persist ( #3126 ) ( #3147 )
...
* feat: flush delete mailbox on persist (#3126 )
* chore: review feedback
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-11-19 09:45:29 +00:00
a5c04e5fe4
feat: framework for compact os chunks
2021-11-17 18:12:51 -05:00
8155747735
feat: add write buffer delete encoding ( #2731 ) ( #3127 )
...
* feat: add write buffer delete encoding (#2731 )
* chore: fix doc
* chore: review feedback
* chore: review feedback
* chore: fmt
* chore: review feedback
2021-11-17 16:12:19 +00:00
4e71de508e
fix: ensure `ConsistenHasher` is consistent
...
The std `DefaultHasher` is NOT guaranteed to stay the same, so let's
directly use the `SipHasher13` which at the moment (2021-11-15) is used
by the standard lib.
Fixes #3063 .
2021-11-15 17:39:17 +01:00
3d091208af
refactor: move delete predicate into data_types ( #2731 ) ( #3094 )
...
* refactor: move delete predicate into dml (#2731 )
* refactor: move DeletePredicate to data_types
* chore: fix doc
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-11-15 10:28:58 +00:00
da7ba511b2
feat: remove legacy per-line partitioning logic ( #3087 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-11-11 16:50:42 +00:00
c1fa41196d
feat: impl `Display` for `ShardId`
2021-11-08 14:09:33 +01:00
5b01f9b3a5
docs: explain btree map decision
2021-11-08 11:11:52 +01:00
11f4a1dee8
feat: add connection management for router
2021-11-08 11:11:52 +01:00
06dfdc4af8
feat: simplify sharding ( #3031 )
...
* feat: simplify sharding
* chore: fix lint
2021-11-04 16:57:19 +00:00
4840dc11cb
feat: add basic CRUD operations for router configs
2021-11-04 12:03:14 +01:00
0d0c0cb42b
refactor: move write buffer configs to new home
...
Write buffer configs will partially be shared by database and router
nodes, so lets move them into a shared home.
2021-11-02 10:17:01 +01:00
f1a6468e7b
feat: migrate write buffer consumer to use DbWrite ( #2724 ) ( #3003 )
...
* feat: migrate write buffer consumer to use DbWrite (#2724 )
* fix: doc
* chore: fmt
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-11-01 16:38:48 +00:00
0bccd35f8c
fix: List details about databases only prints active dbs and their UUIDs
...
We can no longer list deleted databases because the server no longer
knows about them, and we now have UUIDs that are useful to know about,
so change the detailed listing of databases to return UUID instead of
possible deleted at time.
2021-10-28 13:20:29 -04:00
Raphael Taylor-Davies
Marco Neumann
Dom
Andrew Lamb
Edd Robinson
Carol (Nichols || Goulding)
kodiakhq[bot]
Nga Tran
Carol (Nichols || Goulding)