0788892413
feat: add row_count to system.chunks and Chunk managment API ( #1373 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-04-29 13:53:25 +00:00
262bf446ab
feat: change background worker to use backoff instead of poll ( #1339 ) ( #1347 )
...
* feat: change background worker to use backoff instead (#1339 )
* chore: fix comment
* chore: fix tests
* chore: review comments
* chore: review feedback
2021-04-29 11:10:51 +00:00
1f3bf6b5a6
Merge branch 'main' into sharding_refa
2021-04-28 21:58:21 +00:00
b1924d10b4
fix: Only show current database's system.operations table ( #1310 )
...
* test: add test for operations.system_tables
* fix: only show operations for current database
* fix: update test
* fix: improve test
* refactor: filter in Schema provider rather than in job tracker
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-04-28 21:06:19 +00:00
91ee005ada
feat: Add multiple matchers to shard config
2021-04-28 17:02:58 +02:00
b6054f9df9
feat: Make background task period configurable
2021-04-28 15:47:31 +02:00
9b57214344
feat: move closing chunks as soon as possible ( #1329 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-04-28 07:50:15 +00:00
eddc9319ff
docs: deny broken intradoc links
2021-04-27 13:22:28 +02:00
84c863aeb0
refactor: remove unused PersistSegment job ( #1245 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-04-26 19:41:59 +00:00
934bc75855
fix: Remove PartialEq<u32> impls for ServerId, not really necessary
2021-04-26 18:44:32 +00:00
272cdb85ce
fix: Use the ServerId type everywhere, for writing, querying, anything
2021-04-26 18:44:32 +00:00
b8face3335
refactor: Organize use statements
2021-04-26 18:44:32 +00:00
bdb772b243
refactor: Alphabetize module declarations
2021-04-26 18:44:32 +00:00
67f5ad841d
refactor: Introduce ServerId and CurrentServerId types
2021-04-26 18:44:32 +00:00
0a835436ac
feat: use bitmasks within MUB ( #1274 ) ( #1289 )
...
* feat: use bitmasks within MUB (#1274 )
* chore: review feedback
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-04-26 18:00:16 +00:00
9c74074095
feat: List background operations as system.operations table
2021-04-26 16:59:36 +00:00
f12f1eeef7
refactor: make chunks per-table
...
This changes the hierarchy from
```
database -> partition -> chunk -> table
```
to
```
database -> partition -> table -> chunk
```
Only the high-level APIs are changed for now. The chunk states (like
MutableBuffer and ReadBuffer) still multiplex tables, although they will
always only get a single table assigned (or no table if no data was
presented yet).
Closes #1256 .
2021-04-23 09:18:55 +02:00
83d6550316
feat: Implement write_entry_downstream
2021-04-21 20:50:46 +00:00
80995afb70
fix: Change WAL to Write Buffer in comments and documentation
2021-04-21 17:43:03 +00:00
ed6025b400
refactor: Rename Wal metadata endpoint types to WriteBuffer
2021-04-21 17:43:03 +00:00
5f024401a4
refactor: Rename data_types::wal module to data_types::write_buffer
2021-04-21 17:43:03 +00:00
cdb9574beb
fix: Rename wal/meta as wb/meta
2021-04-21 17:43:03 +00:00
0677c544bb
refactor: Rename WalBufferRollover to WriteBufferRollover
2021-04-21 17:43:03 +00:00
ee45db402b
refactor: Rename WalBufferConfig to WriteBufferConfig
2021-04-21 17:43:03 +00:00
8f0ed5e51c
refactor: satisfy new clippy lints
2021-04-20 17:30:50 +00:00
757933afc4
fix: use Self when possible
2021-04-19 08:48:11 -04:00
e2954b918e
fix: set defaults for `persist` and `drop_non_persisted` to `true`
...
This should prevent writer deaths for default configs.
Closes #1225 .
2021-04-19 13:03:04 +02:00
8bfcc1782d
feat: Implement sharding logic
2021-04-16 15:14:57 +00:00
878b1b318e
feat: Initial scaffolding for routing layer
...
Part of #916
Adding first-class concept of ShardId in shard config, fixes #1156
NEXT:
- [ ] implement sharder
- [ ] implement `write_entry_downstream`
- [ ] add tests
2021-04-15 09:02:47 +00:00
4e2d59d9a5
feat: saimplement a few more functions as part of supporting query dfrom parquet files
2021-04-14 16:06:47 -04:00
e516f65cbe
feat: add WriteChunk job
...
For #1083 .
2021-04-14 12:37:43 +02:00
440cef59df
docs: fix docs of `LifecycleRules::buffer_size_soft`
2021-04-14 12:37:43 +02:00
ef7de0aa05
feat: add persist flag to LifecycleRules
...
For #1083 .
2021-04-14 12:37:43 +02:00
bd13c09bad
refactor: make sharder optional when generating entry
2021-04-13 12:52:14 +00:00
7e28f8ef66
feat: Implement Entry writing to Db
...
This removes the old ReplicatedWrite structure and implements the writing of an Entry to the Db. I also call out in `server/lib.rs` and in the `Db` where sharding and replication might happen.
I've also added helpers in various places to write line protocol to chunks, tables, and databases. That enabled removing a good amount of code from the test helpers crate.
2021-04-13 12:52:14 +00:00
1997324344
feat: mutable buffer snapshotting ( #1179 )
...
* feat: mutable buffer snapshotting
* chore: review feedback
2021-04-13 12:14:54 +00:00
7f77a01e61
chore: merged main to branch and resolved conflicts
2021-04-12 12:09:04 -04:00
453aeaf1a0
feat: Add tests for writing RB chunks to Object Store
2021-04-09 17:39:23 -04:00
e76980928b
feat: Implement Update API
2021-04-08 22:25:36 +00:00
f501a74aea
refactor: Address review comments
2021-04-07 21:28:03 -04:00
be6e1e48e4
feat: add writer_id and object_store in Db
2021-04-07 18:36:07 -04:00
82588d5c72
fix: Don't return Result from test functions
2021-04-07 12:40:00 -04:00
a7659addbe
chore: cleanup entry.rs based on PR feedback and add better errors
2021-04-06 17:56:58 -04:00
2a134a3ac1
feat: implment line protocol to flatbuffers
...
This implements a builder for converting line protocol to the Entry flatbuffers. It also contains wrapper structs to make working with the flatbuffers a little easier.
The flatbuffer needed an addition to keep track of how many bits in the null mask should be used.
2021-04-06 17:56:56 -04:00
5cd1d6691d
refactor: use DatabaseName in DatabaseRules ( #1127 )
2021-04-06 13:26:30 +00:00
80d55d0829
chore: rename tracing_deps to observability_deps
...
OpenTelemetry makes this necessary.
2021-04-02 13:14:30 -07:00
0b880d3534
chore: Group all tracing-related crates under one crate for easier upgrade management
2021-04-02 09:54:39 -04:00
18ca5c5292
refactor: Implement `as_str()` directly rather than using a crate
2021-04-01 21:22:04 +00:00
c0abeba792
feat: system tables
2021-04-01 21:22:04 +00:00
49267114d3
chore: merge main into branch and resolve conflicts
2021-04-01 13:22:49 -04:00
Andrew Lamb
Raphael Taylor-Davies
kodiakhq[bot]
Marko Mikulicic
Marco Neumann
Carol (Nichols || Goulding)
Jake Goulding
Andrew Lamb
Edd Robinson
Nga Tran
Paul Dix
Jacob Marble
Raphael Taylor-Davies