6e01fbc382
feat: ause TableSummary as metadata for parquet chunk's tables and read buffer's read_filter ot get data
2021-04-05 15:37:34 -04:00
59bd66a33a
refactor: Merge remote-tracking branch 'origin' into ntran/write_parquet_3
2021-04-05 09:40:08 -04:00
80d55d0829
chore: rename tracing_deps to observability_deps
...
OpenTelemetry makes this necessary.
2021-04-02 13:14:30 -07:00
4bdf8963e6
feat: continue buidling foundation for writing RB chunks to parquet files
2021-04-02 16:06:25 -04:00
5a04c85da5
Merge branch 'main' into ntran/parquet_write_2
2021-04-02 14:43:46 +00:00
0b880d3534
chore: Group all tracing-related crates under one crate for easier upgrade management
2021-04-02 09:54:39 -04:00
eba799e4e7
Merge branch 'main' into ntran/parquet_write_2
2021-04-01 21:54:19 +00:00
839fe514a2
fix: do not make rows for empty partitions or empty tables
2021-04-01 21:22:04 +00:00
18ca5c5292
refactor: Implement `as_str()` directly rather than using a crate
2021-04-01 21:22:04 +00:00
e52c6a8292
test: Add / update query tests
2021-04-01 21:22:04 +00:00
d0e901f503
chore: fix row_estimate to match comment
2021-04-01 21:22:04 +00:00
c0abeba792
feat: system tables
2021-04-01 21:22:04 +00:00
f66c67692c
feat: add 2 more chunk states for writing chunks to object store
2021-04-01 17:04:08 -04:00
49267114d3
chore: merge main into branch and resolve conflicts
2021-04-01 13:22:49 -04:00
1463c6645f
feat: Add ChunkState::ObjectStore and rename ParquetChunk to Chunk
2021-04-01 11:53:03 -04:00
3a150594ab
refactor: push chunk and partition summary into catalog ( #1103 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-04-01 13:34:21 +00:00
a59a6edbb8
feat: add SHOW TABLES and `select * from information_schema.columns` ( #1102 )
...
* chore: update dependencies
* test: add tests for new information schema
2021-04-01 12:50:37 +00:00
0d599eeaf4
refactor: remove Db::writeable ( #1101 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-04-01 11:45:40 +00:00
b0e21e5f9e
feat: automated chunk lifecycle ( #1091 )
...
* feat: automated chunk lifecycle
* chore: use >= for lifecycle timestamp comparisons
* chore: review fixes
2021-04-01 11:05:58 +00:00
8f69a12ec6
refactor: Add rwlock around db rules
2021-04-01 10:43:46 +00:00
de25dc1b53
refactor: remove stuff moved to different crate
2021-03-31 20:02:34 -04:00
19a453a483
feat: finally have some framework with clear todos for writing a chunk into parquet files
2021-03-31 16:21:53 -04:00
b61875e0b2
feat: Add TableSummary calculation to ReadBuffer ( #1092 )
2021-03-31 16:26:37 +00:00
569f90d937
feat: Add ability to get PartitionSummary statistics from a Db ( #1090 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-03-31 14:18:53 +00:00
13612047c0
feat: chunk sorting ( #1077 )
2021-03-31 12:30:59 +00:00
cd409b471f
feat: continue the implementation
2021-03-30 21:31:51 -04:00
bccac45c75
chore: Merge remote-tracking branch 'origin' into ntran/parquet_write
2021-03-30 18:31:32 -04:00
0bcd52d5c9
feat: Add more changes
2021-03-30 18:31:09 -04:00
7154dfd5f6
feat: Add timestamps to ChunkSummary ( #1079 )
...
* refactor: Move timestamps from mutable_buffer::Chunk to catalog::Chunk
* feat: Add timestamps to ChunkSummary
* feat: Add timestamp conversion logic to protobuf types
* test: Add tests
* fix: Update data_types test
* fix: handle negative nanos during conversion
* fix: clippy
* fix: more clippy
* fix: even more clippy
* fix: even more clippy
2021-03-30 19:03:23 +00:00
d8d5b1568a
feat: automated chunk closing ( #1074 )
...
* feat: automated chunk closing
* chore: remove unused error variant
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-03-30 15:31:09 +00:00
1a0e698571
feat: initial of write chunks to parquet files
2021-03-30 09:56:53 -04:00
f0b411cd43
feat: enable information_schema
2021-03-30 09:01:43 -04:00
0b7dea09d5
refactor: Remove `MutableBufferDb` and Partition ( #1068 )
...
* refactor: Move test for missing tags as null to db
* refactor: move last_write_at and create_time to catalog partition
* refactor: remove mutable buffer partition and port tests
* fix: port over partition sorting logic
* fix: remove stale comment
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-03-30 12:51:37 +00:00
18f3124721
refactor: remove DbCatalog newtype wrapper ( #1067 )
...
* refactor: remove DbCatalog newtype wrapper
* chore: review comments
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-03-30 07:52:16 +00:00
6a48001d13
refactor: Manage storage directly in the Catalog ( #1057 )
...
* refactor: Manage mutable buffer chunks directly
* fix: do not use mutable_buffer for listing table names
2021-03-29 17:55:07 +00:00
1a4c793270
feat: lifecycle rules ( #1045 )
2021-03-29 15:32:36 +00:00
eb0122655d
refactor: Remove async from PartitionChunk ( #1062 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-03-29 13:00:36 +00:00
02ae743e8e
refactor: Remove async from Database ( #1063 )
2021-03-29 12:48:12 +00:00
fb130ea99d
feat: use CatalogProvider and SchemaProvider ( #1058 )
...
* feat: use CatalogProvider and SchemaProvider
* refactor: review comments
2021-03-29 11:08:46 +00:00
0ca9ad7285
refactor: Remove async from `PartitionChunk::table_schema` ( #1060 )
2021-03-27 18:08:12 +00:00
790e5d3348
refactor: Add fine grained (object level) catalog locking ( #1053 )
...
* refactor: inline catalog crate to server
* refactor: Add fine grained (object level) catalog locking
* fix: Move mod definition and use to top of file
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-03-26 13:48:24 +00:00
1b3fa7c5e8
fix: increase timing tolerance for tracker timing test ( #1051 ) ( #1052 )
...
* fix: increase timing tolerance for tracker timing test (#1051 )
* fix: fmt
Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
2021-03-25 15:09:09 +00:00
cda7d13110
refactor: unify serialization of database rules ( #973 ) ( #1047 )
...
* refactor: unify serialization of database rules (#973 )
* chore: fix test
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-03-25 14:13:14 +00:00
4cbf7c6ae5
refactor: Make `Db::mutable_buffer_chunks` and `Db::read_buffer_chunks` private ( #1044 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-03-24 20:57:15 +00:00
44d67db733
feat: Rework Db to use Catalog for chunk state ( #1043 )
...
* feat: Rework Db to use Catalog for chunk state
* docs: Update server/src/db.rs
* fix: fmt
* fix: fmt
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-03-24 17:57:11 +00:00
360df39d4c
feat: per-Db background tasks ( #1032 )
...
* feat: per-Db background tasks (#1007 )
* chore: review comments
* fix: ensure tracker registry is drained on shutdown
* chore: formatting
* chore: further review fixes
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-03-24 09:43:52 +00:00
d4eef65f2a
feat: share job registry with Db struct ( #1038 )
2021-03-23 12:34:55 +00:00
1cbfea7096
refactor: don't implement serialize for Db ( #1037 )
2021-03-23 11:36:46 +00:00
9e9387352e
Merge branch 'main' into ntran/unsigned_tests
2021-03-22 16:26:08 -04:00
6a2e5d69f1
chore: e2e test for unsigned types
2021-03-22 15:50:42 -04:00
Nga Tran
Jacob Marble
kodiakhq[bot]
Carol (Nichols || Goulding)
Andrew Lamb
Raphael Taylor-Davies
Raphael Taylor-Davies
Andrew Lamb
Marko Mikulicic