80d55d0829
chore: rename tracing_deps to observability_deps
...
OpenTelemetry makes this necessary.
2021-04-02 13:14:30 -07:00
0b880d3534
chore: Group all tracing-related crates under one crate for easier upgrade management
2021-04-02 09:54:39 -04:00
569f90d937
feat: Add ability to get PartitionSummary statistics from a Db ( #1090 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-03-31 14:18:53 +00:00
7154dfd5f6
feat: Add timestamps to ChunkSummary ( #1079 )
...
* refactor: Move timestamps from mutable_buffer::Chunk to catalog::Chunk
* feat: Add timestamps to ChunkSummary
* feat: Add timestamp conversion logic to protobuf types
* test: Add tests
* fix: Update data_types test
* fix: handle negative nanos during conversion
* fix: clippy
* fix: more clippy
* fix: even more clippy
* fix: even more clippy
2021-03-30 19:03:23 +00:00
0b7dea09d5
refactor: Remove `MutableBufferDb` and Partition ( #1068 )
...
* refactor: Move test for missing tags as null to db
* refactor: move last_write_at and create_time to catalog partition
* refactor: remove mutable buffer partition and port tests
* fix: port over partition sorting logic
* fix: remove stale comment
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-03-30 12:51:37 +00:00
6a48001d13
refactor: Manage storage directly in the Catalog ( #1057 )
...
* refactor: Manage mutable buffer chunks directly
* fix: do not use mutable_buffer for listing table names
2021-03-29 17:55:07 +00:00
1a4c793270
feat: lifecycle rules ( #1045 )
2021-03-29 15:32:36 +00:00
02ae743e8e
refactor: Remove async from Database ( #1063 )
2021-03-29 12:48:12 +00:00
44d67db733
feat: Rework Db to use Catalog for chunk state ( #1043 )
...
* feat: Rework Db to use Catalog for chunk state
* docs: Update server/src/db.rs
* fix: fmt
* fix: fmt
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-03-24 17:57:11 +00:00
c47a252c15
feat: Check in generated WAL flatbuffers code
2021-03-22 09:38:59 -04:00
52e415ae1b
fix: Verify flatbuffers when restoring from a file, and not after that
2021-03-22 09:38:59 -04:00
7c6f543d35
fix: wb::ColumnValue is now a set of associated consts, not a real Rust enum
2021-03-22 09:38:58 -04:00
24af745b23
chore: Upgrade to flatbuffers 0.8
2021-03-22 09:38:58 -04:00
6e1795fda0
refactor: Move some types (not yet exposed to clients) into internal_types ( #1015 )
...
* refactor: Move some types (not yet exposed to clients) into internal_types
* docs: Add README.md explaining the rationale
* refactor: remove some stragglers
* fix: fix benches
* fix: Apply suggestions from code review
Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
* fix: add clippy lints
* fix: fmt
* docs: Apply suggestions from code review
fix typos
Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-03-19 16:27:57 +00:00
4781317647
feat: Management API + CLI command to create a new chunk (rollover partition) ( #990 )
...
* feat: Management API + CLI command to create a new chunk (rollover partition)
* fix: Update tests/end_to_end_cases/management_api.rs
fix typo
* fix: logical merge conflict
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-03-15 16:41:18 +00:00
65f7a1ac5b
fix: use consistent crate versions ( #989 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-03-15 15:42:19 +00:00
6ac7e2c1a7
feat: Add management API and CLI to list chunks ( #968 )
...
* feat: Add management API and CLI to list chunks
* fix: Apply suggestions from code review
Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
* fix: add comment to protobuf
* fix: fix comment
* fix: fmt, fixup merge errors
* fix: fascinating type dance with prost generated types
* fix: clippy
* fix: move command to influxdb_iox database chunk list
Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
2021-03-12 13:56:14 +00:00
488252d9f1
fix: test
2021-03-09 07:52:52 -08:00
09c4f88758
fix: cargo fmt
2021-03-09 07:45:05 -08:00
abb991ffca
fix: add tests to unsigned integer parsing in LP and MUB
2021-03-08 15:29:05 -08:00
ac1b0c04ae
fix(line-protocol): add unsigned integer field type
...
Fixes #904
The line protocol parser was lacking the unsigned integer type, which
suffixes values with `u`. This adds unsigned integer support to the line
protocol parser, and fills a few corresponding gaps in the mutable
buffer.
2021-03-08 09:59:12 -08:00
746373a687
refactor: Remove mutable_buffer crate dependency on query crate ( #927 )
2021-03-05 11:34:27 +00:00
a6965769b4
refactor: Remove impl of `query::PartititionChunk` in mutable_buffer: This PR ( #923 )
2021-03-05 10:51:08 +00:00
b1e5cfedf7
refactor: Move chunk predicate creation from query::Predicate into server crate ( #922 )
...
* refactor: Create `ChunkPredicate` predicate in the Server crate
* refactor: move chunk predicate out of chunk
* fix: remove comment in server/src/db/pred.rs
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-03-04 23:23:43 +00:00
3be5c26f92
refactor: Remove impl of `query::Database` in mutable_buffer ( #914 )
2021-03-04 22:02:42 +00:00
945d2f8d45
refactor: Remove query stuff from mutable buffer ( #912 )
...
* refactor: remove query code from mutable_buffer database.rs
* refactor: remove query code from mutable_buffer table.rs
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-03-04 21:42:56 +00:00
8b1f100df3
feat: make read_group and read_window_aggregate work across chunks ( #905 )
...
* feat: make read_group and read_window_aggregate work across chunks
* refactor: Apply suggestions from code review
Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
* refactor: Update query/src/frontend/influxrpc.rs
Improve logic and use strings directly
Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
* fix: fmt
Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-03-04 17:06:31 +00:00
7d8d00781c
feat: Make read_filter work for mutable buffer and read buffer ( #882 )
...
* feat: port read_filter to InfluxRPCPlanner
* fix: remove commented out vestigal test
* fix: Apply suggestions from code review
Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
* fix: fmt
* fix: Update arrow_deps/src/util.rs
Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
2021-03-01 16:50:29 +00:00
18de3bdcab
chore: merge main into branch
...
Merge branch 'main' into ntran/optimize_column_selection
2021-02-26 15:29:43 -05:00
f37e5846aa
feat: fmt auto fix
2021-02-26 14:56:10 -05:00
eb81975151
feat: Optimize Column Selection
2021-02-26 14:28:46 -05:00
12deacd8a0
refactor: move SeriesSetPlans into its own module ( #878 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-02-25 23:12:39 +00:00
8fb7651719
feat: Port tag_values to the InfluxRPCPlanner ( #859 )
...
* feat: Port tag_values to the InfluxRPCPlanner
* refactor: merge imports
* refactor: rename column_names to tag_column_names for clarity
* fix: Update query/src/frontend/influxrpc.rs
Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
* refactor: use ensure!
Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
* refactor: less silly whitespace
Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
* fix: code review comments
Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-02-24 23:11:22 +00:00
ed7859e182
refactor: rename tag_column_names --> tag_keys in InfluxRPCPlanner ( #860 )
2021-02-23 17:04:53 +00:00
56c515a063
refactor: Simplify partition sorting code
2021-02-22 11:19:47 -05:00
04e2631f26
refactor: Switch to explicit Arc::clone
2021-02-22 11:10:07 -05:00
d0707725cf
Merge remote-tracking branch 'origin/main' into pd-mutable-buffer-data-eviction
2021-02-22 10:21:59 -05:00
92eb8b9e85
refactor: make certain Database method sync
...
A couple of methods don't seem to have any await points in their
implementations, so it feels like they could just be `sync`.
2021-02-19 17:14:17 +00:00
9b91e0624c
feat: implement field_columns plan ( #819 )
...
* feat: implement field_columns plan
* fix: fix doc tests
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-02-17 20:43:24 +00:00
94a93e56ff
feat: implement `tag_keys` in gRPC planner and across mutable buffer ( #795 )
...
* feat: move tag_column_names into rpc planner
* fix: Apply suggestions from code review
Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
* fix: Apply suggestions from code review
Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
* fix: compile error
* refactor: remove PassThrough error type
* fix: Avoid extra layers of errors in mutable buffer chunk
* fix: use HashMap::get rather than values() and find
* fix: push filtering down to chunk in gRPC planner
* fix: fixup trait bounds to be non-silly
* fix: remove incorrect comment
* fix: remove cruft
* fix: clippy + fmt
* fix: correct comment
Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
2021-02-15 16:47:52 +00:00
8a85158a98
refactor: add arc clone lint
2021-02-15 12:40:19 +00:00
dc465e5d02
feat: Add function to check db size and drop partitions
...
Adds functionality to the server Db to check the mutable buffer size and drop partitions based on the database rules.
2021-02-13 17:19:40 -06:00
83bfa6d949
feat: Add created_at, last_write_at tracking to partition and sorting
...
This commit adds created_at and last_write_at instants to partitions in the mutable buffer. It adds a method on the mutable buffer database to get back the partitions in sorted order based on either the created_at or last_write_at instants. Ordering based on the summary stats from a column are still left to do.
Finally, it modifies the helper function to create replicated write to take a Partitioner trait that can generate partition keys based on lines, rather than taking the DatabaseRules struct directly. This makes it easier to write test cases where data is split into multiple partitions in the mutable buffer.
2021-02-13 17:19:40 -06:00
9e39e91139
chore: Cleaning things in prep for rust 2021
...
Also remove a NUL byte in a test string literal; some editors drop them.
2021-02-12 16:48:17 +00:00
a316b16960
feat: Change table_names to return either Some(set) or None, rather than a plan (try 2) ( #776 )
...
* feat: Change table_names to return either Some(set) or None, rather than a plan
* docs: improve comments
* docs: Apply suggestions from code review
Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
* fix: merge conflict
* fix: don't clone a string unless needed
Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
2021-02-09 12:20:59 -05:00
e5da2ab589
feat: add ability to roll up summaries from multiple chunks ( #763 )
2021-02-08 18:11:21 -05:00
47bc28460e
refactor: rename partition, table, and column in parition_meta for clarity ( #757 )
...
* refactor: rename partition, table, and column in parition_meta for clarity
2021-02-05 08:00:22 -05:00
de7bc7d645
feat: add column name to the partition metadata summaries ( #755 )
2021-02-05 07:20:16 -05:00
3ec483b769
refactor: Reduce async in mutable buffer, use std::sync ( #749 )
...
* refactor: Reduce async in mutable buffer, use std::sync
* fix: logical confict with new code
2021-02-05 06:47:40 -05:00
fbf776c6b3
chore: Clean up Cargo.tomls ( #754 )
...
* fix: test_helpers crate should only be a dev-dep
* fix: object_store no longer has a build script, so no longer needs a build dep
* chore: Alphabetize all Cargo.tomls
2021-02-04 18:56:02 -05:00
Jacob Marble
Carol (Nichols || Goulding)
Andrew Lamb
Raphael Taylor-Davies
Jacob Marble
Nga Tran
NGA TRAN
Edd Robinson
Paul Dix
Marko Mikulicic
Paul Dix
Carol (Nichols || Goulding)