a064820a70
fix: code comment should match the code
...
Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
2021-07-15 17:20:46 +02:00
b5428e53a5
refactor: write buffer testing + better mocking
...
This refactors the write buffer a bit for:
- **Testing:** Add generic tests for the Kafka and the mocking
implementation. The same interface can be used easily add new
implementations (e.g. via Redis, filesystem, ...).
- **Partition on Write:** The caller of the writer operation must now
specify the partition/sequencer ID. The implicit partitioning of the
Kafka writer would have lead to broken data since we must never spill
entries w/ the same primary key over multiple partitions. At the
moment we will only use partition 0 but we can easily implement
better logic in the future.
- **Improved Mocking:** The mocked implementation now simulates a system
that feels more real. Especially the handling around multiple streams
and "write while read" has been improved. This will be helpful for
testing and for new features like seeking (during replay). A solid
realistic mock also helps us to ensure that the tests using the mock
do not rely on unrealistic behavior too much.
2021-07-15 17:20:45 +02:00
8d23dd6d6d
fix: Set kafka max message size in client
2021-07-14 14:46:49 +02:00
9cb9ae0874
chore: move write buffer into its own crate
2021-07-14 14:09:18 +02:00
f4a9a5ae56
fix: Remove write buffer
2021-06-04 14:40:17 -04:00
eddc9319ff
docs: deny broken intradoc links
2021-04-27 13:22:28 +02:00
938acb98e8
refactor: Rename WAL within the types, variables, and comments in the write_buffer crate
2021-04-21 17:43:03 +00:00
f0b93c5c8c
refactor: Rename the wal crate to write_buffer
2021-04-21 17:43:03 +00:00
48c43b136c
refactor: rename write_buffer --> mutable_buffer ( #595 )
...
* refactor: git mv write_buffer mutable_buffer
* refactor: update crate name references
* refactor: update some more references
2020-12-22 10:49:53 -05:00
263af1eeac
feat: implement read_group in the write_buffer ( #583 )
...
* feat: implement read_group in the write_buffer
* fix: Apply suggestions from code review
Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
* fix: rustfmt
* fix: adjust tests for min/max
* fix: Update write_buffer/src/table.rs
Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
2020-12-22 09:03:20 -05:00
28eac06d8f
refactor: Organize window_bounds the same as selector functions ( #594 )
...
* refactor: Organize window_bounds the same as selector functions
* fix: add missing file
2020-12-21 12:51:36 -05:00
bb96142564
chore: Update arrow dependencies, remove custom min/max implementation ( #585 )
...
* chore: Update arrow dependency
* fix: Update code for changes in datafusion
* fix: use arrow version of min_boolean
2020-12-21 12:31:39 -05:00
699504a022
feat: implement read_group for Sum, Count and Mean aggregates ( #557 )
...
* feat: Implement read_group for "normal" aggregates
* test: add tests
2020-12-15 09:35:00 -05:00
a6d2c13888
chore: Update arrow + other depenencies ( #540 )
...
* chore: Update arrow + other depenencies
* chore: Update write_buffer and query crate
2020-12-15 08:46:27 -05:00
6f473984d0
style: wrap comments
...
Runs rustfmt with the new config.
2020-12-11 18:22:26 +00:00
ea6b2f6bc8
refactor: remove minor code duplication ( #555 )
2020-12-11 11:18:00 -05:00
fa3ecbd4ed
feat: Implement write buffer to Parquet snapshotting ( #526 )
...
* feat: Implement write buffer to Parquet snapshotting
This introduces snapshot to the server packages to manage snapshotting. It also introduces a new trait for representing a Partition. There is a very crude API wired up in http_routes for testing purposes. Follow on work will bring the server package into http_routes and rework the snapshot API.
2020-12-08 14:20:43 -05:00
4ec75a4f22
fix: Fix gRPC panic` when multiple field selections are provided ( #523 )
...
* fix: do not assert when multiple fields are selected
* fix: clippy
* fix: write unit test, fix bug
* fix: tweak comments
2020-12-03 12:31:02 -05:00
ecc4eee8e1
refactor: Move SQL functions into is own trait ( #511 )
...
* refactor: remove uneeded function table_to_arrow from Trait
* refactor: Move SQL functions into is own trait
2020-12-02 08:23:37 -05:00
5ef499bb63
refactor: rename Database --> TSDatabase to better reflect its purpose ( #510 )
...
* refactor: rename Database --> TSDatabase to better reflect its purpose
* refactor: rename field_columns to field_column_names
* fix: clippy?
2020-12-01 12:37:11 -05:00
1646397891
refactor: consolidate GroupedSeriesSet and SeriesSet ( #502 )
2020-11-30 14:23:58 -05:00
20f421e9c6
fix: Do not send GroupFrames in response to read_window_aggregate ( #497 )
...
* fix: Do not send GroupFrames in response to read_window_aggregate
* fix: clippy and test
2020-11-30 05:59:05 -05:00
8c8af66d4d
refactor: use datafusion expression building code in write_buffer ( #484 )
2020-11-25 14:41:39 -05:00
0eaa90e89d
feat: Hook up read_window_aggregate into the write_buffer, end-to-end tests ( #483 )
...
* feat: read_window_aggregate_plans
* fix: clippy sacrifice
* fix: clippy
* fix: clippy
2020-11-25 10:20:49 -05:00
9f6427c94f
refactor: query/src/groupby.rs -> query/src/group_by.rs ( #477 )
...
* refactor: query/src/groupby.rs -> query/src/group_by.rs
* refactor: update references
2020-11-25 06:43:11 -05:00
cdb26e60e4
refactor: rename `storage` crate to `query` to better reflect what it is ( #475 )
...
* refactor: rename storage --> query
* refactor: update a few more referenes
2020-11-24 14:19:29 -05:00
85921fe401
feat: Implement gRPC and storage interface for read_group_aggregate ( #474 )
...
* feat: Implement gRPC and storage interface for read_group_aggregate
* fix: clippy
* docs: Tweak comments
* fix: moar clippy
* fix: fmt
* docs: Apply doc improvement suggestions from code review
Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
* refactor: improve error creation
* refactor: use match instead of if
* refactor: clearer match
* refactor: clean up storage aggregate matching
Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
2020-11-24 13:41:11 -05:00
597933622d
fix: improve error messages with more context ( #455 )
2020-11-16 16:40:29 -05:00
2fa0e03162
fix: Use datafusion optimizer in IOx query plans ( #439 )
...
* chore: update arrow dep to 8e4d9ebef3
2020-11-11 18:06:21 -05:00
a52e0001c5
refactor: rename all crates that start with`delorean_` in preparation for rename ( #415 )
...
* refactor: rename delorean_cluster --> cluster
* refactor: rebane delorean_generated_types --> generated_types
* refactor: rename delorean_write_buffer --> write_buffer
* refactor: rename delorean_ingest --> ingest
* refactor: rename delorean_storage --> storage
* refactor: rename delorean_tsm --> tsm
* refactor: rename delorean_test_helpers --> test_helpers
* refactor: rename delorean_arrow --> arrow_deps
* refactor: rename delorean_line_parser --> influxdb_line_protocol
2020-11-05 13:44:36 -05:00
Marco Neumann
Marko Mikulicic
Carol (Nichols || Goulding)
Andrew Lamb
Dom
Paul Dix