Commit Graph

5 Commits (7de5d35baba27d104fd407930f18378068a2da58)

Author SHA1 Message Date
Paul Dix 3b1f045f44 feat: add serialization to wal buffer segments
Adds serialization with compression and checksum for WAL buffer segments.

This required a weird structure where the flatbuffer bytes of ReplicatedWrite were kept as a raw payload. I did this because otherwise each of the replicated writes would have been rebuilt in the segment.

The other thing that isn't ideal is that deserializing a segment actually marshals it into a Rust struct as opposed to keeping the entire thing as raw flatbuffers. We could update this later to have a concept of an open segment (regular rust stuct) and closed segments that are just the flatbuffers.
2021-01-13 18:15:34 -05:00
Paul Dix b38fb80c14 chore: ensure sequence increasing in wal segment 2020-11-20 08:35:35 -05:00
Paul Dix 0deee2c0db feat: Implement WAL in-memory buffer
This splits the cluster package out into server and buffer modules. The WAL buffer is in-memory and split into segments. Follow on commits will implement it in the server and add persistence to object storage.
2020-11-19 19:35:17 -05:00
Andrew Lamb b745a180a4
refactor: rename delorean --> InfluDB IOx (#417) 2020-11-05 13:51:04 -05:00
Andrew Lamb a52e0001c5
refactor: rename all crates that start with`delorean_` in preparation for rename (#415)
* refactor: rename delorean_cluster --> cluster

* refactor: rebane delorean_generated_types --> generated_types

* refactor: rename delorean_write_buffer --> write_buffer

* refactor: rename delorean_ingest --> ingest

* refactor: rename delorean_storage --> storage

* refactor: rename delorean_tsm --> tsm

* refactor: rename delorean_test_helpers --> test_helpers

* refactor: rename delorean_arrow --> arrow_deps

* refactor: rename delorean_line_parser --> influxdb_line_protocol
2020-11-05 13:44:36 -05:00