edd606aa3b
feat: Serialize using protobuf instead of json
2022-11-23 17:07:49 -05:00
a85abadfd5
fix: Define and use a sequence number represented as u64 instead
2022-11-23 17:07:49 -05:00
d7ffa65918
feat: Add an id method to OpenSegmentFileWriter
2022-11-23 17:07:49 -05:00
35576b26b5
refactor: Rename placeholder fnamex to build_segment_path
2022-11-23 17:07:49 -05:00
40dd575de9
feat: Add segment_id metadata to WriteSummary
2022-11-23 17:07:48 -05:00
8618264fe6
fix: Remove unused created_at field
2022-11-23 17:07:48 -05:00
1ba0f193a9
fix: fsync parent directory before returning that new segment file was created
2022-11-23 17:07:48 -05:00
d5b439732d
feat: Implement delete WAL file
2022-11-23 17:07:48 -05:00
a2c25f5191
docs: Improve some descriptions
2022-11-23 17:07:48 -05:00
cdec449963
fix: Return single SequencedWalOps for now, will batch later
2022-11-23 17:07:48 -05:00
db9a43f1b3
fix: Create types for 3 different tasks of a WAL
2022-11-23 17:07:48 -05:00
de9c02caad
fix: Use bytes_written rather than file metadata
2022-11-23 17:07:48 -05:00
1354136b93
fix: fsync and panic if it fails
2022-11-23 17:07:47 -05:00
ee99b93d09
fix: Make checksum private
2022-11-23 17:07:47 -05:00
be2207bcd3
refactor: Rename to clarify closed segment types
2022-11-23 17:07:47 -05:00
4335edc6d1
refactor: Rename to clarify open segment types
2022-11-23 17:07:47 -05:00
70b12a8112
fix: Clarify and simplify the public API
2022-11-23 17:07:47 -05:00
1f5fd43829
fix: cross ps and dot qs
2022-11-23 17:07:47 -05:00
0ae40ba025
fix: move writing to blocking and wire up some things
2022-11-23 17:07:47 -05:00
2e6f269840
refactor: move blocking code to new file
2022-11-23 17:07:47 -05:00
49633842a5
test: Rotate and reading of closed segments
2022-11-23 17:07:46 -05:00
58d9ada90c
feat: Stream the reading
2022-11-23 17:07:46 -05:00
51aa2a118b
feat: Add buffering to File reader
2022-11-23 17:07:46 -05:00
5be0315db9
fix: Move blocking read code to separate task
2022-11-23 17:07:46 -05:00
48e5000ec6
fix: Make test data deterministic
2022-11-23 17:07:46 -05:00
4bb8d3a408
fix: Update to DmlWrite only containing IDs
2022-11-23 17:07:46 -05:00
42ed02ea31
feat: Actually write entries to the open segment file
2022-11-23 17:07:46 -05:00
ebe605a246
fix: Notes and questions
2022-11-23 17:07:46 -05:00
497afc5c91
fix: Change name from e to clarify this covers the Ok case too
2022-11-23 17:07:45 -05:00
2df915b189
fix: Notes and questions
2022-11-23 17:07:45 -05:00
1ee14155a5
refactor: associate functions with SegmentFileReader
2022-11-23 17:07:45 -05:00
41aa4f09fc
fix: remove unneeded usage of trait object
2022-11-23 17:07:45 -05:00
e81a84dd40
fix: don't require a reference just to make an owned value
2022-11-23 17:07:45 -05:00
5104dda6fb
fix: Notes and questions
2022-11-23 17:07:45 -05:00
2852d1ae89
fix: simplify error check
2022-11-23 17:07:45 -05:00
05ae070da2
fix: remove impossible failure case
2022-11-23 17:07:45 -05:00
eb6abb5d67
fix: avoid unneeded allocation
2022-11-23 17:07:45 -05:00
651b7a1ce6
fix: use write_all instead of write
2022-11-23 17:07:44 -05:00
b96de5983d
fix: don't require a reference just to make an owned value
2022-11-23 17:07:44 -05:00
3eda17ea8b
fix: Make SegmentId into a newtype; test the open segment file is created
2022-11-23 17:07:44 -05:00
eea7f113fb
fix: Start implementing traits on Wal to enable outside use
2022-11-23 17:07:44 -05:00
d7134d4846
fix: Cleanups, notes, and clarifications noticed while reading
2022-11-23 17:07:44 -05:00
ef6eda6399
fix: Alphabetize deps
2022-11-23 17:07:44 -05:00
8e414b7c35
fix: Remove custom PartialEq impl; derive works for me 🤷
2022-11-23 17:07:44 -05:00
5c3c0accb1
fix: Add workspace-hack to wal crate
2022-11-23 17:07:44 -05:00
28782caea2
fix: Resolve warnings and format wal code
2022-11-23 17:07:43 -05:00
db63dc55a2
feat: Initial WAL implementation
2022-11-23 17:07:43 -05:00
f0b93c5c8c
refactor: Rename the wal crate to write_buffer
2021-04-21 17:43:03 +00:00
1997324344
feat: mutable buffer snapshotting ( #1179 )
...
* feat: mutable buffer snapshotting
* chore: review feedback
2021-04-13 12:14:54 +00:00
82588d5c72
fix: Don't return Result from test functions
2021-04-07 12:40:00 -04:00
80d55d0829
chore: rename tracing_deps to observability_deps
...
OpenTelemetry makes this necessary.
2021-04-02 13:14:30 -07:00
0b880d3534
chore: Group all tracing-related crates under one crate for easier upgrade management
2021-04-02 09:54:39 -04:00
2bd09612bd
refactor: Enable parking lot for dependencies
2021-02-22 13:37:35 -05:00
dfe6e50e26
refactor: add arc clone lint
2021-02-15 12:52:12 +00:00
fbf776c6b3
chore: Clean up Cargo.tomls ( #754 )
...
* fix: test_helpers crate should only be a dev-dep
* fix: object_store no longer has a build script, so no longer needs a build dep
* chore: Alphabetize all Cargo.tomls
2021-02-04 18:56:02 -05:00
f3bd8bd0e3
chore: update deps (tokio 1.0 and ecosystem) ( #707 )
...
* chore: Update arrow + tokio deps
* chore: Use bleeding edge azure
* chore: Update aws + other deps
* fix: fmt
* fix: Switch to in-house version of routerify
* fix: Upgrade to hyper 0.14
The hyper::error module is now private; hyper::Error is the public
re-export
* fix: Upgrade cloud storage to get tokio upgrade
* fix: Upgrade open_telemetry
* fix: Do not call `panic::set_hook` during another panic
Doing so leads to a double panic which aborts the process.
* fix: new h2 error who dis
Co-authored-by: Carol (Nichols || Goulding) <carol.nichols@integer32.com>
Co-authored-by: Jake Goulding <jake.goulding@integer32.com>
2021-01-29 16:11:55 -05:00
5fa77c32cc
feat: Add "Chunks" to the Mutable Buffer ( #596 )
...
* refactor: Update docs, remove unused field
* refactor: rename partition -> chunk
* feat: Introduce new partition, which is a holder for Chunks
* refactor: Remove use of wal from mutable database
* refactor: cleanups, remove last direct use of chunks
* fix: delete old benchmarks
* fix: clippy sacrifice
* docs: tidy up comments
* refactor: remove unused error types
* chore: remove commented out tests
2020-12-28 07:10:25 -05:00
6f473984d0
style: wrap comments
...
Runs rustfmt with the new config.
2020-12-11 18:22:26 +00:00
831a0875d6
chore: update to latest arrow + Rust nightly-2020-11-14 ( #454 )
...
* chore: update to latest arrow + Rust nightly-2020-11-14
* chore: update ci
* fix: update for clippy lints
* fix: Allow redundant_field_names in generated types crate
* fix: clippy about try_for_each
* fix: clippy uneeded-collect
* fix: clippy about default values
* fix: clippy mathces --> matches!
* fix: clippy sort --> sort_by_key
* fix: clippy about default values again
2020-11-16 11:48:42 -05:00
b745a180a4
refactor: rename delorean --> InfluDB IOx ( #417 )
2020-11-05 13:51:04 -05:00
a52e0001c5
refactor: rename all crates that start with`delorean_` in preparation for rename ( #415 )
...
* refactor: rename delorean_cluster --> cluster
* refactor: rebane delorean_generated_types --> generated_types
* refactor: rename delorean_write_buffer --> write_buffer
* refactor: rename delorean_ingest --> ingest
* refactor: rename delorean_storage --> storage
* refactor: rename delorean_tsm --> tsm
* refactor: rename delorean_test_helpers --> test_helpers
* refactor: rename delorean_arrow --> arrow_deps
* refactor: rename delorean_line_parser --> influxdb_line_protocol
2020-11-05 13:44:36 -05:00
ff824a5477
refactor: rename delorean_wal --> wal, conslidate wal_writer ( #411 )
2020-11-05 09:25:29 -05:00
Carol (Nichols || Goulding)
Jake Goulding
Paul Dix
Carol (Nichols || Goulding)
Raphael Taylor-Davies
Jacob Marble
Edd Robinson
Carol (Nichols || Goulding)
Andrew Lamb
Dom