Influxdata
/
influxdb
mirror of https://github.com/influxdata/influxdb.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
influxdb/influxdb3_write/Cargo.toml

41 lines
957 B
TOML
Raw Normal View History

feat: Add basic Edge server structure (#24552) * WIP: basic influxdb3 command and http server * WIP: write lp, buffer, query out * WIP: test write & query on influxdb3_server, fix warnings * WIP: pull write buffer and catalog into separate crate * WIP: sketch out types used for write: buffer, wal, persister * WIP: remove a bunch of old IOx stuff and fmt
2024-01-08 16:50:59 +00:00
[package]
name = "influxdb3_write"
version.workspace = true
authors.workspace = true
edition.workspace = true
license.workspace = true
[dependencies]
feat: Split out shared core crates from Edge (#24714) This commit is a major refactor for the code base. It mainly does four things: 1. Splits code shared between the internal IOx repository and this one into it's own repo over at https://github.com/influxdata/influxdb3_core 2. Removes any docs or anything else that did not relate to this project 3. Reorganizes the Cargo.toml files to use the top level Cargo.toml to declare dependencies and versions to keep all crates in sync and sets all others to use `<dep>.workspace = true` unless it's an optional dependency 4. Set the top level Cargo.toml to point to the core crates as git dependencies With this any changes specific to Edge will be contained here, updating deps will be a PR over in `influxdata/influxdb3_core`, and we can prove out the viability for this model to use for IOx.
2024-02-29 21:21:41 +00:00
# Core Crates
data_types.workspace = true
datafusion_util.workspace = true
influxdb-line-protocol.workspace = true
iox_catalog.workspace = true
iox_query.workspace = true
observability_deps.workspace = true
schema.workspace = true
feat: Add basic Edge server structure (#24552) * WIP: basic influxdb3 command and http server * WIP: write lp, buffer, query out * WIP: test write & query on influxdb3_server, fix warnings * WIP: pull write buffer and catalog into separate crate * WIP: sketch out types used for write: buffer, wal, persister * WIP: remove a bunch of old IOx stuff and fmt
2024-01-08 16:50:59 +00:00
feat: Split out shared core crates from Edge (#24714) This commit is a major refactor for the code base. It mainly does four things: 1. Splits code shared between the internal IOx repository and this one into it's own repo over at https://github.com/influxdata/influxdb3_core 2. Removes any docs or anything else that did not relate to this project 3. Reorganizes the Cargo.toml files to use the top level Cargo.toml to declare dependencies and versions to keep all crates in sync and sets all others to use `<dep>.workspace = true` unless it's an optional dependency 4. Set the top level Cargo.toml to point to the core crates as git dependencies With this any changes specific to Edge will be contained here, updating deps will be a PR over in `influxdata/influxdb3_core`, and we can prove out the viability for this model to use for IOx.
2024-02-29 21:21:41 +00:00
# crates.io dependencies
arrow.workspace = true
async-trait.workspace = true
byteorder.workspace = true
chrono.workspace = true
crc32fast.workspace = true
crossbeam-channel.workspace = true
datafusion.workspace = true
object_store.workspace = true
parking_lot.workspace = true
parquet.workspace = true
thiserror.workspace = true
tokio.workspace = true
serde.workspace = true
serde_json.workspace = true
snap.workspace = true
bytes.workspace = true
futures-util.workspace = true
feat: add basic wal implementation for Edge (#24570) * feat: add basic wal implementation for Edge This WAL implementation uses some of the code from the wal crate, but departs pretty significantly from it in many ways. For now it uses simple JSON encoding for the serialized ops, but we may want to switch that to Protobuf at some point in the future. This version of the wal doesn't have its own buffering. That will be implemented higher up in the BufferImpl, which will use the wal and SegmentWriter to make data in the buffer durable. The write flow will be that writes will come into the buffer and validate/update against an in memory Catalog. Once validated, writes will get buffered up in memory and then flushed into the WAL periodically (likely every 10-20ms). After being flushed to the wal, the entire batch of writes will be put into the in memory queryable buffer. After that responses will be sent back to the clients. This should reduce the write lock pressure on the in-memory buffer considerably. In this PR: - Update the Wal, WalSegmentWriter, and WalSegmentReader traits to line up with new design/understanding - Implement wal (mainly just a way to identify segment files in a directory) - Implement WalSegmentWriter (write header, op batch with crc, and track sequence number in segment, re-open existing file) - Implement WalSegmentReader * refactor: make Wal return impl reader/writer * refactor: clean up wal segment open * fix: WriteBuffer and Wal usage Turn wal and write buffer references into a concrete type, rather than dyn. * fix: have wal loading ignore invalid files
2024-01-12 16:52:28 +00:00
[dev-dependencies]
feat: Split out shared core crates from Edge (#24714) This commit is a major refactor for the code base. It mainly does four things: 1. Splits code shared between the internal IOx repository and this one into it's own repo over at https://github.com/influxdata/influxdb3_core 2. Removes any docs or anything else that did not relate to this project 3. Reorganizes the Cargo.toml files to use the top level Cargo.toml to declare dependencies and versions to keep all crates in sync and sets all others to use `<dep>.workspace = true` unless it's an optional dependency 4. Set the top level Cargo.toml to point to the core crates as git dependencies With this any changes specific to Edge will be contained here, updating deps will be a PR over in `influxdata/influxdb3_core`, and we can prove out the viability for this model to use for IOx.
2024-02-29 21:21:41 +00:00
# Core Crates
arrow_util.workspace = true
test_helpers.workspace = true