Influxdata
/
influxdb
mirror of https://github.com/influxdata/influxdb.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
influxdb/router/Cargo.toml

63 lines
1.8 KiB
TOML
Raw Normal View History

feat: new router2 crate This commit adds an almost-empty router2 crate containing enough of a skeleton to plumb into the IOx CLI/server runner.
2022-01-12 14:13:20 +00:00
[package]
fix: Rename router2 to router
2022-05-06 18:51:52 +00:00
name = "router"
feat: new router2 crate This commit adds an almost-empty router2 crate containing enough of a skeleton to plumb into the IOx CLI/server runner.
2022-01-12 14:13:20 +00:00
version = "0.1.0"
edition = "2021"
[dependencies]
feat: abstract DML handler Defines the DmlHandler trait responsible for processing a request in some abstract way, decoupling the HTTP/gRPC request handlers from the underlying routing logic.
2022-01-14 11:34:42 +00:00
async-trait = "0.1"
feat: router2 HTTP handlers Implements the HTTP v2 write API endpoint for router2.
2022-01-14 13:34:16 +00:00
bytes = "1.1"
fix: Rename data_types2 to data_types
2022-05-05 19:29:24 +00:00
data_types = { path = "../data_types" }
feat: new router2 crate This commit adds an almost-empty router2 crate containing enough of a skeleton to plumb into the IOx CLI/server runner.
2022-01-12 14:13:20 +00:00
dml = { path = "../dml" }
feat: router2 HTTP handlers Implements the HTTP v2 write API endpoint for router2.
2022-01-14 13:34:16 +00:00
flate2 = "1.0"
chore(deps): bump futures from 0.3.19 to 0.3.21 (#3706) Bumps [futures](https://github.com/rust-lang/futures-rs) from 0.3.19 to 0.3.21. - [Release notes](https://github.com/rust-lang/futures-rs/releases) - [Changelog](https://github.com/rust-lang/futures-rs/blob/master/CHANGELOG.md) - [Commits](https://github.com/rust-lang/futures-rs/compare/0.3.19...0.3.21) --- updated-dependencies: - dependency-name: futures dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-02-10 09:19:19 +00:00
futures = "0.3.21"
feat: new router2 crate This commit adds an almost-empty router2 crate containing enough of a skeleton to plumb into the IOx CLI/server runner.
2022-01-12 14:13:20 +00:00
generated_types = { path = "../generated_types" }
chore: Update hashbrown (#3551) * chore: Update hashbrown * fix: hakari Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-01-27 15:34:10 +00:00
hashbrown = "0.12"
feat: new router2 crate This commit adds an almost-empty router2 crate containing enough of a skeleton to plumb into the IOx CLI/server runner.
2022-01-12 14:13:20 +00:00
hyper = "0.14"
feat: schema validation Implements a write schema validation DML handler, denying requests that conflict with the schema within the global catalog. Additive schema changes are accepted, incrementally updating the global catalog schema. Deletes are passed through unchanged and unvalidated.
2022-01-28 16:25:20 +00:00
iox_catalog = { path = "../iox_catalog" }
feat: add initial catalog service to router (#4316) Create new crate for iox_catalog_service. Add rpc to return parquet_file records by partition id. Add CatalogService to router2. The catalog service will be added to over time to provide access to the catalog over gRPC.
2022-04-14 17:39:18 +00:00
iox_catalog_service = { path = "../iox_catalog_service"}
fix: Rename data_types2 to data_types
2022-05-05 19:29:24 +00:00
iox_time = { path = "../iox_time" }
feat: new router2 crate This commit adds an almost-empty router2 crate containing enough of a skeleton to plumb into the IOx CLI/server runner.
2022-01-12 14:13:20 +00:00
metric = { path = "../metric" }
refactor: avoid constructing DmlOperation Instead of converting the set of MutableBatches into a DmlOperation to shard into more DmlOperation instances, the sharder can operate directly on the MutableBatches.
2022-01-14 16:52:03 +00:00
mutable_batch = { path = "../mutable_batch" }
feat: abstract DML handler Defines the DmlHandler trait responsible for processing a request in some abstract way, decoupling the HTTP/gRPC request handlers from the underlying routing logic.
2022-01-14 11:34:42 +00:00
mutable_batch_lp = { path = "../mutable_batch_lp" }
feat: grpc write handler Implements the gRPC write handler endpoint.
2022-02-23 15:41:45 +00:00
mutable_batch_pb = { version = "0.1.0", path = "../mutable_batch_pb" }
feat: add object store service to router (#4338) Add method to catalog to get parquet file by object store id. Add gRPC service for object store to get a file from by its uuid. Add the object store service to router2 with object store config.
2022-04-16 17:58:31 +00:00
object_store = { path = "../object_store" }
feat: abstract DML handler Defines the DmlHandler trait responsible for processing a request in some abstract way, decoupling the HTTP/gRPC request handlers from the underlying routing logic.
2022-01-14 11:34:42 +00:00
observability_deps = { path = "../observability_deps" }
chore: upgrade rskafka and parking_lot (#3592)
2022-02-01 11:50:42 +00:00
parking_lot = "0.12"
feat(router2): implement delete API handler Adds support for the HTTP v2 delete API endpoint.
2022-01-14 16:01:01 +00:00
predicate = { path = "../predicate" }
feat: grpc write handler Implements the gRPC write handler endpoint.
2022-02-23 15:41:45 +00:00
schema = { version = "0.1.0", path = "../schema" }
feat: router2 HTTP handlers Implements the HTTP v2 write API endpoint for router2.
2022-01-14 13:34:16 +00:00
serde = "1.0"
serde_urlencoded = "0.7"
feat: Interrogate schema from querier (as well as router) (#4557) * refactor: move SchemaService into `service_grpc_schema` * feat: implement schema gRPC for querier * chore: Run cargo hakari tasks Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
2022-05-10 20:55:58 +00:00
service_grpc_schema = { path = "../service_grpc_schema" }
refactor: rename `iox_object_store_service` to `service_grpc_object_store` for consisntency (#4572) Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-05-12 14:22:18 +00:00
service_grpc_object_store = { path = "../service_grpc_object_store" }
feat: sharder trait & impl This commit defines the Sharder trait that should allow us to implement multiple sharding strategies over a defined set of input types (such as a MutableBatch for writes, DeletePredicate for deletes, etc). This commit also includes a jump hash implementation that consistently shards (table name, namespace) tuples to a given shard for all input types.
2022-01-18 18:46:58 +00:00
siphasher = "0.3"
feat: new router2 crate This commit adds an almost-empty router2 crate containing enough of a skeleton to plumb into the IOx CLI/server runner.
2022-01-12 14:13:20 +00:00
thiserror = "1.0"
feat: router2 HTTP handlers Implements the HTTP v2 write API endpoint for router2.
2022-01-14 13:34:16 +00:00
tokio = { version = "1", features = ["rt-multi-thread", "macros"] }
chore: Update arrow, arrow-flight, parquet, tonic, prost, etc (#4357) * chore: Update datafusion * chore: Update arrow/arrow-flight/parquet to 12 * chore: update datafusion correctly * chore: Update prost, tonic, and dependents * fix: Fixup some api changes * fix: Update test output in db * fix: Update test output in parquet_file * fix: remove old pbjson types * fix: Add "--experimental_allow_proto3_optional" flag * chore: Run cargo hakari tasks * fix: compile error * chore: Update heappy Co-authored-by: CircleCI[bot] <circleci@influxdata.com> Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-04-20 11:12:17 +00:00
tonic = "0.7"
feat: new router2 crate This commit adds an almost-empty router2 crate containing enough of a skeleton to plumb into the IOx CLI/server runner.
2022-01-12 14:13:20 +00:00
trace = { path = "../trace/" }
build: update workspace-hack
2022-01-12 15:09:06 +00:00
workspace-hack = { path = "../workspace-hack"}
feat(router2): shard writes/deletes into write buffer (#3499) * feat: Sequencer wrapper This type wraps an underlying WriteBufferWriter implementation, tagging it with a sequencer ID it should use when enqueuing operations to the buffer. * feat: mock sharder Implements a mock Sharder impl that returns pre-configured responses to shard(), and captures the input to the call. * feat: sharded write buffer Implements sharding of ops into an underlying WriteBuffer. Writes are sharded by some abstract Sharder impl, collated per shard to maximise the size of each op (and therefore compression efficiency), converted into a DML operation and then enqueued in parallel to the underlying WriteBuffer implementation. Deletes are modelled as being mapped to a single write buffer shard, which is the case while we support sharding based on the table & namespace only. Deletes will be extended to support (potentially) multiple shards when column overrides are implemented. * refactor: runtime write buffers Switch from using static dispatch, to using a runtime specified WriteBufferWriting implementation. Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-01-25 15:19:48 +00:00
write_buffer = { path = "../write_buffer" }
feat: return write_token from HTTP writes to router2 (#4202) * feat: return write_token from HTTP writes to router2 * fix: Update router2/src/dml_handlers/instrumentation.rs Co-authored-by: Dom <dom@itsallbroken.com> * refactor: Use WriteSummary::default more vigorously * fix: fix typo and add links to follow on issues Co-authored-by: Dom <dom@itsallbroken.com> Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-04-02 10:34:51 +00:00
write_summary = { path = "../write_summary" }
feat: abstract DML handler Defines the DmlHandler trait responsible for processing a request in some abstract way, decoupling the HTTP/gRPC request handlers from the underlying routing logic.
2022-01-14 11:34:42 +00:00
[dev-dependencies]
assert_matches = "1.5"
feat: more benchmarks of router2 (#3575)
2022-01-28 17:44:10 +00:00
criterion = { version = "0.3.4", features = ["async_tokio", "html_reports"] }
feat: write partitioner Implements a write partitioning DML handler that splits per-table MutableBatch instances into per-partition, per-table MutableBatch and concurrently calls the inner DML handler with each.
2022-02-09 13:31:13 +00:00
lazy_static = "1.4.0"
chore(deps): Bump paste from 1.0.6 to 1.0.7 (#4140) Bumps [paste](https://github.com/dtolnay/paste) from 1.0.6 to 1.0.7. - [Release notes](https://github.com/dtolnay/paste/releases) - [Commits](https://github.com/dtolnay/paste/compare/1.0.6...1.0.7) --- updated-dependencies: - dependency-name: paste dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-03-28 08:44:01 +00:00
paste = "1.0.7"
chore(deps): Bump pretty_assertions from 1.2.0 to 1.2.1 (#4213) Bumps [pretty_assertions](https://github.com/colin-kiegel/rust-pretty-assertions) from 1.2.0 to 1.2.1. - [Release notes](https://github.com/colin-kiegel/rust-pretty-assertions/releases) - [Changelog](https://github.com/colin-kiegel/rust-pretty-assertions/blob/main/CHANGELOG.md) - [Commits](https://github.com/colin-kiegel/rust-pretty-assertions/compare/v1.2.0...v1.2.1) --- updated-dependencies: - dependency-name: pretty_assertions dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-04-04 10:53:31 +00:00
pretty_assertions = "1.2.1"
feat: router2 sharder benchmarking (#3558) * feat: benchmarking the router2 sharder * chore: added throughput to sharder benchmarks; vary num buckets
2022-01-27 18:09:16 +00:00
rand = "0.8.3"
feat: schema validation Implements a write schema validation DML handler, denying requests that conflict with the schema within the global catalog. Additive schema changes are accepted, incrementally updating the global catalog schema. Deletes are passed through unchanged and unvalidated.
2022-01-28 16:25:20 +00:00
schema = { path = "../schema" }
feat(router2): simultaneous HTTP request limit This commit adds a service protection limit, allowing the router to restrict the number of simultaneous HTTP requests to a hard-coded limit of 200. By dropping additional requests once the router is serving the configured maximum, the router is prevented from accepting more and more requests that stack up waiting on the catalog / postgres connection pool. Currently buffering all these stalled connections consumes so much memory the router is OOM killed. After this commit, a portion of requests will be satisfied and the rest will be dropped early with a HTTP 503 Service Unavailable response.
2022-04-25 14:06:28 +00:00
test_helpers = { version = "0.1.0", path = "../test_helpers", features = ["future_timeout"] }
tokio-stream = { version = "0.1.8", default_features = false, features = [] }
feat: router2 sharder benchmarking (#3558) * feat: benchmarking the router2 sharder * chore: added throughput to sharder benchmarks; vary num buckets
2022-01-27 18:09:16 +00:00
[[bench]]
name = "sharder"
harness = false
feat: more benchmarks of router2 (#3575)
2022-01-28 17:44:10 +00:00
feat: schema validation benchmarks Useful for confirming the scalability of the schema check algorithm.
2022-03-04 00:28:02 +00:00
[[bench]]
name = "schema_validator"
harness = false
feat: more benchmarks of router2 (#3575)
2022-01-28 17:44:10 +00:00
[[bench]]
name = "e2e"
harness = false