Influxdata
/
influxdb
mirror of https://github.com/influxdata/influxdb.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
influxdb/router/Cargo.toml

72 lines
2.2 KiB
TOML
Raw Normal View History

feat: new router2 crate This commit adds an almost-empty router2 crate containing enough of a skeleton to plumb into the IOx CLI/server runner.
2022-01-12 14:13:20 +00:00
[package]
fix: Rename router2 to router
2022-05-06 18:51:52 +00:00
name = "router"
feat: Use workspace package metadata to reduce differences and repetition
2022-09-26 14:43:00 +00:00
version.workspace = true
authors.workspace = true
edition.workspace = true
license.workspace = true
feat: new router2 crate This commit adds an almost-empty router2 crate containing enough of a skeleton to plumb into the IOx CLI/server runner.
2022-01-12 14:13:20 +00:00
[dependencies]
feat: abstract DML handler Defines the DmlHandler trait responsible for processing a request in some abstract way, decoupling the HTTP/gRPC request handlers from the underlying routing logic.
2022-01-14 11:34:42 +00:00
async-trait = "0.1"
chore(deps): Bump bytes from 1.2.1 to 1.3.0 (#6199) Bumps [bytes](https://github.com/tokio-rs/bytes) from 1.2.1 to 1.3.0. - [Release notes](https://github.com/tokio-rs/bytes/releases) - [Changelog](https://github.com/tokio-rs/bytes/blob/master/CHANGELOG.md) - [Commits](https://github.com/tokio-rs/bytes/commits) --- updated-dependencies: - dependency-name: bytes dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-11-22 08:23:24 +00:00
bytes = "1.3"
feat: Add a feature flag to switch to the router RPC write path (#6247) * feat: Add a feature flag to switch to the router RPC write path Fixes #6242. * refactor: Remove a weird arc clone/rename that's not needed I'm sure this was needed at some point, but it doesn't make much sense. I wasn't going to change this, but I'm now trying to minimize the differences between this function and the write path init function, so make this one better too. * fix: Add the namespace autocreation to the RPC write path too The topic/query pool don't really apply to this case, but use them anyway to be able to use the existing catalog methods. Also add a bunch of comments pointing out where the RPC write path initializer and the old router's initializer are the same and where they're different, so that perhaps it'll be easier to keep them in sync while they both exist. Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-12-01 11:05:39 +00:00
client_util = { path = "../client_util" }
fix: Rename data_types2 to data_types
2022-05-05 19:29:24 +00:00
data_types = { path = "../data_types" }
feat: new router2 crate This commit adds an almost-empty router2 crate containing enough of a skeleton to plumb into the IOx CLI/server runner.
2022-01-12 14:13:20 +00:00
dml = { path = "../dml" }
feat: router2 HTTP handlers Implements the HTTP v2 write API endpoint for router2.
2022-01-14 13:34:16 +00:00
flate2 = "1.0"
chore(deps): Bump futures from 0.3.24 to 0.3.25 (#5938) Bumps [futures](https://github.com/rust-lang/futures-rs) from 0.3.24 to 0.3.25. - [Release notes](https://github.com/rust-lang/futures-rs/releases) - [Changelog](https://github.com/rust-lang/futures-rs/blob/master/CHANGELOG.md) - [Commits](https://github.com/rust-lang/futures-rs/compare/0.3.24...0.3.25) --- updated-dependencies: - dependency-name: futures dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-10-21 11:03:20 +00:00
futures = "0.3.25"
feat: new router2 crate This commit adds an almost-empty router2 crate containing enough of a skeleton to plumb into the IOx CLI/server runner.
2022-01-12 14:13:20 +00:00
generated_types = { path = "../generated_types" }
refactor: use a workspace dependency for hashbrown
2022-11-11 17:12:30 +00:00
hashbrown = { workspace = true }
feat: new router2 crate This commit adds an almost-empty router2 crate containing enough of a skeleton to plumb into the IOx CLI/server runner.
2022-01-12 14:13:20 +00:00
hyper = "0.14"
feat: schema validation Implements a write schema validation DML handler, denying requests that conflict with the schema within the global catalog. Additive schema changes are accepted, incrementally updating the global catalog schema. Deletes are passed through unchanged and unvalidated.
2022-01-28 16:25:20 +00:00
iox_catalog = { path = "../iox_catalog" }
fix: Rename data_types2 to data_types
2022-05-05 19:29:24 +00:00
iox_time = { path = "../iox_time" }
feat: new router2 crate This commit adds an almost-empty router2 crate containing enough of a skeleton to plumb into the IOx CLI/server runner.
2022-01-12 14:13:20 +00:00
metric = { path = "../metric" }
refactor: avoid constructing DmlOperation Instead of converting the set of MutableBatches into a DmlOperation to shard into more DmlOperation instances, the sharder can operate directly on the MutableBatches.
2022-01-14 16:52:03 +00:00
mutable_batch = { path = "../mutable_batch" }
feat: abstract DML handler Defines the DmlHandler trait responsible for processing a request in some abstract way, decoupling the HTTP/gRPC request handlers from the underlying routing logic.
2022-01-14 11:34:42 +00:00
mutable_batch_lp = { path = "../mutable_batch_lp" }
feat: grpc write handler Implements the gRPC write handler endpoint.
2022-02-23 15:41:45 +00:00
mutable_batch_pb = { version = "0.1.0", path = "../mutable_batch_pb" }
chore(deps): Bump object_store from 0.5.0 to 0.5.1 Bumps [object_store](https://github.com/apache/arrow-rs) from 0.5.0 to 0.5.1. - [Release notes](https://github.com/apache/arrow-rs/releases) - [Changelog](https://github.com/apache/arrow-rs/blob/master/CHANGELOG-old.md) - [Commits](https://github.com/apache/arrow-rs/compare/object_store_0.5.0...object_store_0.5.1) --- updated-dependencies: - dependency-name: object_store dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com>
2022-10-11 01:19:10 +00:00
object_store = "0.5.1"
feat: abstract DML handler Defines the DmlHandler trait responsible for processing a request in some abstract way, decoupling the HTTP/gRPC request handlers from the underlying routing logic.
2022-01-14 11:34:42 +00:00
observability_deps = { path = "../observability_deps" }
chore: upgrade rskafka and parking_lot (#3592)
2022-02-01 11:50:42 +00:00
parking_lot = "0.12"
feat(router2): implement delete API handler Adds support for the HTTP v2 delete API endpoint.
2022-01-14 16:01:01 +00:00
predicate = { path = "../predicate" }
feat: grpc write handler Implements the gRPC write handler endpoint.
2022-02-23 15:41:45 +00:00
schema = { version = "0.1.0", path = "../schema" }
feat: router2 HTTP handlers Implements the HTTP v2 write API endpoint for router2.
2022-01-14 13:34:16 +00:00
serde = "1.0"
chore(deps): Bump serde_json from 1.0.88 to 1.0.89 (#6203) Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.88 to 1.0.89. - [Release notes](https://github.com/serde-rs/json/releases) - [Commits](https://github.com/serde-rs/json/compare/v1.0.88...v1.0.89) --- updated-dependencies: - dependency-name: serde_json dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-11-22 09:28:31 +00:00
serde_json = "1.0.89"
feat: router2 HTTP handlers Implements the HTTP v2 write API endpoint for router2.
2022-01-14 13:34:16 +00:00
serde_urlencoded = "0.7"
chore: Alphabetize some dependencies I can't handle not knowing where to look for a dependency or knowing where to add a new dependency.
2022-10-28 14:34:23 +00:00
service_grpc_catalog = { path = "../service_grpc_catalog"}
chore: router namespace api (#6151) * chore: move ns api from querier to router * chore: add explanatory comment in querier about moved namespace API * fix: add namespace service to router * fix: querier returns unimplemented error for ns retention, not panic * chore: reuse namespace -> proto in router ns api * chore: grpc namespace - consume ns to avoid clone Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-11-16 15:25:49 +00:00
service_grpc_namespace = { path = "../service_grpc_namespace"}
feat: Interrogate schema from querier (as well as router) (#4557) * refactor: move SchemaService into `service_grpc_schema` * feat: implement schema gRPC for querier * chore: Run cargo hakari tasks Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
2022-05-10 20:55:58 +00:00
service_grpc_schema = { path = "../service_grpc_schema" }
refactor: rename `iox_object_store_service` to `service_grpc_object_store` for consisntency (#4572) Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-05-12 14:22:18 +00:00
service_grpc_object_store = { path = "../service_grpc_object_store" }
feat: Extract sharder to its own crate
2022-06-09 19:10:16 +00:00
sharder = { path = "../sharder" }
chore: Alphabetize some dependencies I can't handle not knowing where to look for a dependency or knowing where to add a new dependency.
2022-10-28 14:34:23 +00:00
snafu = "0.7"
feat: new router2 crate This commit adds an almost-empty router2 crate containing enough of a skeleton to plumb into the IOx CLI/server runner.
2022-01-12 14:13:20 +00:00
thiserror = "1.0"
feat: router2 HTTP handlers Implements the HTTP v2 write API endpoint for router2.
2022-01-14 13:34:16 +00:00
tokio = { version = "1", features = ["rt-multi-thread", "macros"] }
chore: Update datafusion / arrow/parquet/arrow-flight and prost/tonic ecosystem (#5360) * chore: Update datafusion and arrow * chore: Update Cargo.lock * chore: update to Decimal128 * chore: Update tonic/prost/pbjson/etc * chore: Run cargo hakari tasks * fix: doctest in generated types Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
2022-08-09 17:30:44 +00:00
tonic = "0.8"
feat: new router2 crate This commit adds an almost-empty router2 crate containing enough of a skeleton to plumb into the IOx CLI/server runner.
2022-01-12 14:13:20 +00:00
trace = { path = "../trace/" }
build: update workspace-hack
2022-01-12 15:09:06 +00:00
workspace-hack = { path = "../workspace-hack"}
feat(router2): shard writes/deletes into write buffer (#3499) * feat: Sequencer wrapper This type wraps an underlying WriteBufferWriter implementation, tagging it with a sequencer ID it should use when enqueuing operations to the buffer. * feat: mock sharder Implements a mock Sharder impl that returns pre-configured responses to shard(), and captures the input to the call. * feat: sharded write buffer Implements sharding of ops into an underlying WriteBuffer. Writes are sharded by some abstract Sharder impl, collated per shard to maximise the size of each op (and therefore compression efficiency), converted into a DML operation and then enqueued in parallel to the underlying WriteBuffer implementation. Deletes are modelled as being mapped to a single write buffer shard, which is the case while we support sharding based on the table & namespace only. Deletes will be extended to support (potentially) multiple shards when column overrides are implemented. * refactor: runtime write buffers Switch from using static dispatch, to using a runtime specified WriteBufferWriting implementation. Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-01-25 15:19:48 +00:00
write_buffer = { path = "../write_buffer" }
feat: return write_token from HTTP writes to router2 (#4202) * feat: return write_token from HTTP writes to router2 * fix: Update router2/src/dml_handlers/instrumentation.rs Co-authored-by: Dom <dom@itsallbroken.com> * refactor: Use WriteSummary::default more vigorously * fix: fix typo and add links to follow on issues Co-authored-by: Dom <dom@itsallbroken.com> Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-04-02 10:34:51 +00:00
write_summary = { path = "../write_summary" }
feat: abstract DML handler Defines the DmlHandler trait responsible for processing a request in some abstract way, decoupling the HTTP/gRPC request handlers from the underlying routing logic.
2022-01-14 11:34:42 +00:00
[dev-dependencies]
assert_matches = "1.5"
chore: slim down `criterion` (#5611) Criterion comes with some extra cargo tooling called `cargo criterion` which can be used instead of `cargo bench`. The advantage is that we don't need to compile the entire reporting infrastructure into our benchmarks. So let's embrace this separation of concerns. Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-09-12 08:47:35 +00:00
criterion = { version = "0.4", default-features = false, features = ["async_tokio", "rayon"]}
test: Ensure router's HTTP error messages are stable (#6006) * test: Ensure router's HTTP error messages are stable If you change the text of an error, the tests will fail. If you add a new error variant to the `Error` enum but don't add it to the test, test compilation will fail with a "non-exhaustive patterns" message. If you remove an error variant, test compilation will fail with a "no variant named `RemovedError`" message. You can get the list of error variants and their current text via `cargo test -p router -- print_out_error_text --nocapture`. A step towards accomplishing #5863 Co-authored-by: Carol (Nichols || Goulding) <carol.nichols@gmail.com> Co-authored-by: Jake Goulding <jake.goulding@integer32.com> * fix: Remove optional commas and document macro arguments * docs: Clarify the purpose of the tests the check_errors macro generates * fix: Add tests for inner mutable batch LP error variants Co-authored-by: Carol (Nichols || Goulding) <carol.nichols@gmail.com> Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-11-09 09:37:55 +00:00
influxdb_line_protocol = { path = "../influxdb_line_protocol" }
feat: Enforce table column limits from the schema cache (#5819) * fix: Avoid some allocations by collecting instead of inserting into a vec * refactor: Encode that adding columns is for one table at a time * test: Add another test of column limits * test: Add below/above limit tests for create_or_get_many * fix: Explicitly DO NOT check column limits when inserting many columns * feat: Cache the max_columns_per_table on the NamespaceSchema * feat: Add a function to validate column limits in-memory * fix: Provide more useful information when over column limits * fix: Swap types to remove intermediate allocation * docs: Explain the interactions of the cache and the column limits * test: Actually set up test that showcases column limit race condition * fix: Allow writing to existing columns even if table is over column limit Co-authored-by: Dom <dom@itsallbroken.com>
2022-10-14 11:34:17 +00:00
iox_tests = { path = "../iox_tests" }
refactor: Replace all uses of lazy_static with once_cell Went through and remove all lazy_static uses with once_cell (while waiting for the project to compile). There are still dependencies using lazy_static so it is still in the crate graph but at least there isn't an explicit dependency on it (and it is easier to update to `std::lazy::Lazy` once that is stable).
2022-06-29 11:27:43 +00:00
once_cell = "1"
chore(deps): Bump paste from 1.0.8 to 1.0.9 (#5526) Bumps [paste](https://github.com/dtolnay/paste) from 1.0.8 to 1.0.9. - [Release notes](https://github.com/dtolnay/paste/releases) - [Commits](https://github.com/dtolnay/paste/compare/1.0.8...1.0.9) --- updated-dependencies: - dependency-name: paste dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-09-01 12:07:53 +00:00
paste = "1.0.9"
chore(deps): Bump pretty_assertions from 1.2.1 to 1.3.0 (#5517) Bumps [pretty_assertions](https://github.com/rust-pretty-assertions/rust-pretty-assertions) from 1.2.1 to 1.3.0. - [Release notes](https://github.com/rust-pretty-assertions/rust-pretty-assertions/releases) - [Changelog](https://github.com/rust-pretty-assertions/rust-pretty-assertions/blob/main/CHANGELOG.md) - [Commits](https://github.com/rust-pretty-assertions/rust-pretty-assertions/compare/v1.2.1...v1.3.0) --- updated-dependencies: - dependency-name: pretty_assertions dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-09-01 10:20:26 +00:00
pretty_assertions = "1.3.0"
feat: router2 sharder benchmarking (#3558) * feat: benchmarking the router2 sharder * chore: added throughput to sharder benchmarks; vary num buckets
2022-01-27 18:09:16 +00:00
rand = "0.8.3"
feat: schema validation Implements a write schema validation DML handler, denying requests that conflict with the schema within the global catalog. Additive schema changes are accepted, incrementally updating the global catalog schema. Deletes are passed through unchanged and unvalidated.
2022-01-28 16:25:20 +00:00
schema = { path = "../schema" }
feat(router2): simultaneous HTTP request limit This commit adds a service protection limit, allowing the router to restrict the number of simultaneous HTTP requests to a hard-coded limit of 200. By dropping additional requests once the router is serving the configured maximum, the router is prevented from accepting more and more requests that stack up waiting on the catalog / postgres connection pool. Currently buffering all these stalled connections consumes so much memory the router is OOM killed. After this commit, a portion of requests will be satisfied and the rest will be dropped early with a HTTP 503 Service Unavailable response.
2022-04-25 14:06:28 +00:00
test_helpers = { version = "0.1.0", path = "../test_helpers", features = ["future_timeout"] }
chore(deps): Bump tokio-stream from 0.1.10 to 0.1.11 (#5838) Bumps [tokio-stream](https://github.com/tokio-rs/tokio) from 0.1.10 to 0.1.11. - [Release notes](https://github.com/tokio-rs/tokio/releases) - [Commits](https://github.com/tokio-rs/tokio/compare/tokio-stream-0.1.10...tokio-stream-0.1.11) --- updated-dependencies: - dependency-name: tokio-stream dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-10-12 12:37:24 +00:00
tokio-stream = { version = "0.1.11", default_features = false, features = [] }
feat: router2 sharder benchmarking (#3558) * feat: benchmarking the router2 sharder * chore: added throughput to sharder benchmarks; vary num buckets
2022-01-27 18:09:16 +00:00
chore: do not build benchmark binary for lib targets (#5400) Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-08-16 08:11:23 +00:00
[lib]
# Allow --save-baseline to work
# https://github.com/bheisler/criterion.rs/issues/275
bench = false
feat: schema validation benchmarks Useful for confirming the scalability of the schema check algorithm.
2022-03-04 00:28:02 +00:00
[[bench]]
name = "schema_validator"
harness = false
feat: more benchmarks of router2 (#3575)
2022-01-28 17:44:10 +00:00
[[bench]]
name = "e2e"
harness = false