Commit Graph

390 Commits (5027c9a88cf8c6249364d9fd3b2323653e217fe2)

Author SHA1 Message Date
Dom Dwyer 7afa3bfaec feat: expose ShardService over gRPC
Plumbs in the ShardService impl, and exposes it over the router's gRPC
interface.
2022-08-24 15:47:26 +02:00
Dom Dwyer da31f31406 feat: ShardService gRPC endpoint handler
Implements the ShardService to expose the shard mapping produced by
routers to external clients.

This impl uses an internal cache to eliminate unnecessary Catalog
queries, as the Kafka partition/Sequencer/Shard mapping is static once a
router has initialised.
2022-08-24 15:46:33 +02:00
Dom Dwyer abf26767c1 refactor: infallible JumpHash initialisation
This doesn't really need to be fallible but forces propagation of a ton
of error handling - no shards is always a sign of something being very
wrong, and can be caught in the caller if it's for some reason an
acceptable state / can be recovered from.
2022-08-24 13:18:57 +02:00
Dom Dwyer 9b920f1cbb refactor(router): use KafkaPartition in sequencer
The Sequencer (which will be renamed shortly) is a type that represents
a single sequencer/shard/kafka partition in the router.

In order to minimise confusion with all the various IDs floating around,
we have a KafkaPartition - this commit changes the Sequencer to return
the Kafka partition index as a typed value, rather than a usize to help
eliminate any inconsistencies.

As a side effect of these conversion changes, I've tightened up the
casting to ensure we assert on any overflows - we juggle a lot of
numeric types!
2022-08-23 16:02:11 +02:00
Andrew Lamb 7f0ae53d6f
chore: Update to (almost) released object_store 0.4.0 (#5419)
* chore: update object_store

* chore: update hakari config

* chore: Run cargo hakari tasks

Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
2022-08-17 13:44:48 +00:00
Dom Dwyer 180ff9f681 feat: table name in schema validation errors
Scopes all schema validation errors to include the table name in the
error output.
2022-08-16 19:00:44 +02:00
Andrew Lamb 0a7e6919f2
chore: do not build benchmark binary for lib targets (#5400)
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-08-16 08:11:23 +00:00
dependabot[bot] 0a44a42999
chore(deps): Bump futures from 0.3.21 to 0.3.23 (#5394)
Bumps [futures](https://github.com/rust-lang/futures-rs) from 0.3.21 to 0.3.23.
- [Release notes](https://github.com/rust-lang/futures-rs/releases)
- [Changelog](https://github.com/rust-lang/futures-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/futures-rs/compare/0.3.21...0.3.23)

---
updated-dependencies:
- dependency-name: futures
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-08-15 12:19:17 +00:00
Dom Dwyer d3dfc4221a docs: router writes to partitions
Though it can write to different topics, it currently writes to
different partitions.
2022-08-12 13:48:04 +02:00
Carol (Nichols || Goulding) b982bdaf2f
fix: Derive Eq when we derive PartialEq and members can derive Eq
Allow this in generated code that we don't control, though.

Recommended by clippy now. https://rust-lang.github.io/rust-clippy/master/index.html#derive_partial_eq_without_eq
2022-08-11 15:04:06 -04:00
Andrew Lamb 16ddc5efc6
chore: Update datafusion / arrow/parquet/arrow-flight and prost/tonic ecosystem (#5360)
* chore: Update datafusion and arrow

* chore: Update Cargo.lock

* chore: update to Decimal128

* chore: Update tonic/prost/pbjson/etc

* chore: Run cargo hakari tasks

* fix: doctest in generated types

Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
2022-08-09 17:30:44 +00:00
dependabot[bot] 94fe5b4c10
chore(deps): Bump paste from 1.0.7 to 1.0.8 (#5280)
Bumps [paste](https://github.com/dtolnay/paste) from 1.0.7 to 1.0.8.
- [Release notes](https://github.com/dtolnay/paste/releases)
- [Commits](https://github.com/dtolnay/paste/compare/1.0.7...1.0.8)

---
updated-dependencies:
- dependency-name: paste
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-08-03 09:03:25 +00:00
Andrew Lamb fbf672015e
refactor: Reduce ceremony requried to create a `Span` from `SpanContext` (#5181)
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-07-22 11:19:38 +00:00
dependabot[bot] 278a7f91af
chore(deps): Bump bytes from 1.1.0 to 1.2.0 (#5156)
Bumps [bytes](https://github.com/tokio-rs/bytes) from 1.1.0 to 1.2.0.
- [Release notes](https://github.com/tokio-rs/bytes/releases)
- [Changelog](https://github.com/tokio-rs/bytes/blob/master/CHANGELOG.md)
- [Commits](https://github.com/tokio-rs/bytes/compare/v1.1.0...v1.2.0)

---
updated-dependencies:
- dependency-name: bytes
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-07-20 10:00:08 +00:00
dependabot[bot] c443d07a5c
chore(deps): Bump criterion from 0.3.5 to 0.3.6 (#5059)
Bumps [criterion](https://github.com/bheisler/criterion.rs) from 0.3.5 to 0.3.6.
- [Release notes](https://github.com/bheisler/criterion.rs/releases)
- [Changelog](https://github.com/bheisler/criterion.rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/bheisler/criterion.rs/compare/0.3.5...0.3.6)

---
updated-dependencies:
- dependency-name: criterion
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-07-07 07:09:27 +00:00
Andrew Lamb 5944f27e77
refactor: avoid write buffer cloning in `store_operation` (#5042)
* refactor: avoid write buffer cloning in `store_operation`

* fix: update usage

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-07-06 06:57:03 +00:00
Raphael Taylor-Davies 835e1c91c7
chore: update object_store to 0.3.0 (#4707)
* chore: update object_store to 0.3.0

* chore: review feedback

Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>

Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-06-29 21:44:03 +00:00
Markus Westerlind edf3f08e81 refactor: Replace all uses of lazy_static with once_cell
Went through and remove all lazy_static uses with once_cell (while waiting for the project to compile). There are still dependencies using lazy_static so it is still in the crate graph but at least there isn't an explicit dependency on it (and it is easier to update to `std::lazy::Lazy` once that is stable).
2022-06-29 16:22:02 +02:00
Dom Dwyer c1f7154031 feat: propagate partition key through kafka
Changes the kafka message wire format to include the partition key for
serialised DML writes on the wire.

After this commit, the kafka messages will contain the partition key for
each op, but this information will go unused in the ingester - this
enables us to roll out the producer side, before making the value's
presence necessary on the consumer side.

A follow-up PR will change the ingester to utilise this embedded
partition key.

This has the unfortunate side effect of making the partition key part of
the public gRPC write API:

    https://github.com/influxdata/influxdb_iox/issues/4866
2022-06-20 13:42:51 +01:00
Dom Dwyer 2cc2ad6887 test: assert backfill partitioning in router
Assert writes in the router that cover multiple partitions are correctly
split up.
2022-06-17 14:40:53 +01:00
kodiakhq[bot] fa9a094068
Merge branch 'main' into cn/talk-to-ingesters-less 2022-06-15 17:42:40 +00:00
Carol (Nichols || Goulding) 03f6f59a9b
fix: Change the sharder to return error instead of panicking for no shards 2022-06-15 11:23:31 -04:00
Dom Dwyer 4df2964566 refactor: store PartitionKey in DmlWrite
Carry the PartitionKey in the DmlWrite, allowing the batch to be
associated with a specific partition key.
2022-06-15 15:48:54 +01:00
Dom Dwyer 61182f506b refactor: emit PartitionKey from partitioner
Changes the partitioning code to emit a PartitionKey, instead of a bare
String.
2022-06-15 15:38:02 +01:00
Carol (Nichols || Goulding) 6417e7dc2a
feat: Extract sharder to its own crate 2022-06-15 10:01:45 -04:00
Carol (Nichols || Goulding) e875a92cf8
feat: Log time spent requesting ingester partitions (#4806)
* feat: Log time spent requesting ingester partitions

Fixes #4558.

* feat: Record a metric for the duration queriers wait on ingesters

* fix: Use DurationHistogram instead of U64 Histogram

* test: Add a test for the ingester ms metric

* feat: Add back the logging to provide both logging and metrics for ingester duration

* refactor: Use sample_count method on metrics

* feat: Record ingester duration separately for success or failure

* fix: Create a separate test for the ingester metrics

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-06-14 17:58:19 +00:00
Carol (Nichols || Goulding) 1c7cbaf5ae
refactor: Use DurationHistogram in more places 2022-06-09 14:20:51 -04:00
dependabot[bot] a1ea793e13
chore(deps): Bump tokio-stream from 0.1.8 to 0.1.9 (#4785)
Bumps [tokio-stream](https://github.com/tokio-rs/tokio) from 0.1.8 to 0.1.9.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-stream-0.1.8...tokio-stream-0.1.9)

---
updated-dependencies:
- dependency-name: tokio-stream
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-06-06 14:21:54 +00:00
Andrew Lamb dde3c3922c
refactor: use consistent spelling of serialize (#4717) 2022-05-27 14:42:59 +00:00
Carol (Nichols || Goulding) 6ce6a38094
fix: Make metric names potentially less confusing 2022-05-25 10:04:39 -04:00
Raphael Taylor-Davies f2bb0fdf77
feat: update to crates.io object_store version (#4595)
* feat: update to crates.io object_store version

* chore: Run cargo hakari tasks

* fix: tests

* chore: remove object store integration test plumbing

Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
2022-05-13 16:26:07 +00:00
Andrew Lamb 35d80fc512
refactor: rename `iox_catalog_service` to `service_grpc_catalog` for consistency (#4581)
* refactor: rename `iox_catalog_service` to `service_grpc_catalog`

* fix: fmt

* fix: alphabetize crate

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-05-13 14:07:58 +00:00
Andrew Lamb 7eff52d04e
refactor: rename `iox_object_store_service` to `service_grpc_object_store` for consisntency (#4572)
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-05-12 14:22:18 +00:00
Andrew Lamb b8cb4c3f2b
feat: Interrogate schema from querier (as well as router) (#4557)
* refactor: move SchemaService into `service_grpc_schema`

* feat: implement schema gRPC for querier

* chore: Run cargo hakari tasks

Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
2022-05-10 20:55:58 +00:00
Dom Dwyer 776b0e0778 docs: finish incomplete comments 2022-05-09 14:46:32 +01:00
Dom Dwyer 31f805faa5 docs: partial sharded write semantics
Documents the partial write semantics and expected resolution for the
ShardedWriteBuffer in the router.
2022-05-09 14:46:32 +01:00
Dom 5a33749c98 chore: fix test comments
Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
2022-05-09 14:46:32 +01:00
Dom Dwyer 5069e37bd7 feat: shard delete ops with no table predicate
Changes the JumpHash sharder and modifies the ShardedWriteBuffer's
DmlHandler::delete() impl in order to enqueue delete ops across all
shards if no table name is specified.

If a table name is specified, it is sharded as before: a delete for a
given table & namespace always maps to the same shard as a write to the
same table & namespace.
2022-05-09 14:46:30 +01:00
Jake Goulding e07bcd40c2 refactor: Remove unused dependencies
These were found by iterating over all of the dependencies of each
Cargo.toml, then grepping that crate for the dependency's name. If it
didn't show up, I attempted to remove it.

I left a few dependencies that this process flagged:

* generated_types
  - `pbjson`,`serde`. Apparently used by the generated code.

* grpc-router-test-gen
  - `prost`. Apparently used by the generated code.

* influxdb_iox
  - `heappy`. Doesn't appear used, but is behind enough feature
    flags that I don't care to reason about and it's already optional.
  - `tikv_jemalloc_sys`. Appears to be setting a feature flag of an
    indirect dependency.

* iox_gitops_adapter
  - `k8s_openapi`. Appears to be setting a feature flag of an indirect
    dependency.
2022-05-06 15:57:58 -04:00
Carol (Nichols || Goulding) fcd4815645
fix: Rename router2 to router 2022-05-06 14:51:52 -04:00
Carol (Nichols || Goulding) e0bc1801ac
fix: Remove router 2022-05-06 11:30:36 -04:00
dependabot[bot] 420c306caa
chore(deps): Bump tokio from 1.17.0 to 1.18.0 (#4453)
Bumps [tokio](https://github.com/tokio-rs/tokio) from 1.17.0 to 1.18.0.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.17.0...tokio-1.18.0)

---
updated-dependencies:
- dependency-name: tokio
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-04-28 08:21:17 +00:00
二手掉包工程师 4b47d723b1
refactor: Rename time to iox_time (#4416)
Signed-off-by: hi-rustin <rustin.liu@gmail.com>

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-04-26 00:19:59 +00:00
Carol (Nichols || Goulding) e2ebeff39c
feat: Add write tokens to the gRPC write endpoint response
As a header in the same way the HTTP write endpoint does.

Fixes #4208.
2022-04-25 09:46:13 -04:00
dependabot[bot] ad3868ed7c
chore(deps): Bump tokio from 1.16.1 to 1.17.0 (#3814)
* chore(deps): Bump tokio from 1.16.1 to 1.17.0

Bumps [tokio](https://github.com/tokio-rs/tokio) from 1.16.1 to 1.17.0.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.16.1...tokio-1.17.0)

---
updated-dependencies:
- dependency-name: tokio
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* build: update workspace-hack

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Dom Dwyer <dom@itsallbroken.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-02-22 16:27:43 +00:00
dependabot[bot] 6c3b208438
chore(deps): bump cache_loader_async from 0.1.2 to 0.2.0 (#3665)
* chore(deps): bump cache_loader_async from 0.1.2 to 0.2.0

Bumps [cache_loader_async](https://github.com/ZeroTwo-Bot/cache-loader-async-rs) from 0.1.2 to 0.2.0.
- [Release notes](https://github.com/ZeroTwo-Bot/cache-loader-async-rs/releases)
- [Changelog](https://github.com/ZeroTwo-Bot/cache-loader-async-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/ZeroTwo-Bot/cache-loader-async-rs/commits/v0.2.0)

---
updated-dependencies:
- dependency-name: cache_loader_async
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* fix: breaking changes from `cache_loader_async`

* fix: Update workspace hack: tokio full is no longer needed

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Marco Neumann <marco@crepererum.net>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
Co-authored-by: Carol (Nichols || Goulding) <carol.nichols@gmail.com>
2022-02-10 17:01:11 +00:00
Raphael Taylor-Davies ca331503a5
feat: add WriteBufferErrorKind (#3664)
* feat: add WriteBufferErrorKind

* fix: test_offset_after_broken_message

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-02-08 15:34:05 +00:00
Marco Neumann 50cff27b01
chore: remove rdkafka dependency (#3625)
All features are now covered by rskafka. This also removes the need to
specify a server ID for write buffer consumers. This was only used for
rdkafka since there we needed to specify a consumer group, even though
we did not use any transactions.
2022-02-03 13:33:56 +00:00
Marco Neumann 22778a3a80
chore: upgrade rskafka and parking_lot (#3592) 2022-02-01 11:50:42 +00:00
Marco Neumann b326b62b44
feat: buffer writes when writing to RSKafka (#3520) 2022-02-01 10:07:52 +00:00
Andrew Lamb 2062267d0f
chore: Update hashbrown (#3551)
* chore: Update hashbrown

* fix: hakari

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-01-27 15:34:10 +00:00
Paul Dix 16d584b2ff
feat: Add db_name/namespace to DmlWrite and DmlDelete (#3531)
* feat: Add db_name/namespace to DmlWrite and DmlDelete

This is required for the new ingester to be able to work with the write buffer. The protobuf that gets serialized over Kafka already includes the database name, it just wasn't getting carried through to the marshaled Dml operation.

* fix: database != namespace, propagation through write buffer

Co-authored-by: Marco Neumann <marco@crepererum.net>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-01-27 14:12:20 +00:00
Andrew Lamb 9c19cd6cc4
fix: clamp start/end of TimestampRange to min/max valid timestamp values (#3487)
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-01-20 16:08:00 +00:00
Marco Neumann f3f6f335a9
chore: upgrade to snafu 0.7 (#3440) 2022-01-11 19:22:36 +00:00
Carol (Nichols || Goulding) 471c3181bb
feat: Make a top-level influxdb_iox feature for kafka functionality
Thread the feature through router and server to the write buffer crate.
Move an end-to-end test that uses Kafka behind the feature flag.
2021-12-09 08:49:34 -05:00
Carol (Nichols || Goulding) 16d8ae5e04
fix: Match tokio features to what's actually in use in each crate
Some crates listed features they don't use; other crates ware relying on
feature flags enabled by something else. I tested these changes by
disabling the workspace hack crate and testing each crate.
2021-12-06 09:37:16 -05:00
Carol (Nichols || Goulding) 02c297e850
fix: Always specify the parking_lot feature of tokio to get potential perf boost 2021-12-06 09:37:15 -05:00
Carol (Nichols || Goulding) 30c4da7ca7
fix: Be consistent with regex version range specification 2021-12-06 09:37:15 -05:00
Raphael Taylor-Davies f17b670fe9
refactor: cleanup gRPC client error handling (#3251) (#3277)
* refactor: cleanup gRPC client error handling (#3251)

* chore: review feedback

* chore: review feedback

Co-authored-by: Andrew Lamb <alamb@influxdata.com>

Co-authored-by: Andrew Lamb <alamb@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-12-02 20:56:58 +00:00
Marco Neumann ad65687cba refactor: consolidate sharding implementation
Move sharding implementation from router to `dml` (where the types can
be consumed w/o cloning the data) and only support the new sharding
configs.

The old sharding configs will be removed soon.
2021-11-26 13:00:36 +01:00
Marco Neumann 7f2e4f4342 refactor: remove write buffer direction
The direction was required when a database could read or write from/to a
write buffer. Now it is clear from the usage context of a write buffer
context which of the two applications is meant (databases read, routers
write) so the direction flag is no longer required.
2021-11-26 12:38:40 +01:00
Marco Neumann 5ecaa6d8a9 refactor: use serialized predicates instead of strings for gRPC deletes
IOx is the only consumer of this API so we might just use the serialized
form. Cloud2 uses the HTTP API which supports SQL-like predicates.

Fixes #3192.
2021-11-24 09:30:32 +01:00
Marco Neumann 93cc79875a refactor: simplify code 2021-11-23 15:39:53 +01:00
Marco Neumann 18cab1545c test: improve test naming
Co-authored-by: Raphael Taylor-Davies <1781103+tustvold@users.noreply.github.com>
2021-11-23 15:39:53 +01:00
Marco Neumann df19d67532 test: allow to specify `WriteBufferConfigFactory` when creating a test `RouterServer` 2021-11-23 15:39:53 +01:00
Marco Neumann 964411bdbb feat: add delete support to router 2021-11-23 15:39:53 +01:00
Carol (Nichols || Goulding) 3ed9dfc4db
fix: Make all usages of cache_loader_async use the same features 2021-11-19 14:33:55 -05:00
Carol (Nichols || Goulding) 9fd4a560f5
feat: Results of running cargo hakari manage-deps 2021-11-19 09:21:57 -05:00
Raphael Taylor-Davies 8155747735
feat: add write buffer delete encoding (#2731) (#3127)
* feat: add write buffer delete encoding (#2731)

* chore: fix doc

* chore: review feedback

* chore: review feedback

* chore: fmt

* chore: review feedback
2021-11-17 16:12:19 +00:00
Marco Neumann e6fdd79a0f feat: emit Kafka stats as metrics instead of logs
This maps a subset of Kafka stats as metrics. The set can -- of course
-- be changed in the future depending on our needs.

Fixes #3100.
2021-11-16 17:18:41 +01:00
Raphael Taylor-Davies 553e412226
refactor: DMLOperation write path (#2731) (#3121)
* refactor: DMLOperation write path (#2731)

* chore: fmt

* chore: review feedback
2021-11-16 12:42:19 +00:00
Raphael Taylor-Davies 6f268f8260
refactor: extract DML types (#2731) (#3084)
* refactor: extract DML types (#2731)

* chore: fmt
2021-11-11 12:34:07 +00:00
Marco Neumann 09239c4943 feat: improve testing interface for the `router` crate 2021-11-10 17:51:58 +01:00
Marco Neumann ea2604b7b6 docs: explain write sink test 2021-11-08 14:09:33 +01:00
Marco Neumann 4130d508c0 docs: document write buffer variants 2021-11-08 14:09:33 +01:00
Marco Neumann 1d34ba0183 docs: explain router name argument 2021-11-08 14:09:33 +01:00
Marco Neumann 12c684b546 test: extend `test_write` test 2021-11-08 14:09:33 +01:00
Marco Neumann dcc376456a docs: explain iteration order in `Router` 2021-11-08 14:09:33 +01:00
Marco Neumann 6a090b785a fix: use new `WriteMeta` constructor 2021-11-08 14:09:33 +01:00
Marco Neumann b226d629c1 feat: router write logic 2021-11-08 14:09:33 +01:00
Marco Neumann c745a0a94c feat: impl write sinks logic for router 2021-11-08 14:09:33 +01:00
Marco Neumann fc4ea71921 feat: `RouterServer::router` 2021-11-08 14:09:33 +01:00
Marco Neumann 9984e4d386 fix: debug msg
Co-authored-by: Andrew Lamb <alamb@influxdata.com>
2021-11-08 11:11:52 +01:00
Marco Neumann 7f6280305d fix: `ConnectionPool::new` requires tokio 2021-11-08 11:11:52 +01:00
Marco Neumann 11f4a1dee8 feat: add connection management for router 2021-11-08 11:11:52 +01:00
Raphael Taylor-Davies 898567e221
feat: migrate server to DbWrite (#2724) (#3035)
* feat: migrate server to DbWrite (#2724)

* chore: print perf log output

* fix: don't suppress CI status code

* chore: review feedback

* fix: don't error on empty line protocol write payloads

* fix: test

* fix: test
2021-11-05 11:09:33 +00:00
Marco Neumann 92e6acaacd feat: implement sharder for new router 2021-11-05 09:49:33 +01:00
Marco Neumann 78101eb52d feat: implement `RemoteService` for new router mode 2021-11-04 15:35:05 +01:00
Marco Neumann 4840dc11cb feat: add basic CRUD operations for router configs 2021-11-04 12:03:14 +01:00
Marco Neumann a91e572b43 feat: implement basic router server type
This only implements the deployment service and nothing else yet.
2021-11-03 16:34:51 +01:00