c538cac4ef
chore(deps): Bump tokio from 1.25.0 to 1.26.0 ( #7107 )
...
* chore(deps): Bump tokio from 1.25.0 to 1.26.0
Bumps [tokio](https://github.com/tokio-rs/tokio ) from 1.25.0 to 1.26.0.
- [Release notes](https://github.com/tokio-rs/tokio/releases )
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.25.0...tokio-1.26.0 )
---
updated-dependencies:
- dependency-name: tokio
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* chore: Run cargo hakari tasks
---------
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: Dom <dom@itsallbroken.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-03-02 09:50:39 +00:00
faae5eb438
chore: Rerun cargo hakari manage-deps
2023-02-27 11:56:15 +01:00
4cf066680e
chore(deps): Bump tempfile from 3.3.0 to 3.4.0 ( #7069 )
...
* chore(deps): Bump tempfile from 3.3.0 to 3.4.0
Bumps [tempfile](https://github.com/Stebalien/tempfile ) from 3.3.0 to 3.4.0.
- [Release notes](https://github.com/Stebalien/tempfile/releases )
- [Changelog](https://github.com/Stebalien/tempfile/blob/master/NEWS )
- [Commits](https://github.com/Stebalien/tempfile/commits )
---
updated-dependencies:
- dependency-name: tempfile
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* chore: Run cargo hakari tasks
---------
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: Dom <dom@itsallbroken.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-02-27 10:02:21 +00:00
0cbd9f6a82
chore(deps): Bump tokio-util from 0.7.5 to 0.7.7 ( #6964 )
...
---
updated-dependencies:
- dependency-name: tokio-util
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-02-13 10:10:53 +00:00
c0c9b51b9e
chore(deps): Bump tokio-util from 0.7.4 to 0.7.5 ( #6941 )
...
Bumps [tokio-util](https://github.com/tokio-rs/tokio ) from 0.7.4 to 0.7.5.
- [Release notes](https://github.com/tokio-rs/tokio/releases )
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-util-0.7.4...tokio-util-0.7.5 )
---
updated-dependencies:
- dependency-name: tokio-util
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-02-10 09:42:00 +00:00
30fea67701
fix: Move variables within format strings. Thanks clippy!
...
Changes made automatically using `cargo clippy --fix`.
2023-02-03 13:06:17 -05:00
ed7d02a225
chore(deps): Bump tokio from 1.24.2 to 1.25.0
...
Bumps [tokio](https://github.com/tokio-rs/tokio ) from 1.24.2 to 1.25.0.
- [Release notes](https://github.com/tokio-rs/tokio/releases )
- [Commits](https://github.com/tokio-rs/tokio/commits/tokio-1.25.0 )
---
updated-dependencies:
- dependency-name: tokio
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-01-30 01:57:27 +00:00
b49cc2e35e
chore(deps): Bump tokio from 1.24.0 to 1.24.1 ( #6545 )
...
Bumps [tokio](https://github.com/tokio-rs/tokio ) from 1.24.0 to 1.24.1.
- [Release notes](https://github.com/tokio-rs/tokio/releases )
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.24.0...tokio-1.24.1 )
---
updated-dependencies:
- dependency-name: tokio
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-01-10 09:48:44 +00:00
656e9ee689
chore: update error message when running int tests w/out kafka
2022-12-16 16:10:32 +00:00
896a03fdbc
chore: update `rskafka` ( #6258 )
...
Useful because it updates `zstd` to 0.12. With the upcoming `parquet`
update, we can than drop `zstd` 0.11.
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-11-29 10:00:13 +00:00
a9db7581cd
chore(deps): Bump tokio from 1.21.2 to 1.22.0 ( #6183 )
...
Bumps [tokio](https://github.com/tokio-rs/tokio ) from 1.21.2 to 1.22.0.
- [Release notes](https://github.com/tokio-rs/tokio/releases )
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.21.2...tokio-1.22.0 )
---
updated-dependencies:
- dependency-name: tokio
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-11-21 10:21:24 +00:00
02c3083192
fix: Remove table names from Dml operations
2022-11-18 10:40:38 -05:00
a4b277594c
fix: Remove namespace argument from kafka test helper fn
2022-11-14 16:46:05 -05:00
5854ff28e9
fix: Remove namespace argument from write test helper fn
2022-11-14 16:46:04 -05:00
3943faf998
fix: Remove namespace from DmlWrite and DmlDelete constructors
2022-11-14 16:46:04 -05:00
f78195f7c7
fix: Remove namespace name field from DmlWrite and DmlDelete
...
But leave the argument in their constructors for now.
Not all numbers in tests can be 42, Dom.
2022-11-14 16:46:04 -05:00
e2d2e69076
fix: Remove db_name from encode_operation and FileBufferProducer
2022-11-14 16:46:04 -05:00
8dbdab8754
fix: Remove db_name field from DeletePayload
...
Doesn't seem to be used anywhere.
2022-11-14 16:46:04 -05:00
f4b8fe7751
fix: Remove now-unused db_name argument to encode_write
2022-11-14 16:46:04 -05:00
2c4c578df3
fix: Remove namespace name from IoxHeaders in write messages
2022-11-14 16:46:03 -05:00
a969754819
chore(deps): Bump chrono from 0.4.22 to 0.4.23 ( #6129 )
...
* chore(deps): Bump chrono from 0.4.22 to 0.4.23
Bumps [chrono](https://github.com/chronotope/chrono ) from 0.4.22 to 0.4.23.
- [Release notes](https://github.com/chronotope/chrono/releases )
- [Changelog](https://github.com/chronotope/chrono/blob/main/CHANGELOG.md )
- [Commits](https://github.com/chronotope/chrono/compare/v0.4.22...v0.4.23 )
---
updated-dependencies:
- dependency-name: chrono
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* refactor: chrono future compat
Integer->timstamp conversions should not silently panic.
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Marco Neumann <marco@crepererum.net>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-11-14 13:34:09 +00:00
cc17e5a54b
refactor: use a workspace dependency for hashbrown
2022-11-11 13:25:39 -05:00
5024523f00
chore(deps): Bump hashbrown from 0.12.3 to 0.13.1
...
Bumps [hashbrown](https://github.com/rust-lang/hashbrown ) from 0.12.3 to 0.13.1.
- [Release notes](https://github.com/rust-lang/hashbrown/releases )
- [Changelog](https://github.com/rust-lang/hashbrown/blob/master/CHANGELOG.md )
- [Commits](https://github.com/rust-lang/hashbrown/compare/v0.12.3...v0.13.1 )
---
updated-dependencies:
- dependency-name: hashbrown
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2022-11-11 13:24:56 -05:00
694443bb87
chore: Rename DatabaseName to NamespaceName ( #6100 )
...
* chore: Rename DatabaseName to NamespaceName
* fix: fmt
* chore: Updates some more references
* chore: more cleanup
* fix: adjust test
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-11-10 14:13:59 +00:00
43687a86d2
fix: Remove lots of needless borrows that Clippy can now identify
...
Except for in generated code that we don't control.
2022-11-09 10:54:18 -05:00
fa46951524
fix: Remove needless deref done by auto deref, thanks Clippy!
2022-11-09 10:54:18 -05:00
8ebea0df37
feat: table/namespace IDs in write protocol
...
Expose the Table and Namespace IDs encoded within the serialised DML
write (added in #6036 ).
This makes the IDs available for use in the consumers, ending the
transition period. This commit DOES NOT remove the strings sent over the
wire.
2022-11-08 16:57:53 +01:00
09e9b69b85
Merge remote-tracking branch 'origin/main' into dom/dml-delete-namespace-id
2022-11-04 14:56:10 -04:00
6565352a93
feat: support multiple bootstrap_broker server config ( #6045 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-11-03 13:31:23 +00:00
6fa48731aa
feat: NamespaceId in DmlDelete
...
Changes the DmlDelete to contain the NamespaceId for which it should be
applied, propagating this value over the wire.
Like the existing IDs within the DmlWrite, these values are marked
unsafe to use due to avoid the consumers utilising them accidentally
during deployment. Unlike DmlWrite, the DmlDelete is completely unused,
so this is less of an issue.
2022-11-03 13:57:40 +01:00
eec192fa4e
fix: test_dml_write_round_trip
2022-11-02 13:43:29 +01:00
8e14b831a6
refactor: avoid redundant string copying
2022-11-02 13:38:22 +01:00
46c921940a
docs(write_buffer): fix intra-doc links
2022-11-02 13:33:01 +01:00
ddd6ab0ba4
refactor(write_buffer): pass IDs in wire format
...
This commit is part of a two-part change in order to add the table &
namespace IDs to the write buffer wire format. This commit forms the
first half; changing the producer to send the IDs.
In this commit the new ID values are never read on the consumer side,
ensuring there is no consumer dependency on them. This ensures they
remain operational during a rollout, where the consumer may be updated
to the latest code dependent on the IDs before the producer is updated
to send them. This also ensures we have a window of time where where the
consumers can be rolled back after being updated, and still handle
replaying messages in Kafka.
2022-11-02 13:28:56 +01:00
e49f2ca5c7
fix: actually enable zstd compression for write write buffer ( #6035 )
...
Fixes #5981 .
2022-11-02 09:09:35 +00:00
254be59856
feat: enable ZSTD compression for write buffer payload
...
Closes #5981 .
2022-11-01 14:22:33 +01:00
aa4eec9939
chore: update rskafka
...
Mostly upstream dependencies updates8678dfe049de05415929ffec7c1be8921bb057f7.
2022-11-01 14:19:32 +01:00
d6cbae16ac
chore: update rskafka ( #5998 )
...
Includes additional logging to debug https://github.com/influxdata/idpe/issues/16278
2022-11-01 06:39:26 +00:00
72a358e52f
refactor(dml): PartitionKey required for writes
...
Changes the DmlWrite type to require a PartitionKey be specified,
instead of accepting an Option.
This requirement was already in place - the write buffer upheld an
invariant that all writes contained a partition key value (was not
"None") or it panicked at runtime when attempting to enqueue the write.
It is now possible to encode this invariant in the type system, which is
what this change does.
2022-10-28 10:57:30 +02:00
3d524baa9e
fix: update rskafka to support throttling ( #5988 )
...
See https://github.com/influxdata/rskafka/issues/182 .
2022-10-27 12:34:43 +00:00
d466a04ad8
feat: allow deploys to set Kafka client ID ( #5983 )
2022-10-26 15:59:16 +00:00
99f4440658
refactor: parse HTTP trace header even w/o collector ( #5979 )
...
Since we log trace IDs to allow easier correlation of logs no matter
what the `sampled` flag says, we should also parse these logs if we
don't have a tace collector at all.
In practice, this won't make a difference since we always deploy with a
trace collector, but it also makes the code easier to reason about.
Helps with #5975 .
2022-10-26 09:20:52 +00:00
2e83e04eab
feat: Use workspace package metadata to reduce differences and repetition
2022-10-24 13:04:09 -04:00
f3c27c5c71
chore(deps): Bump dotenvy from 0.15.5 to 0.15.6 ( #5881 )
...
Bumps [dotenvy](https://github.com/allan2/dotenvy ) from 0.15.5 to 0.15.6.
- [Release notes](https://github.com/allan2/dotenvy/releases )
- [Changelog](https://github.com/allan2/dotenvy/blob/master/CHANGELOG.md )
- [Commits](https://github.com/allan2/dotenvy/compare/v0.15.5...v0.15.6 )
---
updated-dependencies:
- dependency-name: dotenvy
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-10-18 07:06:40 +00:00
5d835d5047
revert: rdkafka/rskafka swapping ( #5844 )
...
This reverts commit 442a7ff2a4https://github.com/influxdata/influxdb_iox/pull/5800
2022-10-17 12:34:28 +02:00
442a7ff2a4
revert: "revert: rdkafka/rskafka swapping ( #5800 )" ( #5844 )
...
* revert: "revert: rdkafka/rskafka swapping (#5800 )"
This reverts commit b77c3540e1
2022-10-13 09:33:06 +00:00
b77c3540e1
revert: rdkafka/rskafka swapping ( #5800 )
...
This reverts commit 33391af973
2022-10-11 13:01:10 +02:00
33391af973
feat: Swap Kafka Producer implementation back to rdkafka as diagnosis of latency problem ( #5800 )
...
* feat: Add back rdkafka dependency
* feat: Remove RSKafkaProducer
* feat: Remove write buffer RecordAggregator
* feat: Add back rdkafka producer
Using code from 58a2a0b9c8311303c796495db4f167c99a2ea3aa then getting it
to compile with the latest
* feat: Add a metric around enqueue
* fix: Remove unused imports
* fix: Increase Kafka timeout to 20s
* docs: Clarify that Kafka topics should only be created in test/dev envs
* fix: Remove metrics that aren't needed for this experiment
Co-authored-by: Dom <dom@itsallbroken.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-10-11 09:14:45 +00:00
cd4087e00d
style: add no todo!() or dbg!() lints
...
Some crates had theme, some not - lets be consistent and have the
compiler spot dbg!() and todo!() macro calls - they should never be in
prod code!
2022-09-29 13:10:07 +02:00
a5e5dfda6f
refactor: remove dbg!
...
Left over from debugging.
2022-09-29 13:08:47 +02:00
dependabot[bot]
Carol (Nichols || Goulding)
Luke Bond
Marco Neumann
Jake Goulding
Andrew Lamb
Dom Dwyer
YIXIAO SHI
Carol (Nichols || Goulding)