1eab81f47e
chore: ignore `RUSTSEC-2021-0127`
2021-11-30 15:46:34 +01:00
1e515a1dec
feat: load RUB from object store ( #3224 ) ( #3250 )
2021-11-30 14:39:52 +00:00
20da62eb87
chore: Update `arrow`, `arrow-flight`, and `parquet` to 6.3.0 ( #3254 )
...
* chore: Update `arrow`, `arrow-flight`, and `parquet` to 6.3.0
* fix: update metadata size
2021-11-30 12:30:17 +00:00
88acf3788e
feat: rebuild catalog ( #3225 ) ( #3253 )
...
* feat: rebuild catalog (#3225 )
* chore: fix doc
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-11-30 12:17:27 +00:00
6f8d513798
chore: remove temporary logging ( #2297 ) ( #3256 )
2021-11-30 11:11:17 +00:00
f165faf5bc
feat: support forcing persistence of partition irrespective of arrival time ( #3249 )
...
* feat: force persistence of partition irrespective of arrival time
* feat: expose to CLI
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-11-29 22:07:35 +00:00
837e029ed8
refactor: cleanup database recovery ( #3225 ) ( #3245 )
...
* refactor: cleanup database recovery (#3225 )
* chore: fix errors
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-11-29 18:49:46 +00:00
fb9d21cf64
Merge pull request #3248 from influxdata/crepererum/extend_ci_lints
...
ci: run shellcheck and yamllint
2021-11-29 15:59:26 +00:00
5ffb03b83c
Merge branch 'main' into crepererum/extend_ci_lints
2021-11-29 15:51:06 +00:00
0517adf752
feat: test interaction of write buffer and lifecycle ( #2811 ) ( #3211 )
...
* feat: test interaction of write buffer and lifecycle (#2811 )
* chore: more docs
2021-11-29 14:16:43 +00:00
f1e9ac9ef2
ci: run shellcheck
2021-11-29 15:11:15 +01:00
86772a6d20
ci: run yamllint
2021-11-29 15:11:15 +01:00
a5261d4cca
Merge pull request #3246 from influxdata/crepererum/extend_ci_image
...
ci: add `yamllint` and `shellcheck` to CI image
2021-11-29 13:38:04 +00:00
24be4784fb
ci: add `yamllint` and `shellcheck` to CI image
...
This prepares a PR to run these two tools as linters.
2021-11-29 14:10:05 +01:00
c5472973ee
Merge pull request #3244 from influxdata/crepererum/remove_routing_from_database_mode_7
...
refactor: remove old routing / sharding config
2021-11-29 11:46:57 +00:00
23dffefcf8
Merge branch 'main' into crepererum/remove_routing_from_database_mode_7
2021-11-29 11:38:12 +00:00
197634ed50
feat: reload chunk back into read buffer ( #3209 ) ( #3216 )
...
* feat: reload chunk back into read buffer (#3209 )
* chore: fix logical conflict
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-11-29 11:34:55 +00:00
4e043ecb55
refactor: remove old routing / sharding config
...
This is superseded by the new router subsystem.
2021-11-29 12:33:48 +01:00
6ebec4ff71
Merge pull request #3243 from influxdata/crepererum/improve_kafka_client_usage
...
refactor: improve Kafka client usage
2021-11-29 10:54:22 +00:00
8463d941b5
Merge branch 'main' into crepererum/improve_kafka_client_usage
2021-11-29 10:44:11 +00:00
7684794e5c
Merge pull request #3222 from influxdata/crepererum/dml_shard_new_types
...
refactor: consolidate sharding implementation
2021-11-29 10:43:53 +00:00
068b44334d
Merge branch 'main' into crepererum/dml_shard_new_types
2021-11-29 10:33:49 +00:00
d7053cad6b
chore(deps): bump futures-io from 0.3.17 to 0.3.18 ( #3241 )
...
Bumps [futures-io](https://github.com/rust-lang/futures-rs ) from 0.3.17 to 0.3.18.
- [Release notes](https://github.com/rust-lang/futures-rs/releases )
- [Changelog](https://github.com/rust-lang/futures-rs/blob/master/CHANGELOG.md )
- [Commits](https://github.com/rust-lang/futures-rs/compare/0.3.17...0.3.18 )
---
updated-dependencies:
- dependency-name: futures-io
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-11-29 10:33:26 +00:00
0100bfef79
feat: error if persist_age_threshold less than late arrival ( #2005 ) ( #3228 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-11-29 10:21:12 +00:00
24e8c1305c
Merge pull request #3240 from influxdata/dependabot/cargo/futures-channel-0.3.18
...
chore(deps): bump futures-channel from 0.3.17 to 0.3.18
2021-11-29 09:55:32 +00:00
459c14035c
test: ensure that Kafka producers also generate sane metrics
2021-11-29 10:55:08 +01:00
eba6ebd67a
chore(deps): bump futures-channel from 0.3.17 to 0.3.18
...
Bumps [futures-channel](https://github.com/rust-lang/futures-rs ) from 0.3.17 to 0.3.18.
- [Release notes](https://github.com/rust-lang/futures-rs/releases )
- [Changelog](https://github.com/rust-lang/futures-rs/blob/master/CHANGELOG.md )
- [Commits](https://github.com/rust-lang/futures-rs/compare/0.3.17...0.3.18 )
---
updated-dependencies:
- dependency-name: futures-channel
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2021-11-29 09:46:01 +00:00
ef02846fcc
Merge pull request #3239 from influxdata/dependabot/cargo/serde_json-1.0.72
...
chore(deps): bump serde_json from 1.0.71 to 1.0.72
2021-11-29 09:43:49 +00:00
b7952c15a6
refactor: improve Kafka client usage
...
With the new rust-rdkafka release (merged in #3234 ) managing multiple
consumer streams becomes a bit easier. Also we can just reuse consumer
clients for multiple metadata requests. In total that provides:
- use only a single client connection for consumers (we had multiple
connection attempts during startup and one client per stream)
- use only two clients for producers (sadly we need a consumer client to
probe the partitions during startup)
- consumers no longer need to poll the stream to receive statistics
2021-11-29 10:39:09 +01:00
5d784ba866
chore(deps): bump serde_json from 1.0.71 to 1.0.72
...
Bumps [serde_json](https://github.com/serde-rs/json ) from 1.0.71 to 1.0.72.
- [Release notes](https://github.com/serde-rs/json/releases )
- [Commits](https://github.com/serde-rs/json/compare/v1.0.71...v1.0.72 )
---
updated-dependencies:
- dependency-name: serde_json
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2021-11-29 09:28:07 +00:00
0e4afa54bd
Merge pull request #3238 from influxdata/dependabot/cargo/futures-task-0.3.18
...
chore(deps): bump futures-task from 0.3.17 to 0.3.18
2021-11-29 09:21:59 +00:00
0ac4b46ebf
chore(deps): bump futures-task from 0.3.17 to 0.3.18
...
Bumps [futures-task](https://github.com/rust-lang/futures-rs ) from 0.3.17 to 0.3.18.
- [Release notes](https://github.com/rust-lang/futures-rs/releases )
- [Changelog](https://github.com/rust-lang/futures-rs/blob/master/CHANGELOG.md )
- [Commits](https://github.com/rust-lang/futures-rs/compare/0.3.17...0.3.18 )
---
updated-dependencies:
- dependency-name: futures-task
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2021-11-29 09:13:12 +00:00
b3873c946b
Merge pull request #3237 from influxdata/dependabot/cargo/crc32fast-1.2.2
...
chore(deps): bump crc32fast from 1.2.1 to 1.2.2
2021-11-29 09:09:14 +00:00
3bedf6e62a
chore(deps): bump crc32fast from 1.2.1 to 1.2.2
...
Bumps [crc32fast](https://github.com/srijs/rust-crc32fast ) from 1.2.1 to 1.2.2.
- [Release notes](https://github.com/srijs/rust-crc32fast/releases )
- [Commits](https://github.com/srijs/rust-crc32fast/compare/v1.2.1...v1.2.2 )
---
updated-dependencies:
- dependency-name: crc32fast
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2021-11-29 09:00:01 +00:00
5e2dcad821
Merge pull request #3236 from influxdata/dependabot/cargo/predicates-2.1.0
...
chore(deps): bump predicates from 2.0.3 to 2.1.0
2021-11-29 08:57:52 +00:00
b46d07b8a5
chore(deps): bump predicates from 2.0.3 to 2.1.0
...
Bumps [predicates](https://github.com/assert-rs/predicates-rs ) from 2.0.3 to 2.1.0.
- [Release notes](https://github.com/assert-rs/predicates-rs/releases )
- [Changelog](https://github.com/assert-rs/predicates-rs/blob/master/CHANGELOG.md )
- [Commits](https://github.com/assert-rs/predicates-rs/compare/v2.0.3...v2.1.0 )
---
updated-dependencies:
- dependency-name: predicates
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2021-11-29 08:40:45 +00:00
eaff41ac20
Merge pull request #3235 from influxdata/dependabot/cargo/futures-core-0.3.18
...
chore(deps): bump futures-core from 0.3.17 to 0.3.18
2021-11-29 08:38:25 +00:00
b9d804c236
chore(deps): bump futures-core from 0.3.17 to 0.3.18
...
Bumps [futures-core](https://github.com/rust-lang/futures-rs ) from 0.3.17 to 0.3.18.
- [Release notes](https://github.com/rust-lang/futures-rs/releases )
- [Changelog](https://github.com/rust-lang/futures-rs/blob/master/CHANGELOG.md )
- [Commits](https://github.com/rust-lang/futures-rs/compare/0.3.17...0.3.18 )
---
updated-dependencies:
- dependency-name: futures-core
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2021-11-29 08:29:32 +00:00
b74976a8c6
Merge pull request #3234 from influxdata/dependabot/cargo/rdkafka-0.28.0
...
chore(deps): bump rdkafka from 0.27.0 to 0.28.0
2021-11-29 08:27:18 +00:00
c729fc6a25
chore(deps): bump rdkafka from 0.27.0 to 0.28.0
...
Bumps [rdkafka](https://github.com/fede1024/rust-rdkafka ) from 0.27.0 to 0.28.0.
- [Release notes](https://github.com/fede1024/rust-rdkafka/releases )
- [Changelog](https://github.com/fede1024/rust-rdkafka/blob/master/changelog.md )
- [Commits](https://github.com/fede1024/rust-rdkafka/compare/v0.27.0...v0.28.0 )
---
updated-dependencies:
- dependency-name: rdkafka
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2021-11-29 08:18:02 +00:00
82539e625e
Merge pull request #3233 from influxdata/dependabot/cargo/cc-1.0.72
...
chore(deps): bump cc from 1.0.71 to 1.0.72
2021-11-29 08:15:26 +00:00
c0fe972b20
chore(deps): bump cc from 1.0.71 to 1.0.72
...
Bumps [cc](https://github.com/alexcrichton/cc-rs ) from 1.0.71 to 1.0.72.
- [Release notes](https://github.com/alexcrichton/cc-rs/releases )
- [Commits](https://github.com/alexcrichton/cc-rs/compare/1.0.71...1.0.72 )
---
updated-dependencies:
- dependency-name: cc
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2021-11-29 01:29:47 +00:00
ab4f8cdbf2
test: add end to end tests for Storage/Offsets gRPC call ( #3232 )
2021-11-27 12:01:33 +00:00
b5033ce8ef
Merge pull request #3223 from influxdata/offsets-not-implemented
...
fix: Implement a dummy Storage.Offsets method
2021-11-26 16:47:25 +00:00
8fe2c64f5a
fix: Implement a dummy Storage.Offsets method
...
Closes https://github.com/influxdata/conductor/issues/766
2021-11-26 16:39:07 +01:00
a286960285
refactor: remove delete predicates from read filter interface ( #3212 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-11-26 12:15:29 +00:00
f68cbb08ee
Merge pull request #3220 from influxdata/meta-script
...
feat: Add a simple script to decode the IOx metadata of a parquet file
2021-11-26 12:05:42 +00:00
ad65687cba
refactor: consolidate sharding implementation
...
Move sharding implementation from router to `dml` (where the types can
be consumed w/o cloning the data) and only support the new sharding
configs.
The old sharding configs will be removed soon.
2021-11-26 13:00:36 +01:00
473ed75253
Merge branch 'main' into meta-script
2021-11-26 11:57:26 +00:00
f0ace30369
Merge pull request #3221 from influxdata/crepererum/remove_routing_from_database_mode_6
...
refactor: remove write buffer direction
2021-11-26 11:47:46 +00:00
Marco Neumann
Raphael Taylor-Davies
Andrew Lamb
kodiakhq[bot]
dependabot[bot]
Marko Mikulicic