651b7a1ce6
fix: use write_all instead of write
2022-11-23 17:07:44 -05:00
b96de5983d
fix: don't require a reference just to make an owned value
2022-11-23 17:07:44 -05:00
3eda17ea8b
fix: Make SegmentId into a newtype; test the open segment file is created
2022-11-23 17:07:44 -05:00
eea7f113fb
fix: Start implementing traits on Wal to enable outside use
2022-11-23 17:07:44 -05:00
d7134d4846
fix: Cleanups, notes, and clarifications noticed while reading
2022-11-23 17:07:44 -05:00
ef6eda6399
fix: Alphabetize deps
2022-11-23 17:07:44 -05:00
8e414b7c35
fix: Remove custom PartialEq impl; derive works for me 🤷
2022-11-23 17:07:44 -05:00
5c3c0accb1
fix: Add workspace-hack to wal crate
2022-11-23 17:07:44 -05:00
28782caea2
fix: Resolve warnings and format wal code
2022-11-23 17:07:43 -05:00
db63dc55a2
feat: Initial WAL implementation
2022-11-23 17:07:43 -05:00
52d70b060a
test: retention test for querier inthe query_tests ( #6220 )
2022-11-23 17:04:14 +00:00
71b29ae442
chore: Update DataFusion (3 of N) and arrow/parquet to 27.0.0 ( #6208 )
...
* chore: Update DataFusion and arrow/parquet to 27.0.0
* chore: Run cargo hakari tasks
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
2022-11-23 14:21:47 +00:00
9fb1de0428
chore: Update datafusion (2 of N) right before arrow 27 upgrade ( #6207 )
...
* chore: Update datafusion (2 of N) right before arrow 27 upgrade
* fix: Update tests for better unsigned pushdown
* chore: Run cargo hakari tasks
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
2022-11-23 11:04:14 +00:00
2306c383f3
feat: Introduce InfluxQL to Flight ( #6166 )
...
* feat: Introduce InfluxQL to Flight
All InfluxQL queries will fail with an error
* chore: Temper protobuf lint
* chore: Finalize flight.proto changes; fix tests
* chore: Add tests for InfluxQL planner
* chore: Update docs
* chore: Update docs
* chore: Rename back to original
* chore: Use .into() rather than cast
* chore: Use function rather than field
* chore: Improved InfluxQL planner name
* chore: Restore `impl Into<String>` argument
* chore: Add a comment that Go clients are unable to execute InfluxQL
* chore: Add a test for the `--lang` argument and InfluxQL
2022-11-23 00:33:49 +00:00
1a1ea74cb7
chore: Upgrade datafusion again ( #6160 )
...
* Revert "Revert "chore: Update datafusion again (#6108 )""
This reverts commit 766b3bbeb440618cfe332f6ee7d4f8a8217acc48.
* fix: Respect the partition sort key
* chore: update plans
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-11-22 19:28:26 +00:00
f89d542715
refactor: Minor cleanup of retention predicate code ( #6211 )
...
* refactor: Minor cleanup of retention predicate code
* fix: use cow
2022-11-22 18:28:54 +00:00
dd1755b23a
feat: querier filters data outsude retnetion period ( #6209 )
2022-11-22 15:41:00 +00:00
255a168d07
refactor: Refactor ParquetFileCombining into a builder and plan, and add sort exec test ( #6196 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-11-22 14:47:47 +00:00
bf9c1f03d4
Merge pull request #6206 from influxdata/dom/buffer-tree
...
refactor(ingester): split out BufferTree from Shard
2022-11-22 14:41:12 +00:00
9eafa9dbed
style: consistent import ordering
...
Reorder all imports in the ingester to match a consistent order:
* stdlib
* external crates
* intra-crate imports
This helps prevent merge conflicts & keeps everything tidy.
2022-11-22 14:11:10 +01:00
ee8b728c32
refactor: decouple Shard & BufferTree
...
Splits out the nested tree of namespace -> tables -> partitions
(referred to as the "buffer tree") from the Shard which previously held
the namespace map.
This allows the BufferTree to exist without a shard, or many trees to
exist within a shard, etc.
2022-11-22 14:11:10 +01:00
e4c12fa6a5
fix: slice flight response batches ( #6205 )
...
* fix: slice flight response batches
Same as #6094 but for the Apache Flight interface.
Ref https://github.com/influxdata/idpe/issues/16073 .
* refactor: use `RecordBatch::slice`
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-11-22 12:25:23 +00:00
caa595a6fc
chore(deps): Bump serde_json from 1.0.88 to 1.0.89 ( #6203 )
...
Bumps [serde_json](https://github.com/serde-rs/json ) from 1.0.88 to 1.0.89.
- [Release notes](https://github.com/serde-rs/json/releases )
- [Commits](https://github.com/serde-rs/json/compare/v1.0.88...v1.0.89 )
---
updated-dependencies:
- dependency-name: serde_json
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-11-22 09:28:31 +00:00
29bfb4835f
chore(deps): Bump crossbeam-utils from 0.8.12 to 0.8.14 ( #6204 )
...
Bumps [crossbeam-utils](https://github.com/crossbeam-rs/crossbeam ) from 0.8.12 to 0.8.14.
- [Release notes](https://github.com/crossbeam-rs/crossbeam/releases )
- [Changelog](https://github.com/crossbeam-rs/crossbeam/blob/master/CHANGELOG.md )
- [Commits](https://github.com/crossbeam-rs/crossbeam/compare/crossbeam-utils-0.8.12...crossbeam-utils-0.8.14 )
---
updated-dependencies:
- dependency-name: crossbeam-utils
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-11-22 09:21:35 +00:00
0c6afd7dbe
refactor: tune circuit breaker config ( #6202 )
...
At the moment it takes way to long to half-open and close circuits ones
they were opened.
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-11-22 09:03:04 +00:00
63bd302b30
chore(deps): Bump comfy-table from 6.1.2 to 6.1.3 ( #6200 )
...
Bumps [comfy-table](https://github.com/nukesor/comfy-table ) from 6.1.2 to 6.1.3.
- [Release notes](https://github.com/nukesor/comfy-table/releases )
- [Changelog](https://github.com/Nukesor/comfy-table/blob/main/CHANGELOG.md )
- [Commits](https://github.com/nukesor/comfy-table/compare/v6.1.2...v6.1.3 )
---
updated-dependencies:
- dependency-name: comfy-table
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-11-22 08:55:46 +00:00
04c00bbb62
chore(deps): Bump bytes from 1.2.1 to 1.3.0 ( #6199 )
...
Bumps [bytes](https://github.com/tokio-rs/bytes ) from 1.2.1 to 1.3.0.
- [Release notes](https://github.com/tokio-rs/bytes/releases )
- [Changelog](https://github.com/tokio-rs/bytes/blob/master/CHANGELOG.md )
- [Commits](https://github.com/tokio-rs/bytes/commits )
---
updated-dependencies:
- dependency-name: bytes
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-11-22 08:23:24 +00:00
846c8c813b
chore: Update crossbeam-utils ( #6201 )
2022-11-22 08:15:32 +00:00
e5f921fccd
Merge pull request #6193 from influxdata/dom/move-sequence-range
...
refactor: move SequenceNumberRange
2022-11-21 16:25:28 +00:00
097f0acb85
refactor: move SequenceNumberRange
...
Moves the SequenceNumberRange type out of "data" and into the root to be
reused outside of the data module. This construct is universally useful
across all the ingester code.
2022-11-21 16:11:55 +01:00
586c3fe8ea
Merge pull request #6189 from influxdata/dom/dml-sink-error
...
refactor: decouple DmlSink error type
2022-11-21 15:05:03 +00:00
1938c18c50
refactor: decouple DmlSink error type
...
Allows different DmlSink implementations to return different error
types. This allows for small, concise errors that are local to the
DmlSink implementation and specific to it. This helps avoid bloated
"kitchen sink" error types.
2022-11-21 15:29:13 +01:00
3000a2c34a
Merge pull request #6188 from influxdata/dom/move-dml-sink
...
refactor: move DmlSink
2022-11-21 14:28:00 +00:00
64c9d87b9b
refactor: move DmlSink
...
Extracts the DmlSink trait into its own module - it is independent of
the Kafka handler and will be reused.
2022-11-21 15:02:24 +01:00
67af105bde
feat: add `process_start_time_seconds` metric ( #6187 )
...
This is helpful to reason about IOx versions and changes/alerts during
deployments.
2022-11-21 12:27:04 +00:00
a9db7581cd
chore(deps): Bump tokio from 1.21.2 to 1.22.0 ( #6183 )
...
Bumps [tokio](https://github.com/tokio-rs/tokio ) from 1.21.2 to 1.22.0.
- [Release notes](https://github.com/tokio-rs/tokio/releases )
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.21.2...tokio-1.22.0 )
---
updated-dependencies:
- dependency-name: tokio
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-11-21 10:21:24 +00:00
6991b03329
chore(deps): Bump cc from 1.0.76 to 1.0.77 ( #6182 )
...
Bumps [cc](https://github.com/rust-lang/cc-rs ) from 1.0.76 to 1.0.77.
- [Release notes](https://github.com/rust-lang/cc-rs/releases )
- [Commits](https://github.com/rust-lang/cc-rs/compare/1.0.76...1.0.77 )
---
updated-dependencies:
- dependency-name: cc
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-11-21 10:14:52 +00:00
54c8b0a10a
chore(deps): Bump snap from 1.0.5 to 1.1.0 ( #6185 )
...
Bumps [snap](https://github.com/BurntSushi/rust-snappy ) from 1.0.5 to 1.1.0.
- [Release notes](https://github.com/BurntSushi/rust-snappy/releases )
- [Commits](https://github.com/BurntSushi/rust-snappy/compare/1.0.5...1.1.0 )
---
updated-dependencies:
- dependency-name: snap
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-11-21 10:08:11 +00:00
9746ec0b00
chore(deps): Bump crossbeam-utils from 0.8.12 to 0.8.13 ( #6184 )
...
Bumps [crossbeam-utils](https://github.com/crossbeam-rs/crossbeam ) from 0.8.12 to 0.8.13.
- [Release notes](https://github.com/crossbeam-rs/crossbeam/releases )
- [Changelog](https://github.com/crossbeam-rs/crossbeam/blob/master/CHANGELOG.md )
- [Commits](https://github.com/crossbeam-rs/crossbeam/compare/crossbeam-utils-0.8.12...crossbeam-utils-0.8.13 )
---
updated-dependencies:
- dependency-name: crossbeam-utils
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-11-21 10:01:37 +00:00
5ff06060b9
Merge pull request #6186 from influxdata/dependabot/cargo/serde_json-1.0.88
...
chore(deps): Bump serde_json from 1.0.87 to 1.0.88
2022-11-21 09:54:34 +00:00
52c50c16e1
chore(deps): Bump serde_json from 1.0.87 to 1.0.88
...
Bumps [serde_json](https://github.com/serde-rs/json ) from 1.0.87 to 1.0.88.
- [Release notes](https://github.com/serde-rs/json/releases )
- [Commits](https://github.com/serde-rs/json/compare/v1.0.87...v1.0.88 )
---
updated-dependencies:
- dependency-name: serde_json
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2022-11-21 01:52:18 +00:00
4d55efe655
Merge pull request #6178 from influxdata/dom/ingester-rpc-write
...
feat(ingester): rpc write handler
2022-11-18 16:43:53 +00:00
21ca6bff88
Merge branch 'main' into dom/ingester-rpc-write
2022-11-18 16:37:17 +00:00
d41c1290cf
chore: copy iox images skopeo gar registry ( #6172 )
...
* chore: copy the iox images to the gar registry with skopeo
* chore: add quotes
2022-11-18 16:37:08 +00:00
7103ecd733
Merge branch 'main' into dom/ingester-rpc-write
2022-11-18 16:34:36 +00:00
85c8d16680
refactor: add a message to unreachable!()
...
Adds a message to say an impossible thing is impossible.
2022-11-18 17:33:58 +01:00
9dc32f1c16
refactor: remove names from DML init
...
Fixes conflicts introduced by #6170 .
2022-11-18 17:31:56 +01:00
d2b66ff42c
Merge pull request #6174 from influxdata/dom/write-rpc-client
...
feat(router): handler for direct write to ingester
2022-11-18 16:28:38 +00:00
cc55ab384c
Merge branch 'main' into dom/write-rpc-client
2022-11-18 16:22:19 +00:00
4630bbb956
feat: push down all predicates ( #6042 )
...
* feat: push down all predicates
* fix: fmt
* fix: fmt
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-11-18 16:22:01 +00:00
Jake Goulding
Carol (Nichols || Goulding)
Paul Dix
Nga Tran
Andrew Lamb
Stuart Carnie
Dom
Marco Neumann
dependabot[bot]
kodiakhq[bot]
jaymebrd