1f18738203
chore: Run cargo hakari tasks
2023-01-16 02:07:11 +00:00
0a70e9f43f
chore(deps): Bump rustyline from 10.0.0 to 10.1.0
...
Bumps [rustyline](https://github.com/kkawakam/rustyline ) from 10.0.0 to 10.1.0.
- [Release notes](https://github.com/kkawakam/rustyline/releases )
- [Changelog](https://github.com/kkawakam/rustyline/blob/master/History.md )
- [Commits](https://github.com/kkawakam/rustyline/compare/v10.0.0...v10.1.0 )
---
updated-dependencies:
- dependency-name: rustyline
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-01-16 02:06:24 +00:00
3f6bb3e330
feat: Parse IANA timezones in an InfluxQL TZ clause ( #6585 )
...
* feat: Parse IANA timezone strings to chrono_tz::Tz
* feat: Visitors can customise the return error type
This avoids having to remap errors from `&'static str` to the caller's
error type, and will be used in a future PR for time range expressions.
* chore: Run cargo hakari tasks
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
2023-01-15 22:00:41 +00:00
1c9fdd38d6
chore(deps): Bump io-lifetimes from 1.0.3 to 1.0.4 ( #6584 )
...
Bumps [io-lifetimes](https://github.com/sunfishcode/io-lifetimes ) from 1.0.3 to 1.0.4.
- [Release notes](https://github.com/sunfishcode/io-lifetimes/releases )
- [Commits](https://github.com/sunfishcode/io-lifetimes/compare/v1.0.3...1.0.4 )
---
updated-dependencies:
- dependency-name: io-lifetimes
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-01-13 09:21:06 +00:00
e9b3efb33d
refactor: Extract a method for making requests to the ingester onto MiniCluster
2023-01-12 11:03:10 -05:00
0d111c4672
refactor: delegate frontend shutdown to backend
...
Prior to this commit, the (happy path) shutdown sequence of an IOx
process was hard coded to:
1. Stop gRPC & HTTP servers
2. Stop backend server (i.e. ingester2)
After this commit, the execution of step 1 is delegated to the handler
for step 2; the server implementation (router / ingester / querier /
etc) now chooses when to shut down the RPC & HTTP servers.
This allows the server shutdown delegate to correctly sequence the
shutdown of all components of the IOx server. This allows ingester2 to
correctly sequence the shutdown of the query RPC server w.r.t the
graceful stop & persist, ensuring queries continue to be serviced.
2023-01-12 14:59:50 +01:00
8565ca5137
chore(deps): Bump zstd from 0.12.1+zstd.1.5.2 to 0.12.2+zstd.1.5.2 ( #6573 )
...
Bumps [zstd](https://github.com/gyscos/zstd-rs ) from 0.12.1+zstd.1.5.2 to 0.12.2+zstd.1.5.2.
- [Release notes](https://github.com/gyscos/zstd-rs/releases )
- [Commits](https://github.com/gyscos/zstd-rs/commits )
---
updated-dependencies:
- dependency-name: zstd
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Dom <dom@itsallbroken.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-01-12 09:54:28 +00:00
cc6d8709c9
chore(deps): Bump hashbrown from 0.13.1 to 0.13.2 ( #6574 )
...
Bumps [hashbrown](https://github.com/rust-lang/hashbrown ) from 0.13.1 to 0.13.2.
- [Release notes](https://github.com/rust-lang/hashbrown/releases )
- [Changelog](https://github.com/rust-lang/hashbrown/blob/master/CHANGELOG.md )
- [Commits](https://github.com/rust-lang/hashbrown/compare/v0.13.1...v0.13.2 )
---
updated-dependencies:
- dependency-name: hashbrown
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Dom <dom@itsallbroken.com>
2023-01-12 09:47:15 +00:00
be278dc689
chore(deps): Bump prost-types from 0.11.5 to 0.11.6 ( #6559 )
...
Bumps [prost-types](https://github.com/tokio-rs/prost ) from 0.11.5 to 0.11.6.
- [Release notes](https://github.com/tokio-rs/prost/releases )
- [Commits](https://github.com/tokio-rs/prost/compare/v0.11.5...v0.11.6 )
---
updated-dependencies:
- dependency-name: prost-types
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-01-11 16:16:04 +00:00
93a16c78d7
chore(deps): Bump bzip2 from 0.4.3 to 0.4.4 ( #6563 )
...
Bumps [bzip2](https://github.com/alexcrichton/bzip2-rs ) from 0.4.3 to 0.4.4.
- [Release notes](https://github.com/alexcrichton/bzip2-rs/releases )
- [Commits](https://github.com/alexcrichton/bzip2-rs/commits/0.4.4 )
---
updated-dependencies:
- dependency-name: bzip2
dependency-type: indirect
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-01-11 11:11:39 +00:00
e9f9948bea
chore(deps): Bump zstd-sys from 2.0.4+zstd.1.5.2 to 2.0.5+zstd.1.5.2 ( #6557 )
...
Bumps [zstd-sys](https://github.com/gyscos/zstd-rs ) from 2.0.4+zstd.1.5.2 to 2.0.5+zstd.1.5.2.
- [Release notes](https://github.com/gyscos/zstd-rs/releases )
- [Commits](https://github.com/gyscos/zstd-rs/commits )
---
updated-dependencies:
- dependency-name: zstd-sys
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-01-11 10:52:20 +00:00
43a0280365
chore(deps): Bump prost from 0.11.5 to 0.11.6
...
Bumps [prost](https://github.com/tokio-rs/prost ) from 0.11.5 to 0.11.6.
- [Release notes](https://github.com/tokio-rs/prost/releases )
- [Commits](https://github.com/tokio-rs/prost/compare/v0.11.5...v0.11.6 )
---
updated-dependencies:
- dependency-name: prost
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-01-11 02:53:33 +00:00
66047f4372
feat: InfluxQL learns how to plan some InfluxQL queries ( #6520 )
...
* feat: InfluxQL learns how to plan some queries
Also added a means to test the planner and execution
* chore: Update module docs
* chore: Document the planner functions
* chore: Update end_to_end_cases crate
* chore: Clarify why `SLIMIT` and `SOFFSET` return `NotImplemented`
* chore: Address lint issues
* chore: Fix rustdoc link issue
* chore: Remove InfluxQL tests from query_tests crate
Will follow conventions established by @carols10cents when
new query_tests crate is merged.
* chore: `now` field
`now` is a DataFusion built-in scalar function
* chore: remove unused code
* chore: Add additional arithmetic expression tests
* chore: Establish pattern for identifying and tracking InfluxQL issues
* chore: Add tests for case sensitivity issues
* chore: group tests into modules and functions
This avoids mass rewriting of insta snapshots as new
tests are added to each function. When tests are added in the middle,
existing snapshots are renamed (-N+1, -N+2, etc) resulting in
having to review numerous additional snapshots.
2023-01-11 02:50:49 +00:00
c68049c37a
chore(deps): Bump regex from 1.7.0 to 1.7.1 ( #6546 )
...
Bumps [regex](https://github.com/rust-lang/regex ) from 1.7.0 to 1.7.1.
- [Release notes](https://github.com/rust-lang/regex/releases )
- [Changelog](https://github.com/rust-lang/regex/blob/master/CHANGELOG.md )
- [Commits](https://github.com/rust-lang/regex/compare/1.7.0...1.7.1 )
---
updated-dependencies:
- dependency-name: regex
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-01-10 09:55:41 +00:00
b49cc2e35e
chore(deps): Bump tokio from 1.24.0 to 1.24.1 ( #6545 )
...
Bumps [tokio](https://github.com/tokio-rs/tokio ) from 1.24.0 to 1.24.1.
- [Release notes](https://github.com/tokio-rs/tokio/releases )
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.24.0...tokio-1.24.1 )
---
updated-dependencies:
- dependency-name: tokio
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-01-10 09:48:44 +00:00
09746c3aea
chore(deps): Bump assert_cmd from 2.0.7 to 2.0.8
...
Bumps [assert_cmd](https://github.com/assert-rs/assert_cmd ) from 2.0.7 to 2.0.8.
- [Release notes](https://github.com/assert-rs/assert_cmd/releases )
- [Changelog](https://github.com/assert-rs/assert_cmd/blob/master/CHANGELOG.md )
- [Commits](https://github.com/assert-rs/assert_cmd/compare/v2.0.7...v2.0.8 )
---
updated-dependencies:
- dependency-name: assert_cmd
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-01-10 01:06:02 +00:00
828992c9c5
feat: Ingest replica skeleton ( #6529 )
...
* feat: Update replication.proto
* Remove the PartitionId in the replicate request as a single replicate request can have the data for many partitions.
* Add namespace_id and table_id to persist complete request to make data easier to lookup in buffer.
* feat: Initial ingest_replica skeleton
A bunch of copy pasta here from ingester2, but this takes out a ton of stuff that isn't used in replicas.
Also lays the groundwork for the simpler buffer structure to keep the data and a basic cache for catalog information that will be required.
* feat: update replication.proto GetPartitionBufferResponse
* chore: PR cleanup
* chore: PR cleanup
2023-01-09 16:53:49 +00:00
7f7f0ca170
chore: bump tokio 1.22 -> 1.24
...
Fixes a bug on windows:
https://github.com/tokio-rs/tokio/security/advisories/GHSA-7rrj-xr53-82p7
2023-01-09 16:25:35 +01:00
fa6b07f3a0
refactor(trogging): remove atty dependency
...
Replace atty with IsTerminal due to:
https://rustsec.org/advisories/RUSTSEC-2021-0145
2023-01-09 16:14:11 +01:00
c313db8dfe
Merge branch 'main' into dom/seqnum-set
2023-01-09 10:33:53 +00:00
3fac475b63
chore(deps): Bump base64 from 0.20.0 to 0.21.0 ( #6530 )
...
* chore(deps): Bump base64 from 0.20.0 to 0.21.0
Bumps [base64](https://github.com/marshallpierce/rust-base64 ) from 0.20.0 to 0.21.0.
- [Release notes](https://github.com/marshallpierce/rust-base64/releases )
- [Changelog](https://github.com/marshallpierce/rust-base64/blob/master/RELEASE-NOTES.md )
- [Commits](https://github.com/marshallpierce/rust-base64/compare/v0.20.0...v0.21.0 )
---
updated-dependencies:
- dependency-name: base64
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* chore: Run cargo hakari tasks
* fix: deprecations
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: Marco Neumann <marco@crepererum.net>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-01-09 09:39:40 +00:00
dfdc964f64
chore(deps): Bump insta from 1.24.1 to 1.26.0 ( #6534 )
...
* chore(deps): Bump insta from 1.24.1 to 1.26.0
Bumps [insta](https://github.com/mitsuhiko/insta ) from 1.24.1 to 1.26.0.
- [Release notes](https://github.com/mitsuhiko/insta/releases )
- [Changelog](https://github.com/mitsuhiko/insta/blob/master/CHANGELOG.md )
- [Commits](https://github.com/mitsuhiko/insta/compare/1.24.1...1.26.0 )
---
updated-dependencies:
- dependency-name: insta
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* chore: Run cargo hakari tasks
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-01-09 08:02:08 +00:00
e31c84a794
chore(deps): Bump async-trait from 0.1.60 to 0.1.61 ( #6533 )
...
Bumps [async-trait](https://github.com/dtolnay/async-trait ) from 0.1.60 to 0.1.61.
- [Release notes](https://github.com/dtolnay/async-trait/releases )
- [Commits](https://github.com/dtolnay/async-trait/compare/0.1.60...0.1.61 )
---
updated-dependencies:
- dependency-name: async-trait
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-01-09 07:44:35 +00:00
3b672c223c
feat(data_types): SequenceNumberSet
...
Adds a space-efficient encoding of a set of SequenceNumber, backed by
roaring bitmaps.
Memory utilisation will change as the number of elements changes,
according to the underlying roaring bitmap design, but is intended to be
"relatively" cheap.
2023-01-05 18:20:48 +01:00
9ab86fa154
fix(router2): drive ingester node (re)-discovery
...
The tonic / tower load-balance implementation discards failed nodes,
even when using a static list - this causes nodes that fail once to
never be retried.
This doesn't happen for the last node for some reason, and leads to all
the load from one router hitting a single ingester instead of load
balancing across all ingesters.
This commit adds a hack to constantly tell the load balancer to probe
all nodes, hopefully causing them to re-discover previously failed
nodes. I don't have the time to do this properly :(
2023-01-05 14:06:29 +01:00
84485b57b0
chore(deps): Bump insta from 1.24.0 to 1.24.1 ( #6500 )
...
Bumps [insta](https://github.com/mitsuhiko/insta ) from 1.24.0 to 1.24.1.
- [Release notes](https://github.com/mitsuhiko/insta/releases )
- [Changelog](https://github.com/mitsuhiko/insta/blob/master/CHANGELOG.md )
- [Commits](https://github.com/mitsuhiko/insta/compare/1.24.0...1.24.1 )
---
updated-dependencies:
- dependency-name: insta
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-05 08:49:05 +00:00
6843eee1d2
feat: Extract encoding from `RecordBatch` --> `FlightData` from flight implementations ( #6460 )
...
* feat: Extract encoding from `RecordBatch` --> `FlightData` from flight implementations
Refactor existing flight server impl
* fix: Apply suggestions from code review
Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
* fix: fixup code review comments
* fix: update for more details
* fix: Update names / types
Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-01-04 13:36:16 +00:00
8bb56f6ef5
chore(deps): Bump insta from 1.23.0 to 1.24.0 ( #6491 )
...
Bumps [insta](https://github.com/mitsuhiko/insta ) from 1.23.0 to 1.24.0.
- [Release notes](https://github.com/mitsuhiko/insta/releases )
- [Changelog](https://github.com/mitsuhiko/insta/blob/master/CHANGELOG.md )
- [Commits](https://github.com/mitsuhiko/insta/compare/1.23.0...1.24.0 )
---
updated-dependencies:
- dependency-name: insta
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-04 07:11:08 +00:00
dbe52f1ca1
chore: Upgrade datafusion ( #6467 )
...
* chore: Update datafusion
* fix: Update for new apis
* chore: Update expected plan
* fix: Update for new config construction
* chore: update clippy
* fix: Fix error codes
* fix: update another test
* chore: Run cargo hakari tasks
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-01-03 15:29:11 +00:00
90d53ae383
chore(deps): Bump sqlparser from 0.28.0 to 0.30.0 ( #6479 )
...
* chore(deps): Bump sqlparser from 0.28.0 to 0.30.0
Bumps [sqlparser](https://github.com/sqlparser-rs/sqlparser-rs ) from 0.28.0 to 0.30.0.
- [Release notes](https://github.com/sqlparser-rs/sqlparser-rs/releases )
- [Changelog](https://github.com/sqlparser-rs/sqlparser-rs/blob/main/CHANGELOG.md )
- [Commits](https://github.com/sqlparser-rs/sqlparser-rs/compare/v0.28.0...v0.30.0 )
---
updated-dependencies:
- dependency-name: sqlparser
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* fix: Update for new API
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
2023-01-03 14:28:15 +00:00
4add55d39e
feat: InfluxQL planner learns how to normalise InfluxQL AST ( #6236 )
...
* chore: Move logic to context, in line with DataFusion SQL
* chore: Add ordering for InfluxQL data types
Ordering is used to determine automatic casting operations. If two
field columns are present in an expression, one float and one integer,
the integer should be cast to a float, such that the final expression
will be a float.
* chore: Add DerefMut trait to collection types
Will allow these collections to be mutated when traversing the InfluxQL
AST.
* chore: Add influxql module with initial AST normalisation implementation
* chore: Add more unit tests and docs
* chore: Run cargo hakari tasks
* chore: Fix link
* chore: Support regular expression expansion and Call expressions
* chore: Add tests for walk_expr functions
* chore: Add insta snapshot files
* chore: Add docs and make API accessible to the crate
* chore: Move to Arc<dyn SchemaProvider> for use in influxql planner
* chore: Move code back; it is better encapsulated here
* chore: Remove redundant attribute
* chore: Improve regex compatibility with InfluxQL / Go
* chore: Style improvement.
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
2023-01-02 23:48:21 +00:00
0aacef3c59
chore(deps): Bump once_cell from 1.16.0 to 1.17.0 ( #6473 )
...
* chore(deps): Bump once_cell from 1.16.0 to 1.17.0
Bumps [once_cell](https://github.com/matklad/once_cell ) from 1.16.0 to 1.17.0.
- [Release notes](https://github.com/matklad/once_cell/releases )
- [Changelog](https://github.com/matklad/once_cell/blob/master/CHANGELOG.md )
- [Commits](https://github.com/matklad/once_cell/compare/v1.16.0...v1.17.0 )
---
updated-dependencies:
- dependency-name: once_cell
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* chore: Change once_cell version specifier to major.minor for less churn
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Carol (Nichols || Goulding) <carol.nichols@gmail.com>
2023-01-02 17:07:15 +00:00
ec869d2384
chore(deps): Bump comfy-table from 6.1.3 to 6.1.4 ( #6478 )
...
Bumps [comfy-table](https://github.com/nukesor/comfy-table ) from 6.1.3 to 6.1.4.
- [Release notes](https://github.com/nukesor/comfy-table/releases )
- [Changelog](https://github.com/Nukesor/comfy-table/blob/main/CHANGELOG.md )
- [Commits](https://github.com/nukesor/comfy-table/compare/v6.1.3...v6.1.4 )
---
updated-dependencies:
- dependency-name: comfy-table
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-01-02 15:16:11 +00:00
3d7b4e29b9
chore(deps): Bump nom from 7.1.1 to 7.1.2 ( #6477 )
...
Bumps [nom](https://github.com/Geal/nom ) from 7.1.1 to 7.1.2.
- [Release notes](https://github.com/Geal/nom/releases )
- [Changelog](https://github.com/rust-bakery/nom/blob/main/CHANGELOG.md )
- [Commits](https://github.com/Geal/nom/compare/7.1.1...7.1.2 )
---
updated-dependencies:
- dependency-name: nom
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-02 15:09:27 +00:00
0bc4f43319
chore(deps): Bump rustix from 0.36.5 to 0.36.6 ( #6470 )
...
Bumps [rustix](https://github.com/bytecodealliance/rustix ) from 0.36.5 to 0.36.6.
- [Release notes](https://github.com/bytecodealliance/rustix/releases )
- [Commits](https://github.com/bytecodealliance/rustix/compare/v0.36.5...v0.36.6 )
---
updated-dependencies:
- dependency-name: rustix
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-02 15:00:42 +00:00
f56c792f79
chore(deps): Bump serde from 1.0.151 to 1.0.152 ( #6468 )
...
Bumps [serde](https://github.com/serde-rs/serde ) from 1.0.151 to 1.0.152.
- [Release notes](https://github.com/serde-rs/serde/releases )
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.151...v1.0.152 )
---
updated-dependencies:
- dependency-name: serde
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-02 10:27:01 +00:00
febaf3c737
chore(deps): Bump clap from 4.0.31 to 4.0.32 ( #6466 )
...
Bumps [clap](https://github.com/clap-rs/clap ) from 4.0.31 to 4.0.32.
- [Release notes](https://github.com/clap-rs/clap/releases )
- [Changelog](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md )
- [Commits](https://github.com/clap-rs/clap/compare/v4.0.31...v4.0.32 )
---
updated-dependencies:
- dependency-name: clap
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-02 10:14:23 +00:00
7279b9597a
chore(deps): Bump libc from 0.2.138 to 0.2.139 ( #6465 )
...
Bumps [libc](https://github.com/rust-lang/libc ) from 0.2.138 to 0.2.139.
- [Release notes](https://github.com/rust-lang/libc/releases )
- [Commits](https://github.com/rust-lang/libc/compare/0.2.138...0.2.139 )
---
updated-dependencies:
- dependency-name: libc
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-01-02 09:04:29 +00:00
5b9e5b4a92
chore(deps): Bump clap from 4.0.29 to 4.0.31 ( #6464 )
...
Bumps [clap](https://github.com/clap-rs/clap ) from 4.0.29 to 4.0.31.
- [Release notes](https://github.com/clap-rs/clap/releases )
- [Changelog](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md )
- [Commits](https://github.com/clap-rs/clap/compare/v4.0.29...v4.0.31 )
---
updated-dependencies:
- dependency-name: clap
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-12-22 20:00:23 +00:00
858a41c733
chore(deps): Bump handlebars from 4.3.5 to 4.3.6 ( #6458 )
...
Bumps [handlebars](https://github.com/sunng87/handlebars-rust ) from 4.3.5 to 4.3.6.
- [Release notes](https://github.com/sunng87/handlebars-rust/releases )
- [Changelog](https://github.com/sunng87/handlebars-rust/blob/v4.3.6/CHANGELOG.md )
- [Commits](https://github.com/sunng87/handlebars-rust/compare/v4.3.5...v4.3.6 )
---
updated-dependencies:
- dependency-name: handlebars
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-12-22 18:53:45 +00:00
ae296886c2
chore(deps): Bump prost-types from 0.11.2 to 0.11.5 ( #6452 )
...
* chore(deps): Bump prost-types from 0.11.2 to 0.11.5
Bumps [prost-types](https://github.com/tokio-rs/prost ) from 0.11.2 to 0.11.5.
- [Release notes](https://github.com/tokio-rs/prost/releases )
- [Commits](https://github.com/tokio-rs/prost/compare/v0.11.2...v0.11.5 )
---
updated-dependencies:
- dependency-name: prost-types
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* chore: Run cargo hakari tasks
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-12-21 16:31:35 +00:00
ed17069087
chore(deps): Bump num_cpus from 1.14.0 to 1.15.0 ( #6451 )
...
Bumps [num_cpus](https://github.com/seanmonstar/num_cpus ) from 1.14.0 to 1.15.0.
- [Release notes](https://github.com/seanmonstar/num_cpus/releases )
- [Changelog](https://github.com/seanmonstar/num_cpus/blob/master/CHANGELOG.md )
- [Commits](https://github.com/seanmonstar/num_cpus/compare/v1.14.0...v1.15.0 )
---
updated-dependencies:
- dependency-name: num_cpus
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-12-21 15:17:55 +00:00
e1059a9009
feat: FlightSQL Milestone 2 basic FlightSQL client and FlightSQL server implementation and plumbing ( #6398 )
...
* feat: Add basic Flight and FlightSQL client into IOx codebase
Basic flight end to end test
* fix: Apply suggestions from code review
Co-authored-by: Raphael Taylor-Davies <1781103+tustvold@users.noreply.github.com>
Co-authored-by: Raphael Taylor-Davies <1781103+tustvold@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-12-20 17:34:00 +00:00
9d4b2b56ae
chore(deps): Bump snafu from 0.7.3 to 0.7.4
...
Bumps [snafu](https://github.com/shepmaster/snafu ) from 0.7.3 to 0.7.4.
- [Release notes](https://github.com/shepmaster/snafu/releases )
- [Changelog](https://github.com/shepmaster/snafu/blob/main/CHANGELOG.md )
- [Commits](https://github.com/shepmaster/snafu/compare/0.7.3...0.7.4 )
---
updated-dependencies:
- dependency-name: snafu
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2022-12-20 01:04:43 +00:00
d0d5906476
chore: Update datafusion pin ( #6442 )
...
* chore: Update datafusion pin
* refactor: Update iox_query for new apis
* chore: Update some more apis
* chore: Run cargo hakari tasks
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
2022-12-19 20:02:42 +00:00
94c2f94ea1
refactor: Extract common ArrowFlight client into iox_arrow_flight ( #6427 )
...
* refactor: Extract common ArrowFlight client into iox_arrow_flight
* chore: Run cargo hakari tasks
* fix: clarify intent of iox_arrow_flight crate
* refactor: Apply suggestions from code review
Co-authored-by: Raphael Taylor-Davies <1781103+tustvold@users.noreply.github.com>
* fix: loop --> while let
* fix: REmove make_tonic_error in favor of From impl
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: Raphael Taylor-Davies <1781103+tustvold@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-12-19 11:35:20 +00:00
299f0e99f9
chore(deps): Bump thiserror from 1.0.37 to 1.0.38
...
Bumps [thiserror](https://github.com/dtolnay/thiserror ) from 1.0.37 to 1.0.38.
- [Release notes](https://github.com/dtolnay/thiserror/releases )
- [Commits](https://github.com/dtolnay/thiserror/compare/1.0.37...1.0.38 )
---
updated-dependencies:
- dependency-name: thiserror
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2022-12-19 10:33:32 +00:00
8478d41bcb
chore(deps): Bump paste from 1.0.10 to 1.0.11 ( #6430 )
...
Bumps [paste](https://github.com/dtolnay/paste ) from 1.0.10 to 1.0.11.
- [Release notes](https://github.com/dtolnay/paste/releases )
- [Commits](https://github.com/dtolnay/paste/compare/1.0.10...1.0.11 )
---
updated-dependencies:
- dependency-name: paste
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-12-19 10:31:05 +00:00
149de84313
chore(deps): Bump serde from 1.0.150 to 1.0.151 ( #6432 )
...
Bumps [serde](https://github.com/serde-rs/serde ) from 1.0.150 to 1.0.151.
- [Release notes](https://github.com/serde-rs/serde/releases )
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.150...v1.0.151 )
---
updated-dependencies:
- dependency-name: serde
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-12-19 10:24:11 +00:00
ecd6cfa3a8
chore(deps): Bump syn from 1.0.105 to 1.0.107 ( #6434 )
...
Bumps [syn](https://github.com/dtolnay/syn ) from 1.0.105 to 1.0.107.
- [Release notes](https://github.com/dtolnay/syn/releases )
- [Commits](https://github.com/dtolnay/syn/commits )
---
updated-dependencies:
- dependency-name: syn
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-12-19 10:17:11 +00:00
c72734473c
chore(deps): Bump async-trait from 0.1.59 to 0.1.60 ( #6433 )
...
Bumps [async-trait](https://github.com/dtolnay/async-trait ) from 0.1.59 to 0.1.60.
- [Release notes](https://github.com/dtolnay/async-trait/releases )
- [Commits](https://github.com/dtolnay/async-trait/compare/0.1.59...0.1.60 )
---
updated-dependencies:
- dependency-name: async-trait
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-12-19 10:09:23 +00:00
7f2aa8b10c
chore(deps): Bump serde_json from 1.0.89 to 1.0.91
...
Bumps [serde_json](https://github.com/serde-rs/json ) from 1.0.89 to 1.0.91.
- [Release notes](https://github.com/serde-rs/json/releases )
- [Commits](https://github.com/serde-rs/json/compare/v1.0.89...v1.0.91 )
---
updated-dependencies:
- dependency-name: serde_json
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2022-12-19 01:44:18 +00:00
84698b3532
feat: add size_data to mutable batch ( #6425 )
...
This method will be used in the new ingestion pipeline to approximate how much memory a butable batch will take to convert to arrow and persist. It is meant only as a very rough estimate to trigger persistence for hot partitions.
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-12-16 17:20:16 +00:00
cfb7c16bb1
Merge branch 'main' into dom/optimal-persist-parallelism
2022-12-16 09:12:22 +00:00
6a841bdf5a
chore(deps): Bump cc from 1.0.77 to 1.0.78
...
Bumps [cc](https://github.com/rust-lang/cc-rs ) from 1.0.77 to 1.0.78.
- [Release notes](https://github.com/rust-lang/cc-rs/releases )
- [Commits](https://github.com/rust-lang/cc-rs/compare/1.0.77...1.0.78 )
---
updated-dependencies:
- dependency-name: cc
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2022-12-16 01:08:24 +00:00
aec98015d7
fix: Remove the rpc_write feature flag and use INFLUXDB_IOX_MODE env var instead
...
And standardize on ingester2 and router2 for consistency.
Connects to #6402 .
2022-12-15 14:06:59 -05:00
933ab1f8c7
feat(ingester2): optimal persist parallelism
...
This commit changes the behaviour of the persist system to enable
optimal parallelism of persist operations, and improve the accuracy of
the outstanding job bound / back-pressure.
Previously all persist operations for a given partition were
consistently hashed to a single worker task. This serialised persistence
per partition, ensuring all updates to the partition sort key were
serialised. However, this also unnecessarily serialises persist
operations that do not need to update the sort key, reducing the
potential throughput of the system; in the worst case of a single
partition receiving all the writes, only one worker would be persisting,
and the other N-1 workers would be idle.
After this change, the sort key is inspected when enqueuing the persist
operation and if it can be determined that no sort key update is
necessary (the typical case), then the persist task is placed into a
global work queue from which all workers consume. This allows for
maximal parallelisation of these jobs, and the removes the per-worker
head-of-line blocking.
In the case that the sort key does need updating, these jobs continue to
be consistently hashed to a single worker, ensuring serialised sort key
updates only where necessary.
To support these changes, the back-pressure system has been changed to
account for all outstanding persist jobs in the system, regardless of
type or assigned worker - a logical, bounded queue is composed together
of a semaphore limiting the number of persist tasks overall, and a
series of physical, unbounded queues - one to each worker & the global
queue. The overall system remains bounded by the
INFLUXDB_IOX_PERSIST_QUEUE_DEPTH value, and is now simpler to reason
about (it is independent of the number of workers, etc).
2022-12-15 18:30:51 +01:00
6324707110
chore(deps): Bump backtrace from 0.3.66 to 0.3.67 ( #6410 )
...
Bumps [backtrace](https://github.com/rust-lang/backtrace-rs ) from 0.3.66 to 0.3.67.
- [Release notes](https://github.com/rust-lang/backtrace-rs/releases )
- [Commits](https://github.com/rust-lang/backtrace-rs/compare/0.3.66...0.3.67 )
---
updated-dependencies:
- dependency-name: backtrace
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-12-15 09:23:12 +00:00
f82f516c0d
chore(deps): Bump toml from 0.5.9 to 0.5.10 ( #6409 )
...
Bumps [toml](https://github.com/toml-rs/toml ) from 0.5.9 to 0.5.10.
- [Release notes](https://github.com/toml-rs/toml/releases )
- [Commits](https://github.com/toml-rs/toml/commits/toml-v0.5.10 )
---
updated-dependencies:
- dependency-name: toml
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-12-15 08:48:53 +00:00
be45889be1
chore: Upgrade datafusion ( #6407 )
...
* chore: Update datafusion
* chore: Update for API change
2022-12-15 06:51:35 +00:00
d9c72bb93f
feat: optimize wal with batching ( #6399 )
...
* feat: optimize wal with batching
Simplified the wal writer so that it batches up write operations. Currently it waits 10ms between fsync calls. We can pull this out to a config variable later if we want, but I think this is good enough for now.
Also updated the reader to be a more simple blocking reader without the extra tasks and channels as that wasn't really getting us anything that I know of.
* chore: cleanup wal code for PR feedback
2022-12-14 16:07:20 +00:00
8729977851
chore: Upgrade datafusion / arrow to 29.0.0 to get flightsql client ( #6396 )
...
* chore: Update datafusion pin
* chore: Update for API change
* chore: Run cargo hakari tasks
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
2022-12-13 20:16:09 +00:00
9e8ae1485f
Merge branch 'main' into dom/wal-bench
2022-12-13 15:19:32 +00:00
e108a8b6c9
chore(deps): Bump paste from 1.0.9 to 1.0.10 ( #6384 )
...
Bumps [paste](https://github.com/dtolnay/paste ) from 1.0.9 to 1.0.10.
- [Release notes](https://github.com/dtolnay/paste/releases )
- [Commits](https://github.com/dtolnay/paste/compare/1.0.9...1.0.10 )
---
updated-dependencies:
- dependency-name: paste
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-12-13 06:03:05 +00:00
727efcbdee
Merge branch 'main' into cn/ingester2-uuid
2022-12-12 16:21:15 +00:00
336ca761a3
chore: Update datafusion pin (to get sqlparser update) ( #6378 )
...
* chore: Update datafusion pin
* chore: Run cargo hakari tasks
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
2022-12-12 14:39:42 +00:00
7c28a30d1b
test(ingester2): WAL replay benchmark
...
This adds a simple WAL replay benchmark to ingester2 that executes a
replay of a single line of LP.
Unfortunately each file in the benches directory is compiled as it's own
binary/crate, and as such is restricted to importing only "pub" types.
This sucks, as it requires you to either benchmark at a high level
(macro, not microbenchmarks - i.e. benchmarking the ingester startup,
not just the WAL replay) or you are forced to mark the reliant types &
functions as "pub", as well as all the other types/traits they reference
in their signatures. Because the performance sensitive code is usually
towards the lower end of the call stack, this can quickly lead to an
explosion of "pub" types causing a large amount of internal code to be
exported.
Instead this commit uses a middle-ground; benchmarked types & fns are
conditionally marked as "pub" iff the "benches" feature is enabled. This
prevents them from being visible by default, but allows the benchmark
function to call them.
The benchmark itself is also restricted to only run when this feature is
enabled.
2022-12-12 15:02:36 +01:00
e0ecacf6cc
chore: Update DataFusion (get median fix and automatic string to timestamp coercion) ( #6363 )
...
* chore: Update DataFusion pin to get median fix
* chore: Update for new Expr node
* test: add test for median
* test: add test for coercion of strings to timestamps
* chore: Run cargo hakari tasks
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-12-12 12:14:00 +00:00
95969ad24f
chore(deps): Bump base64 from 0.13.1 to 0.20.0 ( #6371 )
...
* chore(deps): Bump base64 from 0.13.1 to 0.20.0
Bumps [base64](https://github.com/marshallpierce/rust-base64 ) from 0.13.1 to 0.20.0.
- [Release notes](https://github.com/marshallpierce/rust-base64/releases )
- [Changelog](https://github.com/marshallpierce/rust-base64/blob/master/RELEASE-NOTES.md )
- [Commits](https://github.com/marshallpierce/rust-base64/compare/v0.13.1...v0.20.0 )
---
updated-dependencies:
- dependency-name: base64
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* chore: Run cargo hakari tasks
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-12-12 07:07:19 +00:00
9305e4c566
chore(deps): Bump insta from 1.22.0 to 1.23.0 ( #6370 )
...
Bumps [insta](https://github.com/mitsuhiko/insta ) from 1.22.0 to 1.23.0.
- [Release notes](https://github.com/mitsuhiko/insta/releases )
- [Changelog](https://github.com/mitsuhiko/insta/blob/master/CHANGELOG.md )
- [Commits](https://github.com/mitsuhiko/insta/compare/1.22.0...1.23.0 )
---
updated-dependencies:
- dependency-name: insta
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-12-12 06:48:35 +00:00
a66895ecdc
chore(deps): Bump serde from 1.0.149 to 1.0.150 ( #6369 )
...
Bumps [serde](https://github.com/serde-rs/serde ) from 1.0.149 to 1.0.150.
- [Release notes](https://github.com/serde-rs/serde/releases )
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.149...v1.0.150 )
---
updated-dependencies:
- dependency-name: serde
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-12-12 06:35:16 +00:00
061d582a9b
chore: patch object_store to get apache#3274 ( #6362 )
...
* chore: patch object_store to get apache#3274
* chore: Run cargo hakari tasks
* fix: add issue breadcrumb
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: Andrew Lamb <alamb@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-12-09 14:52:25 +00:00
0221820123
feat: rate-limit Jaeger UDP messages ( #6354 )
...
* feat: rate-limit Jaeger UDP messages
The Jaeger UDP protocol provides no way to signal backpressure /
overload. In certain situations, we are emitting that many tracing spans
in a short period of time that the OS, the network, or Jaeger drop them.
While a rate limit is not a perfect solution, it for sure helps a lot
(tested locally).
Note that the limiter does NOT lead to unlimited buffering because we
already have a limited outbox queue in place (see
`trace_exporters::export::CHANNEL_SIZE`).
Fixes #5446 .
* fix: only warn ones when the tracing channel is full
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-12-09 09:46:07 +00:00
62db312a8f
feat: Switch to escargot to get more control over running Cargo bins
2022-12-08 15:29:44 -05:00
9175f4a0b5
chore: Upgrade datafusion to get correct support for multi-part identifiers ( #6349 )
...
* test: add tests for periods in measurement names
* chore: Update Datafusion
* chore: Update for changed APIs
* chore: Update expected plan output
* chore: Run cargo hakari tasks
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-12-08 11:27:13 +00:00
8c5c6aee0b
chore(deps): Bump filetime from 0.2.18 to 0.2.19 ( #6350 )
...
Bumps [filetime](https://github.com/alexcrichton/filetime ) from 0.2.18 to 0.2.19.
- [Release notes](https://github.com/alexcrichton/filetime/releases )
- [Commits](https://github.com/alexcrichton/filetime/compare/0.2.18...0.2.19 )
---
updated-dependencies:
- dependency-name: filetime
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-12-08 09:26:52 +00:00
c97b025751
chore(deps): Bump zstd from 0.12.0+zstd.1.5.2 to 0.12.1+zstd.1.5.2 ( #6344 )
...
Bumps [zstd](https://github.com/gyscos/zstd-rs ) from 0.12.0+zstd.1.5.2 to 0.12.1+zstd.1.5.2.
- [Release notes](https://github.com/gyscos/zstd-rs/releases )
- [Commits](https://github.com/gyscos/zstd-rs/compare/v0.12.0...v0.12.1 )
---
updated-dependencies:
- dependency-name: zstd
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-12-07 09:19:35 +00:00
abe60ee0dc
chore(deps): Bump serde from 1.0.148 to 1.0.149 ( #6341 )
...
Bumps [serde](https://github.com/serde-rs/serde ) from 1.0.148 to 1.0.149.
- [Release notes](https://github.com/serde-rs/serde/releases )
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.148...v1.0.149 )
---
updated-dependencies:
- dependency-name: serde
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-12-06 17:10:26 +00:00
687493e8e6
chore(deps): Bump sqlparser from 0.27.0 to 0.28.0 ( #6342 )
...
Bumps [sqlparser](https://github.com/sqlparser-rs/sqlparser-rs ) from 0.27.0 to 0.28.0.
- [Release notes](https://github.com/sqlparser-rs/sqlparser-rs/releases )
- [Changelog](https://github.com/sqlparser-rs/sqlparser-rs/blob/main/CHANGELOG.md )
- [Commits](https://github.com/sqlparser-rs/sqlparser-rs/compare/v0.27.0...v0.28.0 )
---
updated-dependencies:
- dependency-name: sqlparser
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-12-06 12:02:01 +00:00
04e9253ade
chore(deps): Bump rustix from 0.36.4 to 0.36.5 ( #6340 )
...
Bumps [rustix](https://github.com/bytecodealliance/rustix ) from 0.36.4 to 0.36.5.
- [Release notes](https://github.com/bytecodealliance/rustix/releases )
- [Commits](https://github.com/bytecodealliance/rustix/compare/v0.36.4...v0.36.5 )
---
updated-dependencies:
- dependency-name: rustix
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-12-06 09:47:00 +00:00
1d38d400f0
chore(deps): Bump object_store from 0.5.1 to 0.5.2 ( #6339 )
...
* chore(deps): Bump object_store from 0.5.1 to 0.5.2
Bumps [object_store](https://github.com/apache/arrow-rs ) from 0.5.1 to 0.5.2.
- [Release notes](https://github.com/apache/arrow-rs/releases )
- [Changelog](https://github.com/apache/arrow-rs/blob/master/CHANGELOG-old.md )
- [Commits](https://github.com/apache/arrow-rs/compare/object_store_0.5.1...object_store_0.5.2 )
---
updated-dependencies:
- dependency-name: object_store
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* chore: Run cargo hakari tasks
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-12-06 07:53:54 +00:00
2c771ea9f2
chore(deps): Bump mockito from 0.31.0 to 0.31.1 ( #6338 )
...
Bumps [mockito](https://github.com/lipanski/mockito ) from 0.31.0 to 0.31.1.
- [Release notes](https://github.com/lipanski/mockito/releases )
- [Commits](https://github.com/lipanski/mockito/compare/0.31.0...0.31.1 )
---
updated-dependencies:
- dependency-name: mockito
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-12-06 07:44:07 +00:00
c3a2ac99aa
chore(deps): Bump insta from 1.21.2 to 1.22.0 ( #6332 )
...
Bumps [insta](https://github.com/mitsuhiko/insta ) from 1.21.2 to 1.22.0.
- [Release notes](https://github.com/mitsuhiko/insta/releases )
- [Changelog](https://github.com/mitsuhiko/insta/blob/master/CHANGELOG.md )
- [Commits](https://github.com/mitsuhiko/insta/compare/1.21.2...1.22.0 )
---
updated-dependencies:
- dependency-name: insta
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-12-05 10:18:59 +00:00
05cc735849
chore(deps): Bump tower-http from 0.3.4 to 0.3.5 ( #6331 )
...
Bumps [tower-http](https://github.com/tower-rs/tower-http ) from 0.3.4 to 0.3.5.
- [Release notes](https://github.com/tower-rs/tower-http/releases )
- [Commits](https://github.com/tower-rs/tower-http/compare/tower-http-0.3.4...tower-http-0.3.5 )
---
updated-dependencies:
- dependency-name: tower-http
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-12-05 09:18:12 +00:00
fc9431f6ae
chore(deps): Bump assert_cmd from 2.0.6 to 2.0.7 ( #6330 )
...
Bumps [assert_cmd](https://github.com/assert-rs/assert_cmd ) from 2.0.6 to 2.0.7.
- [Release notes](https://github.com/assert-rs/assert_cmd/releases )
- [Changelog](https://github.com/assert-rs/assert_cmd/blob/master/CHANGELOG.md )
- [Commits](https://github.com/assert-rs/assert_cmd/compare/v2.0.6...v2.0.7 )
---
updated-dependencies:
- dependency-name: assert_cmd
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-12-05 09:05:56 +00:00
3740110cb5
chore(deps): Bump libc from 0.2.137 to 0.2.138 ( #6329 )
...
Bumps [libc](https://github.com/rust-lang/libc ) from 0.2.137 to 0.2.138.
- [Release notes](https://github.com/rust-lang/libc/releases )
- [Commits](https://github.com/rust-lang/libc/compare/0.2.137...0.2.138 )
---
updated-dependencies:
- dependency-name: libc
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-12-05 08:49:09 +00:00
66aab55534
feat(ingester2): run persistence task
...
Configures the initialisation of an ingester2 instance to spawn a
persistence task (currently unused) and plumbs in various configuration
parameters.
2022-12-02 17:18:39 +01:00
f524687602
feat(ingester2): parallel partition persistence
...
Implements actor-based, parallel persistence in ingester2 with
controllable fan-out parallelism and queue depths.
This implementation encapsulates the complexity of persistence, queuing
and parallelism - the caller simply uses the handle to persist a
partition, while the actor handles fan-out to a set of persistence
workers, compaction in a separate thread-pool, and optional completion
notifications.
By consistently hashing persist jobs onto workers, parallelism is
achieved across partitions, but serialisation of partition persists is
enforced so that the sort key update is correctly serialised.
2022-12-02 16:34:03 +01:00
9e3d0fcefb
Merge branch 'main' into cn/ingester2
2022-12-02 13:39:55 +00:00
ec2e72d223
test: simplify test executors ( #6312 )
...
Have a single global test executor w/ reasonable defaults. Also don't
require tests to join/await executor shutdowns (most tests forget this
anyways and will get a runtime warning).
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-12-02 11:38:18 +00:00
8742ed6c67
chore(deps): Bump syn from 1.0.104 to 1.0.105 ( #6308 )
...
Bumps [syn](https://github.com/dtolnay/syn ) from 1.0.104 to 1.0.105.
- [Release notes](https://github.com/dtolnay/syn/releases )
- [Commits](https://github.com/dtolnay/syn/compare/1.0.104...1.0.105 )
---
updated-dependencies:
- dependency-name: syn
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-12-02 09:01:19 +00:00
d0f1f6a4fd
chore: Upgrade datafusion to get memory limits ( #6297 )
...
* chore: Update datafusion
* fix: use correctly qualified column names
* chore: Run cargo hakari tasks
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-12-01 16:40:26 +00:00
b9e424582f
refactor: Extract a clap block for the ingester2 RPC write path
...
To be able to share it with the coming all-in-one2 command
2022-12-01 11:39:30 -05:00
0133479122
feat: Create an ingester2 server
2022-12-01 11:39:29 -05:00
99698d0f46
feat: Create an ioxd_ingester2 crate
2022-12-01 11:37:21 -05:00
14a9bc92e9
Revert "Revert "chore: Update Datafusion and arrow/arrow-flight/parquet to `28.0.0` ( #6279 )" ( #6294 )" ( #6296 )
...
This reverts commit b7e52c0d8d
2022-12-01 14:20:43 +00:00
b7e52c0d8d
Revert "chore: Update Datafusion and arrow/arrow-flight/parquet to `28.0.0` ( #6279 )" ( #6294 )
...
This reverts commit 039a45ddd1
2022-12-01 11:38:42 +00:00
c008219692
feat: Add a feature flag to switch to the router RPC write path ( #6247 )
...
* feat: Add a feature flag to switch to the router RPC write path
Fixes #6242 .
* refactor: Remove a weird arc clone/rename that's not needed
I'm sure this was needed at some point, but it doesn't make much sense.
I wasn't going to change this, but I'm now trying to minimize the
differences between this function and the write path init function, so
make this one better too.
* fix: Add the namespace autocreation to the RPC write path too
The topic/query pool don't really apply to this case, but use them
anyway to be able to use the existing catalog methods.
Also add a bunch of comments pointing out where the RPC write path
initializer and the old router's initializer are the same and where
they're different, so that perhaps it'll be easier to keep them in sync
while they both exist.
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-12-01 11:05:39 +00:00
b6b8e6ac10
Merge remote-tracking branch 'origin/main' into dom/wal-write
2022-11-30 13:27:28 -05:00
096d850fd5
fix: Maintain WAL segment file ordering ( #6287 )
...
Rather than naming WAL files with a UUID, give them a number that
indicates the order they were created in so that they can be read back
in order.
Fixes #6227 .
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-11-30 17:37:37 +00:00
d2a3d0920b
feat(ingester2): commit writes to write-ahead log
...
Adds WalSink, an implementation of the DmlSink trait that commits DML
operations to the write-ahead log before passing the DML op into the
decorated inner DmlSink chain.
By structuring the WAL logic as a decorator, the chain of inner DmlSink
handlers within it are only ever invoked after the WAL commit has
successfully completed, which keeps all the WAL commit code in a
singly-responsible component. This also lets us layer on the WAL commit
logic to the DML sink chain after replaying any existing WAL files,
avoiding a circular WAL mess.
The application-logic level WAL code abstracts over the underlying WAL
implementation & codec through the WalAppender trait. This decouples the
business logic from the WAL implementation both for testing purposes,
and for trying different WAL implementations in the future.
2022-11-30 16:37:05 +01:00
039a45ddd1
chore: Update Datafusion and arrow/arrow-flight/parquet to `28.0.0` ( #6279 )
...
* chore: Update Datafusion and arrow/arrow-flight/parquet to `28.0.0`
* chore: Update thrift to 0.17
* fix: use workspace arrow-flight in ingester2
* chore: Update for API changes
* fix: test
* chore: Update hakari
* chore: Update hakari again
* chore: Update trace_exporters to latest thrift
* fix: update test
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-11-30 14:12:30 +00:00
8249396705
Merge branch 'main' into dom/sequence-rpc-write
2022-11-30 12:25:53 +00:00
9356868562
chore(deps): Bump nix from 0.25.0 to 0.26.1 ( #6273 )
...
Bumps [nix](https://github.com/nix-rust/nix ) from 0.25.0 to 0.26.1.
- [Release notes](https://github.com/nix-rust/nix/releases )
- [Changelog](https://github.com/nix-rust/nix/blob/master/CHANGELOG.md )
- [Commits](https://github.com/nix-rust/nix/compare/v0.25.0...v0.26.1 )
---
updated-dependencies:
- dependency-name: nix
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-11-30 10:31:46 +00:00
f9c9e49e10
chore(deps): Bump tonic-reflection from 0.5.0 to 0.6.0 ( #6271 )
...
Bumps [tonic-reflection](https://github.com/hyperium/tonic ) from 0.5.0 to 0.6.0.
- [Release notes](https://github.com/hyperium/tonic/releases )
- [Changelog](https://github.com/hyperium/tonic/blob/master/CHANGELOG.md )
- [Commits](https://github.com/hyperium/tonic/compare/v0.5.0...v0.6.0 )
---
updated-dependencies:
- dependency-name: tonic-reflection
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-11-30 09:57:08 +00:00
ace4b7f669
feat: operation timestamp sequencer
...
Adds a TimestampOracle to provide an ingester-internal ordering to
incoming DmlOperations using a logical clock.
2022-11-30 10:40:22 +01:00
b8e6a89b9b
chore(deps): Bump tonic-build from 0.8.3 to 0.8.4
...
Bumps [tonic-build](https://github.com/hyperium/tonic ) from 0.8.3 to 0.8.4.
- [Release notes](https://github.com/hyperium/tonic/releases )
- [Changelog](https://github.com/hyperium/tonic/blob/master/CHANGELOG.md )
- [Commits](https://github.com/hyperium/tonic/compare/v0.8.3...v0.8.4 )
---
updated-dependencies:
- dependency-name: tonic-build
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2022-11-30 01:18:23 +00:00
3d74790191
chore: update dependencies ( #6267 )
...
* chore: update dependencies
* chore: Run cargo hakari tasks
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-11-29 16:07:20 +00:00
297ea8be55
refactor: make `IOxSessionContext::exec` non-optional ( #6266 )
...
`None` was only used for testing and even than we should probably have a
proper executor instead of panicking for some methods.
Found while working on #6216 .
2022-11-29 14:52:32 +00:00
f22b1e1a09
chore: Update datafusion (to get memory limiting code) ( #6246 )
...
* chore: Update datafusion
* chore: Run cargo hakari tasks
* fix: Update to newer api
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-11-29 12:05:17 +00:00
bce4902e63
chore(deps): Bump serde from 1.0.147 to 1.0.148 ( #6257 )
...
Bumps [serde](https://github.com/serde-rs/serde ) from 1.0.147 to 1.0.148.
- [Release notes](https://github.com/serde-rs/serde/releases )
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.147...v1.0.148 )
---
updated-dependencies:
- dependency-name: serde
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-11-29 11:38:07 +00:00
d1983fc385
chore(deps): Bump zstd-sys from 2.0.3+zstd.1.5.2 to 2.0.4+zstd.1.5.2 ( #6253 )
...
Bumps [zstd-sys](https://github.com/gyscos/zstd-rs ) from 2.0.3+zstd.1.5.2 to 2.0.4+zstd.1.5.2.
- [Release notes](https://github.com/gyscos/zstd-rs/releases )
- [Commits](https://github.com/gyscos/zstd-rs/commits )
---
updated-dependencies:
- dependency-name: zstd-sys
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-11-29 11:28:27 +00:00
be1e5ad8c2
chore(deps): Bump syn from 1.0.103 to 1.0.104 ( #6250 )
...
Bumps [syn](https://github.com/dtolnay/syn ) from 1.0.103 to 1.0.104.
- [Release notes](https://github.com/dtolnay/syn/releases )
- [Commits](https://github.com/dtolnay/syn/compare/1.0.103...1.0.104 )
---
updated-dependencies:
- dependency-name: syn
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-11-29 10:43:21 +00:00
ffd63f564b
chore(deps): Bump tonic-health from 0.7.1 to 0.8.0 ( #6254 )
...
Bumps [tonic-health](https://github.com/hyperium/tonic ) from 0.7.1 to 0.8.0.
- [Release notes](https://github.com/hyperium/tonic/releases )
- [Changelog](https://github.com/hyperium/tonic/blob/master/CHANGELOG.md )
- [Commits](https://github.com/hyperium/tonic/compare/v0.7.1...v0.8.0 )
---
updated-dependencies:
- dependency-name: tonic-health
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-11-29 10:23:48 +00:00
8129887c1f
chore(deps): Bump tonic-build from 0.8.2 to 0.8.3
...
Bumps [tonic-build](https://github.com/hyperium/tonic ) from 0.8.2 to 0.8.3.
- [Release notes](https://github.com/hyperium/tonic/releases )
- [Changelog](https://github.com/hyperium/tonic/blob/master/CHANGELOG.md )
- [Commits](https://github.com/hyperium/tonic/compare/v0.8.2...v0.8.3 )
---
updated-dependencies:
- dependency-name: tonic-build
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2022-11-29 10:02:37 +00:00
896a03fdbc
chore: update `rskafka` ( #6258 )
...
Useful because it updates `zstd` to 0.12. With the upcoming `parquet`
update, we can than drop `zstd` 0.11.
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-11-29 10:00:13 +00:00
b5aa39db4b
chore(deps): Bump tonic from 0.8.2 to 0.8.3 ( #6249 )
...
Bumps [tonic](https://github.com/hyperium/tonic ) from 0.8.2 to 0.8.3.
- [Release notes](https://github.com/hyperium/tonic/releases )
- [Changelog](https://github.com/hyperium/tonic/blob/master/CHANGELOG.md )
- [Commits](https://github.com/hyperium/tonic/compare/v0.8.2...v0.8.3 )
---
updated-dependencies:
- dependency-name: tonic
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-11-29 09:53:30 +00:00
fc5697b8e7
chore: Update datafusion again (N of N) ( #6218 )
...
* chore: Update datafusion again (4 of N)
* fix: Update plans
* fix: Update for renamed API
* fix: Update more plans
* chore: Update to datafusion @ d355f69aae2cc951cfd021e5c0b690861ba0c4ac
* fix: update explain plan tests
* fix: update test after schema error
* chore: Update datafusion again
* fix: Add size() calculation to selectors
* chore: Run cargo hakari tasks
* fix: Update newly added test
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-11-28 17:09:40 +00:00
721dfb9960
chore(deps): Bump insta from 1.21.1 to 1.21.2 ( #6235 )
...
Bumps [insta](https://github.com/mitsuhiko/insta ) from 1.21.1 to 1.21.2.
- [Release notes](https://github.com/mitsuhiko/insta/releases )
- [Changelog](https://github.com/mitsuhiko/insta/blob/master/CHANGELOG.md )
- [Commits](https://github.com/mitsuhiko/insta/compare/1.21.1...1.21.2 )
---
updated-dependencies:
- dependency-name: insta
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-11-28 09:42:03 +00:00
a0ab78298f
feat(ingester2): gRPC methods & type-erased init
...
This commit implements the gRPC direct-write RPC interface (largely
copied from the ingester crate), and adds a much improved RPC query
handler.
Compared to the ingester crate, the query API is now split into two
defined halves - the API handler side, and types necessary to support it
(server/grpc/query.rs) and the Ingester query execution side (a stub in
query/exec.rs). These two halves maintain a separation of concerns, and
are interfaced by an abstract QueryExec trait (in query/trait.rs).
I also added the catalog RPC interface as it is currently exposed on the
ingester, though I am unsure if it is used by anything.
This commit also introduces the "init" module, and the
IngesterRpcInterface trait within it. This trait forms the public
ingester2 crate API, defining the complete set of methods external
crates can expect to utilise in a stable, unchanging and decoupled way.
The IngesterRpcInterface trait also serves as a method of type-erasure
on the underlying handler implementations, avoiding the need to
expose/pub the types, abstractions, and internal implementation details
of the ingester to external crates.
2022-11-25 12:40:01 +01:00
5843e30ba4
Merge branch 'main' into dom/ingester2
2022-11-25 10:07:10 +00:00
74cf99eac7
chore(deps): Bump zstd from 0.11.2+zstd.1.5.2 to 0.12.0+zstd.1.5.2 ( #6225 )
...
* chore(deps): Bump zstd from 0.11.2+zstd.1.5.2 to 0.12.0+zstd.1.5.2
Bumps [zstd](https://github.com/gyscos/zstd-rs ) from 0.11.2+zstd.1.5.2 to 0.12.0+zstd.1.5.2.
- [Release notes](https://github.com/gyscos/zstd-rs/releases )
- [Commits](https://github.com/gyscos/zstd-rs/commits )
---
updated-dependencies:
- dependency-name: zstd
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* chore: Run cargo hakari tasks
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-11-25 10:00:28 +00:00
faf8f33b31
chore(deps): Bump zstd-sys from 2.0.1+zstd.1.5.2 to 2.0.3+zstd.1.5.2 ( #6230 )
...
Bumps [zstd-sys](https://github.com/gyscos/zstd-rs ) from 2.0.1+zstd.1.5.2 to 2.0.3+zstd.1.5.2.
- [Release notes](https://github.com/gyscos/zstd-rs/releases )
- [Commits](https://github.com/gyscos/zstd-rs/commits )
---
updated-dependencies:
- dependency-name: zstd-sys
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-11-25 09:53:48 +00:00
898085300e
chore(deps): Bump flate2 from 1.0.24 to 1.0.25 ( #6231 )
...
Bumps [flate2](https://github.com/rust-lang/flate2-rs ) from 1.0.24 to 1.0.25.
- [Release notes](https://github.com/rust-lang/flate2-rs/releases )
- [Commits](https://github.com/rust-lang/flate2-rs/commits/1.0.25 )
---
updated-dependencies:
- dependency-name: flate2
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-11-25 09:47:24 +00:00
ac6403d347
chore(deps): Bump clap from 4.0.26 to 4.0.27
...
Bumps [clap](https://github.com/clap-rs/clap ) from 4.0.26 to 4.0.27.
- [Release notes](https://github.com/clap-rs/clap/releases )
- [Changelog](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md )
- [Commits](https://github.com/clap-rs/clap/compare/v4.0.26...v4.0.27 )
---
updated-dependencies:
- dependency-name: clap
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2022-11-25 01:06:03 +00:00
44eeab7e2b
chore: Run cargo hakari tasks
2022-11-24 14:51:21 +00:00
a66fc0b645
feat(ingester): ingester2 init
...
Adds an ingester2 crate to hold the MVP of the Kafkaless project.
This was necessary due to the tight coupling of the ingester internals
with tests in external crates, and eases the parallel development of two
version of the ingester.
This commit contains various changes from the "ingester" crate, mostly
removing the concept/references to a "shard" or "ShardId" where
possible.
This commit does not copy over all of the "ingester" crate - only those
components that are definitely needed. I will drag across more as
functionality is implemented.
2022-11-24 15:34:02 +01:00
edd606aa3b
feat: Serialize using protobuf instead of json
2022-11-23 17:07:49 -05:00
5c3c0accb1
fix: Add workspace-hack to wal crate
2022-11-23 17:07:44 -05:00
db63dc55a2
feat: Initial WAL implementation
2022-11-23 17:07:43 -05:00
52d70b060a
test: retention test for querier inthe query_tests ( #6220 )
2022-11-23 17:04:14 +00:00
71b29ae442
chore: Update DataFusion (3 of N) and arrow/parquet to 27.0.0 ( #6208 )
...
* chore: Update DataFusion and arrow/parquet to 27.0.0
* chore: Run cargo hakari tasks
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
2022-11-23 14:21:47 +00:00
9fb1de0428
chore: Update datafusion (2 of N) right before arrow 27 upgrade ( #6207 )
...
* chore: Update datafusion (2 of N) right before arrow 27 upgrade
* fix: Update tests for better unsigned pushdown
* chore: Run cargo hakari tasks
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
2022-11-23 11:04:14 +00:00
2306c383f3
feat: Introduce InfluxQL to Flight ( #6166 )
...
* feat: Introduce InfluxQL to Flight
All InfluxQL queries will fail with an error
* chore: Temper protobuf lint
* chore: Finalize flight.proto changes; fix tests
* chore: Add tests for InfluxQL planner
* chore: Update docs
* chore: Update docs
* chore: Rename back to original
* chore: Use .into() rather than cast
* chore: Use function rather than field
* chore: Improved InfluxQL planner name
* chore: Restore `impl Into<String>` argument
* chore: Add a comment that Go clients are unable to execute InfluxQL
* chore: Add a test for the `--lang` argument and InfluxQL
2022-11-23 00:33:49 +00:00
1a1ea74cb7
chore: Upgrade datafusion again ( #6160 )
...
* Revert "Revert "chore: Update datafusion again (#6108 )""
This reverts commit 766b3bbeb440618cfe332f6ee7d4f8a8217acc48.
* fix: Respect the partition sort key
* chore: update plans
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-11-22 19:28:26 +00:00
caa595a6fc
chore(deps): Bump serde_json from 1.0.88 to 1.0.89 ( #6203 )
...
Bumps [serde_json](https://github.com/serde-rs/json ) from 1.0.88 to 1.0.89.
- [Release notes](https://github.com/serde-rs/json/releases )
- [Commits](https://github.com/serde-rs/json/compare/v1.0.88...v1.0.89 )
---
updated-dependencies:
- dependency-name: serde_json
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-11-22 09:28:31 +00:00
29bfb4835f
chore(deps): Bump crossbeam-utils from 0.8.12 to 0.8.14 ( #6204 )
...
Bumps [crossbeam-utils](https://github.com/crossbeam-rs/crossbeam ) from 0.8.12 to 0.8.14.
- [Release notes](https://github.com/crossbeam-rs/crossbeam/releases )
- [Changelog](https://github.com/crossbeam-rs/crossbeam/blob/master/CHANGELOG.md )
- [Commits](https://github.com/crossbeam-rs/crossbeam/compare/crossbeam-utils-0.8.12...crossbeam-utils-0.8.14 )
---
updated-dependencies:
- dependency-name: crossbeam-utils
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-11-22 09:21:35 +00:00
63bd302b30
chore(deps): Bump comfy-table from 6.1.2 to 6.1.3 ( #6200 )
...
Bumps [comfy-table](https://github.com/nukesor/comfy-table ) from 6.1.2 to 6.1.3.
- [Release notes](https://github.com/nukesor/comfy-table/releases )
- [Changelog](https://github.com/Nukesor/comfy-table/blob/main/CHANGELOG.md )
- [Commits](https://github.com/nukesor/comfy-table/compare/v6.1.2...v6.1.3 )
---
updated-dependencies:
- dependency-name: comfy-table
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-11-22 08:55:46 +00:00
04c00bbb62
chore(deps): Bump bytes from 1.2.1 to 1.3.0 ( #6199 )
...
Bumps [bytes](https://github.com/tokio-rs/bytes ) from 1.2.1 to 1.3.0.
- [Release notes](https://github.com/tokio-rs/bytes/releases )
- [Changelog](https://github.com/tokio-rs/bytes/blob/master/CHANGELOG.md )
- [Commits](https://github.com/tokio-rs/bytes/commits )
---
updated-dependencies:
- dependency-name: bytes
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-11-22 08:23:24 +00:00
846c8c813b
chore: Update crossbeam-utils ( #6201 )
2022-11-22 08:15:32 +00:00
a9db7581cd
chore(deps): Bump tokio from 1.21.2 to 1.22.0 ( #6183 )
...
Bumps [tokio](https://github.com/tokio-rs/tokio ) from 1.21.2 to 1.22.0.
- [Release notes](https://github.com/tokio-rs/tokio/releases )
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.21.2...tokio-1.22.0 )
---
updated-dependencies:
- dependency-name: tokio
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-11-21 10:21:24 +00:00
6991b03329
chore(deps): Bump cc from 1.0.76 to 1.0.77 ( #6182 )
...
Bumps [cc](https://github.com/rust-lang/cc-rs ) from 1.0.76 to 1.0.77.
- [Release notes](https://github.com/rust-lang/cc-rs/releases )
- [Commits](https://github.com/rust-lang/cc-rs/compare/1.0.76...1.0.77 )
---
updated-dependencies:
- dependency-name: cc
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-11-21 10:14:52 +00:00
54c8b0a10a
chore(deps): Bump snap from 1.0.5 to 1.1.0 ( #6185 )
...
Bumps [snap](https://github.com/BurntSushi/rust-snappy ) from 1.0.5 to 1.1.0.
- [Release notes](https://github.com/BurntSushi/rust-snappy/releases )
- [Commits](https://github.com/BurntSushi/rust-snappy/compare/1.0.5...1.1.0 )
---
updated-dependencies:
- dependency-name: snap
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-11-21 10:08:11 +00:00
9746ec0b00
chore(deps): Bump crossbeam-utils from 0.8.12 to 0.8.13 ( #6184 )
...
Bumps [crossbeam-utils](https://github.com/crossbeam-rs/crossbeam ) from 0.8.12 to 0.8.13.
- [Release notes](https://github.com/crossbeam-rs/crossbeam/releases )
- [Changelog](https://github.com/crossbeam-rs/crossbeam/blob/master/CHANGELOG.md )
- [Commits](https://github.com/crossbeam-rs/crossbeam/compare/crossbeam-utils-0.8.12...crossbeam-utils-0.8.13 )
---
updated-dependencies:
- dependency-name: crossbeam-utils
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-11-21 10:01:37 +00:00
52c50c16e1
chore(deps): Bump serde_json from 1.0.87 to 1.0.88
...
Bumps [serde_json](https://github.com/serde-rs/json ) from 1.0.87 to 1.0.88.
- [Release notes](https://github.com/serde-rs/json/releases )
- [Commits](https://github.com/serde-rs/json/compare/v1.0.87...v1.0.88 )
---
updated-dependencies:
- dependency-name: serde_json
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2022-11-21 01:52:18 +00:00
59b3c793d3
Merge branch 'main' into dom/ingester-rpc-write
2022-11-18 16:21:07 +00:00
02c3083192
fix: Remove table names from Dml operations
2022-11-18 10:40:38 -05:00
90dd9906f6
feat(ingester): rpc write endpoint
...
Adds a handler implementation of the gRPC WriteService to receive direct
RPC writes from a router.
This code is currently unused.
2022-11-18 16:36:19 +01:00
84977f63ca
chore(deps): Bump indexmap from 1.9.1 to 1.9.2 ( #6171 )
...
Bumps [indexmap](https://github.com/bluss/indexmap ) from 1.9.1 to 1.9.2.
- [Release notes](https://github.com/bluss/indexmap/releases )
- [Changelog](https://github.com/bluss/indexmap/blob/master/RELEASES.md )
- [Commits](https://github.com/bluss/indexmap/compare/1.9.1...1.9.2 )
---
updated-dependencies:
- dependency-name: indexmap
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-11-18 12:21:01 +00:00
bea3b987aa
chore(deps): Bump digest from 0.10.5 to 0.10.6 ( #6162 )
...
Bumps [digest](https://github.com/RustCrypto/traits ) from 0.10.5 to 0.10.6.
- [Release notes](https://github.com/RustCrypto/traits/releases )
- [Commits](https://github.com/RustCrypto/traits/compare/digest-v0.10.5...digest-v0.10.6 )
---
updated-dependencies:
- dependency-name: digest
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-11-17 10:45:03 +00:00
CircleCI[bot]
dependabot[bot]
Stuart Carnie
Carol (Nichols || Goulding)
Dom Dwyer
Paul Dix
Andrew Lamb
kodiakhq[bot]
Raphael Taylor-Davies
Marco Neumann
Carol (Nichols || Goulding)
Nga Tran