143cb26e87
chore: Update datafusion again ( #6958 )
...
* chore: Update datafusion pin
* chore: Update plans
* chore: Run cargo hakari tasks
---------
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-02-13 10:28:44 +00:00
0cbd9f6a82
chore(deps): Bump tokio-util from 0.7.5 to 0.7.7 ( #6964 )
...
---
updated-dependencies:
- dependency-name: tokio-util
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-02-13 10:10:53 +00:00
083f9bc0a7
chore(deps): Bump zstd-safe from 6.0.3+zstd.1.5.2 to 6.0.4+zstd.1.5.4 ( #6962 )
...
Bumps [zstd-safe](https://github.com/gyscos/zstd-rs ) from 6.0.3+zstd.1.5.2 to 6.0.4+zstd.1.5.4.
- [Release notes](https://github.com/gyscos/zstd-rs/releases )
- [Commits](https://github.com/gyscos/zstd-rs/commits/zstd-safe-6.0.4 )
---
updated-dependencies:
- dependency-name: zstd-safe
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-02-13 09:58:17 +00:00
c805a7527b
chore(deps): Bump zstd-sys from 2.0.6+zstd.1.5.2 to 2.0.7+zstd.1.5.4 ( #6963 )
...
Bumps [zstd-sys](https://github.com/gyscos/zstd-rs ) from 2.0.6+zstd.1.5.2 to 2.0.7+zstd.1.5.4.
- [Release notes](https://github.com/gyscos/zstd-rs/releases )
- [Commits](https://github.com/gyscos/zstd-rs/commits/zstd-sys-2.0.7 )
---
updated-dependencies:
- dependency-name: zstd-sys
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-02-13 09:23:39 +00:00
06d50e8611
chore(deps): Bump croaring from 0.7.0 to 0.8.0 ( #6965 )
...
Bumps [croaring](https://github.com/saulius/croaring-rs ) from 0.7.0 to 0.8.0.
- [Release notes](https://github.com/saulius/croaring-rs/releases )
- [Commits](https://github.com/saulius/croaring-rs/compare/0.7.0...0.8.0 )
---
updated-dependencies:
- dependency-name: croaring
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-02-13 09:09:32 +00:00
501032c2fb
chore: Update datafusion again ( #6929 )
...
* chore: Update DataFusion
* fix: disable repartitioned sorting
* chore: Run cargo hakari tasks
---------
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-02-10 11:51:47 +00:00
c0c9b51b9e
chore(deps): Bump tokio-util from 0.7.4 to 0.7.5 ( #6941 )
...
Bumps [tokio-util](https://github.com/tokio-rs/tokio ) from 0.7.4 to 0.7.5.
- [Release notes](https://github.com/tokio-rs/tokio/releases )
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-util-0.7.4...tokio-util-0.7.5 )
---
updated-dependencies:
- dependency-name: tokio-util
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-02-10 09:42:00 +00:00
7f4fd1013c
feat: measure scratchpad store ( #6925 )
...
* refactor: create `objec_store` submodule
* feat: add gauge for scratchpad size
2023-02-09 13:03:26 +00:00
6327e3d9c0
chore(deps): Bump serde_json from 1.0.92 to 1.0.93 ( #6918 )
...
Bumps [serde_json](https://github.com/serde-rs/json ) from 1.0.92 to 1.0.93.
- [Release notes](https://github.com/serde-rs/json/releases )
- [Commits](https://github.com/serde-rs/json/compare/v1.0.92...v1.0.93 )
---
updated-dependencies:
- dependency-name: serde_json
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-02-09 10:39:33 +00:00
4911f37fe5
chore(deps): Bump filetime from 0.2.19 to 0.2.20 ( #6917 )
...
Bumps [filetime](https://github.com/alexcrichton/filetime ) from 0.2.19 to 0.2.20.
- [Release notes](https://github.com/alexcrichton/filetime/releases )
- [Commits](https://github.com/alexcrichton/filetime/compare/0.2.19...0.2.20 )
---
updated-dependencies:
- dependency-name: filetime
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-02-09 10:13:50 +00:00
2f4d901fbe
chore: Update datafusion ( #6893 )
...
* chore: Update datafusion
* chore: Run cargo hakari tasks
* fix: update for api change
---------
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-02-08 18:27:18 +00:00
0ecde75af5
chore(deps): Bump object_store from 0.5.3 to 0.5.4 ( #6900 )
...
Bumps [object_store](https://github.com/apache/arrow-rs ) from 0.5.3 to 0.5.4.
- [Release notes](https://github.com/apache/arrow-rs/releases )
- [Changelog](https://github.com/apache/arrow-rs/blob/master/CHANGELOG-old.md )
- [Commits](https://github.com/apache/arrow-rs/compare/object_store_0.5.3...object_store_0.5.4 )
---
updated-dependencies:
- dependency-name: object_store
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-02-08 09:40:11 +00:00
bb825faf06
chore(deps): Bump toml from 0.7.1 to 0.7.2 ( #6899 )
...
Bumps [toml](https://github.com/toml-rs/toml ) from 0.7.1 to 0.7.2.
- [Release notes](https://github.com/toml-rs/toml/releases )
- [Commits](https://github.com/toml-rs/toml/compare/toml-v0.7.1...toml-v0.7.2 )
---
updated-dependencies:
- dependency-name: toml
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-02-08 09:11:04 +00:00
f56023b222
feat: use insta_snapshots ( #6884 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-02-07 15:08:47 +00:00
eb245d6774
feat: Initial SQLite catalog schema ( #6851 )
...
* feat: Initial SQLite catalog schema
* chore: Run cargo hakari tasks
* feat: impls, many TODOs
* feat: completed `todo!()`'s
* chore: add remaining tests from postgres module
* feat: add SQLite to get_catalog API
* chore: Add docs
* chore: Placate clippy
* chore: Placate clippy
* chore: PR feedback from @domodwyer
---------
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
2023-02-06 22:55:14 +00:00
b8ec022ea6
test(e2e): HTTP error response
...
Ensure a HTTP error response contains a well-formed JSON structure
containing "code" and "message" fields (for backwards compatibility with
existing InfluxDB versions) and a correct "content-type" header.
2023-02-06 17:10:30 +01:00
ae10b33f0c
chore: Update datafusion ( #6868 )
...
* chore: Update datafusion
* chore: Run cargo hakari tasks
* fix: Update error message
---------
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
2023-02-06 14:22:45 +00:00
d3601a59f8
chore: update DataFusion, upgrade `arrow` `arrow-flight` and `parquet` to `32.0.0` ( #6756 )
...
* chore: update DataFusion
* fix: test
* chore: format
* chore: clippy
* chore: update arrow
* chore: arrow upgrade fallout
* chore: Run cargo hakari tasks
* chore: remove failing warm compaction test
* fix: flight error propagation
* chore: update parquet size
* fix: Update error message
* chore: Update parquet metadata test
---------
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: Andrew Lamb <alamb@influxdata.com>
Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-02-06 11:35:39 +00:00
9f3a068d22
chore(deps): Bump tikv-jemalloc-sys ( #6865 )
...
Bumps [tikv-jemalloc-sys](https://github.com/tikv/jemallocator ) from 0.5.2+5.3.0-patched to 0.5.3+5.3.0-patched.
- [Release notes](https://github.com/tikv/jemallocator/releases )
- [Changelog](https://github.com/tikv/jemallocator/blob/main/CHANGELOG.md )
- [Commits](https://github.com/tikv/jemallocator/compare/tikv-jemalloc-sys-0.5.2...tikv-jemalloc-sys-0.5.3 )
---
updated-dependencies:
- dependency-name: tikv-jemalloc-sys
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-02-06 09:28:19 +00:00
362a18343f
chore(deps): Bump proptest from 1.0.0 to 1.1.0 ( #6863 )
...
Bumps [proptest](https://github.com/proptest-rs/proptest ) from 1.0.0 to 1.1.0.
- [Release notes](https://github.com/proptest-rs/proptest/releases )
- [Changelog](https://github.com/proptest-rs/proptest/blob/master/CHANGELOG.md )
- [Commits](https://github.com/proptest-rs/proptest/compare/1.0.0...v1.1.0 )
---
updated-dependencies:
- dependency-name: proptest
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-02-06 09:11:44 +00:00
a0c551618b
chore(deps): Bump rustix from 0.36.7 to 0.36.8 ( #6862 )
...
Bumps [rustix](https://github.com/bytecodealliance/rustix ) from 0.36.7 to 0.36.8.
- [Release notes](https://github.com/bytecodealliance/rustix/releases )
- [Commits](https://github.com/bytecodealliance/rustix/compare/v0.36.7...v0.36.8 )
---
updated-dependencies:
- dependency-name: rustix
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-02-06 08:57:17 +00:00
d122f0d087
chore(deps): Bump is-terminal from 0.4.2 to 0.4.3 ( #6861 )
...
Bumps [is-terminal](https://github.com/sunfishcode/is-terminal ) from 0.4.2 to 0.4.3.
- [Release notes](https://github.com/sunfishcode/is-terminal/releases )
- [Commits](https://github.com/sunfishcode/is-terminal/compare/v0.4.2...v0.4.3 )
---
updated-dependencies:
- dependency-name: is-terminal
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-02-06 08:46:17 +00:00
6f4e287a3a
chore(deps): Bump serde_json from 1.0.91 to 1.0.92 ( #6860 )
...
Bumps [serde_json](https://github.com/serde-rs/json ) from 1.0.91 to 1.0.92.
- [Release notes](https://github.com/serde-rs/json/releases )
- [Commits](https://github.com/serde-rs/json/compare/v1.0.91...v1.0.92 )
---
updated-dependencies:
- dependency-name: serde_json
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-02-06 08:27:41 +00:00
b55e4ffa21
test: Port table_schema tests to end-to-end and get rid of query_tests ( #6828 )
...
* test: Port table_schema query_tests to end-to-end
* fix: Remove now outdated query_tests crate
---------
Co-authored-by: Andrew Lamb <alamb@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-02-03 17:41:08 +00:00
1b7cf02442
chore(deps): Bump hyper from 0.14.23 to 0.14.24 ( #6827 )
...
Bumps [hyper](https://github.com/hyperium/hyper ) from 0.14.23 to 0.14.24.
- [Release notes](https://github.com/hyperium/hyper/releases )
- [Changelog](https://github.com/hyperium/hyper/blob/v0.14.24/CHANGELOG.md )
- [Commits](https://github.com/hyperium/hyper/compare/v0.14.23...v0.14.24 )
---
updated-dependencies:
- dependency-name: hyper
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-02-03 10:00:37 +00:00
a6d543556d
chore(deps): Bump heck from 0.4.0 to 0.4.1 ( #6812 )
...
Bumps [heck](https://github.com/withoutboats/heck ) from 0.4.0 to 0.4.1.
- [Release notes](https://github.com/withoutboats/heck/releases )
- [Changelog](https://github.com/withoutboats/heck/blob/master/CHANGELOG.md )
- [Commits](https://github.com/withoutboats/heck/commits )
---
updated-dependencies:
- dependency-name: heck
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-02-02 09:46:15 +00:00
00b697d76a
chore(deps): Bump io-lifetimes from 1.0.4 to 1.0.5
...
Bumps [io-lifetimes](https://github.com/sunfishcode/io-lifetimes ) from 1.0.4 to 1.0.5.
- [Release notes](https://github.com/sunfishcode/io-lifetimes/releases )
- [Commits](https://github.com/sunfishcode/io-lifetimes/compare/1.0.4...v1.0.5 )
---
updated-dependencies:
- dependency-name: io-lifetimes
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-02-02 01:05:31 +00:00
bf6ae3dc4d
fix: map retention period 0 values to NULL
...
Treat a retention period of 0 as "infinite", and not "none".
2023-02-01 11:29:13 +01:00
1a9c27cd9a
chore(deps): Bump uuid from 1.2.2 to 1.3.0
...
Bumps [uuid](https://github.com/uuid-rs/uuid ) from 1.2.2 to 1.3.0.
- [Release notes](https://github.com/uuid-rs/uuid/releases )
- [Commits](https://github.com/uuid-rs/uuid/compare/1.2.2...1.3.0 )
---
updated-dependencies:
- dependency-name: uuid
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-02-01 09:32:39 +00:00
d0e6b16450
chore(deps): Bump bytes from 1.3.0 to 1.4.0
...
Bumps [bytes](https://github.com/tokio-rs/bytes ) from 1.3.0 to 1.4.0.
- [Release notes](https://github.com/tokio-rs/bytes/releases )
- [Changelog](https://github.com/tokio-rs/bytes/blob/master/CHANGELOG.md )
- [Commits](https://github.com/tokio-rs/bytes/compare/v1.3.0...v1.4.0 )
---
updated-dependencies:
- dependency-name: bytes
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-02-01 00:30:56 +00:00
57f55e14c8
feat: IOx InfluxQL planner learns how to process time range expressions ( #6772 )
...
* feat: IOx learns InfluxQL time-range expression → DF logical Expr
IOx now understand the how to evaluate an InfluxQL time-range filter
expression and transform that to a DataFusion logical expression.
* chore: move time range expression to independent functions
There is no need for these to be part of the `InfluxQLToLogicalPlan`
struct and makes them easier to test.
* chore: support scalar now on either side of binary expression
* chore: improve error messages
* chore: address clippy concerns
* chore: add tests for time ranges
* chore: add a test where time appears on the right-hand side
Ensure time is correctly identified on the right-hand side of a
conditional expression.
* chore: add tests that specify a timezone
* chore: Run cargo hakari tasks
* chore: fix linting issues
* chore: Remove unnecessary line
* chore: Feedback: Add API to parse a conditional expression
Based on feedback from @alamb, we don't want to hide the error from
parsing a `ConditionalExpression`. To do this, we use the
public API, `parse_statements` as a model and provide a new API,
`parse_conditional_expression`, which returns a `Result` with the error
being a `ParseError`. Additionally, `ConditionalExpression` implements
the `FromStr` API using the `parse_conditional_expression` API.
* chore: PR feedback reverting this change
I believe my intention was to update all instances in the match, but
never completed the change. Will leave for another day.
* chore: PR feedback add additional comments
* chore: rustfmt
---------
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
2023-02-01 00:27:17 +00:00
38e128fad8
chore(deps): Bump zstd from 0.12.2+zstd.1.5.2 to 0.12.3+zstd.1.5.2 ( #6785 )
...
Bumps [zstd](https://github.com/gyscos/zstd-rs ) from 0.12.2+zstd.1.5.2 to 0.12.3+zstd.1.5.2.
- [Release notes](https://github.com/gyscos/zstd-rs/releases )
- [Commits](https://github.com/gyscos/zstd-rs/commits )
---
updated-dependencies:
- dependency-name: zstd
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-31 16:47:39 +00:00
5035017a58
chore(deps): Bump zstd-sys from 2.0.5+zstd.1.5.2 to 2.0.6+zstd.1.5.2 ( #6784 )
...
Bumps [zstd-sys](https://github.com/gyscos/zstd-rs ) from 2.0.5+zstd.1.5.2 to 2.0.6+zstd.1.5.2.
- [Release notes](https://github.com/gyscos/zstd-rs/releases )
- [Commits](https://github.com/gyscos/zstd-rs/commits )
---
updated-dependencies:
- dependency-name: zstd-sys
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-31 15:42:02 +00:00
ad390131c4
chore(deps): Bump zstd-safe from 6.0.2+zstd.1.5.2 to 6.0.3+zstd.1.5.2 ( #6786 )
...
Bumps [zstd-safe](https://github.com/gyscos/zstd-rs ) from 6.0.2+zstd.1.5.2 to 6.0.3+zstd.1.5.2.
- [Release notes](https://github.com/gyscos/zstd-rs/releases )
- [Commits](https://github.com/gyscos/zstd-rs/commits )
---
updated-dependencies:
- dependency-name: zstd-safe
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-01-31 15:19:36 +00:00
fa15a48c31
refactor: use types rather than `Any` in `flightsql` interface ( #6705 )
...
* refactor: use types rather than Any in flightsql interface
* fix: Update service_grpc_flight/src/lib.rs
Co-authored-by: Raphael Taylor-Davies <1781103+tustvold@users.noreply.github.com>
* chore: review feedback
* fix: fmt
* fix: error conversion
---------
Co-authored-by: Raphael Taylor-Davies <1781103+tustvold@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-01-31 14:45:46 +00:00
62697265c1
feat: compactor sharding ( #6729 )
...
I'm not saying we have to use this, but this is a demonstration how easy
it would be to add sharding to the compaction tier and also acts as a
"backup / insurance" if we ever need it.
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-01-31 14:37:06 +00:00
875b6a3e99
chore(deps): Bump futures from 0.3.25 to 0.3.26 ( #6766 )
...
Bumps [futures](https://github.com/rust-lang/futures-rs ) from 0.3.25 to 0.3.26.
- [Release notes](https://github.com/rust-lang/futures-rs/releases )
- [Changelog](https://github.com/rust-lang/futures-rs/blob/master/CHANGELOG.md )
- [Commits](https://github.com/rust-lang/futures-rs/compare/0.3.25...0.3.26 )
---
updated-dependencies:
- dependency-name: futures
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-01-31 11:33:50 +00:00
a90ca290b2
chore(deps): Bump futures-util from 0.3.25 to 0.3.26 ( #6767 )
...
Bumps [futures-util](https://github.com/rust-lang/futures-rs ) from 0.3.25 to 0.3.26.
- [Release notes](https://github.com/rust-lang/futures-rs/releases )
- [Changelog](https://github.com/rust-lang/futures-rs/blob/master/CHANGELOG.md )
- [Commits](https://github.com/rust-lang/futures-rs/compare/0.3.25...0.3.26 )
---
updated-dependencies:
- dependency-name: futures-util
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-31 11:17:32 +00:00
97342fcdc7
chore(deps): Bump futures-io from 0.3.25 to 0.3.26 ( #6765 )
...
Bumps [futures-io](https://github.com/rust-lang/futures-rs ) from 0.3.25 to 0.3.26.
- [Release notes](https://github.com/rust-lang/futures-rs/releases )
- [Changelog](https://github.com/rust-lang/futures-rs/blob/master/CHANGELOG.md )
- [Commits](https://github.com/rust-lang/futures-rs/compare/0.3.25...0.3.26 )
---
updated-dependencies:
- dependency-name: futures-io
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-31 11:15:41 +00:00
51b0483699
chore(deps): Bump futures-task from 0.3.25 to 0.3.26 ( #6768 )
...
Bumps [futures-task](https://github.com/rust-lang/futures-rs ) from 0.3.25 to 0.3.26.
- [Release notes](https://github.com/rust-lang/futures-rs/releases )
- [Changelog](https://github.com/rust-lang/futures-rs/blob/master/CHANGELOG.md )
- [Commits](https://github.com/rust-lang/futures-rs/compare/0.3.25...0.3.26 )
---
updated-dependencies:
- dependency-name: futures-task
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-31 10:58:40 +00:00
6f032b1d57
chore(deps): Bump async-trait from 0.1.63 to 0.1.64 ( #6769 )
...
Bumps [async-trait](https://github.com/dtolnay/async-trait ) from 0.1.63 to 0.1.64.
- [Release notes](https://github.com/dtolnay/async-trait/releases )
- [Commits](https://github.com/dtolnay/async-trait/compare/0.1.63...0.1.64 )
---
updated-dependencies:
- dependency-name: async-trait
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-31 10:18:27 +00:00
17e118d8c3
chore(deps): Bump futures-channel from 0.3.25 to 0.3.26 ( #6770 )
...
Bumps [futures-channel](https://github.com/rust-lang/futures-rs ) from 0.3.25 to 0.3.26.
- [Release notes](https://github.com/rust-lang/futures-rs/releases )
- [Changelog](https://github.com/rust-lang/futures-rs/blob/master/CHANGELOG.md )
- [Commits](https://github.com/rust-lang/futures-rs/compare/0.3.25...0.3.26 )
---
updated-dependencies:
- dependency-name: futures-channel
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-31 10:04:22 +00:00
370a0ef7dd
chore(deps): Bump toml from 0.7.0 to 0.7.1 ( #6771 )
...
Bumps [toml](https://github.com/toml-rs/toml ) from 0.7.0 to 0.7.1.
- [Release notes](https://github.com/toml-rs/toml/releases )
- [Commits](https://github.com/toml-rs/toml/compare/toml-v0.7.0...toml-v0.7.1 )
---
updated-dependencies:
- dependency-name: toml
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-31 09:53:48 +00:00
5b14caa780
chore: Update DataFusion ( #6753 )
...
* chore: Update datafusion
* fix: Update for changes
* chore: Run cargo hakari tasks
---------
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-01-30 14:48:52 +00:00
1d46fa1a6f
chore(deps): Bump cc from 1.0.78 to 1.0.79
...
Bumps [cc](https://github.com/rust-lang/cc-rs ) from 1.0.78 to 1.0.79.
- [Release notes](https://github.com/rust-lang/cc-rs/releases )
- [Commits](https://github.com/rust-lang/cc-rs/compare/1.0.78...1.0.79 )
---
updated-dependencies:
- dependency-name: cc
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-01-30 09:45:58 +00:00
ed7d02a225
chore(deps): Bump tokio from 1.24.2 to 1.25.0
...
Bumps [tokio](https://github.com/tokio-rs/tokio ) from 1.24.2 to 1.25.0.
- [Release notes](https://github.com/tokio-rs/tokio/releases )
- [Commits](https://github.com/tokio-rs/tokio/commits/tokio-1.25.0 )
---
updated-dependencies:
- dependency-name: tokio
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-01-30 01:57:27 +00:00
55257b46c9
chore: validate ingester URIs on querier CLI ( #6740 )
...
* chore: add validate for ingesters on querier CLI
* chore: fix typo and tests
* chore: clippy
* chore: review feedback
---------
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-01-27 21:13:52 +00:00
56825f119e
chore(deps): Bump toml from 0.6.0 to 0.7.0 ( #6739 )
...
Bumps [toml](https://github.com/toml-rs/toml ) from 0.6.0 to 0.7.0.
- [Release notes](https://github.com/toml-rs/toml/releases )
- [Commits](https://github.com/toml-rs/toml/compare/toml-v0.6.0...toml-v0.7.0 )
---
updated-dependencies:
- dependency-name: toml
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-01-27 20:44:53 +00:00
0d32662eea
chore: Update datafusion again ( #6722 )
...
* chore: Update datafusion
* fix: Update for API
* chore: Run cargo hakari tasks
---------
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-01-27 18:59:27 +00:00
13fc93a1b2
chore(deps): Bump either from 1.8.0 to 1.8.1 ( #6724 )
...
Bumps [either](https://github.com/bluss/either ) from 1.8.0 to 1.8.1.
- [Release notes](https://github.com/bluss/either/releases )
- [Commits](https://github.com/bluss/either/compare/1.8.0...1.8.1 )
---
updated-dependencies:
- dependency-name: either
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-01-27 10:12:49 +00:00
30d411dc95
feat: shadow mode ( #6712 )
...
* refactor: remove untyped durations from `compactor2`
* feat: shadow mode
Closes #6645 .
* refactor: split input and output store
2023-01-26 14:20:55 +00:00
f310e01b1a
test: Start of porting InfluxRpc query_tests
...
Make a new trait, `InfluxRpcTest`, that types can implement to define
how to run a test on a specific Storage gRPC API. `InfluxRpcTest` takes
care of iterating through the two architectures, running the setups, and
creating the custom test step.
Implementers of the trait can define aspects of the tests that differ
per run, to make the parameters of the test clearer and highlight what
different tests are testing.
2023-01-25 10:27:42 -05:00
f72a999fb3
chore(deps): Bump clap from 4.1.3 to 4.1.4 ( #6694 )
...
Bumps [clap](https://github.com/clap-rs/clap ) from 4.1.3 to 4.1.4.
- [Release notes](https://github.com/clap-rs/clap/releases )
- [Changelog](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md )
- [Commits](https://github.com/clap-rs/clap/compare/v4.1.3...v4.1.4 )
---
updated-dependencies:
- dependency-name: clap
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-01-25 11:03:41 +00:00
40e6a1a437
feat: job semaphore ( #6696 )
...
* refactor: avoid too-many-arguments
* refactor: extract `fetch_partition_info`
* feat: job semaphore
2023-01-25 10:35:07 +00:00
cae3071776
chore(deps): Bump ahash from 0.8.2 to 0.8.3
...
Bumps [ahash](https://github.com/tkaitchuck/ahash ) from 0.8.2 to 0.8.3.
- [Release notes](https://github.com/tkaitchuck/ahash/releases )
- [Commits](https://github.com/tkaitchuck/ahash/compare/v0.8.2...v0.8.3 )
---
updated-dependencies:
- dependency-name: ahash
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-01-25 01:08:55 +00:00
c3bc61f10e
refactor: Move `flightsql` code into its own module, add docs and tests ( #6640 )
...
* refactor: Move `flightsql` code into its own module
* fix: get schema from LogicalPlan
* refactor: use arrow_flight::sql::Any instead of prost_types::any
* fix: cleanup docs and avoid as_ref
* fix: Use Bytes
* fix: use Any::pack
* fix: doclink
2023-01-24 18:24:32 +00:00
32df24e057
feat: compactor2 error classification ( #6676 )
...
* feat: add error kinds
* refactor: sink proper error type
* fix: ignore object store errors
See <https://github.com/influxdata/idpe/issues/16984 >.
* feat: log error kind
* feat: per-kind error metric
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-01-24 13:50:19 +00:00
1e0a52eeb6
Merge branch 'main' into dom/circuit-breaker
2023-01-24 11:12:58 +00:00
0e304efc28
chore(deps): Bump toml from 0.5.11 to 0.6.0 ( #6670 )
...
Bumps [toml](https://github.com/toml-rs/toml ) from 0.5.11 to 0.6.0.
- [Release notes](https://github.com/toml-rs/toml/releases )
- [Commits](https://github.com/toml-rs/toml/compare/toml-v0.5.11...toml-v0.6.0 )
---
updated-dependencies:
- dependency-name: toml
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-24 10:08:20 +00:00
681d4d940f
chore(deps): Bump clap from 4.1.1 to 4.1.3 ( #6669 )
...
Bumps [clap](https://github.com/clap-rs/clap ) from 4.1.1 to 4.1.3.
- [Release notes](https://github.com/clap-rs/clap/releases )
- [Changelog](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md )
- [Commits](https://github.com/clap-rs/clap/compare/clap_complete-v4.1.1...v4.1.3 )
---
updated-dependencies:
- dependency-name: clap
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-01-24 09:53:06 +00:00
67b73d90dd
feat: low-overhead circuit breaker
...
Implements a "circuit breaker", a construct that tracks the error &
success of requests to a remote node, and uses this information to allow
or deny further requests.
This circuit breaker stops sending requests to the remote when the error
count exceeds 80% of requests in a 5 second window. Once this happens,
up to 10 "probe" requests per second are allowed, and when they succeed,
normal operation resumes (though concurrent requests may still be
completing during the probe regime and are counted towards the probe
results).
In the happy path, this circuit breaker is very cheap (lock free; WFPO)
to evaluate and record request results in, minimising the throughput
penalty. Once the breaker enters an unhealthy state (hopefully a rare
occurrence) it uses a mutex to manage the probe state (with a higher
overhead) for simplicity; it's definitely possible to optimise this away
if high latencies are observed during upstream outages when the circuit
breaker is open/unhealthy.
2023-01-23 13:55:12 +01:00
9a61f36a53
chore: Update datafusion again ( #6656 )
...
* chore: Update datafusion pin
* chore: Run cargo hakari tasks
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
2023-01-23 12:13:07 +00:00
b09691dc6b
chore: Upgrade datafusion (again, I know) ( #6639 )
...
* chore: Update datafusion
* chore: Run cargo hakari tasks
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-01-23 11:24:22 +00:00
d1379e9747
chore(deps): Bump toml from 0.5.10 to 0.5.11
...
Bumps [toml](https://github.com/toml-rs/toml ) from 0.5.10 to 0.5.11.
- [Release notes](https://github.com/toml-rs/toml/releases )
- [Commits](https://github.com/toml-rs/toml/compare/toml-v0.5.10...toml-v0.5.11 )
---
updated-dependencies:
- dependency-name: toml
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-01-23 08:43:52 +00:00
0114e7ee50
chore(deps): Bump async-trait from 0.1.61 to 0.1.63 ( #6660 )
...
Bumps [async-trait](https://github.com/dtolnay/async-trait ) from 0.1.61 to 0.1.63.
- [Release notes](https://github.com/dtolnay/async-trait/releases )
- [Commits](https://github.com/dtolnay/async-trait/compare/0.1.61...0.1.63 )
---
updated-dependencies:
- dependency-name: async-trait
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-23 08:41:27 +00:00
840923abab
refactor: execute compaction plan ( #6654 )
...
* chore: address review comment of previous PR
* refactor: execute compact plan
* refactor: we will now compact all L0 and L1 files of a partition and split them as needed
* chore: comnents
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-01-20 22:34:50 +00:00
24ca1e6f8c
Merge branch 'main' into cn/test-old-with-new
2023-01-20 15:20:40 +00:00
8aeded32d6
refactor: reorganize compact_files core function ( #6636 )
...
* refactor: reorganize compact_files core function
* chore: smore more in-progress structure
* refactor: further reorganization for compact_files
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-01-20 13:41:23 +00:00
d808c57cdc
chore: Remove `iox_arrow_flight` ( #6621 )
...
* chore: Remove iox_arrow_flight
* fix: hack around tonic status errors
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-01-20 11:16:19 +00:00
8ac93a58a6
chore(deps): Bump reqwest from 0.11.13 to 0.11.14 ( #6643 )
...
* chore(deps): Bump reqwest from 0.11.13 to 0.11.14
Bumps [reqwest](https://github.com/seanmonstar/reqwest ) from 0.11.13 to 0.11.14.
- [Release notes](https://github.com/seanmonstar/reqwest/releases )
- [Changelog](https://github.com/seanmonstar/reqwest/blob/master/CHANGELOG.md )
- [Commits](https://github.com/seanmonstar/reqwest/compare/v0.11.13...v0.11.14 )
---
updated-dependencies:
- dependency-name: reqwest
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* chore: Run cargo hakari tasks
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-01-20 07:21:58 +00:00
33045489f9
chore(deps): Bump rustyline from 10.1.0 to 10.1.1 ( #6642 )
...
* chore(deps): Bump rustyline from 10.1.0 to 10.1.1
Bumps [rustyline](https://github.com/kkawakam/rustyline ) from 10.1.0 to 10.1.1.
- [Release notes](https://github.com/kkawakam/rustyline/releases )
- [Changelog](https://github.com/kkawakam/rustyline/blob/master/History.md )
- [Commits](https://github.com/kkawakam/rustyline/compare/v10.1.0...v10.1.1 )
---
updated-dependencies:
- dependency-name: rustyline
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* chore: Run cargo hakari tasks
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
2023-01-20 07:13:12 +00:00
6afd782b3f
fix: Move query_tests2 into influxdb_iox/tests so that the code rebuilds
2023-01-19 16:44:31 -05:00
b265036f33
refactor: remove iox_arrow_flight use in ingester_replica ( #6634 )
...
* refactor: remove iox_arrow_flight use in ingester_replica
* chore: fixlu
* fix: update doclink
* fix: Undo rustix downgrade
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-01-19 18:55:57 +00:00
65c020c9f2
refactor: remove iox_arrow_flight use in `influxdb_iox_client ` and `querier` ( #6624 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-01-19 18:48:23 +00:00
5b6d261396
refactor: remove iox_arrow_flight use in ingester2 ( #6623 )
...
* refactor: remove iox_arrow_flight use in ingester2
* fix: Update ingester2/src/server/grpc/query.rs
Co-authored-by: Dom <dom@itsallbroken.com>
* chore: remove unused Error enums
Co-authored-by: Dom <dom@itsallbroken.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-01-19 15:27:23 +00:00
f639bf3e23
chore: refactor ingester to use upstream arrow-flight ( #6622 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-01-19 15:16:13 +00:00
ad69a554d6
refactor: extract "fetch files" component ( #6632 )
2023-01-19 13:11:44 +00:00
e4bf9e85d9
refactor: more components for compactor2 ( #6630 )
...
* refactor: rename "rules" to "components"
* refactor: replace `Trait::name` w/ `impl Display`
* refactor: remodel partitions source
* refactor: add TODO
* refactor: fully components-based partitions source
* test: add tests for partitions sources
* docs: improve
Co-authored-by: Dom <dom@itsallbroken.com>
Co-authored-by: Dom <dom@itsallbroken.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-01-19 12:29:38 +00:00
9ae03b16d6
feat: invokes catalog functions for compactor2 ( #6619 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-01-19 10:33:57 +00:00
2db763508d
chore(deps): Bump rustix from 0.36.6 to 0.36.7 ( #6627 )
...
Bumps [rustix](https://github.com/bytecodealliance/rustix ) from 0.36.6 to 0.36.7.
- [Release notes](https://github.com/bytecodealliance/rustix/releases )
- [Commits](https://github.com/bytecodealliance/rustix/compare/v0.36.6...v0.36.7 )
---
updated-dependencies:
- dependency-name: rustix
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-01-19 09:18:27 +00:00
6a4fa11dba
chore: Update to latest datafusion / arrow `31.0.0` ( #6618 )
...
* chore: Update datafusion and arrow/parquet/arrow-flight `31.0.0`
* chore: Update for new API
* chore: Run cargo hakari tasks
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-01-18 20:21:10 +00:00
33168b97f0
Merge branch 'main' into cn/query-tests-grpc
2023-01-18 19:03:51 +00:00
380a855aab
feat: basic compactor2 algo layout ( #6616 )
2023-01-18 18:51:59 +00:00
e72173d58d
feat: very basic compactor2 skeleton ( #6614 )
...
Sets up crate and wires up the main binary. No tests yet, no algorithm
framework, just the bare minimum.
Also I decided to not offer a gRPC server in `compactor2` at the moment
and hence did not implement any handle/delegate infrastructure. We add
this later if we need it. This also means compactor2 does NOT provide a
catalog service for now.
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-01-18 16:36:40 +00:00
3608d2881b
chore: Update dependencies ( #6612 )
...
* chore: Update dependencies
* chore: Run cargo hakari tasks
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
2023-01-18 16:19:50 +00:00
8410998408
chore: Update datafusion to Jan 17, 2023 (2 / 2) and arrow/parquet `30.0.1` ( #6604 )
...
* chore: Update datafusion to Jan 9, 2023 (2 / 2) and arrow/parquet `30.0.1`
* chore: Update for changes in arrow ipc
* chore: Run cargo hakari tasks
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
2023-01-18 15:51:24 +00:00
f3b5dcaab7
feat: Reimagining query_tests
2023-01-18 10:24:17 -05:00
57f08dbccd
chore: Update datafusion to Jan 9, 2023 (1 / 2) ( #6603 )
...
* refactor: Update DataFusion pin to early Jan 2023
* fix: Update tests now that planning is async
* fix: Updates for API changes
* chore: Run cargo hakari tasks
* fix: Update comment
* refactor: nicer config setup
* fix: gapfill async
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
2023-01-18 12:19:32 +00:00
4e0fd0645b
chore(deps): Bump nix from 0.26.1 to 0.26.2 ( #6606 )
...
Bumps [nix](https://github.com/nix-rust/nix ) from 0.26.1 to 0.26.2.
- [Release notes](https://github.com/nix-rust/nix/releases )
- [Changelog](https://github.com/nix-rust/nix/blob/v0.26.2/CHANGELOG.md )
- [Commits](https://github.com/nix-rust/nix/compare/v0.26.1...v0.26.2 )
---
updated-dependencies:
- dependency-name: nix
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Dom <dom@itsallbroken.com>
2023-01-18 10:12:54 +00:00
4344c3df53
chore(deps): Bump tokio from 1.24.1 to 1.24.2 ( #6605 )
...
Bumps [tokio](https://github.com/tokio-rs/tokio ) from 1.24.1 to 1.24.2.
- [Release notes](https://github.com/tokio-rs/tokio/releases )
- [Commits](https://github.com/tokio-rs/tokio/commits )
---
updated-dependencies:
- dependency-name: tokio
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-18 08:00:27 +00:00
586912ad73
Merge branch 'main' into dependabot/cargo/rustyline-10.1.0
2023-01-16 10:04:47 +00:00
f7797b0358
chore(deps): Bump clap from 4.0.32 to 4.1.1
...
Bumps [clap](https://github.com/clap-rs/clap ) from 4.0.32 to 4.1.1.
- [Release notes](https://github.com/clap-rs/clap/releases )
- [Changelog](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md )
- [Commits](https://github.com/clap-rs/clap/compare/v4.0.32...v4.1.1 )
---
updated-dependencies:
- dependency-name: clap
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-01-16 09:31:58 +00:00
3b2c172dc1
chore(deps): Bump nom from 7.1.2 to 7.1.3 ( #6596 )
...
Bumps [nom](https://github.com/Geal/nom ) from 7.1.2 to 7.1.3.
- [Release notes](https://github.com/Geal/nom/releases )
- [Changelog](https://github.com/rust-bakery/nom/blob/7.1.3/CHANGELOG.md )
- [Commits](https://github.com/Geal/nom/compare/7.1.2...7.1.3 )
---
updated-dependencies:
- dependency-name: nom
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-16 09:29:09 +00:00
1f18738203
chore: Run cargo hakari tasks
2023-01-16 02:07:11 +00:00
0a70e9f43f
chore(deps): Bump rustyline from 10.0.0 to 10.1.0
...
Bumps [rustyline](https://github.com/kkawakam/rustyline ) from 10.0.0 to 10.1.0.
- [Release notes](https://github.com/kkawakam/rustyline/releases )
- [Changelog](https://github.com/kkawakam/rustyline/blob/master/History.md )
- [Commits](https://github.com/kkawakam/rustyline/compare/v10.0.0...v10.1.0 )
---
updated-dependencies:
- dependency-name: rustyline
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-01-16 02:06:24 +00:00
3f6bb3e330
feat: Parse IANA timezones in an InfluxQL TZ clause ( #6585 )
...
* feat: Parse IANA timezone strings to chrono_tz::Tz
* feat: Visitors can customise the return error type
This avoids having to remap errors from `&'static str` to the caller's
error type, and will be used in a future PR for time range expressions.
* chore: Run cargo hakari tasks
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
2023-01-15 22:00:41 +00:00
1c9fdd38d6
chore(deps): Bump io-lifetimes from 1.0.3 to 1.0.4 ( #6584 )
...
Bumps [io-lifetimes](https://github.com/sunfishcode/io-lifetimes ) from 1.0.3 to 1.0.4.
- [Release notes](https://github.com/sunfishcode/io-lifetimes/releases )
- [Commits](https://github.com/sunfishcode/io-lifetimes/compare/v1.0.3...1.0.4 )
---
updated-dependencies:
- dependency-name: io-lifetimes
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-01-13 09:21:06 +00:00
e9b3efb33d
refactor: Extract a method for making requests to the ingester onto MiniCluster
2023-01-12 11:03:10 -05:00
0d111c4672
refactor: delegate frontend shutdown to backend
...
Prior to this commit, the (happy path) shutdown sequence of an IOx
process was hard coded to:
1. Stop gRPC & HTTP servers
2. Stop backend server (i.e. ingester2)
After this commit, the execution of step 1 is delegated to the handler
for step 2; the server implementation (router / ingester / querier /
etc) now chooses when to shut down the RPC & HTTP servers.
This allows the server shutdown delegate to correctly sequence the
shutdown of all components of the IOx server. This allows ingester2 to
correctly sequence the shutdown of the query RPC server w.r.t the
graceful stop & persist, ensuring queries continue to be serviced.
2023-01-12 14:59:50 +01:00
8565ca5137
chore(deps): Bump zstd from 0.12.1+zstd.1.5.2 to 0.12.2+zstd.1.5.2 ( #6573 )
...
Bumps [zstd](https://github.com/gyscos/zstd-rs ) from 0.12.1+zstd.1.5.2 to 0.12.2+zstd.1.5.2.
- [Release notes](https://github.com/gyscos/zstd-rs/releases )
- [Commits](https://github.com/gyscos/zstd-rs/commits )
---
updated-dependencies:
- dependency-name: zstd
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Dom <dom@itsallbroken.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-01-12 09:54:28 +00:00
cc6d8709c9
chore(deps): Bump hashbrown from 0.13.1 to 0.13.2 ( #6574 )
...
Bumps [hashbrown](https://github.com/rust-lang/hashbrown ) from 0.13.1 to 0.13.2.
- [Release notes](https://github.com/rust-lang/hashbrown/releases )
- [Changelog](https://github.com/rust-lang/hashbrown/blob/master/CHANGELOG.md )
- [Commits](https://github.com/rust-lang/hashbrown/compare/v0.13.1...v0.13.2 )
---
updated-dependencies:
- dependency-name: hashbrown
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Dom <dom@itsallbroken.com>
2023-01-12 09:47:15 +00:00
be278dc689
chore(deps): Bump prost-types from 0.11.5 to 0.11.6 ( #6559 )
...
Bumps [prost-types](https://github.com/tokio-rs/prost ) from 0.11.5 to 0.11.6.
- [Release notes](https://github.com/tokio-rs/prost/releases )
- [Commits](https://github.com/tokio-rs/prost/compare/v0.11.5...v0.11.6 )
---
updated-dependencies:
- dependency-name: prost-types
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-01-11 16:16:04 +00:00
93a16c78d7
chore(deps): Bump bzip2 from 0.4.3 to 0.4.4 ( #6563 )
...
Bumps [bzip2](https://github.com/alexcrichton/bzip2-rs ) from 0.4.3 to 0.4.4.
- [Release notes](https://github.com/alexcrichton/bzip2-rs/releases )
- [Commits](https://github.com/alexcrichton/bzip2-rs/commits/0.4.4 )
---
updated-dependencies:
- dependency-name: bzip2
dependency-type: indirect
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-01-11 11:11:39 +00:00
e9f9948bea
chore(deps): Bump zstd-sys from 2.0.4+zstd.1.5.2 to 2.0.5+zstd.1.5.2 ( #6557 )
...
Bumps [zstd-sys](https://github.com/gyscos/zstd-rs ) from 2.0.4+zstd.1.5.2 to 2.0.5+zstd.1.5.2.
- [Release notes](https://github.com/gyscos/zstd-rs/releases )
- [Commits](https://github.com/gyscos/zstd-rs/commits )
---
updated-dependencies:
- dependency-name: zstd-sys
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-01-11 10:52:20 +00:00
43a0280365
chore(deps): Bump prost from 0.11.5 to 0.11.6
...
Bumps [prost](https://github.com/tokio-rs/prost ) from 0.11.5 to 0.11.6.
- [Release notes](https://github.com/tokio-rs/prost/releases )
- [Commits](https://github.com/tokio-rs/prost/compare/v0.11.5...v0.11.6 )
---
updated-dependencies:
- dependency-name: prost
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-01-11 02:53:33 +00:00
66047f4372
feat: InfluxQL learns how to plan some InfluxQL queries ( #6520 )
...
* feat: InfluxQL learns how to plan some queries
Also added a means to test the planner and execution
* chore: Update module docs
* chore: Document the planner functions
* chore: Update end_to_end_cases crate
* chore: Clarify why `SLIMIT` and `SOFFSET` return `NotImplemented`
* chore: Address lint issues
* chore: Fix rustdoc link issue
* chore: Remove InfluxQL tests from query_tests crate
Will follow conventions established by @carols10cents when
new query_tests crate is merged.
* chore: `now` field
`now` is a DataFusion built-in scalar function
* chore: remove unused code
* chore: Add additional arithmetic expression tests
* chore: Establish pattern for identifying and tracking InfluxQL issues
* chore: Add tests for case sensitivity issues
* chore: group tests into modules and functions
This avoids mass rewriting of insta snapshots as new
tests are added to each function. When tests are added in the middle,
existing snapshots are renamed (-N+1, -N+2, etc) resulting in
having to review numerous additional snapshots.
2023-01-11 02:50:49 +00:00
c68049c37a
chore(deps): Bump regex from 1.7.0 to 1.7.1 ( #6546 )
...
Bumps [regex](https://github.com/rust-lang/regex ) from 1.7.0 to 1.7.1.
- [Release notes](https://github.com/rust-lang/regex/releases )
- [Changelog](https://github.com/rust-lang/regex/blob/master/CHANGELOG.md )
- [Commits](https://github.com/rust-lang/regex/compare/1.7.0...1.7.1 )
---
updated-dependencies:
- dependency-name: regex
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-01-10 09:55:41 +00:00
b49cc2e35e
chore(deps): Bump tokio from 1.24.0 to 1.24.1 ( #6545 )
...
Bumps [tokio](https://github.com/tokio-rs/tokio ) from 1.24.0 to 1.24.1.
- [Release notes](https://github.com/tokio-rs/tokio/releases )
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.24.0...tokio-1.24.1 )
---
updated-dependencies:
- dependency-name: tokio
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-01-10 09:48:44 +00:00
09746c3aea
chore(deps): Bump assert_cmd from 2.0.7 to 2.0.8
...
Bumps [assert_cmd](https://github.com/assert-rs/assert_cmd ) from 2.0.7 to 2.0.8.
- [Release notes](https://github.com/assert-rs/assert_cmd/releases )
- [Changelog](https://github.com/assert-rs/assert_cmd/blob/master/CHANGELOG.md )
- [Commits](https://github.com/assert-rs/assert_cmd/compare/v2.0.7...v2.0.8 )
---
updated-dependencies:
- dependency-name: assert_cmd
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-01-10 01:06:02 +00:00
828992c9c5
feat: Ingest replica skeleton ( #6529 )
...
* feat: Update replication.proto
* Remove the PartitionId in the replicate request as a single replicate request can have the data for many partitions.
* Add namespace_id and table_id to persist complete request to make data easier to lookup in buffer.
* feat: Initial ingest_replica skeleton
A bunch of copy pasta here from ingester2, but this takes out a ton of stuff that isn't used in replicas.
Also lays the groundwork for the simpler buffer structure to keep the data and a basic cache for catalog information that will be required.
* feat: update replication.proto GetPartitionBufferResponse
* chore: PR cleanup
* chore: PR cleanup
2023-01-09 16:53:49 +00:00
7f7f0ca170
chore: bump tokio 1.22 -> 1.24
...
Fixes a bug on windows:
https://github.com/tokio-rs/tokio/security/advisories/GHSA-7rrj-xr53-82p7
2023-01-09 16:25:35 +01:00
fa6b07f3a0
refactor(trogging): remove atty dependency
...
Replace atty with IsTerminal due to:
https://rustsec.org/advisories/RUSTSEC-2021-0145
2023-01-09 16:14:11 +01:00
c313db8dfe
Merge branch 'main' into dom/seqnum-set
2023-01-09 10:33:53 +00:00
3fac475b63
chore(deps): Bump base64 from 0.20.0 to 0.21.0 ( #6530 )
...
* chore(deps): Bump base64 from 0.20.0 to 0.21.0
Bumps [base64](https://github.com/marshallpierce/rust-base64 ) from 0.20.0 to 0.21.0.
- [Release notes](https://github.com/marshallpierce/rust-base64/releases )
- [Changelog](https://github.com/marshallpierce/rust-base64/blob/master/RELEASE-NOTES.md )
- [Commits](https://github.com/marshallpierce/rust-base64/compare/v0.20.0...v0.21.0 )
---
updated-dependencies:
- dependency-name: base64
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* chore: Run cargo hakari tasks
* fix: deprecations
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: Marco Neumann <marco@crepererum.net>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-01-09 09:39:40 +00:00
dfdc964f64
chore(deps): Bump insta from 1.24.1 to 1.26.0 ( #6534 )
...
* chore(deps): Bump insta from 1.24.1 to 1.26.0
Bumps [insta](https://github.com/mitsuhiko/insta ) from 1.24.1 to 1.26.0.
- [Release notes](https://github.com/mitsuhiko/insta/releases )
- [Changelog](https://github.com/mitsuhiko/insta/blob/master/CHANGELOG.md )
- [Commits](https://github.com/mitsuhiko/insta/compare/1.24.1...1.26.0 )
---
updated-dependencies:
- dependency-name: insta
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* chore: Run cargo hakari tasks
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-01-09 08:02:08 +00:00
e31c84a794
chore(deps): Bump async-trait from 0.1.60 to 0.1.61 ( #6533 )
...
Bumps [async-trait](https://github.com/dtolnay/async-trait ) from 0.1.60 to 0.1.61.
- [Release notes](https://github.com/dtolnay/async-trait/releases )
- [Commits](https://github.com/dtolnay/async-trait/compare/0.1.60...0.1.61 )
---
updated-dependencies:
- dependency-name: async-trait
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-01-09 07:44:35 +00:00
3b672c223c
feat(data_types): SequenceNumberSet
...
Adds a space-efficient encoding of a set of SequenceNumber, backed by
roaring bitmaps.
Memory utilisation will change as the number of elements changes,
according to the underlying roaring bitmap design, but is intended to be
"relatively" cheap.
2023-01-05 18:20:48 +01:00
9ab86fa154
fix(router2): drive ingester node (re)-discovery
...
The tonic / tower load-balance implementation discards failed nodes,
even when using a static list - this causes nodes that fail once to
never be retried.
This doesn't happen for the last node for some reason, and leads to all
the load from one router hitting a single ingester instead of load
balancing across all ingesters.
This commit adds a hack to constantly tell the load balancer to probe
all nodes, hopefully causing them to re-discover previously failed
nodes. I don't have the time to do this properly :(
2023-01-05 14:06:29 +01:00
84485b57b0
chore(deps): Bump insta from 1.24.0 to 1.24.1 ( #6500 )
...
Bumps [insta](https://github.com/mitsuhiko/insta ) from 1.24.0 to 1.24.1.
- [Release notes](https://github.com/mitsuhiko/insta/releases )
- [Changelog](https://github.com/mitsuhiko/insta/blob/master/CHANGELOG.md )
- [Commits](https://github.com/mitsuhiko/insta/compare/1.24.0...1.24.1 )
---
updated-dependencies:
- dependency-name: insta
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-05 08:49:05 +00:00
6843eee1d2
feat: Extract encoding from `RecordBatch` --> `FlightData` from flight implementations ( #6460 )
...
* feat: Extract encoding from `RecordBatch` --> `FlightData` from flight implementations
Refactor existing flight server impl
* fix: Apply suggestions from code review
Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
* fix: fixup code review comments
* fix: update for more details
* fix: Update names / types
Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-01-04 13:36:16 +00:00
8bb56f6ef5
chore(deps): Bump insta from 1.23.0 to 1.24.0 ( #6491 )
...
Bumps [insta](https://github.com/mitsuhiko/insta ) from 1.23.0 to 1.24.0.
- [Release notes](https://github.com/mitsuhiko/insta/releases )
- [Changelog](https://github.com/mitsuhiko/insta/blob/master/CHANGELOG.md )
- [Commits](https://github.com/mitsuhiko/insta/compare/1.23.0...1.24.0 )
---
updated-dependencies:
- dependency-name: insta
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-04 07:11:08 +00:00
dbe52f1ca1
chore: Upgrade datafusion ( #6467 )
...
* chore: Update datafusion
* fix: Update for new apis
* chore: Update expected plan
* fix: Update for new config construction
* chore: update clippy
* fix: Fix error codes
* fix: update another test
* chore: Run cargo hakari tasks
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-01-03 15:29:11 +00:00
90d53ae383
chore(deps): Bump sqlparser from 0.28.0 to 0.30.0 ( #6479 )
...
* chore(deps): Bump sqlparser from 0.28.0 to 0.30.0
Bumps [sqlparser](https://github.com/sqlparser-rs/sqlparser-rs ) from 0.28.0 to 0.30.0.
- [Release notes](https://github.com/sqlparser-rs/sqlparser-rs/releases )
- [Changelog](https://github.com/sqlparser-rs/sqlparser-rs/blob/main/CHANGELOG.md )
- [Commits](https://github.com/sqlparser-rs/sqlparser-rs/compare/v0.28.0...v0.30.0 )
---
updated-dependencies:
- dependency-name: sqlparser
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* fix: Update for new API
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
2023-01-03 14:28:15 +00:00
4add55d39e
feat: InfluxQL planner learns how to normalise InfluxQL AST ( #6236 )
...
* chore: Move logic to context, in line with DataFusion SQL
* chore: Add ordering for InfluxQL data types
Ordering is used to determine automatic casting operations. If two
field columns are present in an expression, one float and one integer,
the integer should be cast to a float, such that the final expression
will be a float.
* chore: Add DerefMut trait to collection types
Will allow these collections to be mutated when traversing the InfluxQL
AST.
* chore: Add influxql module with initial AST normalisation implementation
* chore: Add more unit tests and docs
* chore: Run cargo hakari tasks
* chore: Fix link
* chore: Support regular expression expansion and Call expressions
* chore: Add tests for walk_expr functions
* chore: Add insta snapshot files
* chore: Add docs and make API accessible to the crate
* chore: Move to Arc<dyn SchemaProvider> for use in influxql planner
* chore: Move code back; it is better encapsulated here
* chore: Remove redundant attribute
* chore: Improve regex compatibility with InfluxQL / Go
* chore: Style improvement.
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
2023-01-02 23:48:21 +00:00
0aacef3c59
chore(deps): Bump once_cell from 1.16.0 to 1.17.0 ( #6473 )
...
* chore(deps): Bump once_cell from 1.16.0 to 1.17.0
Bumps [once_cell](https://github.com/matklad/once_cell ) from 1.16.0 to 1.17.0.
- [Release notes](https://github.com/matklad/once_cell/releases )
- [Changelog](https://github.com/matklad/once_cell/blob/master/CHANGELOG.md )
- [Commits](https://github.com/matklad/once_cell/compare/v1.16.0...v1.17.0 )
---
updated-dependencies:
- dependency-name: once_cell
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* chore: Change once_cell version specifier to major.minor for less churn
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Carol (Nichols || Goulding) <carol.nichols@gmail.com>
2023-01-02 17:07:15 +00:00
ec869d2384
chore(deps): Bump comfy-table from 6.1.3 to 6.1.4 ( #6478 )
...
Bumps [comfy-table](https://github.com/nukesor/comfy-table ) from 6.1.3 to 6.1.4.
- [Release notes](https://github.com/nukesor/comfy-table/releases )
- [Changelog](https://github.com/Nukesor/comfy-table/blob/main/CHANGELOG.md )
- [Commits](https://github.com/nukesor/comfy-table/compare/v6.1.3...v6.1.4 )
---
updated-dependencies:
- dependency-name: comfy-table
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-01-02 15:16:11 +00:00
3d7b4e29b9
chore(deps): Bump nom from 7.1.1 to 7.1.2 ( #6477 )
...
Bumps [nom](https://github.com/Geal/nom ) from 7.1.1 to 7.1.2.
- [Release notes](https://github.com/Geal/nom/releases )
- [Changelog](https://github.com/rust-bakery/nom/blob/main/CHANGELOG.md )
- [Commits](https://github.com/Geal/nom/compare/7.1.1...7.1.2 )
---
updated-dependencies:
- dependency-name: nom
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-02 15:09:27 +00:00
0bc4f43319
chore(deps): Bump rustix from 0.36.5 to 0.36.6 ( #6470 )
...
Bumps [rustix](https://github.com/bytecodealliance/rustix ) from 0.36.5 to 0.36.6.
- [Release notes](https://github.com/bytecodealliance/rustix/releases )
- [Commits](https://github.com/bytecodealliance/rustix/compare/v0.36.5...v0.36.6 )
---
updated-dependencies:
- dependency-name: rustix
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-02 15:00:42 +00:00
f56c792f79
chore(deps): Bump serde from 1.0.151 to 1.0.152 ( #6468 )
...
Bumps [serde](https://github.com/serde-rs/serde ) from 1.0.151 to 1.0.152.
- [Release notes](https://github.com/serde-rs/serde/releases )
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.151...v1.0.152 )
---
updated-dependencies:
- dependency-name: serde
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-02 10:27:01 +00:00
febaf3c737
chore(deps): Bump clap from 4.0.31 to 4.0.32 ( #6466 )
...
Bumps [clap](https://github.com/clap-rs/clap ) from 4.0.31 to 4.0.32.
- [Release notes](https://github.com/clap-rs/clap/releases )
- [Changelog](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md )
- [Commits](https://github.com/clap-rs/clap/compare/v4.0.31...v4.0.32 )
---
updated-dependencies:
- dependency-name: clap
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-02 10:14:23 +00:00
7279b9597a
chore(deps): Bump libc from 0.2.138 to 0.2.139 ( #6465 )
...
Bumps [libc](https://github.com/rust-lang/libc ) from 0.2.138 to 0.2.139.
- [Release notes](https://github.com/rust-lang/libc/releases )
- [Commits](https://github.com/rust-lang/libc/compare/0.2.138...0.2.139 )
---
updated-dependencies:
- dependency-name: libc
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-01-02 09:04:29 +00:00
5b9e5b4a92
chore(deps): Bump clap from 4.0.29 to 4.0.31 ( #6464 )
...
Bumps [clap](https://github.com/clap-rs/clap ) from 4.0.29 to 4.0.31.
- [Release notes](https://github.com/clap-rs/clap/releases )
- [Changelog](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md )
- [Commits](https://github.com/clap-rs/clap/compare/v4.0.29...v4.0.31 )
---
updated-dependencies:
- dependency-name: clap
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-12-22 20:00:23 +00:00
858a41c733
chore(deps): Bump handlebars from 4.3.5 to 4.3.6 ( #6458 )
...
Bumps [handlebars](https://github.com/sunng87/handlebars-rust ) from 4.3.5 to 4.3.6.
- [Release notes](https://github.com/sunng87/handlebars-rust/releases )
- [Changelog](https://github.com/sunng87/handlebars-rust/blob/v4.3.6/CHANGELOG.md )
- [Commits](https://github.com/sunng87/handlebars-rust/compare/v4.3.5...v4.3.6 )
---
updated-dependencies:
- dependency-name: handlebars
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-12-22 18:53:45 +00:00
ae296886c2
chore(deps): Bump prost-types from 0.11.2 to 0.11.5 ( #6452 )
...
* chore(deps): Bump prost-types from 0.11.2 to 0.11.5
Bumps [prost-types](https://github.com/tokio-rs/prost ) from 0.11.2 to 0.11.5.
- [Release notes](https://github.com/tokio-rs/prost/releases )
- [Commits](https://github.com/tokio-rs/prost/compare/v0.11.2...v0.11.5 )
---
updated-dependencies:
- dependency-name: prost-types
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* chore: Run cargo hakari tasks
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-12-21 16:31:35 +00:00
ed17069087
chore(deps): Bump num_cpus from 1.14.0 to 1.15.0 ( #6451 )
...
Bumps [num_cpus](https://github.com/seanmonstar/num_cpus ) from 1.14.0 to 1.15.0.
- [Release notes](https://github.com/seanmonstar/num_cpus/releases )
- [Changelog](https://github.com/seanmonstar/num_cpus/blob/master/CHANGELOG.md )
- [Commits](https://github.com/seanmonstar/num_cpus/compare/v1.14.0...v1.15.0 )
---
updated-dependencies:
- dependency-name: num_cpus
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-12-21 15:17:55 +00:00
e1059a9009
feat: FlightSQL Milestone 2 basic FlightSQL client and FlightSQL server implementation and plumbing ( #6398 )
...
* feat: Add basic Flight and FlightSQL client into IOx codebase
Basic flight end to end test
* fix: Apply suggestions from code review
Co-authored-by: Raphael Taylor-Davies <1781103+tustvold@users.noreply.github.com>
Co-authored-by: Raphael Taylor-Davies <1781103+tustvold@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-12-20 17:34:00 +00:00
9d4b2b56ae
chore(deps): Bump snafu from 0.7.3 to 0.7.4
...
Bumps [snafu](https://github.com/shepmaster/snafu ) from 0.7.3 to 0.7.4.
- [Release notes](https://github.com/shepmaster/snafu/releases )
- [Changelog](https://github.com/shepmaster/snafu/blob/main/CHANGELOG.md )
- [Commits](https://github.com/shepmaster/snafu/compare/0.7.3...0.7.4 )
---
updated-dependencies:
- dependency-name: snafu
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2022-12-20 01:04:43 +00:00
d0d5906476
chore: Update datafusion pin ( #6442 )
...
* chore: Update datafusion pin
* refactor: Update iox_query for new apis
* chore: Update some more apis
* chore: Run cargo hakari tasks
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
2022-12-19 20:02:42 +00:00
94c2f94ea1
refactor: Extract common ArrowFlight client into iox_arrow_flight ( #6427 )
...
* refactor: Extract common ArrowFlight client into iox_arrow_flight
* chore: Run cargo hakari tasks
* fix: clarify intent of iox_arrow_flight crate
* refactor: Apply suggestions from code review
Co-authored-by: Raphael Taylor-Davies <1781103+tustvold@users.noreply.github.com>
* fix: loop --> while let
* fix: REmove make_tonic_error in favor of From impl
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: Raphael Taylor-Davies <1781103+tustvold@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-12-19 11:35:20 +00:00
299f0e99f9
chore(deps): Bump thiserror from 1.0.37 to 1.0.38
...
Bumps [thiserror](https://github.com/dtolnay/thiserror ) from 1.0.37 to 1.0.38.
- [Release notes](https://github.com/dtolnay/thiserror/releases )
- [Commits](https://github.com/dtolnay/thiserror/compare/1.0.37...1.0.38 )
---
updated-dependencies:
- dependency-name: thiserror
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2022-12-19 10:33:32 +00:00
8478d41bcb
chore(deps): Bump paste from 1.0.10 to 1.0.11 ( #6430 )
...
Bumps [paste](https://github.com/dtolnay/paste ) from 1.0.10 to 1.0.11.
- [Release notes](https://github.com/dtolnay/paste/releases )
- [Commits](https://github.com/dtolnay/paste/compare/1.0.10...1.0.11 )
---
updated-dependencies:
- dependency-name: paste
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-12-19 10:31:05 +00:00
149de84313
chore(deps): Bump serde from 1.0.150 to 1.0.151 ( #6432 )
...
Bumps [serde](https://github.com/serde-rs/serde ) from 1.0.150 to 1.0.151.
- [Release notes](https://github.com/serde-rs/serde/releases )
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.150...v1.0.151 )
---
updated-dependencies:
- dependency-name: serde
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-12-19 10:24:11 +00:00
ecd6cfa3a8
chore(deps): Bump syn from 1.0.105 to 1.0.107 ( #6434 )
...
Bumps [syn](https://github.com/dtolnay/syn ) from 1.0.105 to 1.0.107.
- [Release notes](https://github.com/dtolnay/syn/releases )
- [Commits](https://github.com/dtolnay/syn/commits )
---
updated-dependencies:
- dependency-name: syn
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-12-19 10:17:11 +00:00
c72734473c
chore(deps): Bump async-trait from 0.1.59 to 0.1.60 ( #6433 )
...
Bumps [async-trait](https://github.com/dtolnay/async-trait ) from 0.1.59 to 0.1.60.
- [Release notes](https://github.com/dtolnay/async-trait/releases )
- [Commits](https://github.com/dtolnay/async-trait/compare/0.1.59...0.1.60 )
---
updated-dependencies:
- dependency-name: async-trait
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-12-19 10:09:23 +00:00
7f2aa8b10c
chore(deps): Bump serde_json from 1.0.89 to 1.0.91
...
Bumps [serde_json](https://github.com/serde-rs/json ) from 1.0.89 to 1.0.91.
- [Release notes](https://github.com/serde-rs/json/releases )
- [Commits](https://github.com/serde-rs/json/compare/v1.0.89...v1.0.91 )
---
updated-dependencies:
- dependency-name: serde_json
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2022-12-19 01:44:18 +00:00
84698b3532
feat: add size_data to mutable batch ( #6425 )
...
This method will be used in the new ingestion pipeline to approximate how much memory a butable batch will take to convert to arrow and persist. It is meant only as a very rough estimate to trigger persistence for hot partitions.
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-12-16 17:20:16 +00:00
cfb7c16bb1
Merge branch 'main' into dom/optimal-persist-parallelism
2022-12-16 09:12:22 +00:00
6a841bdf5a
chore(deps): Bump cc from 1.0.77 to 1.0.78
...
Bumps [cc](https://github.com/rust-lang/cc-rs ) from 1.0.77 to 1.0.78.
- [Release notes](https://github.com/rust-lang/cc-rs/releases )
- [Commits](https://github.com/rust-lang/cc-rs/compare/1.0.77...1.0.78 )
---
updated-dependencies:
- dependency-name: cc
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2022-12-16 01:08:24 +00:00
aec98015d7
fix: Remove the rpc_write feature flag and use INFLUXDB_IOX_MODE env var instead
...
And standardize on ingester2 and router2 for consistency.
Connects to #6402 .
2022-12-15 14:06:59 -05:00
933ab1f8c7
feat(ingester2): optimal persist parallelism
...
This commit changes the behaviour of the persist system to enable
optimal parallelism of persist operations, and improve the accuracy of
the outstanding job bound / back-pressure.
Previously all persist operations for a given partition were
consistently hashed to a single worker task. This serialised persistence
per partition, ensuring all updates to the partition sort key were
serialised. However, this also unnecessarily serialises persist
operations that do not need to update the sort key, reducing the
potential throughput of the system; in the worst case of a single
partition receiving all the writes, only one worker would be persisting,
and the other N-1 workers would be idle.
After this change, the sort key is inspected when enqueuing the persist
operation and if it can be determined that no sort key update is
necessary (the typical case), then the persist task is placed into a
global work queue from which all workers consume. This allows for
maximal parallelisation of these jobs, and the removes the per-worker
head-of-line blocking.
In the case that the sort key does need updating, these jobs continue to
be consistently hashed to a single worker, ensuring serialised sort key
updates only where necessary.
To support these changes, the back-pressure system has been changed to
account for all outstanding persist jobs in the system, regardless of
type or assigned worker - a logical, bounded queue is composed together
of a semaphore limiting the number of persist tasks overall, and a
series of physical, unbounded queues - one to each worker & the global
queue. The overall system remains bounded by the
INFLUXDB_IOX_PERSIST_QUEUE_DEPTH value, and is now simpler to reason
about (it is independent of the number of workers, etc).
2022-12-15 18:30:51 +01:00
Andrew Lamb
dependabot[bot]
Marco Neumann
Stuart Carnie
Dom Dwyer
Raphael Taylor-Davies
Carol (Nichols || Goulding)
Christopher M. Wolff
Carol (Nichols || Goulding)
kodiakhq[bot]
Nga Tran
CircleCI[bot]
Paul Dix