9af93ca9ba
chore(deps): Bump pretty_assertions from 1.2.1 to 1.3.0 ( #5517 )
...
Bumps [pretty_assertions](https://github.com/rust-pretty-assertions/rust-pretty-assertions ) from 1.2.1 to 1.3.0.
- [Release notes](https://github.com/rust-pretty-assertions/rust-pretty-assertions/releases )
- [Changelog](https://github.com/rust-pretty-assertions/rust-pretty-assertions/blob/main/CHANGELOG.md )
- [Commits](https://github.com/rust-pretty-assertions/rust-pretty-assertions/compare/v1.2.1...v1.3.0 )
---
updated-dependencies:
- dependency-name: pretty_assertions
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-09-01 10:20:26 +00:00
00ed79ff1b
chore(deps): Bump thiserror from 1.0.32 to 1.0.33 ( #5524 )
...
Bumps [thiserror](https://github.com/dtolnay/thiserror ) from 1.0.32 to 1.0.33.
- [Release notes](https://github.com/dtolnay/thiserror/releases )
- [Commits](https://github.com/dtolnay/thiserror/compare/1.0.32...1.0.33 )
---
updated-dependencies:
- dependency-name: thiserror
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-09-01 09:11:31 +00:00
6669d85fb4
chore: Update datafusion + arrow/parquet to `21.0.0` ( #5519 )
...
* chore: Update arrow/arrow-flight/parquet to 21.0.0
* chore: Update datafusion pin
* chore: Fix arrow update script
* chore: Update Cargo.lock
* chore: Update for new API
2022-08-31 13:30:47 +00:00
711ba77341
chore: update object_store to test IMDSv1 fallback ( #5509 )
...
* chore: update object_store to test IMDSv1 fallback
* chore: Run cargo hakari tasks
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-08-30 12:31:49 +00:00
89af2f2b1d
Merge branch 'main' into dom/revert-object-store-bump
2022-08-30 09:47:02 +01:00
66f0b59dbb
revert: remove Azure SDK / bump object_store
...
This reverts commit c2f8efa03a
2022-08-30 10:41:29 +02:00
dcc0f9d34f
build: bump dotenvy
...
I fixed this while waiting for my build to deploy. I think that says
more about our build than anything else!
2022-08-30 10:34:26 +02:00
0137db9adc
chore(deps): Bump futures from 0.3.23 to 0.3.24
...
Bumps [futures](https://github.com/rust-lang/futures-rs ) from 0.3.23 to 0.3.24.
- [Release notes](https://github.com/rust-lang/futures-rs/releases )
- [Changelog](https://github.com/rust-lang/futures-rs/blob/master/CHANGELOG.md )
- [Commits](https://github.com/rust-lang/futures-rs/compare/0.3.23...0.3.24 )
---
updated-dependencies:
- dependency-name: futures
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2022-08-30 01:24:21 +00:00
de47f5605b
chore: Update datafusion (with new sqlparser release) - option 1 ( #5433 )
...
* chore: Update datafusion pin
* chore: Update now that user is a reserved word
* chore: Update cargo.lock
* fix: update query for user function
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-08-29 19:10:00 +00:00
74c9529062
fix: Rename KafkaPartition to ShardIndex
2022-08-29 14:07:18 -04:00
e441b5b307
feat: add deadline config to backoff system ( #5489 )
...
This will simplify event emission in #5464 .
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-08-29 14:51:41 +00:00
c2f8efa03a
build: remove Azure SDK / bump object_store
...
Bumps the object_store pin to master to pick up:
https://github.com/apache/arrow-rs/pull/2509
This removes the Azure SDK.
2022-08-29 14:06:51 +02:00
05e599228b
chore(deps): Bump sqlparser from 0.21.0 to 0.22.0 ( #5482 )
...
Bumps [sqlparser](https://github.com/sqlparser-rs/sqlparser-rs ) from 0.21.0 to 0.22.0.
- [Release notes](https://github.com/sqlparser-rs/sqlparser-rs/releases )
- [Changelog](https://github.com/sqlparser-rs/sqlparser-rs/blob/main/CHANGELOG.md )
- [Commits](https://github.com/sqlparser-rs/sqlparser-rs/compare/v0.21.0...v0.22.0 )
---
updated-dependencies:
- dependency-name: sqlparser
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-08-29 09:07:19 +00:00
9e2fc11d04
chore(deps): Bump time from 0.3.13 to 0.3.14 ( #5472 )
...
Bumps [time](https://github.com/time-rs/time ) from 0.3.13 to 0.3.14.
- [Release notes](https://github.com/time-rs/time/releases )
- [Changelog](https://github.com/time-rs/time/blob/main/CHANGELOG.md )
- [Commits](https://github.com/time-rs/time/compare/v0.3.13...v0.3.14 )
---
updated-dependencies:
- dependency-name: time
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-08-29 08:42:28 +00:00
14a3215b7b
chore(deps): Bump lock_api from 0.4.7 to 0.4.8 ( #5481 )
...
Bumps [lock_api](https://github.com/Amanieu/parking_lot ) from 0.4.7 to 0.4.8.
- [Release notes](https://github.com/Amanieu/parking_lot/releases )
- [Changelog](https://github.com/Amanieu/parking_lot/blob/master/CHANGELOG.md )
- [Commits](https://github.com/Amanieu/parking_lot/compare/lock_api-0.4.7...lock_api-0.4.8 )
---
updated-dependencies:
- dependency-name: lock_api
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-08-29 08:08:27 +00:00
6b81d4c0be
chore(deps): Bump comfy-table from 6.0.0 to 6.1.0 ( #5480 )
...
Bumps [comfy-table](https://github.com/nukesor/comfy-table ) from 6.0.0 to 6.1.0.
- [Release notes](https://github.com/nukesor/comfy-table/releases )
- [Changelog](https://github.com/Nukesor/comfy-table/blob/main/CHANGELOG.md )
- [Commits](https://github.com/nukesor/comfy-table/compare/v6.0.0...v6.1.0 )
---
updated-dependencies:
- dependency-name: comfy-table
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-08-29 07:38:37 +00:00
5c20d6248f
chore: Upgrade lz4-sys due to RUSTSEC-2022-0051 ( #5479 )
...
https://rustsec.org/advisories/RUSTSEC-2022-0051
2022-08-26 20:30:19 +00:00
3950ca3a17
feat: upsert partition & update sort key for each day in bulk ingest ( #5447 )
...
* feat: upsert partition & update sort key for each day in bulk ingest
feat: import schema now supports earliest/latest time merging
chore: tests & tidying up for bulk ingest catalog update
* fix: always sort time last in PK in import schema update catalog
* chore: additional test for computing sort key in bulk ingest
* chore: bulk import catalog update gets sequencer from sharder service
chore: import update schema tests refactor using sharder svc mock
* chore: dead code fix
* chore: import schema sequencer lookup test
* chore: clarifying comment in import schema catalog update
2022-08-25 10:47:12 +00:00
abf26767c1
refactor: infallible JumpHash initialisation
...
This doesn't really need to be fallible but forces propagation of a ton
of error handling - no shards is always a sign of something being very
wrong, and can be caught in the caller if it's for some reason an
acceptable state / can be recovered from.
2022-08-24 13:18:57 +02:00
4beb721a9a
fix: Revert Bump dotenvy from 0.15.1 to 0.15.2 ( #5450 ) ( #5455 )
...
This reverts commit 84acbd2fadCloses #5454
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-08-24 09:10:09 +00:00
84acbd2fad
chore(deps): Bump dotenvy from 0.15.1 to 0.15.2 ( #5450 )
...
Bumps [dotenvy](https://github.com/allan2/dotenvy ) from 0.15.1 to 0.15.2.
- [Release notes](https://github.com/allan2/dotenvy/releases )
- [Changelog](https://github.com/allan2/dotenvy/blob/master/CHANGELOG.md )
- [Commits](https://github.com/allan2/dotenvy/commits/v0.15.2 )
---
updated-dependencies:
- dependency-name: dotenvy
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-08-23 11:24:42 +00:00
064606380b
feat: refresh policy for caches ( #5431 )
...
* feat: refresh policy for caches
For #5318 we want to have a policy that refreshes keys before they are
too old. I initially tried to fold both TTL and the refresh system into
a single policy but than decided that they will basically be two
policies in one with a harder-to-test interface. Semantically TTL and
refresh are also a bit different (but will usually be used together):
- **TTL:** Prevents that a users gets data that is too old. It is some kind
of "soft correctness". In some sense this is related to the "remove
if" policy where some part of the system knows for sure (or with
reasonable likelyhood) that a cache entry is outdated. Note that TTL's
primary job is NOT to clean up memory from old keys (even though it
indirectly does that). There is no reason cached entries should be
removed except for correctness (TTL and remove-if) or resource
pressure -- and the latter is handled by the LRU policy.
- **Refresh:** While TTL is some kind of deadline, we often have good
reason to refresh the key before we pull the plug, namely when an
entry is used and a bit old (but not too old). The concrete mechanism
to archive this is flexible. At the moment the policy is rather
simple -- just start a refresh task if a key is old and we receive a
GET request -- but can be extended in the future.
This also adds some integration tests for TTL+refresh. There will be
follow-up changes to test the interaction with LRU as well, althouh I am
pretty certain that there won't be any surprises due to the excessive
testing we have in place for the policy backend itself as well as all
the policies.
This change also does NOT integrate the refresh with the querier for the
sake of keeping the changeset "small" (i.e. it is already rather large).
* docs: improve
Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
2022-08-22 08:45:22 +00:00
2d795bd604
chore: cargo update ( #5439 )
...
* pest 2.2.1 → 2.3.0
* serde 1.0.143 → 1.0.144
* serde-json 1.0.83 → 1.0.85
pest_meta and pest_generator 2.2.1 were yanked
2022-08-22 05:15:48 +00:00
ed38b01e91
chore(deps): Bump sqlparser from 0.20.0 to 0.21.0 ( #5429 )
...
Bumps [sqlparser](https://github.com/sqlparser-rs/sqlparser-rs ) from 0.20.0 to 0.21.0.
- [Release notes](https://github.com/sqlparser-rs/sqlparser-rs/releases )
- [Changelog](https://github.com/sqlparser-rs/sqlparser-rs/blob/main/CHANGELOG.md )
- [Commits](https://github.com/sqlparser-rs/sqlparser-rs/compare/v0.20.0...v0.21.0 )
---
updated-dependencies:
- dependency-name: sqlparser
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-08-19 10:41:38 +00:00
5ce0618b8f
chore: `cargo update` ( #5432 )
...
```
cpufeatures v0.2.2 -> v0.2.3
plotters v0.3.2 -> v0.3.3
plotters-svg v0.3.2 -> v0.3.3
```
`plotters v0.3.2` was yanked.
2022-08-19 09:56:23 +00:00
b4e5895d7a
feat: Add influxdb_influxql_parser crate ( #5415 )
...
* feat: Add crate; parse quoted identifiers
* chore: Run cargo hakari tasks
* chore: satisfy linter
* chore: Use `test_helpers::Result`
* feat: Add all InfluxQL keywords
* chore: Update influxdb_influxql_parser/src/lib.rs
Co-authored-by: Marco Neumann <marco@crepererum.net>
* chore: PR feedback
* chore: PR Feedback, remove Result<()>
* chore: Update Cargo.lock
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: Marco Neumann <marco@crepererum.net>
2022-08-18 23:09:45 +00:00
d75df2b610
chore: `cargo update` ( #5426 )
...
```
bumpalo v3.10.0 -> v3.11.0
either v1.7.0 -> v1.8.0
iana-time-zone v0.1.45 -> v0.1.46
rustix v0.35.8 -> v0.35.9
```
`rustix` is important because `0.35.8` was yanked.
2022-08-18 08:53:00 +00:00
3dca9c1b43
feat: async sleep with `TimeProvider` ( #5417 )
...
* feat: async sleep with `TimeProvider`
This is helpful to mock "ticked" loops or to control certain async time
periods.
Will be used to test the refresh policy developed in #5318 .
* refactor: use a single `TimeProvider::sleep` impl
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-08-17 14:50:35 +00:00
7f0ae53d6f
chore: Update to (almost) released object_store 0.4.0 ( #5419 )
...
* chore: update object_store
* chore: update hakari config
* chore: Run cargo hakari tasks
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
2022-08-17 13:44:48 +00:00
bd4b708055
chore: Update datafusion pin + other dependencies ( #5418 )
...
* chore: Update datafusion pin
* chore: Update other depdecies
* fix: Update for changes in API
2022-08-17 10:42:37 +00:00
2e638fe19c
chore(deps): Bump libc from 0.2.131 to 0.2.132 ( #5414 )
...
Bumps [libc](https://github.com/rust-lang/libc ) from 0.2.131 to 0.2.132.
- [Release notes](https://github.com/rust-lang/libc/releases )
- [Commits](https://github.com/rust-lang/libc/compare/0.2.131...0.2.132 )
---
updated-dependencies:
- dependency-name: libc
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-08-17 08:40:57 +00:00
78665d3092
chore(deps): Bump once_cell from 1.13.0 to 1.13.1 ( #5413 )
...
Bumps [once_cell](https://github.com/matklad/once_cell ) from 1.13.0 to 1.13.1.
- [Release notes](https://github.com/matklad/once_cell/releases )
- [Changelog](https://github.com/matklad/once_cell/blob/master/CHANGELOG.md )
- [Commits](https://github.com/matklad/once_cell/compare/v1.13.0...v1.13.1 )
---
updated-dependencies:
- dependency-name: once_cell
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-08-17 08:31:46 +00:00
180ff9f681
feat: table name in schema validation errors
...
Scopes all schema validation errors to include the table name in the
error output.
2022-08-16 19:00:44 +02:00
7e97620b37
chore: `cargo-update` ( #5405 )
...
```
anyhow v1.0.60 -> v1.0.61
bytemuck v1.11.0 -> v1.12.0
hdrhistogram v7.5.0 -> v7.5.1
iana-time-zone v0.1.44 -> v0.1.45
memmap2 v0.5.5 -> v0.5.7
os_str_bytes v6.2.0 -> v6.3.0
```
`iana-time-zone` is important because `0.1.44` was yanked.
2022-08-16 12:51:46 +00:00
10fee5535a
feat: import schema updates iox catalog ( #5385 )
...
* feat: import schema updates iox catalog
- renamed import/schema module to aggregate_tsm_schema to not conflic
with schema crate
- fetch schema from iox catalog, and validate/merge/create as needed
chore: add catalog dsn config to import schema command
chore: import schema command connects to catalog
chore: import schema merge validation errors return non-zero code
chore: simplified and tidies import update catalog code
chore: tests and refactoring of import schema catalog update
* chore: require retention on ns creation in import
* chore: fixed bad test in import schema validation
* chore: friendlier errors & more tests in import schema catalog update
2022-08-16 11:05:27 +00:00
724759862f
chore(deps): Bump pin-project from 1.0.11 to 1.0.12 ( #5402 )
...
Bumps [pin-project](https://github.com/taiki-e/pin-project ) from 1.0.11 to 1.0.12.
- [Release notes](https://github.com/taiki-e/pin-project/releases )
- [Changelog](https://github.com/taiki-e/pin-project/blob/main/CHANGELOG.md )
- [Commits](https://github.com/taiki-e/pin-project/compare/v1.0.11...v1.0.12 )
---
updated-dependencies:
- dependency-name: pin-project
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-08-16 08:04:25 +00:00
0ccefa0d0c
refactor: port TTL backend to policy framework ( #5396 )
...
* refactor: port TTL backend to policy framework
Note that this is "just" a port, it does NOT change how TTL works. This
will be done in #5318 .
Helps with #5320 .
* fix: ensure inner backend is empty
* test: add some smoke test
2022-08-15 16:48:16 +00:00
0a44a42999
chore(deps): Bump futures from 0.3.21 to 0.3.23 ( #5394 )
...
Bumps [futures](https://github.com/rust-lang/futures-rs ) from 0.3.21 to 0.3.23.
- [Release notes](https://github.com/rust-lang/futures-rs/releases )
- [Changelog](https://github.com/rust-lang/futures-rs/blob/master/CHANGELOG.md )
- [Commits](https://github.com/rust-lang/futures-rs/compare/0.3.21...0.3.23 )
---
updated-dependencies:
- dependency-name: futures
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-08-15 12:19:17 +00:00
69ccfafd14
chore(deps): Bump futures-task from 0.3.21 to 0.3.23 ( #5393 )
...
Bumps [futures-task](https://github.com/rust-lang/futures-rs ) from 0.3.21 to 0.3.23.
- [Release notes](https://github.com/rust-lang/futures-rs/releases )
- [Changelog](https://github.com/rust-lang/futures-rs/blob/master/CHANGELOG.md )
- [Commits](https://github.com/rust-lang/futures-rs/compare/0.3.21...0.3.23 )
---
updated-dependencies:
- dependency-name: futures-task
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-08-15 10:27:22 +00:00
2c934addf5
chore(deps): Bump futures-channel from 0.3.21 to 0.3.23 ( #5388 )
...
Bumps [futures-channel](https://github.com/rust-lang/futures-rs ) from 0.3.21 to 0.3.23.
- [Release notes](https://github.com/rust-lang/futures-rs/releases )
- [Changelog](https://github.com/rust-lang/futures-rs/blob/master/CHANGELOG.md )
- [Commits](https://github.com/rust-lang/futures-rs/compare/0.3.21...0.3.23 )
---
updated-dependencies:
- dependency-name: futures-channel
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-08-15 09:42:53 +00:00
a14e161109
chore(deps): Bump futures-core from 0.3.21 to 0.3.23 ( #5387 )
...
Bumps [futures-core](https://github.com/rust-lang/futures-rs ) from 0.3.21 to 0.3.23.
- [Release notes](https://github.com/rust-lang/futures-rs/releases )
- [Changelog](https://github.com/rust-lang/futures-rs/blob/master/CHANGELOG.md )
- [Commits](https://github.com/rust-lang/futures-rs/compare/0.3.21...0.3.23 )
---
updated-dependencies:
- dependency-name: futures-core
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-08-15 08:59:53 +00:00
6a4425ae08
chore(deps): Bump chrono from 0.4.21 to 0.4.22 ( #5392 )
...
Bumps [chrono](https://github.com/chronotope/chrono ) from 0.4.21 to 0.4.22.
- [Release notes](https://github.com/chronotope/chrono/releases )
- [Changelog](https://github.com/chronotope/chrono/blob/main/CHANGELOG.md )
- [Commits](https://github.com/chronotope/chrono/compare/v0.4.21...v0.4.22 )
---
updated-dependencies:
- dependency-name: chrono
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-08-15 08:31:23 +00:00
38277cf0fb
chore(deps): Bump nix from 0.24.2 to 0.25.0 ( #5391 )
...
* chore(deps): Bump nix from 0.24.2 to 0.25.0
Bumps [nix](https://github.com/nix-rust/nix ) from 0.24.2 to 0.25.0.
- [Release notes](https://github.com/nix-rust/nix/releases )
- [Changelog](https://github.com/nix-rust/nix/blob/master/CHANGELOG.md )
- [Commits](https://github.com/nix-rust/nix/compare/v0.24.2...v0.25.0 )
---
updated-dependencies:
- dependency-name: nix
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* chore: Run cargo hakari tasks
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-08-15 08:10:09 +00:00
822ec18fcd
chore(deps): Bump clap from 3.2.16 to 3.2.17 ( #5390 )
...
Bumps [clap](https://github.com/clap-rs/clap ) from 3.2.16 to 3.2.17.
- [Release notes](https://github.com/clap-rs/clap/releases )
- [Changelog](https://github.com/clap-rs/clap/blob/v3.2.17/CHANGELOG.md )
- [Commits](https://github.com/clap-rs/clap/compare/v3.2.16...v3.2.17 )
---
updated-dependencies:
- dependency-name: clap
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-08-15 07:56:11 +00:00
dbe6b4947c
Merge branch 'main' into dom/bump-rskafka
2022-08-12 09:20:37 +01:00
7118334774
build: bump rskafka
...
Bump rskafka to pick up connection pre-warming:
https://github.com/influxdata/rskafka/pull/166
2022-08-12 10:13:25 +02:00
f0ae8c093b
chore(deps): Bump libc from 0.2.129 to 0.2.131 ( #5379 )
...
Bumps [libc](https://github.com/rust-lang/libc ) from 0.2.129 to 0.2.131.
- [Release notes](https://github.com/rust-lang/libc/releases )
- [Commits](https://github.com/rust-lang/libc/commits )
---
updated-dependencies:
- dependency-name: libc
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-08-12 07:53:06 +00:00
faa1db9a24
build: bump rskafka
...
Bump rskafka & fix minor breakage in order to pick up client
pre-warming:
https://github.com/influxdata/rskafka/pull/165
2022-08-11 17:26:06 +02:00
7174f38f3f
build: bump rskafka
...
Bumps rskafka to HEAD to pick up:
https://github.com/influxdata/rskafka/pull/164
2022-08-11 14:00:21 +02:00
ae6ac27960
chore(deps): Bump console-subscriber from 0.1.6 to 0.1.7 ( #5374 )
...
Bumps [console-subscriber](https://github.com/tokio-rs/console ) from 0.1.6 to 0.1.7.
- [Release notes](https://github.com/tokio-rs/console/releases )
- [Commits](https://github.com/tokio-rs/console/compare/console-subscriber-v0.1.6...tokio-console-v0.1.7 )
---
updated-dependencies:
- dependency-name: console-subscriber
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-08-11 11:08:10 +00:00
692ec97c2f
chore(deps): Bump ahash from 0.7.6 to 0.8.0 ( #5373 )
...
* chore(deps): Bump ahash from 0.7.6 to 0.8.0
Bumps [ahash](https://github.com/tkaitchuck/ahash ) from 0.7.6 to 0.8.0.
- [Release notes](https://github.com/tkaitchuck/ahash/releases )
- [Commits](https://github.com/tkaitchuck/ahash/compare/v0.7.6...v0.8.0 )
---
updated-dependencies:
- dependency-name: ahash
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* chore: Run cargo hakari tasks
* fix: `ahash` features
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: Marco Neumann <marco@crepererum.net>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-08-11 08:37:31 +00:00
7787c51b57
feat: add new CLI command to run the compactor once
2022-08-10 11:28:51 -04:00
ce908c8678
refactor: Use CompactorHandlerImpl::new_with_compactor in service
2022-08-10 11:28:51 -04:00
22037b2461
chore: update libm to 0.2.5 ( #5371 )
...
The former version (0.2.4) was yanked, so our CI is now failing.
2022-08-10 15:03:27 +00:00
7e9918f067
chore: import validate merged schema ( #5367 )
...
* feat: import schema merge now outputs validation results
chore: refactor import crate
chore: renamed some structs for clarity in import crate
* chore: tests for import schema merge validation
* chore: Run cargo hakari tasks
* chore: clippy
* chore: make hashmap loop easier to read in import schema validation
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-08-10 12:15:37 +00:00
624b1b33af
chore(deps): Bump libc from 0.2.127 to 0.2.129 ( #5363 )
...
Bumps [libc](https://github.com/rust-lang/libc ) from 0.2.127 to 0.2.129.
- [Release notes](https://github.com/rust-lang/libc/releases )
- [Commits](https://github.com/rust-lang/libc/compare/0.2.127...0.2.129 )
---
updated-dependencies:
- dependency-name: libc
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-08-10 09:23:29 +00:00
4da124d862
feat: concurrent garbage collector deletes ( #5364 )
...
This should speed up the prod process a bit.
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-08-10 09:14:46 +00:00
c5f062bba0
feat: initial commit of schema merge bulk import tool ( #5344 )
...
* feat: initial commit of schema merge bulk import tool
* chore: use observability depds instead of tracing-*
* chore: removed debug printlns
* chore: fix feature decls for cloud providers for import crate
* chore: use println instead of info in import- no need for a simple CLI
* chore: tidy whitespace
* chore: remove unused dep in import
* chore: Run cargo hakari tasks
* chore: removed unimpld import job subcommand
* chore: clarifying comment about custom serialisation code
* chore: clarifying comment about schema merge code in import
* chore: fix wrong comment in import command
* chore: bump object store dep to get bugfix
* chore: rename import schema struct for clarity
* chore: run `cargo hakari generate`
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-08-10 09:07:38 +00:00
4bc16356bc
chore(deps): Bump serde from 1.0.142 to 1.0.143 ( #5362 )
...
Bumps [serde](https://github.com/serde-rs/serde ) from 1.0.142 to 1.0.143.
- [Release notes](https://github.com/serde-rs/serde/releases )
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.142...v1.0.143 )
---
updated-dependencies:
- dependency-name: serde
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-08-10 08:37:06 +00:00
7975d70632
chore(deps): Bump chrono from 0.4.20 to 0.4.21 ( #5361 )
...
* chore(deps): Bump chrono from 0.4.20 to 0.4.21
Bumps [chrono](https://github.com/chronotope/chrono ) from 0.4.20 to 0.4.21.
- [Release notes](https://github.com/chronotope/chrono/releases )
- [Changelog](https://github.com/chronotope/chrono/blob/main/CHANGELOG.md )
- [Commits](https://github.com/chronotope/chrono/compare/v0.4.20...v0.4.21 )
---
updated-dependencies:
- dependency-name: chrono
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* chore: Run cargo hakari tasks
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
2022-08-10 08:25:45 +00:00
16ddc5efc6
chore: Update datafusion / arrow/parquet/arrow-flight and prost/tonic ecosystem ( #5360 )
...
* chore: Update datafusion and arrow
* chore: Update Cargo.lock
* chore: update to Decimal128
* chore: Update tonic/prost/pbjson/etc
* chore: Run cargo hakari tasks
* fix: doctest in generated types
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
2022-08-09 17:30:44 +00:00
dadcc369b1
chore: update object_store to fix credentials client ( #5359 )
...
* chore: update object_store to fix credentials client
* chore: Run cargo hakari tasks
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
2022-08-09 13:17:43 +00:00
b21799acae
chore: Update datafusion, get `date_bin` ( #5340 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-08-09 11:01:37 +00:00
ccb45d7bac
chore: update to rusoto-less object_store ( #5342 )
...
* chore: update to rusoto-less object_store
* chore: Run cargo hakari tasks
* chore: further fixes
* chore: document workaround
* chore: review feedback
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
2022-08-09 09:06:03 +00:00
284a3069ce
feat: Kafka client produce() instrumentation
...
Adds a decorator over the underlying kafka client to capture the latency
distribution of the low-level kafka writes, independent of the
aggregation/DML batching framework that sits "above" this client.
The latency measurements include the serialisation overhead, protocol
overhead, and actual network I/O.
2022-08-08 15:24:35 +02:00
f3913f89e3
chore: Update datafusion (to get fix for pruning bug) ( #5339 )
...
* chore: Update datafusion
* chore: Update AggregateSelector API
2022-08-08 12:28:21 +00:00
3a697df261
chore(deps): Bump sqlparser from 0.19.0 to 0.20.0 ( #5335 )
...
Bumps [sqlparser](https://github.com/sqlparser-rs/sqlparser-rs ) from 0.19.0 to 0.20.0.
- [Release notes](https://github.com/sqlparser-rs/sqlparser-rs/releases )
- [Changelog](https://github.com/sqlparser-rs/sqlparser-rs/blob/main/CHANGELOG.md )
- [Commits](https://github.com/sqlparser-rs/sqlparser-rs/compare/v0.19.0...v0.20.0 )
---
updated-dependencies:
- dependency-name: sqlparser
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-08-08 08:23:31 +00:00
e82214ed38
chore: fix `cargo audit`, update deps to get new chrono ( #5316 )
...
* chore: update deps to get new chrono
* chore: Run cargo hakari tasks
* chore: migrate away from deprecated API
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
2022-08-04 20:49:28 +00:00
0ba3ae1e0d
Merge branch 'main' into dom/instrument-kafka-produce
2022-08-04 15:13:49 +00:00
e8231b2986
chore(deps): Bump serde_json from 1.0.82 to 1.0.83 ( #5297 )
...
* chore(deps): Bump serde_json from 1.0.82 to 1.0.83
Bumps [serde_json](https://github.com/serde-rs/json ) from 1.0.82 to 1.0.83.
- [Release notes](https://github.com/serde-rs/json/releases )
- [Commits](https://github.com/serde-rs/json/compare/v1.0.82...v1.0.83 )
---
updated-dependencies:
- dependency-name: serde_json
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* chore: Run cargo hakari tasks
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-08-04 14:28:29 +00:00
1cad7e13ec
build: bump rskafka to latest
...
Includes minor code changes needed to support the rskafka HEAD commit.
Breaking changes made in
https://github.com/influxdata/rskafka/issues/160
2022-08-04 15:02:11 +02:00
251af3f844
chore(deps): Bump serde from 1.0.140 to 1.0.142 ( #5296 )
...
* chore(deps): Bump serde from 1.0.140 to 1.0.142
Bumps [serde](https://github.com/serde-rs/serde ) from 1.0.140 to 1.0.142.
- [Release notes](https://github.com/serde-rs/serde/releases )
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.140...v1.0.142 )
---
updated-dependencies:
- dependency-name: serde
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* chore: Run cargo hakari tasks
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-08-04 08:49:12 +00:00
07fe5c8af3
chore(deps): Bump libc from 0.2.126 to 0.2.127 ( #5306 )
...
Bumps [libc](https://github.com/rust-lang/libc ) from 0.2.126 to 0.2.127.
- [Release notes](https://github.com/rust-lang/libc/releases )
- [Commits](https://github.com/rust-lang/libc/compare/0.2.126...0.2.127 )
---
updated-dependencies:
- dependency-name: libc
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-08-04 08:40:23 +00:00
55e1e2ec2b
chore(deps): Bump thiserror from 1.0.31 to 1.0.32 ( #5294 )
...
* chore(deps): Bump thiserror from 1.0.31 to 1.0.32
Bumps [thiserror](https://github.com/dtolnay/thiserror ) from 1.0.31 to 1.0.32.
- [Release notes](https://github.com/dtolnay/thiserror/releases )
- [Commits](https://github.com/dtolnay/thiserror/compare/1.0.31...1.0.32 )
---
updated-dependencies:
- dependency-name: thiserror
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* chore: Run cargo hakari tasks
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-08-03 16:20:36 +00:00
6011c4cd1f
chore: update datafusion pin ( #5290 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-08-03 15:43:18 +00:00
5fd59fcc43
chore(deps): Bump syn from 1.0.98 to 1.0.99 ( #5284 )
...
Bumps [syn](https://github.com/dtolnay/syn ) from 1.0.98 to 1.0.99.
- [Release notes](https://github.com/dtolnay/syn/releases )
- [Commits](https://github.com/dtolnay/syn/compare/1.0.98...1.0.99 )
---
updated-dependencies:
- dependency-name: syn
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-08-03 12:50:56 +00:00
273b3cc165
chore: replace `dotenv` with `dotenvy` ( #5285 )
...
The latter one is a maintained fork. This avoids having both crates
after #5282 .
2022-08-03 12:41:38 +00:00
7c67b93015
chore(deps): Bump sqlx from 0.6.0 to 0.6.1 ( #5282 )
...
Bumps [sqlx](https://github.com/launchbadge/sqlx ) from 0.6.0 to 0.6.1.
- [Release notes](https://github.com/launchbadge/sqlx/releases )
- [Changelog](https://github.com/launchbadge/sqlx/blob/main/CHANGELOG.md )
- [Commits](https://github.com/launchbadge/sqlx/commits )
---
updated-dependencies:
- dependency-name: sqlx
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-08-03 12:19:16 +00:00
94fe5b4c10
chore(deps): Bump paste from 1.0.7 to 1.0.8 ( #5280 )
...
Bumps [paste](https://github.com/dtolnay/paste ) from 1.0.7 to 1.0.8.
- [Release notes](https://github.com/dtolnay/paste/releases )
- [Commits](https://github.com/dtolnay/paste/compare/1.0.7...1.0.8 )
---
updated-dependencies:
- dependency-name: paste
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-08-03 09:03:25 +00:00
a4e2f880be
feat: Expose a C API for the IOx LP parser ( #5267 )
...
Can be useful to call the IOx LP parser from other processes, for example from Go.
I used it to run an online comparison of IOx and influxdb Go LP parser in order to identify compatibility
issues.
2022-08-02 15:44:41 +00:00
e57ae07db7
chore(deps): Bump serde from 1.0.140 to 1.0.141 ( #5260 )
...
Bumps [serde](https://github.com/serde-rs/serde ) from 1.0.140 to 1.0.141.
- [Release notes](https://github.com/serde-rs/serde/releases )
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.140...v1.0.141 )
---
updated-dependencies:
- dependency-name: serde
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-08-02 07:52:06 +00:00
de340cfa53
chore(deps): Bump sqlparser from 0.18.0 to 0.19.0 ( #5237 )
...
Bumps [sqlparser](https://github.com/sqlparser-rs/sqlparser-rs ) from 0.18.0 to 0.19.0.
- [Release notes](https://github.com/sqlparser-rs/sqlparser-rs/releases )
- [Changelog](https://github.com/sqlparser-rs/sqlparser-rs/blob/main/CHANGELOG.md )
- [Commits](https://github.com/sqlparser-rs/sqlparser-rs/compare/v0.18.0...v0.19.0 )
---
updated-dependencies:
- dependency-name: sqlparser
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-08-01 08:58:32 +00:00
fbd39844d8
chore(deps): Bump async-trait from 0.1.56 to 0.1.57 ( #5247 )
...
Bumps [async-trait](https://github.com/dtolnay/async-trait ) from 0.1.56 to 0.1.57.
- [Release notes](https://github.com/dtolnay/async-trait/releases )
- [Commits](https://github.com/dtolnay/async-trait/compare/0.1.56...0.1.57 )
---
updated-dependencies:
- dependency-name: async-trait
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-08-01 08:30:33 +00:00
d0db73d8de
chore(deps): Bump clap from 3.2.15 to 3.2.16 ( #5239 )
...
Bumps [clap](https://github.com/clap-rs/clap ) from 3.2.15 to 3.2.16.
- [Release notes](https://github.com/clap-rs/clap/releases )
- [Changelog](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md )
- [Commits](https://github.com/clap-rs/clap/compare/v3.2.15...v3.2.16 )
---
updated-dependencies:
- dependency-name: clap
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-08-01 08:00:47 +00:00
a687844ddf
chore(deps): Bump bytes from 1.2.0 to 1.2.1 ( #5244 )
...
Bumps [bytes](https://github.com/tokio-rs/bytes ) from 1.2.0 to 1.2.1.
- [Release notes](https://github.com/tokio-rs/bytes/releases )
- [Changelog](https://github.com/tokio-rs/bytes/blob/master/CHANGELOG.md )
- [Commits](https://github.com/tokio-rs/bytes/commits )
---
updated-dependencies:
- dependency-name: bytes
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-08-01 07:50:06 +00:00
87bdabb38a
feat: log external span for query gRPC requests ( #5187 )
...
* feat: log external span for query gRPC requests
This should simplify the correlation with our binlog data.
* refactor: address review comments
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-07-28 12:53:12 +00:00
9215a534d0
chore: Update datafusion and `arrow`/`parquet`/`arrow-flight` to `19.0.0` ( #5229 )
...
* chore: Update datafusion and `arrow`/`parquet`/`arrow-flight` to `19.0.0`
* chore: Run cargo hakari tasks
* fix: Update for API changes
* fix: clippy
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-07-28 08:10:47 +00:00
727f0152d0
chore(deps): Bump tokio from 1.20.0 to 1.20.1 ( #5209 )
...
Bumps [tokio](https://github.com/tokio-rs/tokio ) from 1.20.0 to 1.20.1.
- [Release notes](https://github.com/tokio-rs/tokio/releases )
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.20.0...tokio-1.20.1 )
---
updated-dependencies:
- dependency-name: tokio
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-07-26 15:36:25 +00:00
246b294b05
chore(deps): Bump clap from 3.2.14 to 3.2.15 ( #5208 )
...
Bumps [clap](https://github.com/clap-rs/clap ) from 3.2.14 to 3.2.15.
- [Release notes](https://github.com/clap-rs/clap/releases )
- [Changelog](https://github.com/clap-rs/clap/blob/v3.2.15/CHANGELOG.md )
- [Commits](https://github.com/clap-rs/clap/compare/v3.2.14...v3.2.15 )
---
updated-dependencies:
- dependency-name: clap
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-07-26 08:44:02 +00:00
e4dc8c2067
refactor: rename garbage collector crates for consistency ( #5196 )
...
* refactor: rename garbage collector crates for consistency
* fix: cargo fmt
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-07-25 12:44:37 +00:00
465a69c41d
chore: Update datafusion ( #5193 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-07-22 18:07:38 +00:00
c5451afa08
feat: Implement LP builder ( #5183 )
...
Helps with https://github.com/influxdata/influxrpc_compare/issues/179
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-07-22 14:41:10 +00:00
e6d68f90b4
chore(deps): Bump permutation from 0.4.0 to 0.4.1 ( #5182 )
...
Bumps [permutation](https://github.com/jeremysalwen/rust-permutations ) from 0.4.0 to 0.4.1.
- [Release notes](https://github.com/jeremysalwen/rust-permutations/releases )
- [Commits](https://github.com/jeremysalwen/rust-permutations/commits )
---
updated-dependencies:
- dependency-name: permutation
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-07-22 08:38:23 +00:00
b35502ce61
feat: cache tracing ( #5164 )
...
* feat: cache tracing
Add tracing to the metrics cache wrapper. The extra arguments for GET
and PEEK make this quite simple, because the wrapper can just extend the
inner args with the trace information.
We currently terminate the span in `querier::cache` (i.e. only pass in
`None`, so no tracing will occur) to keep this PR rather small. This
will be changed in subsequent PRs.
For #5129 .
* fix: typo
Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-07-21 11:54:22 +00:00
bf82ed6899
chore(deps): Bump serde from 1.0.139 to 1.0.140 ( #5169 )
...
Bumps [serde](https://github.com/serde-rs/serde ) from 1.0.139 to 1.0.140.
- [Release notes](https://github.com/serde-rs/serde/releases )
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.139...v1.0.140 )
---
updated-dependencies:
- dependency-name: serde
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-07-21 06:33:10 +00:00
826c1995ae
chore(deps): Bump handlebars from 4.3.2 to 4.3.3 ( #5168 )
...
Bumps [handlebars](https://github.com/sunng87/handlebars-rust ) from 4.3.2 to 4.3.3.
- [Release notes](https://github.com/sunng87/handlebars-rust/releases )
- [Changelog](https://github.com/sunng87/handlebars-rust/blob/master/CHANGELOG.md )
- [Commits](https://github.com/sunng87/handlebars-rust/compare/v4.3.2...v4.3.3 )
---
updated-dependencies:
- dependency-name: handlebars
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-07-21 06:23:27 +00:00
d7838e357f
chore(deps): Bump clap from 3.2.13 to 3.2.14 ( #5167 )
...
Bumps [clap](https://github.com/clap-rs/clap ) from 3.2.13 to 3.2.14.
- [Release notes](https://github.com/clap-rs/clap/releases )
- [Changelog](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md )
- [Commits](https://github.com/clap-rs/clap/compare/v3.2.13...v3.2.14 )
---
updated-dependencies:
- dependency-name: clap
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-07-21 06:01:10 +00:00
9fed013848
chore: Update datafusion pin ( #5162 )
...
* chore: Update datafusion pin
* fix: Update expected output
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-07-20 14:34:08 +00:00
95c734dc4c
feat: Improve logging for garbage collector service ( #5146 )
...
* feat: clean up logging for garbage collector service
* fix: restore unintended diff
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-07-20 13:29:44 +00:00
b9c60d4db0
chore(deps): Bump clap from 3.2.12 to 3.2.13 ( #5157 )
...
Bumps [clap](https://github.com/clap-rs/clap ) from 3.2.12 to 3.2.13.
- [Release notes](https://github.com/clap-rs/clap/releases )
- [Changelog](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md )
- [Commits](https://github.com/clap-rs/clap/compare/v3.2.12...v3.2.13 )
---
updated-dependencies:
- dependency-name: clap
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-07-20 11:32:58 +00:00
278a7f91af
chore(deps): Bump bytes from 1.1.0 to 1.2.0 ( #5156 )
...
Bumps [bytes](https://github.com/tokio-rs/bytes ) from 1.1.0 to 1.2.0.
- [Release notes](https://github.com/tokio-rs/bytes/releases )
- [Changelog](https://github.com/tokio-rs/bytes/blob/master/CHANGELOG.md )
- [Commits](https://github.com/tokio-rs/bytes/compare/v1.1.0...v1.2.0 )
---
updated-dependencies:
- dependency-name: bytes
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-07-20 10:00:08 +00:00
449b4cf1d6
chore: Update datafusion pin ( #5148 )
...
* chore: Update datafusion pin
* fix: Update for changes upstream
2022-07-19 13:13:37 +00:00
f7a0fd43d2
feat: make object store garbage collector into a long-running service ( #5135 )
...
* refactor: remove unused logging config
* chore: remove the object store garbage collector CLI tool
* refactor: accept an object store and catalog
* refactor: make Result type alias public like the error
* refactor: remove public modifier from modules
* refactor: allow shutting down the object store garbage collector
* feat: Introduce the object-store garbage collection server
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-07-18 21:27:38 +00:00
e2d871b00b
chore: Update datafusion and arrow/parquet/arrow-flight to `18.0.0` ( #5079 )
...
* chore: Update datafusion to 10.0.0, arrow/parquet/arrow-flight to 18
* chore: Run cargo hakari tasks
* fix: update cargo pin
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-07-18 15:01:03 +00:00
1eeee9809c
chore(deps): Bump rustyline from 9.1.2 to 10.0.0 ( #5139 )
...
Bumps [rustyline](https://github.com/kkawakam/rustyline ) from 9.1.2 to 10.0.0.
- [Release notes](https://github.com/kkawakam/rustyline/releases )
- [Changelog](https://github.com/kkawakam/rustyline/blob/master/History.md )
- [Commits](https://github.com/kkawakam/rustyline/compare/v9.1.2...v10.0.0 )
---
updated-dependencies:
- dependency-name: rustyline
dependency-type: direct:production
update-type: version-update:semver-major
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-07-18 08:10:22 +00:00
fd2d262862
chore(deps): Bump nix from 0.24.1 to 0.24.2 ( #5140 )
...
Bumps [nix](https://github.com/nix-rust/nix ) from 0.24.1 to 0.24.2.
- [Release notes](https://github.com/nix-rust/nix/releases )
- [Changelog](https://github.com/nix-rust/nix/blob/v0.24.2/CHANGELOG.md )
- [Commits](https://github.com/nix-rust/nix/compare/v0.24.1...v0.24.2 )
---
updated-dependencies:
- dependency-name: nix
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-07-18 06:15:49 +00:00
0b4afad08f
chore(deps): Bump hashbrown from 0.12.2 to 0.12.3 ( #5141 )
...
Bumps [hashbrown](https://github.com/rust-lang/hashbrown ) from 0.12.2 to 0.12.3.
- [Release notes](https://github.com/rust-lang/hashbrown/releases )
- [Changelog](https://github.com/rust-lang/hashbrown/blob/master/CHANGELOG.md )
- [Commits](https://github.com/rust-lang/hashbrown/compare/v0.12.2...v0.12.3 )
---
updated-dependencies:
- dependency-name: hashbrown
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-07-18 06:06:24 +00:00
f870c8a887
chore(deps): Bump crypto-common from 0.1.5 to 0.1.6 ( #5138 )
...
Bumps [crypto-common](https://github.com/RustCrypto/traits ) from 0.1.5 to 0.1.6.
- [Release notes](https://github.com/RustCrypto/traits/releases )
- [Commits](https://github.com/RustCrypto/traits/compare/crypto-common-v0.1.5...crypto-common-v0.1.6 )
---
updated-dependencies:
- dependency-name: crypto-common
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-07-18 05:36:20 +00:00
c0ac01c67b
chore(deps): Bump handlebars from 4.3.1 to 4.3.2 ( #5133 )
...
Bumps [handlebars](https://github.com/sunng87/handlebars-rust ) from 4.3.1 to 4.3.2.
- [Release notes](https://github.com/sunng87/handlebars-rust/releases )
- [Changelog](https://github.com/sunng87/handlebars-rust/blob/master/CHANGELOG.md )
- [Commits](https://github.com/sunng87/handlebars-rust/compare/v4.3.1...v4.3.2 )
---
updated-dependencies:
- dependency-name: handlebars
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-07-15 21:18:50 +00:00
9359539acf
chore(deps): Bump clap from 3.2.11 to 3.2.12 ( #5132 )
...
Bumps [clap](https://github.com/clap-rs/clap ) from 3.2.11 to 3.2.12.
- [Release notes](https://github.com/clap-rs/clap/releases )
- [Changelog](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md )
- [Commits](https://github.com/clap-rs/clap/compare/v3.2.11...v3.2.12 )
---
updated-dependencies:
- dependency-name: clap
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-07-15 21:10:51 +00:00
f0bd278652
feat: add tracing to instrumented semaphores ( #5130 )
...
This will allow us to easily see how much time we spend during query
processing waiting for the query semaphore.
Ref #5129 .
2022-07-15 07:50:28 +00:00
14472c4e66
Merge branch 'main' into dependabot/cargo/clap-3.2.11
2022-07-14 14:26:56 +00:00
9b67de2f43
chore(deps): Bump tokio from 1.19.2 to 1.20.0
...
Bumps [tokio](https://github.com/tokio-rs/tokio ) from 1.19.2 to 1.20.0.
- [Release notes](https://github.com/tokio-rs/tokio/releases )
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.19.2...tokio-1.20.0 )
---
updated-dependencies:
- dependency-name: tokio
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2022-07-14 01:21:43 +00:00
8179f113b9
chore(deps): Bump clap from 3.2.10 to 3.2.11
...
Bumps [clap](https://github.com/clap-rs/clap ) from 3.2.10 to 3.2.11.
- [Release notes](https://github.com/clap-rs/clap/releases )
- [Changelog](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md )
- [Commits](https://github.com/clap-rs/clap/compare/v3.2.10...v3.2.11 )
---
updated-dependencies:
- dependency-name: clap
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2022-07-14 01:21:09 +00:00
b42dd1c45e
chore: update all dependencies (esp. `time`) ( #5114 )
...
Our CI found a potential panic:
https://app.circleci.com/pipelines/github/influxdata/influxdb_iox/21307/workflows/7c1df458-3c6a-4f5d-bfda-51612273bbc7/jobs/183140
See https://github.com/time-rs/time/issues/481 , so let's update
`time` and since we've accumulated quite a backlog, just run
`cargo update --workspace`.
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-07-13 14:30:18 +00:00
3a1934e7af
chore(deps): Bump clap from 3.2.8 to 3.2.10 ( #5111 )
...
Bumps [clap](https://github.com/clap-rs/clap ) from 3.2.8 to 3.2.10.
- [Release notes](https://github.com/clap-rs/clap/releases )
- [Changelog](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md )
- [Commits](https://github.com/clap-rs/clap/compare/v3.2.8...v3.2.10 )
---
updated-dependencies:
- dependency-name: clap
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-07-13 08:24:51 +00:00
60df069a10
fix: Bump object_store crate for EKS support
2022-07-13 01:12:29 +02:00
23ad60b35f
chore(deps): Bump serde from 1.0.138 to 1.0.139 ( #5095 )
...
Bumps [serde](https://github.com/serde-rs/serde ) from 1.0.138 to 1.0.139.
- [Release notes](https://github.com/serde-rs/serde/releases )
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.138...v1.0.139 )
---
updated-dependencies:
- dependency-name: serde
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-07-12 15:34:51 +00:00
607831585c
refactor: use less executors and threads during tests ( #5086 )
...
`Executor` is only used as a performance boundary, not as a correctness
or data boundary so let's try to re-use it. This also simplifies
profiling of tests since we don't end up with hundreds (or even
thousands) of threads.
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-07-11 16:23:22 +00:00
5703d12595
chore(deps): Bump hashbrown from 0.12.1 to 0.12.2 ( #5084 )
...
Bumps [hashbrown](https://github.com/rust-lang/hashbrown ) from 0.12.1 to 0.12.2.
- [Release notes](https://github.com/rust-lang/hashbrown/releases )
- [Changelog](https://github.com/rust-lang/hashbrown/blob/master/CHANGELOG.md )
- [Commits](https://github.com/rust-lang/hashbrown/compare/v0.12.1...v0.12.2 )
---
updated-dependencies:
- dependency-name: hashbrown
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-07-11 12:56:31 +00:00
fd3ee7e942
chore(deps): Bump crypto-common from 0.1.4 to 0.1.5 ( #5083 )
...
Bumps [crypto-common](https://github.com/RustCrypto/traits ) from 0.1.4 to 0.1.5.
- [Release notes](https://github.com/RustCrypto/traits/releases )
- [Commits](https://github.com/RustCrypto/traits/compare/crypto-common-v0.1.4...crypto-common-v0.1.5 )
---
updated-dependencies:
- dependency-name: crypto-common
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-07-11 09:05:26 +00:00
93d1a0f5be
chore(deps): Bump hyper from 0.14.19 to 0.14.20 ( #5073 )
...
Bumps [hyper](https://github.com/hyperium/hyper ) from 0.14.19 to 0.14.20.
- [Release notes](https://github.com/hyperium/hyper/releases )
- [Changelog](https://github.com/hyperium/hyper/blob/v0.14.20/CHANGELOG.md )
- [Commits](https://github.com/hyperium/hyper/compare/v0.14.19...v0.14.20 )
---
updated-dependencies:
- dependency-name: hyper
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-07-08 13:24:34 +00:00
9a3d31bc63
chore(deps): Bump backtrace from 0.3.65 to 0.3.66 ( #5072 )
...
Bumps [backtrace](https://github.com/rust-lang/backtrace-rs ) from 0.3.65 to 0.3.66.
- [Release notes](https://github.com/rust-lang/backtrace-rs/releases )
- [Commits](https://github.com/rust-lang/backtrace-rs/compare/0.3.65...0.3.66 )
---
updated-dependencies:
- dependency-name: backtrace
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-07-08 08:00:49 +00:00
6c1b70ba2d
chore(deps): Bump filetime from 0.2.16 to 0.2.17 ( #5071 )
...
Bumps [filetime](https://github.com/alexcrichton/filetime ) from 0.2.16 to 0.2.17.
- [Release notes](https://github.com/alexcrichton/filetime/releases )
- [Commits](https://github.com/alexcrichton/filetime/commits/0.2.17 )
---
updated-dependencies:
- dependency-name: filetime
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-07-08 07:50:40 +00:00
c46e1c6347
chore: Update datafusion + arrow/parquet/arrow-flight to `17.0.0` ( #5021 )
...
* fix: correct nullability declaration of system tables
* chore: Update datafusion and arrow/parquet/arrow-flight
* chore: Run cargo hakari tasks
* fix: Update tests
* fix: Update tests
* fix: predicate pruning
* fix: add some tests
* fix: query_functions
* fix: fix read_buffer test
* fix: fix clippy
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-07-07 19:22:15 +00:00
70dd6009e8
fix: Integrate the object store garbage collector into the main binary
2022-07-07 09:48:06 -04:00
4369431e03
chore: Run cargo hakari tasks
2022-07-07 09:48:05 -04:00
6fc17164bd
test: Add basic end-to-end tests of the garbage collector
2022-07-07 09:48:05 -04:00
72d1ed9b73
feat: Support configurable cutoff time for too new object store files
2022-07-07 09:48:05 -04:00
3fcaf34d54
feat: Add trogging logging config to object store garbage collector
2022-07-07 09:48:04 -04:00
90d7b22d86
fix: Use dotenv and the right features in the garbage collector
2022-07-07 09:48:04 -04:00
c541ca68a2
fix: Test logic for whether to delete object store files
...
And fix the bugs the tests found
2022-07-07 09:48:04 -04:00
98e7133ebf
refactor: Reuse clap blocks rather than creating new ones
2022-07-07 09:48:04 -04:00
428f41f747
feat: Walking skeleton of the object store GC tool
2022-07-07 09:48:04 -04:00
c443d07a5c
chore(deps): Bump criterion from 0.3.5 to 0.3.6 ( #5059 )
...
Bumps [criterion](https://github.com/bheisler/criterion.rs ) from 0.3.5 to 0.3.6.
- [Release notes](https://github.com/bheisler/criterion.rs/releases )
- [Changelog](https://github.com/bheisler/criterion.rs/blob/master/CHANGELOG.md )
- [Commits](https://github.com/bheisler/criterion.rs/compare/0.3.5...0.3.6 )
---
updated-dependencies:
- dependency-name: criterion
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-07-07 07:09:27 +00:00
2b527bbf64
chore(deps): Bump regex from 1.5.6 to 1.6.0 ( #5048 )
...
Bumps [regex](https://github.com/rust-lang/regex ) from 1.5.6 to 1.6.0.
- [Release notes](https://github.com/rust-lang/regex/releases )
- [Changelog](https://github.com/rust-lang/regex/blob/master/CHANGELOG.md )
- [Commits](https://github.com/rust-lang/regex/compare/1.5.6...1.6.0 )
---
updated-dependencies:
- dependency-name: regex
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-07-06 10:25:28 +00:00
b3522086a8
chore(deps): Bump regex-syntax from 0.6.26 to 0.6.27 ( #5047 )
...
Bumps [regex-syntax](https://github.com/rust-lang/regex ) from 0.6.26 to 0.6.27.
- [Release notes](https://github.com/rust-lang/regex/releases )
- [Changelog](https://github.com/rust-lang/regex/blob/master/CHANGELOG.md )
- [Commits](https://github.com/rust-lang/regex/commits )
---
updated-dependencies:
- dependency-name: regex-syntax
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-07-06 10:19:07 +00:00
68eff79594
chore(deps): Bump once_cell from 1.12.0 to 1.13.0 ( #5033 )
...
Bumps [once_cell](https://github.com/matklad/once_cell ) from 1.12.0 to 1.13.0.
- [Release notes](https://github.com/matklad/once_cell/releases )
- [Changelog](https://github.com/matklad/once_cell/blob/master/CHANGELOG.md )
- [Commits](https://github.com/matklad/once_cell/compare/v1.12.0...v1.13.0 )
---
updated-dependencies:
- dependency-name: once_cell
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-07-05 08:54:51 +00:00
1f5863ebcb
chore(deps): Bump serde from 1.0.137 to 1.0.138 ( #5029 )
...
Bumps [serde](https://github.com/serde-rs/serde ) from 1.0.137 to 1.0.138.
- [Release notes](https://github.com/serde-rs/serde/releases )
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.137...v1.0.138 )
---
updated-dependencies:
- dependency-name: serde
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-07-04 10:07:46 +00:00
b8f2f3b9e8
chore(deps): Bump pin-project from 1.0.10 to 1.0.11 ( #5028 )
...
Bumps [pin-project](https://github.com/taiki-e/pin-project ) from 1.0.10 to 1.0.11.
- [Release notes](https://github.com/taiki-e/pin-project/releases )
- [Changelog](https://github.com/taiki-e/pin-project/blob/main/CHANGELOG.md )
- [Commits](https://github.com/taiki-e/pin-project/compare/v1.0.10...v1.0.11 )
---
updated-dependencies:
- dependency-name: pin-project
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-07-04 09:45:31 +00:00
98ac454e88
chore(deps): Bump crypto-common from 0.1.3 to 0.1.4 ( #5027 )
...
Bumps [crypto-common](https://github.com/RustCrypto/traits ) from 0.1.3 to 0.1.4.
- [Release notes](https://github.com/RustCrypto/traits/releases )
- [Commits](https://github.com/RustCrypto/traits/compare/crypto-common-v0.1.3...crypto-common-v0.1.4 )
---
updated-dependencies:
- dependency-name: crypto-common
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-07-04 09:07:37 +00:00
9982d96578
chore(deps): Bump fixedbitset from 0.4.1 to 0.4.2 ( #5026 )
...
Bumps [fixedbitset](https://github.com/petgraph/fixedbitset ) from 0.4.1 to 0.4.2.
- [Release notes](https://github.com/petgraph/fixedbitset/releases )
- [Commits](https://github.com/petgraph/fixedbitset/commits )
---
updated-dependencies:
- dependency-name: fixedbitset
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-07-04 07:47:53 +00:00
c4c251129e
chore: Update datafusion ( #5020 )
...
* chore: Update datafusion
* fix: Update plan
* fix: update explain plans
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-07-01 19:59:41 +00:00
09aaf18fb4
chore(deps): Bump clap from 3.2.6 to 3.2.8 ( #5015 )
...
Bumps [clap](https://github.com/clap-rs/clap ) from 3.2.6 to 3.2.8.
- [Release notes](https://github.com/clap-rs/clap/releases )
- [Changelog](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md )
- [Commits](https://github.com/clap-rs/clap/compare/v3.2.6...v3.2.8 )
---
updated-dependencies:
- dependency-name: clap
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-07-01 08:17:04 +00:00
3049479b78
feat: Implement new querier to ingester config design
2022-06-30 08:26:50 -04:00
4e91121e29
feat: Allow specification of sequencer to ingester mappings in a JSON file
2022-06-30 08:22:46 -04:00
f37f8013ec
feat: Assign a sequencer id to QuerierTables to know which ingester to query
2022-06-30 08:22:46 -04:00
40a8525520
chore(deps): Bump serde_json from 1.0.81 to 1.0.82 ( #4992 )
...
Bumps [serde_json](https://github.com/serde-rs/json ) from 1.0.81 to 1.0.82.
- [Release notes](https://github.com/serde-rs/json/releases )
- [Commits](https://github.com/serde-rs/json/compare/v1.0.81...v1.0.82 )
---
updated-dependencies:
- dependency-name: serde_json
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-06-30 09:54:08 +00:00
d1e3b009c9
chore(deps): Bump smallvec from 1.8.1 to 1.9.0 ( #4991 )
...
Bumps [smallvec](https://github.com/servo/rust-smallvec ) from 1.8.1 to 1.9.0.
- [Release notes](https://github.com/servo/rust-smallvec/releases )
- [Commits](https://github.com/servo/rust-smallvec/compare/v1.8.1...v1.9.0 )
---
updated-dependencies:
- dependency-name: smallvec
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-06-30 09:25:20 +00:00
65a6ba7d45
chore(deps): Bump either from 1.6.1 to 1.7.0 ( #4990 )
...
Bumps [either](https://github.com/bluss/either ) from 1.6.1 to 1.7.0.
- [Release notes](https://github.com/bluss/either/releases )
- [Commits](https://github.com/bluss/either/compare/1.6.1...1.7.0 )
---
updated-dependencies:
- dependency-name: either
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-06-30 08:21:20 +00:00
4cf7bb8dee
chore(deps): Bump clap from 3.2.6 to 3.2.7 ( #4989 )
...
Bumps [clap](https://github.com/clap-rs/clap ) from 3.2.6 to 3.2.7.
- [Release notes](https://github.com/clap-rs/clap/releases )
- [Changelog](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md )
- [Commits](https://github.com/clap-rs/clap/compare/v3.2.6...v3.2.7 )
---
updated-dependencies:
- dependency-name: clap
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-06-30 08:04:57 +00:00
05189fdb00
fix: ensure panic is propagated throw `AdapterStream` ( #4980 )
...
Prior to this change background tasks that we feed into `AdapterStream`
can panic but that would just end the stream without any user-visible
error (except for the panic message on stdout/stderr).
This was found while developing #4964 . I have proposed another fix in #4966
but found that I actually developed an existing solution a 2nd time:
`watch_task`. But I also see a major issue with the existing API: one
can create `AdapterStream` with ordinary tokio tasks that are not
watched at all, leaving the burden to the implementor to check for that
(and actually we forgot that in `parquet_file`).
So this change takes a slightly different approach:
The `AdapterStream` does NOT accept ordinary join handles any longer but
requires that you pass a "watched task". The newly introduced
`WatchedTask` does the same as we did manually before: wrapping a future
into a tokio task, watch it and wrap the watcher into a task.
It is now way more difficult to do anything stupid (sure you can still
mix up the tasks and the channels, but we need at least some flexibility
here to allow for "split" and potential future fan-in/out constructs).
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-06-30 07:57:58 +00:00
835e1c91c7
chore: update object_store to 0.3.0 ( #4707 )
...
* chore: update object_store to 0.3.0
* chore: review feedback
Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-06-29 21:44:03 +00:00
01fb2e132d
chore: Update datafusion pin ( #4969 )
...
* chore: Update datafusion pin
* fix: Update for api
* fix: Explicitly set coalsce batch size
* fix: Update batch size as well
* fix: update tests for new explain plan, and improved coercion
2022-06-29 17:52:37 +00:00
edf3f08e81
refactor: Replace all uses of lazy_static with once_cell
...
Went through and remove all lazy_static uses with once_cell (while waiting for the project to compile). There are still dependencies using lazy_static so it is still in the crate graph but at least there isn't an explicit dependency on it (and it is easier to update to `std::lazy::Lazy` once that is stable).
2022-06-29 16:22:02 +02:00
7546476e15
chore(deps): Bump smallvec from 1.8.0 to 1.8.1 ( #4947 )
...
Bumps [smallvec](https://github.com/servo/rust-smallvec ) from 1.8.0 to 1.8.1.
- [Release notes](https://github.com/servo/rust-smallvec/releases )
- [Commits](https://github.com/servo/rust-smallvec/compare/v1.8.0...v1.8.1 )
---
updated-dependencies:
- dependency-name: smallvec
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-06-27 07:17:32 +00:00
bd6c4659af
refactor: slim down parquet chunk (remove Metadata) ( #4934 )
...
* feat: conversion from `ParquetFile` to `ParquetFilePath`
* refactor: slim down parquet chunk
- ensure it works without binary parquet metadata
- timestamp range is no longer optional (ensured by the NG type system)
- remove table summary: this is only needed for SOME API users. The
compactor can perfectly work without statistics since has the timestamp
range which is sufficient for the current overlap check (we don't use
any other primary key stats at the moment). The querier currently does
NOT use parquet chunks (was replaced by read buffer) but if it will
again in some future it will likely need to find a way to fetch and
cache the statistics.
- the schema is now provided by the API user since it can be
reconstructed using the NG catalog only (and "wrong" column orders are
tolerated as of #4921 )
Ref #4124
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-06-23 10:55:16 +00:00
6f9d8b54cf
chore(deps): Bump integer-encoding from 3.0.3 to 3.0.4 ( #4932 )
...
Bumps [integer-encoding](https://github.com/dermesser/integer-encoding-rs ) from 3.0.3 to 3.0.4.
- [Release notes](https://github.com/dermesser/integer-encoding-rs/releases )
- [Commits](https://github.com/dermesser/integer-encoding-rs/compare/v3.0.3...v3.0.4 )
---
updated-dependencies:
- dependency-name: integer-encoding
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-06-23 09:42:34 +00:00
776c34e03d
chore: Update datafusion ( #4927 )
...
* chore: Update datafusion
* fix: Update for API changes
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-06-23 09:30:43 +00:00
9591bed696
refactor: make querier internals private ( #4922 )
...
Queries internals are not meant to be used by other crates. Only a
handful selected interfaces should be used by IOxD and the query tests.
The compactor only used a very small subset just to read parquet files
back into memory. It shall rather use the official `parquet_file`
interface instead.
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-06-22 13:00:08 +00:00
f7d83ea581
chore(deps): Bump clap from 3.2.5 to 3.2.6 ( #4920 )
...
Bumps [clap](https://github.com/clap-rs/clap ) from 3.2.5 to 3.2.6.
- [Release notes](https://github.com/clap-rs/clap/releases )
- [Changelog](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md )
- [Commits](https://github.com/clap-rs/clap/compare/v3.2.5...v3.2.6 )
---
updated-dependencies:
- dependency-name: clap
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-06-22 10:28:44 +00:00
50ffca791a
chore(deps): Bump indexmap from 1.9.0 to 1.9.1 ( #4919 )
...
Bumps [indexmap](https://github.com/bluss/indexmap ) from 1.9.0 to 1.9.1.
- [Release notes](https://github.com/bluss/indexmap/releases )
- [Changelog](https://github.com/bluss/indexmap/blob/master/RELEASES.md )
- [Commits](https://github.com/bluss/indexmap/compare/1.9.0...1.9.1 )
---
updated-dependencies:
- dependency-name: indexmap
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-06-22 10:18:06 +00:00
087dbd3eca
fix: fix heappy + update docs ( #4917 )
...
* docs: Update heap profiling documentation
* fix: fix heappy builds
* fix: do not run cli tests with heappy
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-06-21 19:53:28 +00:00
1962fcc229
chore: reduce dependencies and run `cargo update` ( #4906 )
...
* chore: reduce proptest features
* chore: remove `grpc-router`
This crate is currently unused and we don't have immediate plans to use
it. And there's GIT, so it can always be restored.
* chore: `cargo update`
2022-06-20 12:18:28 +00:00
cc324613cc
chore(deps): Bump syn from 1.0.96 to 1.0.98 ( #4905 )
...
Bumps [syn](https://github.com/dtolnay/syn ) from 1.0.96 to 1.0.98.
- [Release notes](https://github.com/dtolnay/syn/releases )
- [Commits](https://github.com/dtolnay/syn/compare/1.0.96...1.0.98 )
---
updated-dependencies:
- dependency-name: syn
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-06-20 08:49:02 +00:00
2ddc78bc41
chore(deps): Bump uuid from 1.0.0 to 1.1.2 ( #4904 )
...
Bumps [uuid](https://github.com/uuid-rs/uuid ) from 1.0.0 to 1.1.2.
- [Release notes](https://github.com/uuid-rs/uuid/releases )
- [Commits](https://github.com/uuid-rs/uuid/compare/1.0.0...1.1.2 )
---
updated-dependencies:
- dependency-name: uuid
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-06-20 08:34:33 +00:00
8b28721627
chore(deps): Bump tower from 0.4.12 to 0.4.13 ( #4903 )
...
Bumps [tower](https://github.com/tower-rs/tower ) from 0.4.12 to 0.4.13.
- [Release notes](https://github.com/tower-rs/tower/releases )
- [Commits](https://github.com/tower-rs/tower/compare/tower-0.4.12...tower-0.4.13 )
---
updated-dependencies:
- dependency-name: tower
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-06-20 08:25:29 +00:00
0fbff981ec
chore(deps): Bump sqlx to 0.6.0 and uuid to 1 ( #4894 )
...
Closes #4889 .
Closes #4890 .
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-06-17 10:28:28 +00:00
d9ab157797
chore(deps): Bump indexmap from 1.8.2 to 1.9.0 ( #4891 )
...
* chore(deps): Bump indexmap from 1.8.2 to 1.9.0
Bumps [indexmap](https://github.com/bluss/indexmap ) from 1.8.2 to 1.9.0.
- [Release notes](https://github.com/bluss/indexmap/releases )
- [Changelog](https://github.com/bluss/indexmap/blob/master/RELEASES.md )
- [Commits](https://github.com/bluss/indexmap/compare/1.8.2...1.9.0 )
---
updated-dependencies:
- dependency-name: indexmap
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* chore: Run cargo hakari tasks
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
2022-06-17 07:42:36 +00:00
73e1b3a47e
chore(deps): Bump clap from 3.2.4 to 3.2.5 ( #4872 )
...
Bumps [clap](https://github.com/clap-rs/clap ) from 3.2.4 to 3.2.5.
- [Release notes](https://github.com/clap-rs/clap/releases )
- [Changelog](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md )
- [Commits](https://github.com/clap-rs/clap/compare/v3.2.4...v3.2.5 )
---
updated-dependencies:
- dependency-name: clap
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-06-16 09:27:04 +00:00
66c7d95312
refactor: use new ingester<>querier wire protocol ( #4867 )
...
* refactor: use new ingester<>querier wire protocol
Use and document the new and more flexible ingester<>querier wire
protocol.
Note that the ingester does NOT stream the response data yet, but the
internal data structures would allow that. A follow-up change will
adjust the ingester code to stream the data.
Ref #4849 .
* fix: typos
Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
* refactor: clarify naming and public interface
* test: add schema assertion to `ingester_response_to_record_batches`
Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
2022-06-16 08:02:28 +00:00
03f6f59a9b
fix: Change the sharder to return error instead of panicking for no shards
2022-06-15 11:23:31 -04:00
e9cdaffe74
fix: Create querier sharder from catalog sequencer info
...
Panic if there are no sharders in the catalog.
2022-06-15 10:18:54 -04:00
874ef89daa
feat: Make specifying the write buffer, and thus getting a sharder, optional in querier
2022-06-15 10:01:45 -04:00
127467b5c4
feat: Create a sharder in the querier
2022-06-15 10:01:45 -04:00
148bc57e7b
refactor: Make the querier server constructor more like other server constructors
2022-06-15 10:01:45 -04:00
6417e7dc2a
feat: Extract sharder to its own crate
2022-06-15 10:01:45 -04:00
232dc897df
chore(deps): Bump clap from 3.2.1 to 3.2.4 ( #4860 )
...
Bumps [clap](https://github.com/clap-rs/clap ) from 3.2.1 to 3.2.4.
- [Release notes](https://github.com/clap-rs/clap/releases )
- [Changelog](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md )
- [Commits](https://github.com/clap-rs/clap/compare/clap_complete-v3.2.1...v3.2.4 )
---
updated-dependencies:
- dependency-name: clap
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-06-15 07:28:53 +00:00
b682dbbc2e
chore: Add debug info of sort_key for ingester ( #4859 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-06-14 20:39:17 +00:00
e91d00b10c
chore: Update datafusion + `arrow`/`parquet`/`arrow-flight` to `16.0.0 ( #4851 )
...
* chore: TEMP Update DataFusion to pre-release
* chore: update arrow et al to 16.0.0
* chore: Run cargo hakari tasks
* fix: update reader read_dictionary API
* chore: Update to real Datafusion release
* fix: Update parquet API
* fix: update test
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
2022-06-14 16:31:40 +00:00
23c9e38ea7
chore(deps): Bump clap from 3.1.18 to 3.2.1 ( #4848 )
...
* chore(deps): Bump clap from 3.1.18 to 3.2.1
Bumps [clap](https://github.com/clap-rs/clap ) from 3.1.18 to 3.2.1.
- [Release notes](https://github.com/clap-rs/clap/releases )
- [Changelog](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md )
- [Commits](https://github.com/clap-rs/clap/compare/v3.1.18...clap_complete-v3.2.1 )
---
updated-dependencies:
- dependency-name: clap
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* chore: fix clap deprecations
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Marco Neumann <marco@crepererum.net>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-06-14 15:42:18 +00:00
4a94a21b4a
chore(deps): Bump getrandom from 0.2.6 to 0.2.7 ( #4847 )
...
Bumps [getrandom](https://github.com/rust-random/getrandom ) from 0.2.6 to 0.2.7.
- [Release notes](https://github.com/rust-random/getrandom/releases )
- [Changelog](https://github.com/rust-random/getrandom/blob/master/CHANGELOG.md )
- [Commits](https://github.com/rust-random/getrandom/compare/v0.2.6...v0.2.7 )
---
updated-dependencies:
- dependency-name: getrandom
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-06-14 07:46:23 +00:00
78ae4caf06
chore: Update DataFusion ( #4842 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-06-14 07:36:55 +00:00
7381d4f489
chore(deps): Bump reqwest from 0.11.10 to 0.11.11 ( #4843 )
...
Bumps [reqwest](https://github.com/seanmonstar/reqwest ) from 0.11.10 to 0.11.11.
- [Release notes](https://github.com/seanmonstar/reqwest/releases )
- [Changelog](https://github.com/seanmonstar/reqwest/blob/master/CHANGELOG.md )
- [Commits](https://github.com/seanmonstar/reqwest/compare/v0.11.10...v0.11.11 )
---
updated-dependencies:
- dependency-name: reqwest
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-06-14 07:29:33 +00:00
6b9a712ba8
fix: Use debug rather than dbg in test helpers ( #4841 )
2022-06-13 20:17:24 +00:00
2b84e5c087
feat: measure "probably reloaded" cache loads ( #4813 )
...
To roughly gauge how much data we re-load into cached (i.e. data that
was already loaded but was later evicted due to LRU pressure or TTL
eviction) this change introduces a new metric that estimates if a cache
entry that is requested from the loader was already seen before (using a
probabilistic filter).
2022-06-13 13:51:45 +00:00
66623fe0cd
feat: expose query semaphore metrics ( #4836 )
...
The groundwork for that was already done, just needed a bit of wiring.
This might help us to judge timeouts.
2022-06-13 09:36:50 +00:00
c4bd52ee6f
chore(deps): Bump handlebars from 4.3.0 to 4.3.1 ( #4824 )
...
Bumps [handlebars](https://github.com/sunng87/handlebars-rust ) from 4.3.0 to 4.3.1.
- [Release notes](https://github.com/sunng87/handlebars-rust/releases )
- [Changelog](https://github.com/sunng87/handlebars-rust/blob/master/CHANGELOG.md )
- [Commits](https://github.com/sunng87/handlebars-rust/compare/v4.3.0...v4.3.1 )
---
updated-dependencies:
- dependency-name: handlebars
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-06-10 11:04:19 +00:00
82f6696516
chore: remove some unused deps ( #4803 )
...
Found by `cargo udeps`.
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-06-08 11:15:18 +00:00
3ecb1ee056
chore(deps): Bump http from 0.2.7 to 0.2.8 ( #4796 )
...
Bumps [http](https://github.com/hyperium/http ) from 0.2.7 to 0.2.8.
- [Release notes](https://github.com/hyperium/http/releases )
- [Changelog](https://github.com/hyperium/http/blob/master/CHANGELOG.md )
- [Commits](https://github.com/hyperium/http/compare/v0.2.7...v0.2.8 )
---
updated-dependencies:
- dependency-name: http
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-06-07 13:35:01 +00:00
5d98988c9f
chore(deps): Bump tokio from 1.19.1 to 1.19.2 ( #4795 )
...
Bumps [tokio](https://github.com/tokio-rs/tokio ) from 1.19.1 to 1.19.2.
- [Release notes](https://github.com/tokio-rs/tokio/releases )
- [Commits](https://github.com/tokio-rs/tokio/commits )
---
updated-dependencies:
- dependency-name: tokio
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-06-07 12:37:55 +00:00
1fa4ef9bac
chore(deps): Bump tower-http from 0.3.3 to 0.3.4 ( #4793 )
...
Bumps [tower-http](https://github.com/tower-rs/tower-http ) from 0.3.3 to 0.3.4.
- [Release notes](https://github.com/tower-rs/tower-http/releases )
- [Commits](https://github.com/tower-rs/tower-http/compare/tower-http-0.3.3...tower-http-0.3.4 )
---
updated-dependencies:
- dependency-name: tower-http
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-06-07 11:03:00 +00:00
20e832a5d1
chore(deps): Bump sqlparser from 0.17.0 to 0.18.0 ( #4792 )
...
Bumps [sqlparser](https://github.com/sqlparser-rs/sqlparser-rs ) from 0.17.0 to 0.18.0.
- [Release notes](https://github.com/sqlparser-rs/sqlparser-rs/releases )
- [Changelog](https://github.com/sqlparser-rs/sqlparser-rs/blob/main/CHANGELOG.md )
- [Commits](https://github.com/sqlparser-rs/sqlparser-rs/compare/v0.17.0...v0.18.0 )
---
updated-dependencies:
- dependency-name: sqlparser
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-06-07 10:00:53 +00:00
8e96a2721d
chore: Update datafusion (again) ( #4788 )
...
* chore: Update datafusion
* chore: Update imports
* refactor: update API usage
* refactor: clean up some uses of binary_expr
* fix: remove unused export
* fix: update explain output
* chore: update more explain tests
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-06-07 08:17:56 +00:00
04c685b3b7
chore(deps): Bump tokio-util from 0.7.2 to 0.7.3 ( #4784 )
...
Bumps [tokio-util](https://github.com/tokio-rs/tokio ) from 0.7.2 to 0.7.3.
- [Release notes](https://github.com/tokio-rs/tokio/releases )
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-util-0.7.2...tokio-util-0.7.3 )
---
updated-dependencies:
- dependency-name: tokio-util
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-06-06 14:46:27 +00:00
a1ea793e13
chore(deps): Bump tokio-stream from 0.1.8 to 0.1.9 ( #4785 )
...
Bumps [tokio-stream](https://github.com/tokio-rs/tokio ) from 0.1.8 to 0.1.9.
- [Release notes](https://github.com/tokio-rs/tokio/releases )
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-stream-0.1.8...tokio-stream-0.1.9 )
---
updated-dependencies:
- dependency-name: tokio-stream
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-06-06 14:21:54 +00:00
e03bf94420
chore(deps): Bump tokio from 1.18.2 to 1.19.1 ( #4783 )
...
Bumps [tokio](https://github.com/tokio-rs/tokio ) from 1.18.2 to 1.19.1.
- [Release notes](https://github.com/tokio-rs/tokio/releases )
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.18.2...tokio-1.19.1 )
---
updated-dependencies:
- dependency-name: tokio
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-06-06 14:15:12 +00:00
3592aa52d8
chore: Update datafusion + `arrow`/`parquet`/`arrow-flight` to `15.0.0` ( #4743 )
...
* chore: Update datafusion + `arrow`/`parquet`/`arrow-flight` to `15.0.0`
* chore: Update APIs
* chore: Run cargo hakari tasks
* feat: normalize parquet file metadata
* chore: update size tests
* chore: add docs on metadata stripping
* chore: TEMP UPDATE TO DF BRANCH
* chore: Update for new API
* fix: Update to latest DF
* fix: cargo hakari
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: Raphael Taylor-Davies <r.taylordavies@googlemail.com>
2022-06-03 10:32:26 +00:00
9a21292db8
chore(deps): Bump async-trait from 0.1.53 to 0.1.56 ( #4774 )
...
Bumps [async-trait](https://github.com/dtolnay/async-trait ) from 0.1.53 to 0.1.56.
- [Release notes](https://github.com/dtolnay/async-trait/releases )
- [Commits](https://github.com/dtolnay/async-trait/compare/0.1.53...0.1.56 )
---
updated-dependencies:
- dependency-name: async-trait
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-06-03 09:10:40 +00:00
73a7e6f0a5
chore(deps): Bump syn from 1.0.95 to 1.0.96 ( #4773 )
...
Bumps [syn](https://github.com/dtolnay/syn ) from 1.0.95 to 1.0.96.
- [Release notes](https://github.com/dtolnay/syn/releases )
- [Commits](https://github.com/dtolnay/syn/compare/1.0.95...1.0.96 )
---
updated-dependencies:
- dependency-name: syn
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-06-03 08:02:56 +00:00
9e30a3eb29
refactor: rework querier concurrency limiting ( #4760 )
...
* refactor: rework querier concurrency limiting
With #4752 we introduced a concurrency limit into the querier. It works
by drawing permits from a central semaphore whenever we create a
`QuerierNamespace`. This however only limits concurrency during query
planning and not query execution, because the objects contained within
the plan (chunks and some metadata) neither reference the permit nor the
`QuerierNamespace`.
Now one approach to fix that would be to wire up the permit all the down
into all the query-related data structures. This however is very fiddly
and potentially will get lost at some point, because as soon as we
transform these data structures -- e.g. into streams -- the permit might
get lost again. This will be potentially query-dependent and very hard
to debug.
So instead we reverse the approach and track the permits at the upper
layer of the stack: the gRPC service entry points. There we also need to
be careful -- e.g. when we return streams to tonic -- but it's way
easier to review that then the deeply nested object hierarchy that is
involved with queries. Also the separation of concerns is a bit clearer,
because why would a "chunk" care about the "query concurrency" as a
whole.
* refactor: improve gRPC permit keeping and prepare tests
2022-06-02 09:49:58 +00:00
e638385782
chore(deps): Bump pbjson-types from 0.3.1 to 0.3.2 ( #4750 )
...
Bumps [pbjson-types](https://github.com/influxdata/pbjson ) from 0.3.1 to 0.3.2.
- [Release notes](https://github.com/influxdata/pbjson/releases )
- [Commits](https://github.com/influxdata/pbjson/compare/0.3.1...0.3.2 )
---
updated-dependencies:
- dependency-name: pbjson-types
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-06-01 09:02:37 +00:00
21ec05a6ee
chore(deps): Bump pbjson from 0.3.1 to 0.3.2 ( #4749 )
...
Bumps [pbjson](https://github.com/influxdata/pbjson ) from 0.3.1 to 0.3.2.
- [Release notes](https://github.com/influxdata/pbjson/releases )
- [Commits](https://github.com/influxdata/pbjson/compare/0.3.1...0.3.2 )
---
updated-dependencies:
- dependency-name: pbjson
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-06-01 08:12:46 +00:00
043dee43c8
chore(deps): Bump pbjson-build from 0.3.1 to 0.3.2 ( #4748 )
...
Bumps [pbjson-build](https://github.com/influxdata/pbjson ) from 0.3.1 to 0.3.2.
- [Release notes](https://github.com/influxdata/pbjson/releases )
- [Commits](https://github.com/influxdata/pbjson/compare/0.3.1...0.3.2 )
---
updated-dependencies:
- dependency-name: pbjson-build
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-06-01 07:54:13 +00:00
c91dbe062e
test: "optimize" ingesterrecord batches in query tests ( #4700 )
...
* test: "optimize" ingesterrecord batches in query tests
It seems that I had the right idea in #4656 but wasn't able to trigger
https://github.com/influxdata/conductor/issues/955 because the query
tests do not "optimize" the record batches in the same way the actual
gRPC implementation does. If we apply the same transformation we indeed
end up with the same error.
* fix: all batches within the ingester flight response must have same schema
* refactor: simplify and reuse code
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-06-01 07:37:11 +00:00
988bd38e93
refactor: remove unused code ( #4742 )
2022-05-31 11:36:02 +00:00
5a95da7327
refactor: do NOT use ANY file IO for parquet reading ( #4741 )
2022-05-31 11:18:24 +00:00
01fc550034
chore(deps): Bump parking_lot from 0.12.0 to 0.12.1 ( #4732 )
...
Bumps [parking_lot](https://github.com/Amanieu/parking_lot ) from 0.12.0 to 0.12.1.
- [Release notes](https://github.com/Amanieu/parking_lot/releases )
- [Changelog](https://github.com/Amanieu/parking_lot/blob/master/CHANGELOG.md )
- [Commits](https://github.com/Amanieu/parking_lot/compare/0.12.0...0.12.1 )
---
updated-dependencies:
- dependency-name: parking_lot
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-05-31 08:01:58 +00:00
5b5f0efef5
chore(deps): Bump rayon from 1.5.2 to 1.5.3 ( #4731 )
...
Bumps [rayon](https://github.com/rayon-rs/rayon ) from 1.5.2 to 1.5.3.
- [Release notes](https://github.com/rayon-rs/rayon/releases )
- [Changelog](https://github.com/rayon-rs/rayon/blob/master/RELEASES.md )
- [Commits](https://github.com/rayon-rs/rayon/compare/v1.5.2...v1.5.3 )
---
updated-dependencies:
- dependency-name: rayon
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-05-31 07:50:42 +00:00
642aef103d
chore(deps): Bump comfy-table from 5.0.1 to 6.0.0 ( #4730 )
...
Bumps [comfy-table](https://github.com/nukesor/comfy-table ) from 5.0.1 to 6.0.0.
- [Release notes](https://github.com/nukesor/comfy-table/releases )
- [Changelog](https://github.com/Nukesor/comfy-table/blob/main/CHANGELOG.md )
- [Commits](https://github.com/nukesor/comfy-table/compare/v5.0.1...v6.0.0 )
---
updated-dependencies:
- dependency-name: comfy-table
dependency-type: direct:production
update-type: version-update:semver-major
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-05-31 07:10:37 +00:00
79c054ffc9
fix: do NOT block in parquet file IO ( #4727 )
...
* fix: do NOT block in parquet file IO
I think for historical reason we were using blocking IO to read parquet
files. With the current streaming `SendableRecordStream` approach this
is technically NOT required anymore.
Now one might think that the sync-async dance that we did is kinda
harmless, but looking at our producition querier I think it is really
bad. The querier seems to be stuck but looking at `strace` and other
health signal it seems it is not entirely dead. Looking at GDB
backtraces it seems that nearly all threads are busy in
`download_and_scan_parquet`. Looking at the tokio docs
(<https://docs.rs/tokio/1.18.2/tokio/task/fn.spawn_blocking.html >)
for `spawn_blocking` (which is used to start the sync download) this
makes sense: tokio only starts replacement threads for the current
runtime thread (which calls `spawn_blocking`) if this does NOT exceed the
runtime thread limit. However we set the runtime thread limit to the
number of CPU cores available to IOx, so this is a limiting factor. This
means that there are only a few threads left to do actual work (I've
seen postgres data flowing back and forth for example) but tokio is not
able to use its full potential anymore. This is esp. bad because the
sync code in `download_and_scan_parquet` then uses `futures` `block_on`
functionality to call back into async code, so it waits for tokio
itself.
The change is rather simple: just use async task spawns.
* fix: use async IO to write stream to temp file
* fix: do not block tokio thread during parquet file reading
* refactor: ensure parquet IO tasks are cancelled if they are not needed anymore
There is no REAL way to cancel sync tasks, but at least we can try our
best.
2022-05-30 13:32:20 +00:00
73168f7989
chore(deps): Bump flate2 from 1.0.23 to 1.0.24 ( #4726 )
...
Bumps [flate2](https://github.com/rust-lang/flate2-rs ) from 1.0.23 to 1.0.24.
- [Release notes](https://github.com/rust-lang/flate2-rs/releases )
- [Commits](https://github.com/rust-lang/flate2-rs/commits )
---
updated-dependencies:
- dependency-name: flate2
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-05-30 08:26:12 +00:00
29069be7d4
chore(deps): Bump hyper from 0.14.18 to 0.14.19 ( #4725 )
...
Bumps [hyper](https://github.com/hyperium/hyper ) from 0.14.18 to 0.14.19.
- [Release notes](https://github.com/hyperium/hyper/releases )
- [Changelog](https://github.com/hyperium/hyper/blob/master/CHANGELOG.md )
- [Commits](https://github.com/hyperium/hyper/compare/v0.14.18...v0.14.19 )
---
updated-dependencies:
- dependency-name: hyper
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-05-30 07:58:12 +00:00
7d4670e171
chore(deps): Bump indexmap from 1.8.1 to 1.8.2 ( #4724 )
...
Bumps [indexmap](https://github.com/bluss/indexmap ) from 1.8.1 to 1.8.2.
- [Release notes](https://github.com/bluss/indexmap/releases )
- [Changelog](https://github.com/bluss/indexmap/blob/1.8.2/RELEASES.rst )
- [Commits](https://github.com/bluss/indexmap/compare/1.8.1...1.8.2 )
---
updated-dependencies:
- dependency-name: indexmap
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-05-30 07:47:05 +00:00
cddd6d9b6d
chore: Update datafusion ( #4723 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-05-28 19:00:54 +00:00
25b8260b72
feat: Implement ReadBufferCache::new
2022-05-25 17:19:10 -04:00
31d1b37d73
refactor: de-duplicate low-level arrow code ( #4697 )
...
It seems that during prototyping NG we've copied low level code (w/o
tests!) and never cleaned up. Let's not have this functionality twice.
2022-05-25 16:24:28 +00:00
cdbe546e50
fix: return gRPC error on panic ( #4686 )
2022-05-25 07:06:25 +00:00
24ee251080
chore(deps): Bump prost from 0.10.3 to 0.10.4 ( #4688 )
...
Bumps [prost](https://github.com/tokio-rs/prost ) from 0.10.3 to 0.10.4.
- [Release notes](https://github.com/tokio-rs/prost/releases )
- [Commits](https://github.com/tokio-rs/prost/compare/v0.10.3...v0.10.4 )
---
updated-dependencies:
- dependency-name: prost
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-05-25 06:07:05 +00:00
a8d3fe619c
chore(deps): Bump prost-build from 0.10.3 to 0.10.4 ( #4687 )
...
Bumps [prost-build](https://github.com/tokio-rs/prost ) from 0.10.3 to 0.10.4.
- [Release notes](https://github.com/tokio-rs/prost/releases )
- [Commits](https://github.com/tokio-rs/prost/compare/v0.10.3...v0.10.4 )
---
updated-dependencies:
- dependency-name: prost-build
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-05-25 06:00:27 +00:00
95e6a8ed46
chore: Update datafusion (again) ( #4679 )
...
* chore: Update datafusion deps
* fix: fix for changes in ScalarValue
* fix: fix for using TableSource rather than TableProvider
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-05-24 15:54:39 +00:00
b76a0080d5
chore: remove unused iox_gitops_adapter ( #4675 )
...
* chore: remove unused iox_gitops_adapter
* chore: Run cargo hakari tasks
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
2022-05-24 10:28:43 +00:00
ca49820a0f
chore(deps): Bump console-subscriber from 0.1.5 to 0.1.6 ( #4670 )
...
Bumps [console-subscriber](https://github.com/tokio-rs/console ) from 0.1.5 to 0.1.6.
- [Release notes](https://github.com/tokio-rs/console/releases )
- [Commits](https://github.com/tokio-rs/console/compare/console-subscriber-v0.1.5...console-subscriber-v0.1.6 )
---
updated-dependencies:
- dependency-name: console-subscriber
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-05-24 08:24:12 +00:00
76f7043417
chore(deps): Bump once_cell from 1.11.0 to 1.12.0 ( #4666 )
...
Bumps [once_cell](https://github.com/matklad/once_cell ) from 1.11.0 to 1.12.0.
- [Release notes](https://github.com/matklad/once_cell/releases )
- [Changelog](https://github.com/matklad/once_cell/blob/master/CHANGELOG.md )
- [Commits](https://github.com/matklad/once_cell/compare/v1.11.0...v1.12.0 )
---
updated-dependencies:
- dependency-name: once_cell
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-05-24 08:14:03 +00:00
2029bd16ba
feat: enable debugging of failed querier->ingester requests ( #4659 )
...
* feat: enable debugging of failed querier->ingester requests
- extend `query-ingester` CLI to allow usage of predicates
- on failed requests: log all information that required for the CLI
- test the "ingester fails" scenario
* test: explain
Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
* docs: improve
Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
* refactor: move b64 pred. serde into a single crate
Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
2022-05-23 15:37:31 +00:00
f0d0f1ba0c
Merge branch 'main' into dom/codec-object-store
2022-05-23 15:39:54 +01:00
5c033b462e
chore(deps): Bump regex from 1.5.5 to 1.5.6 ( #4655 )
...
Bumps [regex](https://github.com/rust-lang/regex ) from 1.5.5 to 1.5.6.
- [Release notes](https://github.com/rust-lang/regex/releases )
- [Changelog](https://github.com/rust-lang/regex/blob/master/CHANGELOG.md )
- [Commits](https://github.com/rust-lang/regex/compare/1.5.5...1.5.6 )
---
updated-dependencies:
- dependency-name: regex
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-05-23 08:39:01 +00:00
292f71759e
chore(deps): Bump http-body from 0.4.4 to 0.4.5 ( #4654 )
...
Bumps [http-body](https://github.com/hyperium/http-body ) from 0.4.4 to 0.4.5.
- [Release notes](https://github.com/hyperium/http-body/releases )
- [Changelog](https://github.com/hyperium/http-body/blob/master/CHANGELOG.md )
- [Commits](https://github.com/hyperium/http-body/compare/v0.4.4...v0.4.5 )
---
updated-dependencies:
- dependency-name: http-body
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-05-23 08:30:49 +00:00
1bc02b1487
chore(deps): Bump regex-syntax from 0.6.25 to 0.6.26 ( #4653 )
...
Bumps [regex-syntax](https://github.com/rust-lang/regex ) from 0.6.25 to 0.6.26.
- [Release notes](https://github.com/rust-lang/regex/releases )
- [Changelog](https://github.com/rust-lang/regex/blob/master/CHANGELOG.md )
- [Commits](https://github.com/rust-lang/regex/commits )
---
updated-dependencies:
- dependency-name: regex-syntax
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-05-23 08:21:39 +00:00
b9a745d42d
feat: RecordBatch stream to Parquet file upload
...
Implements an upload() method on the ParquetStorage type, consuming a
stream of RecordBatch, serialising the Parquet file, and uploading the
result to object storage. Returns the IOx-specific file metadata.
Currently while the upload() method accepts a stream of RecordBatch, the
actual resulting Parquet file is buffered in memory before uploading to
object store, due to lack of streaming upload functionality in the
ObjectStore abstraction - this isn't the end of the world, as the files
tend to be relatively small with our current usage.
This impl should be easily modified to be fully streaming once streaming
object store puts are implemented:
https://github.com/influxdata/object_store_rs/issues/9
2022-05-20 15:17:40 +01:00
70856a645f
feat: streaming RecordBatch -> parquet encoding
...
Implements a streaming RecordBatch to Parquet file serialiser.
This impl automatically discovers the schema of the RecordBatch stream,
and accepts &mut destination types (internalising the handle
cloning/etc) to simplify caller usage.
This encoder returns the resulting FileMetaData to allow callers to
inspect the resulting metadata without reading back the file.
Currently unused / not yet plumbed in.
2022-05-20 15:09:26 +01:00
7010af30b7
chore(deps): Bump prometheus from 0.13.0 to 0.13.1 ( #4648 )
...
Bumps [prometheus](https://github.com/tikv/rust-prometheus ) from 0.13.0 to 0.13.1.
- [Release notes](https://github.com/tikv/rust-prometheus/releases )
- [Changelog](https://github.com/tikv/rust-prometheus/blob/master/CHANGELOG.md )
- [Commits](https://github.com/tikv/rust-prometheus/compare/v0.13.0...v0.13.1 )
---
updated-dependencies:
- dependency-name: prometheus
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-05-20 08:12:45 +00:00
40a69c6e29
chore(deps): Bump pprof from 0.9.0 to 0.9.1 ( #4647 )
...
Bumps [pprof](https://github.com/tikv/pprof-rs ) from 0.9.0 to 0.9.1.
- [Release notes](https://github.com/tikv/pprof-rs/releases )
- [Changelog](https://github.com/tikv/pprof-rs/blob/master/CHANGELOG.md )
- [Commits](https://github.com/tikv/pprof-rs/commits )
---
updated-dependencies:
- dependency-name: pprof
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-05-20 08:04:46 +00:00
6bc0c74c7d
chore(deps): Bump once_cell from 1.10.0 to 1.11.0 ( #4646 )
...
* chore(deps): Bump once_cell from 1.10.0 to 1.11.0
Bumps [once_cell](https://github.com/matklad/once_cell ) from 1.10.0 to 1.11.0.
- [Release notes](https://github.com/matklad/once_cell/releases )
- [Changelog](https://github.com/matklad/once_cell/blob/master/CHANGELOG.md )
- [Commits](https://github.com/matklad/once_cell/compare/v1.10.0...v1.11.0 )
---
updated-dependencies:
- dependency-name: once_cell
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* chore: Run cargo hakari tasks
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-05-20 07:40:38 +00:00
addc45327e
fix: ensure that query tokio background tasks are canceled ( #4643 )
...
* fix: ensure that query tokio background tasks are canceled
While I am not entirely sure if this explains some of the memory leaks I
am seeing in prod, not canceling the tasks correctly certainly makes
debugging way harder and also renders certain form of throttling (e.g.
max. concurrent queries) somewhat ineffective.
Note that parquet file downloads are currently NOT canceled because
tokios `spawn_blocking` cannot be canceled.
* refactor: `Vec` -> `Option`
* refactor: `spawn_blocking` creates a join handle, even though it is useless
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-05-20 07:18:52 +00:00
baa86d846f
refactor: use ParquetStore instead of ObjectStore
...
Changes the code paths that interact with Parquet files in the object
store to reference the ParquetStorage directly (DRY refactor).
This change takes us from a dependency graph of:
┌─────────────────┐
│ │
▼ │
Parquet Consumer │
│ ┌──────────────┐
├────────▶│ParquetStorage│
▼ └──────────────┘
┌──────────────┐
│ ObjectStore │
└──────────────┘
│
┌────┴────┐
▼ ▼
File s3
System (etc)
to:
Parquet Consumer
│
▼
┌──────────────┐
│ParquetStorage│
└──────────────┘
│
▼
┌──────────────┐
│ ObjectStore │
└──────────────┘
│
┌────┴────┐
▼ ▼
File s3
System (etc)
With the ParquetStorage being solely responsible for managing
interactions with the object store when dealing with Parquet files.
2022-05-19 13:52:51 +01:00
409ae0ee0d
chore(deps): Bump handlebars from 4.2.2 to 4.3.0 ( #4639 )
...
Bumps [handlebars](https://github.com/sunng87/handlebars-rust ) from 4.2.2 to 4.3.0.
- [Release notes](https://github.com/sunng87/handlebars-rust/releases )
- [Changelog](https://github.com/sunng87/handlebars-rust/blob/master/CHANGELOG.md )
- [Commits](https://github.com/sunng87/handlebars-rust/compare/v4.2.2...v4.3.0 )
---
updated-dependencies:
- dependency-name: handlebars
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-05-19 08:15:51 +00:00
770293a973
feat: add LRU cache metrics ( #4632 )
...
* refactor: require `Resource`s to be convertible to `u64`
* refactor: require `Resource`s to have a unit name
* refactor: make LRU cache IDs static
* feat: add LRU cache metrics
* docs: improve type names in LRU doctest
* docs: epxlain `MeasuredT`
Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
* docs: explain `test_metrics`
Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
2022-05-19 08:05:17 +00:00
43300878bc
fix(pb): encoding entirely NULL columns ( #4272 )
...
This commit changes the protobuf record batch encoding to skip entirely
NULL columns when serialising. This prevents the deserialisation from
erroring due to a column type inference failure.
Prior to this commit, when the system was presented a record batch such
as this:
| time | A | B |
| ---------- | ---- | ---- |
| 1970-01-01 | 1 | NULL |
| 1970-07-05 | NULL | 1 |
Which would be partitioned by YMD into two separate partitions:
| time | A | B |
| ---------- | ---- | ---- |
| 1970-01-01 | 1 | NULL |
and:
| time | A | B |
| ---------- | ---- | ---- |
| 1970-07-05 | NULL | 1 |
Both partitions would contain an entirely NULL column.
Both of these partitioned record batches would be successfully encoded,
but decoding the partition fails due to the inability to infer a column
type from the serialised format which contains no values, which on the
wire, looks like:
Column {
column_name: "B",
semantic_type: Field,
values: Some(
Values {
i64_values: [],
f64_values: [],
u64_values: [],
string_values: [],
bool_values: [],
bytes_values: [],
packed_string_values: None,
interned_string_values: None,
},
),
null_mask: [
1,
],
},
In a column that is not entirely NULL, one of the "Values" fields would
be non-empty, and the decoder would use this to infer the type of the
column.
Because we have chosen to not differentiate between "NULL" and "empty"
in our proto encoding, the decoder cannot infer which field within the
"Values" struct the column belongs to - all are valid, but empty.
This commit prevents this type inference failure by skipping any columns
that are entirely NULL during serialisation, preventing the deserialiser
from having to process columns with ambiguous types.
2022-05-18 13:33:26 +01:00
c99ad9f4a4
chore(deps): Bump schemars from 0.8.9 to 0.8.10 ( #4627 )
...
Bumps [schemars](https://github.com/GREsau/schemars ) from 0.8.9 to 0.8.10.
- [Release notes](https://github.com/GREsau/schemars/releases )
- [Changelog](https://github.com/GREsau/schemars/blob/master/CHANGELOG.md )
- [Commits](https://github.com/GREsau/schemars/compare/v0.8.9...v0.8.10 )
---
updated-dependencies:
- dependency-name: schemars
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-05-18 10:23:34 +00:00
b542e453e8
chore(deps): Bump rustls from 0.20.4 to 0.20.6
...
Bumps [rustls](https://github.com/rustls/rustls ) from 0.20.4 to 0.20.6.
- [Release notes](https://github.com/rustls/rustls/releases )
- [Changelog](https://github.com/rustls/rustls/blob/main/RELEASE_NOTES.md )
- [Commits](https://github.com/rustls/rustls/compare/v/0.20.4...v/0.20.6 )
---
updated-dependencies:
- dependency-name: rustls
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2022-05-18 09:54:48 +00:00
db384ba387
chore(deps): Bump libc from 0.2.125 to 0.2.126 ( #4626 )
...
Bumps [libc](https://github.com/rust-lang/libc ) from 0.2.125 to 0.2.126.
- [Release notes](https://github.com/rust-lang/libc/releases )
- [Commits](https://github.com/rust-lang/libc/compare/0.2.125...0.2.126 )
---
updated-dependencies:
- dependency-name: libc
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-05-18 09:52:35 +00:00
52346642a0
ci: fix cargo deny ( #4629 )
...
* ci: fix cargo deny
* chore: downgrade `socket2`, version 0.4.5 was yanked
* chore: rename `query` to `iox_query`
`query` is already taken on crates.io and yanked and I am getting tired
of working around that.
2022-05-18 09:38:35 +00:00
12937ee724
feat: add SOCKS5 support to Kafka write buffer ( #4623 )
2022-05-17 15:21:35 +00:00
3a33e806c7
chore: Update datafusion + `arrow`/`parquet`/`arrow-flight` to `14.0.0` ( #4619 )
...
* chore: Update datafusion deps
* chore: update arrow/parquet/arrow flight deps
* chore: Run cargo hakari tasks
* chore: Update location of utils
* chore: Update some more APIs
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
2022-05-17 14:13:03 +00:00
87d8a8b684
chore(deps): Bump schemars from 0.8.8 to 0.8.9 ( #4614 )
...
Bumps [schemars](https://github.com/GREsau/schemars ) from 0.8.8 to 0.8.9.
- [Release notes](https://github.com/GREsau/schemars/releases )
- [Changelog](https://github.com/GREsau/schemars/blob/master/CHANGELOG.md )
- [Commits](https://github.com/GREsau/schemars/compare/v0.8.8...v0.8.9 )
---
updated-dependencies:
- dependency-name: schemars
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-05-17 09:49:27 +00:00
5a0006e40b
chore(deps): Bump syn from 1.0.94 to 1.0.95 ( #4610 )
...
Bumps [syn](https://github.com/dtolnay/syn ) from 1.0.94 to 1.0.95.
- [Release notes](https://github.com/dtolnay/syn/releases )
- [Commits](https://github.com/dtolnay/syn/compare/1.0.94...1.0.95 )
---
updated-dependencies:
- dependency-name: syn
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-05-17 09:40:36 +00:00
a91a9d5789
Merge remote-tracking branch 'origin/dependabot/cargo/kube-runtime-0.72.0' into cn/kube-update
2022-05-16 10:50:51 -04:00
42b52da0d4
Merge remote-tracking branch 'origin/dependabot/cargo/kube-derive-0.72.0' into cn/kube-update
2022-05-16 10:49:51 -04:00
edd3cce73c
Merge remote-tracking branch 'origin/dependabot/cargo/kube-0.72.0' into cn/kube-update
2022-05-16 10:26:06 -04:00
dependabot[bot]
Andrew Lamb
Raphael Taylor-Davies
Dom
Carol (Nichols || Goulding)
Marco Neumann
Carol (Nichols || Goulding)
Luke Bond
Marko Mikulicic
Stuart Carnie
Jake Goulding
kodiakhq[bot]
CircleCI[bot]
Markus Westerlind
Nga Tran