Commit Graph

1528 Commits (7fbd1a4d0c745c02ecdb5ea1e7336f756b67c89f)

Author SHA1 Message Date
Andrew Lamb 9fed013848
chore: Update datafusion pin (#5162)
* chore: Update datafusion pin

* fix: Update expected output

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-07-20 14:34:08 +00:00
Andrew Lamb 95c734dc4c
feat: Improve logging for garbage collector service (#5146)
* feat: clean up logging for garbage collector service

* fix: restore unintended diff

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-07-20 13:29:44 +00:00
dependabot[bot] b9c60d4db0
chore(deps): Bump clap from 3.2.12 to 3.2.13 (#5157)
Bumps [clap](https://github.com/clap-rs/clap) from 3.2.12 to 3.2.13.
- [Release notes](https://github.com/clap-rs/clap/releases)
- [Changelog](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md)
- [Commits](https://github.com/clap-rs/clap/compare/v3.2.12...v3.2.13)

---
updated-dependencies:
- dependency-name: clap
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-07-20 11:32:58 +00:00
dependabot[bot] 278a7f91af
chore(deps): Bump bytes from 1.1.0 to 1.2.0 (#5156)
Bumps [bytes](https://github.com/tokio-rs/bytes) from 1.1.0 to 1.2.0.
- [Release notes](https://github.com/tokio-rs/bytes/releases)
- [Changelog](https://github.com/tokio-rs/bytes/blob/master/CHANGELOG.md)
- [Commits](https://github.com/tokio-rs/bytes/compare/v1.1.0...v1.2.0)

---
updated-dependencies:
- dependency-name: bytes
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-07-20 10:00:08 +00:00
Andrew Lamb 449b4cf1d6
chore: Update datafusion pin (#5148)
* chore: Update datafusion pin

* fix: Update for changes upstream
2022-07-19 13:13:37 +00:00
Jake Goulding f7a0fd43d2
feat: make object store garbage collector into a long-running service (#5135)
* refactor: remove unused logging config

* chore: remove the object store garbage collector CLI tool

* refactor: accept an object store and catalog

* refactor: make Result type alias public like the error

* refactor: remove public modifier from modules

* refactor: allow shutting down the object store garbage collector

* feat: Introduce the object-store garbage collection server

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-07-18 21:27:38 +00:00
Andrew Lamb e2d871b00b
chore: Update datafusion and arrow/parquet/arrow-flight to `18.0.0` (#5079)
* chore: Update datafusion to 10.0.0, arrow/parquet/arrow-flight to 18

* chore: Run cargo hakari tasks

* fix: update cargo pin

Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-07-18 15:01:03 +00:00
dependabot[bot] 1eeee9809c
chore(deps): Bump rustyline from 9.1.2 to 10.0.0 (#5139)
Bumps [rustyline](https://github.com/kkawakam/rustyline) from 9.1.2 to 10.0.0.
- [Release notes](https://github.com/kkawakam/rustyline/releases)
- [Changelog](https://github.com/kkawakam/rustyline/blob/master/History.md)
- [Commits](https://github.com/kkawakam/rustyline/compare/v9.1.2...v10.0.0)

---
updated-dependencies:
- dependency-name: rustyline
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-07-18 08:10:22 +00:00
dependabot[bot] fd2d262862
chore(deps): Bump nix from 0.24.1 to 0.24.2 (#5140)
Bumps [nix](https://github.com/nix-rust/nix) from 0.24.1 to 0.24.2.
- [Release notes](https://github.com/nix-rust/nix/releases)
- [Changelog](https://github.com/nix-rust/nix/blob/v0.24.2/CHANGELOG.md)
- [Commits](https://github.com/nix-rust/nix/compare/v0.24.1...v0.24.2)

---
updated-dependencies:
- dependency-name: nix
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-07-18 06:15:49 +00:00
dependabot[bot] 0b4afad08f
chore(deps): Bump hashbrown from 0.12.2 to 0.12.3 (#5141)
Bumps [hashbrown](https://github.com/rust-lang/hashbrown) from 0.12.2 to 0.12.3.
- [Release notes](https://github.com/rust-lang/hashbrown/releases)
- [Changelog](https://github.com/rust-lang/hashbrown/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/hashbrown/compare/v0.12.2...v0.12.3)

---
updated-dependencies:
- dependency-name: hashbrown
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-07-18 06:06:24 +00:00
dependabot[bot] f870c8a887
chore(deps): Bump crypto-common from 0.1.5 to 0.1.6 (#5138)
Bumps [crypto-common](https://github.com/RustCrypto/traits) from 0.1.5 to 0.1.6.
- [Release notes](https://github.com/RustCrypto/traits/releases)
- [Commits](https://github.com/RustCrypto/traits/compare/crypto-common-v0.1.5...crypto-common-v0.1.6)

---
updated-dependencies:
- dependency-name: crypto-common
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-07-18 05:36:20 +00:00
dependabot[bot] c0ac01c67b
chore(deps): Bump handlebars from 4.3.1 to 4.3.2 (#5133)
Bumps [handlebars](https://github.com/sunng87/handlebars-rust) from 4.3.1 to 4.3.2.
- [Release notes](https://github.com/sunng87/handlebars-rust/releases)
- [Changelog](https://github.com/sunng87/handlebars-rust/blob/master/CHANGELOG.md)
- [Commits](https://github.com/sunng87/handlebars-rust/compare/v4.3.1...v4.3.2)

---
updated-dependencies:
- dependency-name: handlebars
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-07-15 21:18:50 +00:00
dependabot[bot] 9359539acf
chore(deps): Bump clap from 3.2.11 to 3.2.12 (#5132)
Bumps [clap](https://github.com/clap-rs/clap) from 3.2.11 to 3.2.12.
- [Release notes](https://github.com/clap-rs/clap/releases)
- [Changelog](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md)
- [Commits](https://github.com/clap-rs/clap/compare/v3.2.11...v3.2.12)

---
updated-dependencies:
- dependency-name: clap
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-07-15 21:10:51 +00:00
Marco Neumann f0bd278652
feat: add tracing to instrumented semaphores (#5130)
This will allow us to easily see how much time we spend during query
processing waiting for the query semaphore.

Ref #5129.
2022-07-15 07:50:28 +00:00
kodiakhq[bot] 14472c4e66
Merge branch 'main' into dependabot/cargo/clap-3.2.11 2022-07-14 14:26:56 +00:00
dependabot[bot] 9b67de2f43
chore(deps): Bump tokio from 1.19.2 to 1.20.0
Bumps [tokio](https://github.com/tokio-rs/tokio) from 1.19.2 to 1.20.0.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.19.2...tokio-1.20.0)

---
updated-dependencies:
- dependency-name: tokio
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-07-14 01:21:43 +00:00
dependabot[bot] 8179f113b9
chore(deps): Bump clap from 3.2.10 to 3.2.11
Bumps [clap](https://github.com/clap-rs/clap) from 3.2.10 to 3.2.11.
- [Release notes](https://github.com/clap-rs/clap/releases)
- [Changelog](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md)
- [Commits](https://github.com/clap-rs/clap/compare/v3.2.10...v3.2.11)

---
updated-dependencies:
- dependency-name: clap
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-07-14 01:21:09 +00:00
Marco Neumann b42dd1c45e
chore: update all dependencies (esp. `time`) (#5114)
Our CI found a potential panic:
https://app.circleci.com/pipelines/github/influxdata/influxdb_iox/21307/workflows/7c1df458-3c6a-4f5d-bfda-51612273bbc7/jobs/183140

See https://github.com/time-rs/time/issues/481 , so let's update
`time` and since we've accumulated quite a backlog, just run
`cargo update --workspace`.

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-07-13 14:30:18 +00:00
dependabot[bot] 3a1934e7af
chore(deps): Bump clap from 3.2.8 to 3.2.10 (#5111)
Bumps [clap](https://github.com/clap-rs/clap) from 3.2.8 to 3.2.10.
- [Release notes](https://github.com/clap-rs/clap/releases)
- [Changelog](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md)
- [Commits](https://github.com/clap-rs/clap/compare/v3.2.8...v3.2.10)

---
updated-dependencies:
- dependency-name: clap
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-07-13 08:24:51 +00:00
Marko Mikulicic 60df069a10 fix: Bump object_store crate for EKS support 2022-07-13 01:12:29 +02:00
dependabot[bot] 23ad60b35f
chore(deps): Bump serde from 1.0.138 to 1.0.139 (#5095)
Bumps [serde](https://github.com/serde-rs/serde) from 1.0.138 to 1.0.139.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.138...v1.0.139)

---
updated-dependencies:
- dependency-name: serde
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-07-12 15:34:51 +00:00
Marco Neumann 607831585c
refactor: use less executors and threads during tests (#5086)
`Executor` is only used as a performance boundary, not as a correctness
or data boundary so let's try to re-use it. This also simplifies
profiling of tests since we don't end up with hundreds (or even
thousands) of threads.

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-07-11 16:23:22 +00:00
dependabot[bot] 5703d12595
chore(deps): Bump hashbrown from 0.12.1 to 0.12.2 (#5084)
Bumps [hashbrown](https://github.com/rust-lang/hashbrown) from 0.12.1 to 0.12.2.
- [Release notes](https://github.com/rust-lang/hashbrown/releases)
- [Changelog](https://github.com/rust-lang/hashbrown/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/hashbrown/compare/v0.12.1...v0.12.2)

---
updated-dependencies:
- dependency-name: hashbrown
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-07-11 12:56:31 +00:00
dependabot[bot] fd3ee7e942
chore(deps): Bump crypto-common from 0.1.4 to 0.1.5 (#5083)
Bumps [crypto-common](https://github.com/RustCrypto/traits) from 0.1.4 to 0.1.5.
- [Release notes](https://github.com/RustCrypto/traits/releases)
- [Commits](https://github.com/RustCrypto/traits/compare/crypto-common-v0.1.4...crypto-common-v0.1.5)

---
updated-dependencies:
- dependency-name: crypto-common
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-07-11 09:05:26 +00:00
dependabot[bot] 93d1a0f5be
chore(deps): Bump hyper from 0.14.19 to 0.14.20 (#5073)
Bumps [hyper](https://github.com/hyperium/hyper) from 0.14.19 to 0.14.20.
- [Release notes](https://github.com/hyperium/hyper/releases)
- [Changelog](https://github.com/hyperium/hyper/blob/v0.14.20/CHANGELOG.md)
- [Commits](https://github.com/hyperium/hyper/compare/v0.14.19...v0.14.20)

---
updated-dependencies:
- dependency-name: hyper
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-07-08 13:24:34 +00:00
dependabot[bot] 9a3d31bc63
chore(deps): Bump backtrace from 0.3.65 to 0.3.66 (#5072)
Bumps [backtrace](https://github.com/rust-lang/backtrace-rs) from 0.3.65 to 0.3.66.
- [Release notes](https://github.com/rust-lang/backtrace-rs/releases)
- [Commits](https://github.com/rust-lang/backtrace-rs/compare/0.3.65...0.3.66)

---
updated-dependencies:
- dependency-name: backtrace
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-07-08 08:00:49 +00:00
dependabot[bot] 6c1b70ba2d
chore(deps): Bump filetime from 0.2.16 to 0.2.17 (#5071)
Bumps [filetime](https://github.com/alexcrichton/filetime) from 0.2.16 to 0.2.17.
- [Release notes](https://github.com/alexcrichton/filetime/releases)
- [Commits](https://github.com/alexcrichton/filetime/commits/0.2.17)

---
updated-dependencies:
- dependency-name: filetime
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-07-08 07:50:40 +00:00
Andrew Lamb c46e1c6347
chore: Update datafusion + arrow/parquet/arrow-flight to `17.0.0` (#5021)
* fix: correct nullability declaration of system tables

* chore: Update datafusion and arrow/parquet/arrow-flight

* chore: Run cargo hakari tasks

* fix: Update tests

* fix: Update tests

* fix: predicate pruning

* fix: add some tests

* fix: query_functions

* fix: fix read_buffer test

* fix: fix clippy

Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-07-07 19:22:15 +00:00
Carol (Nichols || Goulding) 70dd6009e8
fix: Integrate the object store garbage collector into the main binary 2022-07-07 09:48:06 -04:00
CircleCI[bot] 4369431e03
chore: Run cargo hakari tasks 2022-07-07 09:48:05 -04:00
Jake Goulding 6fc17164bd
test: Add basic end-to-end tests of the garbage collector 2022-07-07 09:48:05 -04:00
Carol (Nichols || Goulding) 72d1ed9b73
feat: Support configurable cutoff time for too new object store files 2022-07-07 09:48:05 -04:00
Carol (Nichols || Goulding) 3fcaf34d54
feat: Add trogging logging config to object store garbage collector 2022-07-07 09:48:04 -04:00
Carol (Nichols || Goulding) 90d7b22d86
fix: Use dotenv and the right features in the garbage collector 2022-07-07 09:48:04 -04:00
Carol (Nichols || Goulding) c541ca68a2
fix: Test logic for whether to delete object store files
And fix the bugs the tests found
2022-07-07 09:48:04 -04:00
Carol (Nichols || Goulding) 98e7133ebf
refactor: Reuse clap blocks rather than creating new ones 2022-07-07 09:48:04 -04:00
Jake Goulding 428f41f747
feat: Walking skeleton of the object store GC tool 2022-07-07 09:48:04 -04:00
dependabot[bot] c443d07a5c
chore(deps): Bump criterion from 0.3.5 to 0.3.6 (#5059)
Bumps [criterion](https://github.com/bheisler/criterion.rs) from 0.3.5 to 0.3.6.
- [Release notes](https://github.com/bheisler/criterion.rs/releases)
- [Changelog](https://github.com/bheisler/criterion.rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/bheisler/criterion.rs/compare/0.3.5...0.3.6)

---
updated-dependencies:
- dependency-name: criterion
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-07-07 07:09:27 +00:00
dependabot[bot] 2b527bbf64
chore(deps): Bump regex from 1.5.6 to 1.6.0 (#5048)
Bumps [regex](https://github.com/rust-lang/regex) from 1.5.6 to 1.6.0.
- [Release notes](https://github.com/rust-lang/regex/releases)
- [Changelog](https://github.com/rust-lang/regex/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/regex/compare/1.5.6...1.6.0)

---
updated-dependencies:
- dependency-name: regex
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-07-06 10:25:28 +00:00
dependabot[bot] b3522086a8
chore(deps): Bump regex-syntax from 0.6.26 to 0.6.27 (#5047)
Bumps [regex-syntax](https://github.com/rust-lang/regex) from 0.6.26 to 0.6.27.
- [Release notes](https://github.com/rust-lang/regex/releases)
- [Changelog](https://github.com/rust-lang/regex/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/regex/commits)

---
updated-dependencies:
- dependency-name: regex-syntax
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-07-06 10:19:07 +00:00
dependabot[bot] 68eff79594
chore(deps): Bump once_cell from 1.12.0 to 1.13.0 (#5033)
Bumps [once_cell](https://github.com/matklad/once_cell) from 1.12.0 to 1.13.0.
- [Release notes](https://github.com/matklad/once_cell/releases)
- [Changelog](https://github.com/matklad/once_cell/blob/master/CHANGELOG.md)
- [Commits](https://github.com/matklad/once_cell/compare/v1.12.0...v1.13.0)

---
updated-dependencies:
- dependency-name: once_cell
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-07-05 08:54:51 +00:00
dependabot[bot] 1f5863ebcb
chore(deps): Bump serde from 1.0.137 to 1.0.138 (#5029)
Bumps [serde](https://github.com/serde-rs/serde) from 1.0.137 to 1.0.138.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.137...v1.0.138)

---
updated-dependencies:
- dependency-name: serde
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-07-04 10:07:46 +00:00
dependabot[bot] b8f2f3b9e8
chore(deps): Bump pin-project from 1.0.10 to 1.0.11 (#5028)
Bumps [pin-project](https://github.com/taiki-e/pin-project) from 1.0.10 to 1.0.11.
- [Release notes](https://github.com/taiki-e/pin-project/releases)
- [Changelog](https://github.com/taiki-e/pin-project/blob/main/CHANGELOG.md)
- [Commits](https://github.com/taiki-e/pin-project/compare/v1.0.10...v1.0.11)

---
updated-dependencies:
- dependency-name: pin-project
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-07-04 09:45:31 +00:00
dependabot[bot] 98ac454e88
chore(deps): Bump crypto-common from 0.1.3 to 0.1.4 (#5027)
Bumps [crypto-common](https://github.com/RustCrypto/traits) from 0.1.3 to 0.1.4.
- [Release notes](https://github.com/RustCrypto/traits/releases)
- [Commits](https://github.com/RustCrypto/traits/compare/crypto-common-v0.1.3...crypto-common-v0.1.4)

---
updated-dependencies:
- dependency-name: crypto-common
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-07-04 09:07:37 +00:00
dependabot[bot] 9982d96578
chore(deps): Bump fixedbitset from 0.4.1 to 0.4.2 (#5026)
Bumps [fixedbitset](https://github.com/petgraph/fixedbitset) from 0.4.1 to 0.4.2.
- [Release notes](https://github.com/petgraph/fixedbitset/releases)
- [Commits](https://github.com/petgraph/fixedbitset/commits)

---
updated-dependencies:
- dependency-name: fixedbitset
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-07-04 07:47:53 +00:00
Andrew Lamb c4c251129e
chore: Update datafusion (#5020)
* chore: Update datafusion

* fix: Update plan

* fix: update explain plans

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-07-01 19:59:41 +00:00
dependabot[bot] 09aaf18fb4
chore(deps): Bump clap from 3.2.6 to 3.2.8 (#5015)
Bumps [clap](https://github.com/clap-rs/clap) from 3.2.6 to 3.2.8.
- [Release notes](https://github.com/clap-rs/clap/releases)
- [Changelog](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md)
- [Commits](https://github.com/clap-rs/clap/compare/v3.2.6...v3.2.8)

---
updated-dependencies:
- dependency-name: clap
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-07-01 08:17:04 +00:00
Carol (Nichols || Goulding) 3049479b78
feat: Implement new querier to ingester config design 2022-06-30 08:26:50 -04:00
Carol (Nichols || Goulding) 4e91121e29
feat: Allow specification of sequencer to ingester mappings in a JSON file 2022-06-30 08:22:46 -04:00
Carol (Nichols || Goulding) f37f8013ec
feat: Assign a sequencer id to QuerierTables to know which ingester to query 2022-06-30 08:22:46 -04:00
dependabot[bot] 40a8525520
chore(deps): Bump serde_json from 1.0.81 to 1.0.82 (#4992)
Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.81 to 1.0.82.
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/v1.0.81...v1.0.82)

---
updated-dependencies:
- dependency-name: serde_json
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-06-30 09:54:08 +00:00
dependabot[bot] d1e3b009c9
chore(deps): Bump smallvec from 1.8.1 to 1.9.0 (#4991)
Bumps [smallvec](https://github.com/servo/rust-smallvec) from 1.8.1 to 1.9.0.
- [Release notes](https://github.com/servo/rust-smallvec/releases)
- [Commits](https://github.com/servo/rust-smallvec/compare/v1.8.1...v1.9.0)

---
updated-dependencies:
- dependency-name: smallvec
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-06-30 09:25:20 +00:00
dependabot[bot] 65a6ba7d45
chore(deps): Bump either from 1.6.1 to 1.7.0 (#4990)
Bumps [either](https://github.com/bluss/either) from 1.6.1 to 1.7.0.
- [Release notes](https://github.com/bluss/either/releases)
- [Commits](https://github.com/bluss/either/compare/1.6.1...1.7.0)

---
updated-dependencies:
- dependency-name: either
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-06-30 08:21:20 +00:00
dependabot[bot] 4cf7bb8dee
chore(deps): Bump clap from 3.2.6 to 3.2.7 (#4989)
Bumps [clap](https://github.com/clap-rs/clap) from 3.2.6 to 3.2.7.
- [Release notes](https://github.com/clap-rs/clap/releases)
- [Changelog](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md)
- [Commits](https://github.com/clap-rs/clap/compare/v3.2.6...v3.2.7)

---
updated-dependencies:
- dependency-name: clap
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-06-30 08:04:57 +00:00
Marco Neumann 05189fdb00
fix: ensure panic is propagated throw `AdapterStream` (#4980)
Prior to this change background tasks that we feed into `AdapterStream`
can panic but that would just end the stream without any user-visible
error (except for the panic message on stdout/stderr).

This was found while developing #4964. I have proposed another fix in #4966
but found that I actually developed an existing solution a 2nd time:
`watch_task`. But I also see a major issue with the existing API: one
can create `AdapterStream` with ordinary tokio tasks that are not
watched at all, leaving the burden to the implementor to check for that
(and actually we forgot that in `parquet_file`).

So this change takes a slightly different approach:
The `AdapterStream` does NOT accept ordinary join handles any longer but
requires that you pass a "watched task". The newly introduced
`WatchedTask` does the same as we did manually before: wrapping a future
into a tokio task, watch it and wrap the watcher into a task.

It is now way more difficult to do anything stupid (sure you can still
mix up the tasks and the channels, but we need at least some flexibility
here to allow for "split" and potential future fan-in/out constructs).

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-06-30 07:57:58 +00:00
Raphael Taylor-Davies 835e1c91c7
chore: update object_store to 0.3.0 (#4707)
* chore: update object_store to 0.3.0

* chore: review feedback

Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>

Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-06-29 21:44:03 +00:00
Andrew Lamb 01fb2e132d
chore: Update datafusion pin (#4969)
* chore: Update datafusion pin

* fix: Update for api

* fix: Explicitly set coalsce batch size

* fix: Update batch size as well

* fix: update tests for new explain plan, and improved coercion
2022-06-29 17:52:37 +00:00
Markus Westerlind edf3f08e81 refactor: Replace all uses of lazy_static with once_cell
Went through and remove all lazy_static uses with once_cell (while waiting for the project to compile). There are still dependencies using lazy_static so it is still in the crate graph but at least there isn't an explicit dependency on it (and it is easier to update to `std::lazy::Lazy` once that is stable).
2022-06-29 16:22:02 +02:00
dependabot[bot] 7546476e15
chore(deps): Bump smallvec from 1.8.0 to 1.8.1 (#4947)
Bumps [smallvec](https://github.com/servo/rust-smallvec) from 1.8.0 to 1.8.1.
- [Release notes](https://github.com/servo/rust-smallvec/releases)
- [Commits](https://github.com/servo/rust-smallvec/compare/v1.8.0...v1.8.1)

---
updated-dependencies:
- dependency-name: smallvec
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-06-27 07:17:32 +00:00
Marco Neumann bd6c4659af
refactor: slim down parquet chunk (remove Metadata) (#4934)
* feat: conversion from `ParquetFile` to `ParquetFilePath`

* refactor: slim down parquet chunk

- ensure it works without binary parquet metadata
- timestamp range is no longer optional (ensured by the NG type system)
- remove table summary: this is only needed for SOME API users. The
  compactor can perfectly work without statistics since has the timestamp
  range which is sufficient for the current overlap check (we don't use
  any other primary key stats at the moment). The querier currently does
  NOT use parquet chunks (was replaced by read buffer) but if it will
  again in some future it will likely need to find a way to fetch and
  cache the statistics.
- the schema is now provided by the API user since it can be
  reconstructed using the NG catalog only (and "wrong" column orders are
  tolerated as of #4921)

Ref #4124

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-06-23 10:55:16 +00:00
dependabot[bot] 6f9d8b54cf
chore(deps): Bump integer-encoding from 3.0.3 to 3.0.4 (#4932)
Bumps [integer-encoding](https://github.com/dermesser/integer-encoding-rs) from 3.0.3 to 3.0.4.
- [Release notes](https://github.com/dermesser/integer-encoding-rs/releases)
- [Commits](https://github.com/dermesser/integer-encoding-rs/compare/v3.0.3...v3.0.4)

---
updated-dependencies:
- dependency-name: integer-encoding
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-06-23 09:42:34 +00:00
Andrew Lamb 776c34e03d
chore: Update datafusion (#4927)
* chore: Update datafusion

* fix: Update for API changes

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-06-23 09:30:43 +00:00
Marco Neumann 9591bed696
refactor: make querier internals private (#4922)
Queries internals are not meant to be used by other crates. Only a
handful selected interfaces should be used by IOxD and the query tests.

The compactor only used a very small subset just to read parquet files
back into memory. It shall rather use the official `parquet_file`
interface instead.

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-06-22 13:00:08 +00:00
dependabot[bot] f7d83ea581
chore(deps): Bump clap from 3.2.5 to 3.2.6 (#4920)
Bumps [clap](https://github.com/clap-rs/clap) from 3.2.5 to 3.2.6.
- [Release notes](https://github.com/clap-rs/clap/releases)
- [Changelog](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md)
- [Commits](https://github.com/clap-rs/clap/compare/v3.2.5...v3.2.6)

---
updated-dependencies:
- dependency-name: clap
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-06-22 10:28:44 +00:00
dependabot[bot] 50ffca791a
chore(deps): Bump indexmap from 1.9.0 to 1.9.1 (#4919)
Bumps [indexmap](https://github.com/bluss/indexmap) from 1.9.0 to 1.9.1.
- [Release notes](https://github.com/bluss/indexmap/releases)
- [Changelog](https://github.com/bluss/indexmap/blob/master/RELEASES.md)
- [Commits](https://github.com/bluss/indexmap/compare/1.9.0...1.9.1)

---
updated-dependencies:
- dependency-name: indexmap
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-06-22 10:18:06 +00:00
Andrew Lamb 087dbd3eca
fix: fix heappy + update docs (#4917)
* docs: Update heap profiling documentation

* fix: fix heappy builds

* fix: do not run cli tests with heappy

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-06-21 19:53:28 +00:00
Marco Neumann 1962fcc229
chore: reduce dependencies and run `cargo update` (#4906)
* chore: reduce proptest features

* chore: remove `grpc-router`

This crate is currently unused and we don't have immediate plans to use
it. And there's GIT, so it can always be restored.

* chore: `cargo update`
2022-06-20 12:18:28 +00:00
dependabot[bot] cc324613cc
chore(deps): Bump syn from 1.0.96 to 1.0.98 (#4905)
Bumps [syn](https://github.com/dtolnay/syn) from 1.0.96 to 1.0.98.
- [Release notes](https://github.com/dtolnay/syn/releases)
- [Commits](https://github.com/dtolnay/syn/compare/1.0.96...1.0.98)

---
updated-dependencies:
- dependency-name: syn
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-06-20 08:49:02 +00:00
dependabot[bot] 2ddc78bc41
chore(deps): Bump uuid from 1.0.0 to 1.1.2 (#4904)
Bumps [uuid](https://github.com/uuid-rs/uuid) from 1.0.0 to 1.1.2.
- [Release notes](https://github.com/uuid-rs/uuid/releases)
- [Commits](https://github.com/uuid-rs/uuid/compare/1.0.0...1.1.2)

---
updated-dependencies:
- dependency-name: uuid
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-06-20 08:34:33 +00:00
dependabot[bot] 8b28721627
chore(deps): Bump tower from 0.4.12 to 0.4.13 (#4903)
Bumps [tower](https://github.com/tower-rs/tower) from 0.4.12 to 0.4.13.
- [Release notes](https://github.com/tower-rs/tower/releases)
- [Commits](https://github.com/tower-rs/tower/compare/tower-0.4.12...tower-0.4.13)

---
updated-dependencies:
- dependency-name: tower
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-06-20 08:25:29 +00:00
Marco Neumann 0fbff981ec
chore(deps): Bump sqlx to 0.6.0 and uuid to 1 (#4894)
Closes #4889.
Closes #4890.

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-06-17 10:28:28 +00:00
dependabot[bot] d9ab157797
chore(deps): Bump indexmap from 1.8.2 to 1.9.0 (#4891)
* chore(deps): Bump indexmap from 1.8.2 to 1.9.0

Bumps [indexmap](https://github.com/bluss/indexmap) from 1.8.2 to 1.9.0.
- [Release notes](https://github.com/bluss/indexmap/releases)
- [Changelog](https://github.com/bluss/indexmap/blob/master/RELEASES.md)
- [Commits](https://github.com/bluss/indexmap/compare/1.8.2...1.9.0)

---
updated-dependencies:
- dependency-name: indexmap
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* chore: Run cargo hakari tasks

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
2022-06-17 07:42:36 +00:00
dependabot[bot] 73e1b3a47e
chore(deps): Bump clap from 3.2.4 to 3.2.5 (#4872)
Bumps [clap](https://github.com/clap-rs/clap) from 3.2.4 to 3.2.5.
- [Release notes](https://github.com/clap-rs/clap/releases)
- [Changelog](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md)
- [Commits](https://github.com/clap-rs/clap/compare/v3.2.4...v3.2.5)

---
updated-dependencies:
- dependency-name: clap
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-06-16 09:27:04 +00:00
Marco Neumann 66c7d95312
refactor: use new ingester<>querier wire protocol (#4867)
* refactor: use new ingester<>querier wire protocol

Use and document the new and more flexible ingester<>querier wire
protocol.

Note that the ingester does NOT stream the response data yet, but the
internal data structures would allow that. A follow-up change will
adjust the ingester code to stream the data.

Ref #4849.

* fix: typos

Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>

* refactor: clarify naming and public interface

* test: add schema assertion to `ingester_response_to_record_batches`

Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
2022-06-16 08:02:28 +00:00
Carol (Nichols || Goulding) 03f6f59a9b
fix: Change the sharder to return error instead of panicking for no shards 2022-06-15 11:23:31 -04:00
Carol (Nichols || Goulding) e9cdaffe74
fix: Create querier sharder from catalog sequencer info
Panic if there are no sharders in the catalog.
2022-06-15 10:18:54 -04:00
Carol (Nichols || Goulding) 874ef89daa
feat: Make specifying the write buffer, and thus getting a sharder, optional in querier 2022-06-15 10:01:45 -04:00
Carol (Nichols || Goulding) 127467b5c4
feat: Create a sharder in the querier 2022-06-15 10:01:45 -04:00
Carol (Nichols || Goulding) 148bc57e7b
refactor: Make the querier server constructor more like other server constructors 2022-06-15 10:01:45 -04:00
Carol (Nichols || Goulding) 6417e7dc2a
feat: Extract sharder to its own crate 2022-06-15 10:01:45 -04:00
dependabot[bot] 232dc897df
chore(deps): Bump clap from 3.2.1 to 3.2.4 (#4860)
Bumps [clap](https://github.com/clap-rs/clap) from 3.2.1 to 3.2.4.
- [Release notes](https://github.com/clap-rs/clap/releases)
- [Changelog](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md)
- [Commits](https://github.com/clap-rs/clap/compare/clap_complete-v3.2.1...v3.2.4)

---
updated-dependencies:
- dependency-name: clap
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-06-15 07:28:53 +00:00
Nga Tran b682dbbc2e
chore: Add debug info of sort_key for ingester (#4859)
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-06-14 20:39:17 +00:00
Andrew Lamb e91d00b10c
chore: Update datafusion + `arrow`/`parquet`/`arrow-flight` to `16.0.0 (#4851)
* chore: TEMP Update DataFusion to pre-release

* chore: update arrow et al to 16.0.0

* chore: Run cargo hakari tasks

* fix: update reader read_dictionary API

* chore: Update to real Datafusion release

* fix: Update parquet API

* fix: update test

Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
2022-06-14 16:31:40 +00:00
dependabot[bot] 23c9e38ea7
chore(deps): Bump clap from 3.1.18 to 3.2.1 (#4848)
* chore(deps): Bump clap from 3.1.18 to 3.2.1

Bumps [clap](https://github.com/clap-rs/clap) from 3.1.18 to 3.2.1.
- [Release notes](https://github.com/clap-rs/clap/releases)
- [Changelog](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md)
- [Commits](https://github.com/clap-rs/clap/compare/v3.1.18...clap_complete-v3.2.1)

---
updated-dependencies:
- dependency-name: clap
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* chore: fix clap deprecations

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Marco Neumann <marco@crepererum.net>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-06-14 15:42:18 +00:00
dependabot[bot] 4a94a21b4a
chore(deps): Bump getrandom from 0.2.6 to 0.2.7 (#4847)
Bumps [getrandom](https://github.com/rust-random/getrandom) from 0.2.6 to 0.2.7.
- [Release notes](https://github.com/rust-random/getrandom/releases)
- [Changelog](https://github.com/rust-random/getrandom/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-random/getrandom/compare/v0.2.6...v0.2.7)

---
updated-dependencies:
- dependency-name: getrandom
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-06-14 07:46:23 +00:00
Andrew Lamb 78ae4caf06
chore: Update DataFusion (#4842)
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-06-14 07:36:55 +00:00
dependabot[bot] 7381d4f489
chore(deps): Bump reqwest from 0.11.10 to 0.11.11 (#4843)
Bumps [reqwest](https://github.com/seanmonstar/reqwest) from 0.11.10 to 0.11.11.
- [Release notes](https://github.com/seanmonstar/reqwest/releases)
- [Changelog](https://github.com/seanmonstar/reqwest/blob/master/CHANGELOG.md)
- [Commits](https://github.com/seanmonstar/reqwest/compare/v0.11.10...v0.11.11)

---
updated-dependencies:
- dependency-name: reqwest
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-06-14 07:29:33 +00:00
Carol (Nichols || Goulding) 6b9a712ba8
fix: Use debug rather than dbg in test helpers (#4841) 2022-06-13 20:17:24 +00:00
Marco Neumann 2b84e5c087
feat: measure "probably reloaded" cache loads (#4813)
To roughly gauge how much data we re-load into cached (i.e. data that
was already loaded but was later evicted due to LRU pressure or TTL
eviction) this change introduces a new metric that estimates if a cache
entry that is requested from the loader was already seen before (using a
probabilistic filter).
2022-06-13 13:51:45 +00:00
Marco Neumann 66623fe0cd
feat: expose query semaphore metrics (#4836)
The groundwork for that was already done, just needed a bit of wiring.
This might help us to judge timeouts.
2022-06-13 09:36:50 +00:00
dependabot[bot] c4bd52ee6f
chore(deps): Bump handlebars from 4.3.0 to 4.3.1 (#4824)
Bumps [handlebars](https://github.com/sunng87/handlebars-rust) from 4.3.0 to 4.3.1.
- [Release notes](https://github.com/sunng87/handlebars-rust/releases)
- [Changelog](https://github.com/sunng87/handlebars-rust/blob/master/CHANGELOG.md)
- [Commits](https://github.com/sunng87/handlebars-rust/compare/v4.3.0...v4.3.1)

---
updated-dependencies:
- dependency-name: handlebars
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-06-10 11:04:19 +00:00
Marco Neumann 82f6696516
chore: remove some unused deps (#4803)
Found by `cargo udeps`.

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-06-08 11:15:18 +00:00
dependabot[bot] 3ecb1ee056
chore(deps): Bump http from 0.2.7 to 0.2.8 (#4796)
Bumps [http](https://github.com/hyperium/http) from 0.2.7 to 0.2.8.
- [Release notes](https://github.com/hyperium/http/releases)
- [Changelog](https://github.com/hyperium/http/blob/master/CHANGELOG.md)
- [Commits](https://github.com/hyperium/http/compare/v0.2.7...v0.2.8)

---
updated-dependencies:
- dependency-name: http
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-06-07 13:35:01 +00:00
dependabot[bot] 5d98988c9f
chore(deps): Bump tokio from 1.19.1 to 1.19.2 (#4795)
Bumps [tokio](https://github.com/tokio-rs/tokio) from 1.19.1 to 1.19.2.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/commits)

---
updated-dependencies:
- dependency-name: tokio
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-06-07 12:37:55 +00:00
dependabot[bot] 1fa4ef9bac
chore(deps): Bump tower-http from 0.3.3 to 0.3.4 (#4793)
Bumps [tower-http](https://github.com/tower-rs/tower-http) from 0.3.3 to 0.3.4.
- [Release notes](https://github.com/tower-rs/tower-http/releases)
- [Commits](https://github.com/tower-rs/tower-http/compare/tower-http-0.3.3...tower-http-0.3.4)

---
updated-dependencies:
- dependency-name: tower-http
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-06-07 11:03:00 +00:00
dependabot[bot] 20e832a5d1
chore(deps): Bump sqlparser from 0.17.0 to 0.18.0 (#4792)
Bumps [sqlparser](https://github.com/sqlparser-rs/sqlparser-rs) from 0.17.0 to 0.18.0.
- [Release notes](https://github.com/sqlparser-rs/sqlparser-rs/releases)
- [Changelog](https://github.com/sqlparser-rs/sqlparser-rs/blob/main/CHANGELOG.md)
- [Commits](https://github.com/sqlparser-rs/sqlparser-rs/compare/v0.17.0...v0.18.0)

---
updated-dependencies:
- dependency-name: sqlparser
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-06-07 10:00:53 +00:00
Andrew Lamb 8e96a2721d
chore: Update datafusion (again) (#4788)
* chore: Update datafusion

* chore: Update imports

* refactor: update API usage

* refactor: clean up some uses of binary_expr

* fix: remove unused export

* fix: update explain output

* chore: update more explain tests

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-06-07 08:17:56 +00:00
dependabot[bot] 04c685b3b7
chore(deps): Bump tokio-util from 0.7.2 to 0.7.3 (#4784)
Bumps [tokio-util](https://github.com/tokio-rs/tokio) from 0.7.2 to 0.7.3.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-util-0.7.2...tokio-util-0.7.3)

---
updated-dependencies:
- dependency-name: tokio-util
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-06-06 14:46:27 +00:00
dependabot[bot] a1ea793e13
chore(deps): Bump tokio-stream from 0.1.8 to 0.1.9 (#4785)
Bumps [tokio-stream](https://github.com/tokio-rs/tokio) from 0.1.8 to 0.1.9.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-stream-0.1.8...tokio-stream-0.1.9)

---
updated-dependencies:
- dependency-name: tokio-stream
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-06-06 14:21:54 +00:00
dependabot[bot] e03bf94420
chore(deps): Bump tokio from 1.18.2 to 1.19.1 (#4783)
Bumps [tokio](https://github.com/tokio-rs/tokio) from 1.18.2 to 1.19.1.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.18.2...tokio-1.19.1)

---
updated-dependencies:
- dependency-name: tokio
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-06-06 14:15:12 +00:00
Andrew Lamb 3592aa52d8
chore: Update datafusion + `arrow`/`parquet`/`arrow-flight` to `15.0.0` (#4743)
* chore: Update datafusion + `arrow`/`parquet`/`arrow-flight` to `15.0.0`

* chore: Update APIs

* chore: Run cargo hakari tasks

* feat: normalize parquet file metadata

* chore: update size tests

* chore: add docs on metadata stripping

* chore: TEMP UPDATE TO DF BRANCH

* chore: Update for new API

* fix: Update to latest DF

* fix: cargo hakari

Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: Raphael Taylor-Davies <r.taylordavies@googlemail.com>
2022-06-03 10:32:26 +00:00
dependabot[bot] 9a21292db8
chore(deps): Bump async-trait from 0.1.53 to 0.1.56 (#4774)
Bumps [async-trait](https://github.com/dtolnay/async-trait) from 0.1.53 to 0.1.56.
- [Release notes](https://github.com/dtolnay/async-trait/releases)
- [Commits](https://github.com/dtolnay/async-trait/compare/0.1.53...0.1.56)

---
updated-dependencies:
- dependency-name: async-trait
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-06-03 09:10:40 +00:00
dependabot[bot] 73a7e6f0a5
chore(deps): Bump syn from 1.0.95 to 1.0.96 (#4773)
Bumps [syn](https://github.com/dtolnay/syn) from 1.0.95 to 1.0.96.
- [Release notes](https://github.com/dtolnay/syn/releases)
- [Commits](https://github.com/dtolnay/syn/compare/1.0.95...1.0.96)

---
updated-dependencies:
- dependency-name: syn
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-06-03 08:02:56 +00:00
Marco Neumann 9e30a3eb29
refactor: rework querier concurrency limiting (#4760)
* refactor: rework querier concurrency limiting

With #4752 we introduced a concurrency limit into the querier. It works
by drawing permits from a central semaphore whenever we create a
`QuerierNamespace`. This however only limits concurrency during query
planning and not query execution, because the objects contained within
the plan (chunks and some metadata) neither reference the permit nor the
`QuerierNamespace`.

Now one approach to fix that would be to wire up the permit all the down
into all the query-related data structures. This however is very fiddly
and potentially will get lost at some point, because as soon as we
transform these data structures -- e.g. into streams -- the permit might
get lost again. This will be potentially query-dependent and very hard
to debug.

So instead we reverse the approach and track the permits at the upper
layer of the stack: the gRPC service entry points. There we also need to
be careful -- e.g. when we return streams to tonic -- but it's way
easier to review that then the deeply nested object hierarchy that is
involved with queries. Also the separation of concerns is a bit clearer,
because why would a "chunk" care about the "query concurrency" as a
whole.

* refactor: improve gRPC permit keeping and prepare tests
2022-06-02 09:49:58 +00:00
dependabot[bot] e638385782
chore(deps): Bump pbjson-types from 0.3.1 to 0.3.2 (#4750)
Bumps [pbjson-types](https://github.com/influxdata/pbjson) from 0.3.1 to 0.3.2.
- [Release notes](https://github.com/influxdata/pbjson/releases)
- [Commits](https://github.com/influxdata/pbjson/compare/0.3.1...0.3.2)

---
updated-dependencies:
- dependency-name: pbjson-types
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-06-01 09:02:37 +00:00
dependabot[bot] 21ec05a6ee
chore(deps): Bump pbjson from 0.3.1 to 0.3.2 (#4749)
Bumps [pbjson](https://github.com/influxdata/pbjson) from 0.3.1 to 0.3.2.
- [Release notes](https://github.com/influxdata/pbjson/releases)
- [Commits](https://github.com/influxdata/pbjson/compare/0.3.1...0.3.2)

---
updated-dependencies:
- dependency-name: pbjson
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-06-01 08:12:46 +00:00
dependabot[bot] 043dee43c8
chore(deps): Bump pbjson-build from 0.3.1 to 0.3.2 (#4748)
Bumps [pbjson-build](https://github.com/influxdata/pbjson) from 0.3.1 to 0.3.2.
- [Release notes](https://github.com/influxdata/pbjson/releases)
- [Commits](https://github.com/influxdata/pbjson/compare/0.3.1...0.3.2)

---
updated-dependencies:
- dependency-name: pbjson-build
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-06-01 07:54:13 +00:00
Marco Neumann c91dbe062e
test: "optimize" ingesterrecord batches in query tests (#4700)
* test: "optimize" ingesterrecord batches in query tests

It seems that I had the right idea in #4656 but wasn't able to trigger
https://github.com/influxdata/conductor/issues/955 because the query
tests do not "optimize" the record batches in the same way the actual
gRPC implementation does. If we apply the same transformation we indeed
end up with the same error.

* fix: all batches within the ingester flight response must have same schema

* refactor: simplify and reuse code

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-06-01 07:37:11 +00:00
Marco Neumann 988bd38e93
refactor: remove unused code (#4742) 2022-05-31 11:36:02 +00:00
Marco Neumann 5a95da7327
refactor: do NOT use ANY file IO for parquet reading (#4741) 2022-05-31 11:18:24 +00:00
dependabot[bot] 01fc550034
chore(deps): Bump parking_lot from 0.12.0 to 0.12.1 (#4732)
Bumps [parking_lot](https://github.com/Amanieu/parking_lot) from 0.12.0 to 0.12.1.
- [Release notes](https://github.com/Amanieu/parking_lot/releases)
- [Changelog](https://github.com/Amanieu/parking_lot/blob/master/CHANGELOG.md)
- [Commits](https://github.com/Amanieu/parking_lot/compare/0.12.0...0.12.1)

---
updated-dependencies:
- dependency-name: parking_lot
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-05-31 08:01:58 +00:00
dependabot[bot] 5b5f0efef5
chore(deps): Bump rayon from 1.5.2 to 1.5.3 (#4731)
Bumps [rayon](https://github.com/rayon-rs/rayon) from 1.5.2 to 1.5.3.
- [Release notes](https://github.com/rayon-rs/rayon/releases)
- [Changelog](https://github.com/rayon-rs/rayon/blob/master/RELEASES.md)
- [Commits](https://github.com/rayon-rs/rayon/compare/v1.5.2...v1.5.3)

---
updated-dependencies:
- dependency-name: rayon
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-05-31 07:50:42 +00:00
dependabot[bot] 642aef103d
chore(deps): Bump comfy-table from 5.0.1 to 6.0.0 (#4730)
Bumps [comfy-table](https://github.com/nukesor/comfy-table) from 5.0.1 to 6.0.0.
- [Release notes](https://github.com/nukesor/comfy-table/releases)
- [Changelog](https://github.com/Nukesor/comfy-table/blob/main/CHANGELOG.md)
- [Commits](https://github.com/nukesor/comfy-table/compare/v5.0.1...v6.0.0)

---
updated-dependencies:
- dependency-name: comfy-table
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-05-31 07:10:37 +00:00
Marco Neumann 79c054ffc9
fix: do NOT block in parquet file IO (#4727)
* fix: do NOT block in parquet file IO

I think for historical reason we were using blocking IO to read parquet
files. With the current streaming `SendableRecordStream` approach this
is technically NOT required anymore.

Now one might think that the sync-async dance that we did is kinda
harmless, but looking at our producition querier I think it is really
bad. The querier seems to be stuck but looking at `strace` and other
health signal it seems it is not entirely dead. Looking at GDB
backtraces it seems that nearly all threads are busy in
`download_and_scan_parquet`. Looking at the tokio docs
(<https://docs.rs/tokio/1.18.2/tokio/task/fn.spawn_blocking.html>)
for `spawn_blocking` (which is used to start the sync download) this
makes sense: tokio only starts replacement threads for the current
runtime thread (which calls `spawn_blocking`) if this does NOT exceed the
runtime thread limit. However we set the runtime thread limit to the
number of CPU cores available to IOx, so this is a limiting factor. This
means that there are only a few threads left to do actual work (I've
seen postgres data flowing back and forth for example) but tokio is not
able to use its full potential anymore. This is esp. bad because the
sync code in `download_and_scan_parquet` then uses `futures` `block_on`
functionality to call back into async code, so it waits for tokio
itself.

The change is rather simple: just use async task spawns.

* fix: use async IO to write stream to temp file

* fix: do not block tokio thread during parquet file reading

* refactor: ensure parquet IO tasks are cancelled if they are not needed anymore

There is no REAL way to cancel sync tasks, but at least we can try our
best.
2022-05-30 13:32:20 +00:00
dependabot[bot] 73168f7989
chore(deps): Bump flate2 from 1.0.23 to 1.0.24 (#4726)
Bumps [flate2](https://github.com/rust-lang/flate2-rs) from 1.0.23 to 1.0.24.
- [Release notes](https://github.com/rust-lang/flate2-rs/releases)
- [Commits](https://github.com/rust-lang/flate2-rs/commits)

---
updated-dependencies:
- dependency-name: flate2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-05-30 08:26:12 +00:00
dependabot[bot] 29069be7d4
chore(deps): Bump hyper from 0.14.18 to 0.14.19 (#4725)
Bumps [hyper](https://github.com/hyperium/hyper) from 0.14.18 to 0.14.19.
- [Release notes](https://github.com/hyperium/hyper/releases)
- [Changelog](https://github.com/hyperium/hyper/blob/master/CHANGELOG.md)
- [Commits](https://github.com/hyperium/hyper/compare/v0.14.18...v0.14.19)

---
updated-dependencies:
- dependency-name: hyper
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-05-30 07:58:12 +00:00
dependabot[bot] 7d4670e171
chore(deps): Bump indexmap from 1.8.1 to 1.8.2 (#4724)
Bumps [indexmap](https://github.com/bluss/indexmap) from 1.8.1 to 1.8.2.
- [Release notes](https://github.com/bluss/indexmap/releases)
- [Changelog](https://github.com/bluss/indexmap/blob/1.8.2/RELEASES.rst)
- [Commits](https://github.com/bluss/indexmap/compare/1.8.1...1.8.2)

---
updated-dependencies:
- dependency-name: indexmap
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-05-30 07:47:05 +00:00
Andrew Lamb cddd6d9b6d
chore: Update datafusion (#4723)
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-05-28 19:00:54 +00:00
Carol (Nichols || Goulding) 25b8260b72
feat: Implement ReadBufferCache::new 2022-05-25 17:19:10 -04:00
Marco Neumann 31d1b37d73
refactor: de-duplicate low-level arrow code (#4697)
It seems that during prototyping NG we've copied low level code (w/o
tests!) and never cleaned up. Let's not have this functionality twice.
2022-05-25 16:24:28 +00:00
Marko Mikulicic cdbe546e50
fix: return gRPC error on panic (#4686) 2022-05-25 07:06:25 +00:00
dependabot[bot] 24ee251080
chore(deps): Bump prost from 0.10.3 to 0.10.4 (#4688)
Bumps [prost](https://github.com/tokio-rs/prost) from 0.10.3 to 0.10.4.
- [Release notes](https://github.com/tokio-rs/prost/releases)
- [Commits](https://github.com/tokio-rs/prost/compare/v0.10.3...v0.10.4)

---
updated-dependencies:
- dependency-name: prost
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-05-25 06:07:05 +00:00
dependabot[bot] a8d3fe619c
chore(deps): Bump prost-build from 0.10.3 to 0.10.4 (#4687)
Bumps [prost-build](https://github.com/tokio-rs/prost) from 0.10.3 to 0.10.4.
- [Release notes](https://github.com/tokio-rs/prost/releases)
- [Commits](https://github.com/tokio-rs/prost/compare/v0.10.3...v0.10.4)

---
updated-dependencies:
- dependency-name: prost-build
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-05-25 06:00:27 +00:00
Andrew Lamb 95e6a8ed46
chore: Update datafusion (again) (#4679)
* chore: Update datafusion deps

* fix: fix for changes in ScalarValue

* fix: fix for using TableSource rather than TableProvider

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-05-24 15:54:39 +00:00
Luke Bond b76a0080d5
chore: remove unused iox_gitops_adapter (#4675)
* chore: remove unused iox_gitops_adapter

* chore: Run cargo hakari tasks

Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
2022-05-24 10:28:43 +00:00
dependabot[bot] ca49820a0f
chore(deps): Bump console-subscriber from 0.1.5 to 0.1.6 (#4670)
Bumps [console-subscriber](https://github.com/tokio-rs/console) from 0.1.5 to 0.1.6.
- [Release notes](https://github.com/tokio-rs/console/releases)
- [Commits](https://github.com/tokio-rs/console/compare/console-subscriber-v0.1.5...console-subscriber-v0.1.6)

---
updated-dependencies:
- dependency-name: console-subscriber
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-05-24 08:24:12 +00:00
dependabot[bot] 76f7043417
chore(deps): Bump once_cell from 1.11.0 to 1.12.0 (#4666)
Bumps [once_cell](https://github.com/matklad/once_cell) from 1.11.0 to 1.12.0.
- [Release notes](https://github.com/matklad/once_cell/releases)
- [Changelog](https://github.com/matklad/once_cell/blob/master/CHANGELOG.md)
- [Commits](https://github.com/matklad/once_cell/compare/v1.11.0...v1.12.0)

---
updated-dependencies:
- dependency-name: once_cell
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-05-24 08:14:03 +00:00
Marco Neumann 2029bd16ba
feat: enable debugging of failed querier->ingester requests (#4659)
* feat: enable debugging of failed querier->ingester requests

- extend `query-ingester` CLI to allow usage of predicates
- on failed requests: log all information that required for the CLI
- test the "ingester fails" scenario

* test: explain

Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>

* docs: improve

Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>

* refactor: move b64 pred. serde into a single crate

Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
2022-05-23 15:37:31 +00:00
Dom f0d0f1ba0c
Merge branch 'main' into dom/codec-object-store 2022-05-23 15:39:54 +01:00
dependabot[bot] 5c033b462e
chore(deps): Bump regex from 1.5.5 to 1.5.6 (#4655)
Bumps [regex](https://github.com/rust-lang/regex) from 1.5.5 to 1.5.6.
- [Release notes](https://github.com/rust-lang/regex/releases)
- [Changelog](https://github.com/rust-lang/regex/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/regex/compare/1.5.5...1.5.6)

---
updated-dependencies:
- dependency-name: regex
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-05-23 08:39:01 +00:00
dependabot[bot] 292f71759e
chore(deps): Bump http-body from 0.4.4 to 0.4.5 (#4654)
Bumps [http-body](https://github.com/hyperium/http-body) from 0.4.4 to 0.4.5.
- [Release notes](https://github.com/hyperium/http-body/releases)
- [Changelog](https://github.com/hyperium/http-body/blob/master/CHANGELOG.md)
- [Commits](https://github.com/hyperium/http-body/compare/v0.4.4...v0.4.5)

---
updated-dependencies:
- dependency-name: http-body
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-05-23 08:30:49 +00:00
dependabot[bot] 1bc02b1487
chore(deps): Bump regex-syntax from 0.6.25 to 0.6.26 (#4653)
Bumps [regex-syntax](https://github.com/rust-lang/regex) from 0.6.25 to 0.6.26.
- [Release notes](https://github.com/rust-lang/regex/releases)
- [Changelog](https://github.com/rust-lang/regex/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/regex/commits)

---
updated-dependencies:
- dependency-name: regex-syntax
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-05-23 08:21:39 +00:00
Dom Dwyer b9a745d42d feat: RecordBatch stream to Parquet file upload
Implements an upload() method on the ParquetStorage type, consuming a
stream of RecordBatch, serialising the Parquet file, and uploading the
result to object storage. Returns the IOx-specific file metadata.

Currently while the upload() method accepts a stream of RecordBatch, the
actual resulting Parquet file is buffered in memory before uploading to
object store, due to lack of streaming upload functionality in the
ObjectStore abstraction - this isn't the end of the world, as the files
tend to be relatively small with our current usage.

This impl should be easily modified to be fully streaming once streaming
object store puts are implemented:

    https://github.com/influxdata/object_store_rs/issues/9
2022-05-20 15:17:40 +01:00
Dom Dwyer 70856a645f feat: streaming RecordBatch -> parquet encoding
Implements a streaming RecordBatch to Parquet file serialiser.

This impl automatically discovers the schema of the RecordBatch stream,
and accepts &mut destination types (internalising the handle
cloning/etc) to simplify caller usage.

This encoder returns the resulting FileMetaData to allow callers to
inspect the resulting metadata without reading back the file.

Currently unused / not yet plumbed in.
2022-05-20 15:09:26 +01:00
dependabot[bot] 7010af30b7
chore(deps): Bump prometheus from 0.13.0 to 0.13.1 (#4648)
Bumps [prometheus](https://github.com/tikv/rust-prometheus) from 0.13.0 to 0.13.1.
- [Release notes](https://github.com/tikv/rust-prometheus/releases)
- [Changelog](https://github.com/tikv/rust-prometheus/blob/master/CHANGELOG.md)
- [Commits](https://github.com/tikv/rust-prometheus/compare/v0.13.0...v0.13.1)

---
updated-dependencies:
- dependency-name: prometheus
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-05-20 08:12:45 +00:00
dependabot[bot] 40a69c6e29
chore(deps): Bump pprof from 0.9.0 to 0.9.1 (#4647)
Bumps [pprof](https://github.com/tikv/pprof-rs) from 0.9.0 to 0.9.1.
- [Release notes](https://github.com/tikv/pprof-rs/releases)
- [Changelog](https://github.com/tikv/pprof-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/tikv/pprof-rs/commits)

---
updated-dependencies:
- dependency-name: pprof
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-05-20 08:04:46 +00:00
dependabot[bot] 6bc0c74c7d
chore(deps): Bump once_cell from 1.10.0 to 1.11.0 (#4646)
* chore(deps): Bump once_cell from 1.10.0 to 1.11.0

Bumps [once_cell](https://github.com/matklad/once_cell) from 1.10.0 to 1.11.0.
- [Release notes](https://github.com/matklad/once_cell/releases)
- [Changelog](https://github.com/matklad/once_cell/blob/master/CHANGELOG.md)
- [Commits](https://github.com/matklad/once_cell/compare/v1.10.0...v1.11.0)

---
updated-dependencies:
- dependency-name: once_cell
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* chore: Run cargo hakari tasks

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-05-20 07:40:38 +00:00
Marco Neumann addc45327e
fix: ensure that query tokio background tasks are canceled (#4643)
* fix: ensure that query tokio background tasks are canceled

While I am not entirely sure if this explains some of the memory leaks I
am seeing in prod, not canceling the tasks correctly certainly makes
debugging way harder and also renders certain form of throttling (e.g.
max. concurrent queries) somewhat ineffective.

Note that parquet file downloads are currently NOT canceled because
tokios `spawn_blocking` cannot be canceled.

* refactor: `Vec` -> `Option`

* refactor: `spawn_blocking` creates a join handle, even though it is useless

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-05-20 07:18:52 +00:00
Dom Dwyer baa86d846f refactor: use ParquetStore instead of ObjectStore
Changes the code paths that interact with Parquet files in the object
store to reference the ParquetStorage directly (DRY refactor).

This change takes us from a dependency graph of:

                    ┌─────────────────┐
                    │                 │
                    ▼                 │
            Parquet Consumer          │
                    │         ┌──────────────┐
                    ├────────▶│ParquetStorage│
                    ▼         └──────────────┘
            ┌──────────────┐
            │ ObjectStore  │
            └──────────────┘
                    │
               ┌────┴────┐
               ▼         ▼
             File       s3
            System    (etc)

to:

                Parquet Consumer
                        │
                        ▼
                ┌──────────────┐
                │ParquetStorage│
                └──────────────┘
                        │
                        ▼
                ┌──────────────┐
                │ ObjectStore  │
                └──────────────┘
                        │
                   ┌────┴────┐
                   ▼         ▼
                 File       s3
                System    (etc)

With the ParquetStorage being solely responsible for managing
interactions with the object store when dealing with Parquet files.
2022-05-19 13:52:51 +01:00
dependabot[bot] 409ae0ee0d
chore(deps): Bump handlebars from 4.2.2 to 4.3.0 (#4639)
Bumps [handlebars](https://github.com/sunng87/handlebars-rust) from 4.2.2 to 4.3.0.
- [Release notes](https://github.com/sunng87/handlebars-rust/releases)
- [Changelog](https://github.com/sunng87/handlebars-rust/blob/master/CHANGELOG.md)
- [Commits](https://github.com/sunng87/handlebars-rust/compare/v4.2.2...v4.3.0)

---
updated-dependencies:
- dependency-name: handlebars
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-05-19 08:15:51 +00:00
Marco Neumann 770293a973
feat: add LRU cache metrics (#4632)
* refactor: require `Resource`s to be convertible to `u64`

* refactor: require `Resource`s to have a unit name

* refactor: make LRU cache IDs static

* feat: add LRU cache metrics

* docs: improve type names in LRU doctest

* docs: epxlain `MeasuredT`

Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>

* docs: explain `test_metrics`

Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
2022-05-19 08:05:17 +00:00
Dom Dwyer 43300878bc fix(pb): encoding entirely NULL columns (#4272)
This commit changes the protobuf record batch encoding to skip entirely
NULL columns when serialising. This prevents the deserialisation from
erroring due to a column type inference failure.

Prior to this commit, when the system was presented a record batch such
as this:

            | time       | A    | B    |
            | ---------- | ---- | ---- |
            | 1970-01-01 | 1    | NULL |
            | 1970-07-05 | NULL | 1    |

Which would be partitioned by YMD into two separate partitions:

            | time       | A    | B    |
            | ---------- | ---- | ---- |
            | 1970-01-01 | 1    | NULL |

and:

            | time       | A    | B    |
            | ---------- | ---- | ---- |
            | 1970-07-05 | NULL | 1    |

Both partitions would contain an entirely NULL column.

Both of these partitioned record batches would be successfully encoded,
but decoding the partition fails due to the inability to infer a column
type from the serialised format which contains no values, which on the
wire, looks like:

            Column {
                column_name: "B",
                semantic_type: Field,
                values: Some(
                    Values {
                        i64_values: [],
                        f64_values: [],
                        u64_values: [],
                        string_values: [],
                        bool_values: [],
                        bytes_values: [],
                        packed_string_values: None,
                        interned_string_values: None,
                    },
                ),
                null_mask: [
                    1,
                ],
            },

In a column that is not entirely NULL, one of the "Values" fields would
be non-empty, and the decoder would use this to infer the type of the
column.

Because we have chosen to not differentiate between "NULL" and "empty"
in our proto encoding, the decoder cannot infer which field within the
"Values" struct the column belongs to - all are valid, but empty.

This commit prevents this type inference failure by skipping any columns
that are entirely NULL during serialisation, preventing the deserialiser
from having to process columns with ambiguous types.
2022-05-18 13:33:26 +01:00
dependabot[bot] c99ad9f4a4
chore(deps): Bump schemars from 0.8.9 to 0.8.10 (#4627)
Bumps [schemars](https://github.com/GREsau/schemars) from 0.8.9 to 0.8.10.
- [Release notes](https://github.com/GREsau/schemars/releases)
- [Changelog](https://github.com/GREsau/schemars/blob/master/CHANGELOG.md)
- [Commits](https://github.com/GREsau/schemars/compare/v0.8.9...v0.8.10)

---
updated-dependencies:
- dependency-name: schemars
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-05-18 10:23:34 +00:00
dependabot[bot] b542e453e8
chore(deps): Bump rustls from 0.20.4 to 0.20.6
Bumps [rustls](https://github.com/rustls/rustls) from 0.20.4 to 0.20.6.
- [Release notes](https://github.com/rustls/rustls/releases)
- [Changelog](https://github.com/rustls/rustls/blob/main/RELEASE_NOTES.md)
- [Commits](https://github.com/rustls/rustls/compare/v/0.20.4...v/0.20.6)

---
updated-dependencies:
- dependency-name: rustls
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-05-18 09:54:48 +00:00
dependabot[bot] db384ba387
chore(deps): Bump libc from 0.2.125 to 0.2.126 (#4626)
Bumps [libc](https://github.com/rust-lang/libc) from 0.2.125 to 0.2.126.
- [Release notes](https://github.com/rust-lang/libc/releases)
- [Commits](https://github.com/rust-lang/libc/compare/0.2.125...0.2.126)

---
updated-dependencies:
- dependency-name: libc
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-05-18 09:52:35 +00:00
Marco Neumann 52346642a0
ci: fix cargo deny (#4629)
* ci: fix cargo deny

* chore: downgrade `socket2`, version 0.4.5 was yanked

* chore: rename `query` to `iox_query`

`query` is already taken on crates.io and yanked and I am getting tired
of working around that.
2022-05-18 09:38:35 +00:00
Marco Neumann 12937ee724
feat: add SOCKS5 support to Kafka write buffer (#4623) 2022-05-17 15:21:35 +00:00
Andrew Lamb 3a33e806c7
chore: Update datafusion + `arrow`/`parquet`/`arrow-flight` to `14.0.0` (#4619)
* chore: Update datafusion deps

* chore: update arrow/parquet/arrow flight deps

* chore: Run cargo hakari tasks

* chore: Update location of utils

* chore: Update some more APIs

Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
2022-05-17 14:13:03 +00:00
dependabot[bot] 87d8a8b684
chore(deps): Bump schemars from 0.8.8 to 0.8.9 (#4614)
Bumps [schemars](https://github.com/GREsau/schemars) from 0.8.8 to 0.8.9.
- [Release notes](https://github.com/GREsau/schemars/releases)
- [Changelog](https://github.com/GREsau/schemars/blob/master/CHANGELOG.md)
- [Commits](https://github.com/GREsau/schemars/compare/v0.8.8...v0.8.9)

---
updated-dependencies:
- dependency-name: schemars
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-05-17 09:49:27 +00:00
dependabot[bot] 5a0006e40b
chore(deps): Bump syn from 1.0.94 to 1.0.95 (#4610)
Bumps [syn](https://github.com/dtolnay/syn) from 1.0.94 to 1.0.95.
- [Release notes](https://github.com/dtolnay/syn/releases)
- [Commits](https://github.com/dtolnay/syn/compare/1.0.94...1.0.95)

---
updated-dependencies:
- dependency-name: syn
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-05-17 09:40:36 +00:00
Carol (Nichols || Goulding) a91a9d5789
Merge remote-tracking branch 'origin/dependabot/cargo/kube-runtime-0.72.0' into cn/kube-update 2022-05-16 10:50:51 -04:00
Carol (Nichols || Goulding) 42b52da0d4
Merge remote-tracking branch 'origin/dependabot/cargo/kube-derive-0.72.0' into cn/kube-update 2022-05-16 10:49:51 -04:00
Carol (Nichols || Goulding) edd3cce73c
Merge remote-tracking branch 'origin/dependabot/cargo/kube-0.72.0' into cn/kube-update 2022-05-16 10:26:06 -04:00
dependabot[bot] 93d9db0ac8
chore(deps): Bump kube-derive from 0.71.0 to 0.72.0
Bumps [kube-derive](https://github.com/kube-rs/kube-rs) from 0.71.0 to 0.72.0.
- [Release notes](https://github.com/kube-rs/kube-rs/releases)
- [Changelog](https://github.com/kube-rs/kube-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/kube-rs/kube-rs/compare/0.71.0...0.72.0)

---
updated-dependencies:
- dependency-name: kube-derive
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-05-16 11:44:54 +00:00
dependabot[bot] 259d2486c1
chore(deps): Bump tokio-util from 0.7.1 to 0.7.2 (#4605)
Bumps [tokio-util](https://github.com/tokio-rs/tokio) from 0.7.1 to 0.7.2.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-util-0.7.1...tokio-util-0.7.2)

---
updated-dependencies:
- dependency-name: tokio-util
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-05-16 11:42:31 +00:00
dependabot[bot] f668a11f3e
chore(deps): Bump syn from 1.0.93 to 1.0.94 (#4606)
Bumps [syn](https://github.com/dtolnay/syn) from 1.0.93 to 1.0.94.
- [Release notes](https://github.com/dtolnay/syn/releases)
- [Commits](https://github.com/dtolnay/syn/compare/1.0.93...1.0.94)

---
updated-dependencies:
- dependency-name: syn
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-05-16 11:21:37 +00:00
dependabot[bot] 59a671125e
chore(deps): Bump rustls from 0.20.4 to 0.20.5 (#4602)
Bumps [rustls](https://github.com/rustls/rustls) from 0.20.4 to 0.20.5.
- [Release notes](https://github.com/rustls/rustls/releases)
- [Changelog](https://github.com/rustls/rustls/blob/main/RELEASE_NOTES.md)
- [Commits](https://github.com/rustls/rustls/compare/v/0.20.4...v/0.20.5)

---
updated-dependencies:
- dependency-name: rustls
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-05-16 11:13:40 +00:00
CircleCI[bot] ca19567710 chore: Run cargo hakari tasks 2022-05-16 01:38:59 +00:00
CircleCI[bot] ca1edc3b4d chore: Run cargo hakari tasks 2022-05-16 01:38:11 +00:00
dependabot[bot] 6ba1a5f253
chore(deps): Bump kube-runtime from 0.71.0 to 0.72.0
Bumps [kube-runtime](https://github.com/kube-rs/kube-rs) from 0.71.0 to 0.72.0.
- [Release notes](https://github.com/kube-rs/kube-rs/releases)
- [Changelog](https://github.com/kube-rs/kube-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/kube-rs/kube-rs/compare/0.71.0...0.72.0)

---
updated-dependencies:
- dependency-name: kube-runtime
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-05-16 01:37:57 +00:00
dependabot[bot] d7a0ca7710
chore(deps): Bump kube from 0.71.0 to 0.72.0
Bumps [kube](https://github.com/kube-rs/kube-rs) from 0.71.0 to 0.72.0.
- [Release notes](https://github.com/kube-rs/kube-rs/releases)
- [Changelog](https://github.com/kube-rs/kube-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/kube-rs/kube-rs/compare/0.71.0...0.72.0)

---
updated-dependencies:
- dependency-name: kube
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-05-16 01:37:13 +00:00
Andrew Lamb 22c1853f3e
chore: Update datafusion dependency (#4599)
* chore: Update datafusion dependency

* chore: Update for new API
2022-05-13 19:10:22 +00:00
Raphael Taylor-Davies f2bb0fdf77
feat: update to crates.io object_store version (#4595)
* feat: update to crates.io object_store version

* chore: Run cargo hakari tasks

* fix: tests

* chore: remove object store integration test plumbing

Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
2022-05-13 16:26:07 +00:00
Andrew Lamb 35d80fc512
refactor: rename `iox_catalog_service` to `service_grpc_catalog` for consistency (#4581)
* refactor: rename `iox_catalog_service` to `service_grpc_catalog`

* fix: fmt

* fix: alphabetize crate

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-05-13 14:07:58 +00:00
kodiakhq[bot] 542ec97b66
Merge branch 'main' into cn/rename-no-ng 2022-05-13 13:47:48 +00:00
Marco Neumann cb0a4176fd
refactor: move `querier::cache_system` into its own crate (#4592) 2022-05-13 13:12:07 +00:00
dependabot[bot] d15abc5e2d
chore(deps): Bump zstd-safe from 5.0.1+zstd.1.5.2 to 5.0.2+zstd.1.5.2 (#4591)
Bumps [zstd-safe](https://github.com/gyscos/zstd-rs) from 5.0.1+zstd.1.5.2 to 5.0.2+zstd.1.5.2.
- [Release notes](https://github.com/gyscos/zstd-rs/releases)
- [Commits](https://github.com/gyscos/zstd-rs/commits)

---
updated-dependencies:
- dependency-name: zstd-safe
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-05-13 07:27:24 +00:00
Carol (Nichols || Goulding) 48e6e5713d
fix: Rename test_helpers_end_to_end_ng to test_helpers_end_to_end
Connects to #4450.
2022-05-12 16:09:07 -04:00
Andrew Lamb 7eff52d04e
refactor: rename `iox_object_store_service` to `service_grpc_object_store` for consisntency (#4572)
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-05-12 14:22:18 +00:00
dependabot[bot] 51a1067425
chore(deps): Bump zstd from 0.11.1+zstd.1.5.2 to 0.11.2+zstd.1.5.2 (#4576)
Bumps [zstd](https://github.com/gyscos/zstd-rs) from 0.11.1+zstd.1.5.2 to 0.11.2+zstd.1.5.2.
- [Release notes](https://github.com/gyscos/zstd-rs/releases)
- [Commits](https://github.com/gyscos/zstd-rs/commits/v0.11.2)

---
updated-dependencies:
- dependency-name: zstd
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-05-12 13:53:10 +00:00
Nga Tran 0d5e3c97f0
chore: dump out the diff for SQL tests (#4571)
* chore: dump out the diff for SQL tests

* refactor: more prety assertion to a specific test to avoid affecting other available tests

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-05-11 18:43:36 +00:00
dependabot[bot] 3912328497
chore(deps): Bump pprof from 0.8.0 to 0.9.0 (#4562)
Bumps [pprof](https://github.com/tikv/pprof-rs) from 0.8.0 to 0.9.0.
- [Release notes](https://github.com/tikv/pprof-rs/releases)
- [Changelog](https://github.com/tikv/pprof-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/tikv/pprof-rs/commits)

---
updated-dependencies:
- dependency-name: pprof
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-05-11 18:30:49 +00:00
Andrew Lamb 381ad3b81d chore: Update heappy 2022-05-11 09:49:10 -04:00
dependabot[bot] 5c894db70f
chore(deps): Bump syn from 1.0.92 to 1.0.93 (#4563)
Bumps [syn](https://github.com/dtolnay/syn) from 1.0.92 to 1.0.93.
- [Release notes](https://github.com/dtolnay/syn/releases)
- [Commits](https://github.com/dtolnay/syn/compare/1.0.92...1.0.93)

---
updated-dependencies:
- dependency-name: syn
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-05-11 13:43:54 +00:00
dependabot[bot] 8a81bffee5
chore(deps): Bump clap from 3.1.17 to 3.1.18 (#4561)
Bumps [clap](https://github.com/clap-rs/clap) from 3.1.17 to 3.1.18.
- [Release notes](https://github.com/clap-rs/clap/releases)
- [Changelog](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md)
- [Commits](https://github.com/clap-rs/clap/compare/v3.1.17...v3.1.18)

---
updated-dependencies:
- dependency-name: clap
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-05-11 13:11:22 +00:00
Raphael Taylor-Davies 613cec4513
chore: remove azure git pin (#4564)
* chore: remove azure git pin

* chore: fix hakari

* chore: Run cargo hakari tasks

* chore: remove outdated comment

Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-05-11 10:21:17 +00:00
dependabot[bot] d4e1380ade
chore(deps): Bump sqlparser from 0.16.0 to 0.17.0 (#4560)
Bumps [sqlparser](https://github.com/sqlparser-rs/sqlparser-rs) from 0.16.0 to 0.17.0.
- [Release notes](https://github.com/sqlparser-rs/sqlparser-rs/releases)
- [Changelog](https://github.com/sqlparser-rs/sqlparser-rs/blob/main/CHANGELOG.md)
- [Commits](https://github.com/sqlparser-rs/sqlparser-rs/compare/v0.16.0...v0.17.0)

---
updated-dependencies:
- dependency-name: sqlparser
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-05-11 10:09:22 +00:00
Andrew Lamb b8cb4c3f2b
feat: Interrogate schema from querier (as well as router) (#4557)
* refactor: move SchemaService into `service_grpc_schema`

* feat: implement schema gRPC for querier

* chore: Run cargo hakari tasks

Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
2022-05-10 20:55:58 +00:00
Raphael Taylor-Davies 99b1a9b83f
refactor: split out ObjectStoreMetrics (#4547)
* refactor: split out ObjectStoreMetrics

* chore: add workspace hack

* fix: compile
2022-05-10 10:56:28 +00:00
Andrew Lamb 9b00e4c0da
chore: update deps (#4545)
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-05-09 18:18:02 +00:00
dependabot[bot] 2a15b525a9
chore(deps): Bump clap from 3.1.12 to 3.1.17 (#4537)
* chore(deps): Bump clap from 3.1.12 to 3.1.17

Bumps [clap](https://github.com/clap-rs/clap) from 3.1.12 to 3.1.17.
- [Release notes](https://github.com/clap-rs/clap/releases)
- [Changelog](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md)
- [Commits](https://github.com/clap-rs/clap/compare/v3.1.12...v3.1.17)

---
updated-dependencies:
- dependency-name: clap
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* chore: update lockfile

* chore: Run cargo hakari tasks

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
Co-authored-by: Raphael Taylor-Davies <r.taylordavies@googlemail.com>
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
2022-05-09 10:13:28 +00:00
kodiakhq[bot] a12172dfe8
Merge branch 'main' into cn/delete-more 2022-05-09 08:47:03 +00:00
dependabot[bot] 1c7ef3a3f8
chore(deps): Bump memchr from 2.4.1 to 2.5.0 (#4542)
Bumps [memchr](https://github.com/BurntSushi/memchr) from 2.4.1 to 2.5.0.
- [Release notes](https://github.com/BurntSushi/memchr/releases)
- [Commits](https://github.com/BurntSushi/memchr/compare/2.4.1...2.5.0)

---
updated-dependencies:
- dependency-name: memchr
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-05-09 08:46:44 +00:00
dependabot[bot] 47d35beaf5
chore(deps): Bump croaring from 0.6.0 to 0.6.1 (#4541)
Bumps [croaring](https://github.com/saulius/croaring-rs) from 0.6.0 to 0.6.1.
- [Release notes](https://github.com/saulius/croaring-rs/releases)
- [Commits](https://github.com/saulius/croaring-rs/compare/0.6.0...0.6.1)

---
updated-dependencies:
- dependency-name: croaring
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-05-09 08:40:08 +00:00
dependabot[bot] 83aea40c9a
chore(deps): Bump num-integer from 0.1.44 to 0.1.45 (#4539)
Bumps [num-integer](https://github.com/rust-num/num-integer) from 0.1.44 to 0.1.45.
- [Release notes](https://github.com/rust-num/num-integer/releases)
- [Changelog](https://github.com/rust-num/num-integer/blob/master/RELEASES.md)
- [Commits](https://github.com/rust-num/num-integer/compare/num-integer-0.1.44...num-integer-0.1.45)

---
updated-dependencies:
- dependency-name: num-integer
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-05-09 08:33:10 +00:00
dependabot[bot] 07ceb5a9cd
chore(deps): Bump log from 0.4.16 to 0.4.17 (#4540)
Bumps [log](https://github.com/rust-lang/log) from 0.4.16 to 0.4.17.
- [Release notes](https://github.com/rust-lang/log/releases)
- [Changelog](https://github.com/rust-lang/log/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/log/commits/0.4.17)

---
updated-dependencies:
- dependency-name: log
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-05-09 08:26:29 +00:00
dependabot[bot] 65189f56f4
chore(deps): Bump cloud-storage from 0.11.0 to 0.11.1 (#4538)
Bumps [cloud-storage](https://github.com/ThouCheese/cloud-storage-rs) from 0.11.0 to 0.11.1.
- [Release notes](https://github.com/ThouCheese/cloud-storage-rs/releases)
- [Changelog](https://github.com/ThouCheese/cloud-storage-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/ThouCheese/cloud-storage-rs/commits)

---
updated-dependencies:
- dependency-name: cloud-storage
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-05-09 08:19:08 +00:00
dependabot[bot] f420034bb7
chore(deps): Bump libc from 0.2.124 to 0.2.125 (#4536)
Bumps [libc](https://github.com/rust-lang/libc) from 0.2.124 to 0.2.125.
- [Release notes](https://github.com/rust-lang/libc/releases)
- [Commits](https://github.com/rust-lang/libc/compare/0.2.124...0.2.125)

---
updated-dependencies:
- dependency-name: libc
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-05-09 08:08:21 +00:00
dependabot[bot] 3f5a3e3545
chore(deps): Bump tokio from 1.18.1 to 1.18.2 (#4535)
Bumps [tokio](https://github.com/tokio-rs/tokio) from 1.18.1 to 1.18.2.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.18.1...tokio-1.18.2)

---
updated-dependencies:
- dependency-name: tokio
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-05-09 08:01:20 +00:00
Jake Goulding e07bcd40c2 refactor: Remove unused dependencies
These were found by iterating over all of the dependencies of each
Cargo.toml, then grepping that crate for the dependency's name. If it
didn't show up, I attempted to remove it.

I left a few dependencies that this process flagged:

* generated_types
  - `pbjson`,`serde`. Apparently used by the generated code.

* grpc-router-test-gen
  - `prost`. Apparently used by the generated code.

* influxdb_iox
  - `heappy`. Doesn't appear used, but is behind enough feature
    flags that I don't care to reason about and it's already optional.
  - `tikv_jemalloc_sys`. Appears to be setting a feature flag of an
    indirect dependency.

* iox_gitops_adapter
  - `k8s_openapi`. Appears to be setting a feature flag of an indirect
    dependency.
2022-05-06 15:57:58 -04:00
Jake Goulding 90502c844b refactor: Re-export crates used by the ioxd_common macros
This allows the consumers of the macros to avoid including the
dependencies themselves, which they otherwise wouldn't need to.
2022-05-06 15:57:58 -04:00
Carol (Nichols || Goulding) 1759443a13
fix: Remove unused dependencies in influxdb_iox found by manual inspection 2022-05-06 14:51:54 -04:00
Carol (Nichols || Goulding) 6681298a93
fix: Remove unused dependencies found with cargo-udeps 2022-05-06 14:51:54 -04:00
Carol (Nichols || Goulding) fcd4815645
fix: Rename router2 to router 2022-05-06 14:51:52 -04:00
Carol (Nichols || Goulding) 0650a9bb77
fix: Rename ioxd_router2 to ioxd_router 2022-05-06 14:45:39 -04:00
Carol (Nichols || Goulding) 068096e7e1
fix: Rename data_types2 to data_types 2022-05-06 14:45:39 -04:00
Carol (Nichols || Goulding) e1bef1c218
fix: Remove OG data_types crate 2022-05-06 14:45:39 -04:00
Carol (Nichols || Goulding) 0541c6e40f
fix: Remove data_types crate where it's no longer used 2022-05-06 14:45:39 -04:00
Carol (Nichols || Goulding) bf1d08333d
fix: Move ChunkColumnSummary to read_buffer, the only place it's used 2022-05-06 14:45:39 -04:00
Carol (Nichols || Goulding) d7304c1114
fix: Move TimestampSummary to the only place it's used 2022-05-06 14:45:38 -04:00