Commit Graph

975 Commits (f01382f578703c3203936e13014e1827607ca826)

Author SHA1 Message Date
Dom Dwyer bfc473b8cd feat: object store instrumentation
Implements a decorator of the ObjectStoreAPI trait, recording:

    * Bytes uploaded / downloaded through the instrumented API
    * Call latencies, broken down by operation & success / error state

All the current implementations that return a Stream from the get
operation actually return a "fake" stream containing all the data in one
go rather than streaming chunks from the upstream. I've instrumented the
Stream to future-proof any actual streaming impls in the future.
2022-03-11 13:38:48 +00:00
dependabot[bot] bd2824aa59
chore(deps): Bump pprof from 0.6.2 to 0.7.0 (#4004)
* chore(deps): Bump pprof from 0.6.2 to 0.7.0

Bumps [pprof](https://github.com/tikv/pprof-rs) from 0.6.2 to 0.7.0.
- [Release notes](https://github.com/tikv/pprof-rs/releases)
- [Changelog](https://github.com/tikv/pprof-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/tikv/pprof-rs/commits)

---
updated-dependencies:
- dependency-name: pprof
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* fix: `pprof` features for version 0.7

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Marco Neumann <marco@crepererum.net>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-03-11 11:08:59 +00:00
Nga Tran f03ebd79ab
refactor: move querier's test utils to a new crate to get reused by tests in other crates (#4013)
* refactor: move querier's test utils to a new crate to be able resued by tests in other crates

* chore: remove unused import
2022-03-10 18:17:58 +00:00
dependabot[bot] 67209fce7c
chore(deps): Bump clap from 3.1.5 to 3.1.6 (#4010)
Bumps [clap](https://github.com/clap-rs/clap) from 3.1.5 to 3.1.6.
- [Release notes](https://github.com/clap-rs/clap/releases)
- [Changelog](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md)
- [Commits](https://github.com/clap-rs/clap/compare/v3.1.5...v3.1.6)

---
updated-dependencies:
- dependency-name: clap
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-03-10 14:38:53 +00:00
dependabot[bot] e1344f7ca8
chore(deps): Bump sqlparser from 0.14.0 to 0.15.0 (#4007)
Bumps [sqlparser](https://github.com/sqlparser-rs/sqlparser-rs) from 0.14.0 to 0.15.0.
- [Release notes](https://github.com/sqlparser-rs/sqlparser-rs/releases)
- [Changelog](https://github.com/sqlparser-rs/sqlparser-rs/blob/main/CHANGELOG.md)
- [Commits](https://github.com/sqlparser-rs/sqlparser-rs/compare/v0.14.0...v0.15.0)

---
updated-dependencies:
- dependency-name: sqlparser
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-03-10 14:28:48 +00:00
dependabot[bot] 68e09264f2
chore(deps): Bump pbjson from 0.2.2 to 0.2.3 (#4006)
Bumps [pbjson](https://github.com/influxdata/pbjson) from 0.2.2 to 0.2.3.
- [Release notes](https://github.com/influxdata/pbjson/releases)
- [Commits](https://github.com/influxdata/pbjson/compare/0.2.2...0.2.3)

---
updated-dependencies:
- dependency-name: pbjson
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-03-10 14:20:10 +00:00
dependabot[bot] f2834c4101
chore(deps): Bump handlebars from 4.2.1 to 4.2.2 (#4009)
Bumps [handlebars](https://github.com/sunng87/handlebars-rust) from 4.2.1 to 4.2.2.
- [Release notes](https://github.com/sunng87/handlebars-rust/releases)
- [Changelog](https://github.com/sunng87/handlebars-rust/blob/master/CHANGELOG.md)
- [Commits](https://github.com/sunng87/handlebars-rust/compare/v4.2.1...v4.2.2)

---
updated-dependencies:
- dependency-name: handlebars
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-03-10 14:11:19 +00:00
dependabot[bot] c051e944f2
chore(deps): Bump tracing from 0.1.31 to 0.1.32 (#4008)
Bumps [tracing](https://github.com/tokio-rs/tracing) from 0.1.31 to 0.1.32.
- [Release notes](https://github.com/tokio-rs/tracing/releases)
- [Commits](https://github.com/tokio-rs/tracing/compare/tracing-0.1.31...tracing-0.1.32)

---
updated-dependencies:
- dependency-name: tracing
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-03-10 14:01:00 +00:00
dependabot[bot] cef4090bce
chore(deps): Bump pbjson-types from 0.2.2 to 0.2.3 (#4005)
Bumps [pbjson-types](https://github.com/influxdata/pbjson) from 0.2.2 to 0.2.3.
- [Release notes](https://github.com/influxdata/pbjson/releases)
- [Commits](https://github.com/influxdata/pbjson/compare/0.2.2...0.2.3)

---
updated-dependencies:
- dependency-name: pbjson-types
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-03-10 13:52:08 +00:00
dependabot[bot] ed84b453d3
chore(deps): Bump tracing-core from 0.1.22 to 0.1.23 (#4003)
Bumps [tracing-core](https://github.com/tokio-rs/tracing) from 0.1.22 to 0.1.23.
- [Release notes](https://github.com/tokio-rs/tracing/releases)
- [Commits](https://github.com/tokio-rs/tracing/compare/tracing-core-0.1.22...tracing-core-0.1.23)

---
updated-dependencies:
- dependency-name: tracing-core
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-03-10 13:43:15 +00:00
dependabot[bot] 0756642137
chore(deps): Bump pbjson-build from 0.2.2 to 0.2.3 (#4002)
Bumps [pbjson-build](https://github.com/influxdata/pbjson) from 0.2.2 to 0.2.3.
- [Release notes](https://github.com/influxdata/pbjson/releases)
- [Commits](https://github.com/influxdata/pbjson/compare/0.2.2...0.2.3)

---
updated-dependencies:
- dependency-name: pbjson-build
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-03-10 13:34:00 +00:00
Andrew Lamb 2c3d30ca32
chore: Update datafusion, arrow, flight and parquet (#4000)
* chore: Update datafusion, arrow, flight and parquet

* fix: api change

* fix: fmt

* fix: update test metadata size

* fix: Update sizes in parquet test

* fix: more metadata size update
2022-03-10 12:24:47 +00:00
Carol (Nichols || Goulding) 944f628e29
fix: Remove data_types as a dependency of ng compactor (#3993)
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-03-09 17:03:02 +00:00
Carol (Nichols || Goulding) 93b0cdbcc4
fix: Create the test database as part of ng server fixture startup 2022-03-09 09:55:43 -05:00
Andrew Lamb d2c0acdd46
refactor: Remove serving readiness gate (#3986)
* refactor: Remove serving_readiness

* fix: remove more

* fix: remove test

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-03-09 12:17:44 +00:00
Nga Tran 09fba1d2c0
feat: NG Compactor - main function for finding and compacting parquet files (#3973)
* feat: main function for finding and compacting parquet files

* chore: Apply suggestions from code review

Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>

* refactor: rename file and struct

Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-03-08 16:34:43 +00:00
Dom Dwyer 3278e35b41 build: bump regex
Bump regex version due to RUSTSEC-2022-0013, see:

    https://rustsec.org/advisories/RUSTSEC-2022-0013
2022-03-08 15:37:23 +00:00
Dom Dwyer baae6910f2 build: remove openssl dependency
Switches a stray reqwest dependency to use rustls.
2022-03-07 13:47:39 +00:00
kodiakhq[bot] e47bfc40be
Merge branch 'main' into dom/router-grpc-write 2022-03-07 11:13:19 +00:00
dependabot[bot] d3321135eb
chore(deps): Bump clap from 3.1.2 to 3.1.5 (#3957)
Bumps [clap](https://github.com/clap-rs/clap) from 3.1.2 to 3.1.5.
- [Release notes](https://github.com/clap-rs/clap/releases)
- [Changelog](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md)
- [Commits](https://github.com/clap-rs/clap/compare/v3.1.2...v3.1.5)

---
updated-dependencies:
- dependency-name: clap
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-03-07 09:58:46 +00:00
dependabot[bot] 3c58a08431
chore(deps): Bump siphasher from 0.3.9 to 0.3.10 (#3956)
Bumps [siphasher](https://github.com/jedisct1/rust-siphash) from 0.3.9 to 0.3.10.
- [Release notes](https://github.com/jedisct1/rust-siphash/releases)
- [Commits](https://github.com/jedisct1/rust-siphash/compare/0.3.9...0.3.10)

---
updated-dependencies:
- dependency-name: siphasher
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-03-07 09:10:09 +00:00
dependabot[bot] 48908054d1
chore(deps): Bump once_cell from 1.9.0 to 1.10.0 (#3955)
* chore(deps): Bump once_cell from 1.9.0 to 1.10.0

Bumps [once_cell](https://github.com/matklad/once_cell) from 1.9.0 to 1.10.0.
- [Release notes](https://github.com/matklad/once_cell/releases)
- [Changelog](https://github.com/matklad/once_cell/blob/master/CHANGELOG.md)
- [Commits](https://github.com/matklad/once_cell/compare/v1.9.0...v1.10.0)

---
updated-dependencies:
- dependency-name: once_cell
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* chore: Run cargo hakari tasks

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
2022-03-07 08:54:20 +00:00
Andrew Lamb e09f39d6a0
chore: Update datafusion (#3943)
* chore: Update datafusion

* refactor: update for new datafusion

* chore: Run cargo hakari tasks

Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
2022-03-04 19:37:46 +00:00
Andrew Lamb b870b9340b
chore: remove uneeded dependencies (#3929)
* chore: remove unused deps in compactor

* chore: remove unused deps in influxdb_ioxd

* chore: remove unused deps in object_store

* chore: remove unused deps in server

* fix: object_store needs observability deps when compiled with aws

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-03-04 17:39:46 +00:00
Dom Dwyer 26c43f0a2c feat: grpc write handler
Implements the gRPC write handler endpoint.
2022-03-04 14:51:42 +00:00
Marco Neumann 8d00aaba90
feat: sync chunks in querier (#3911)
* feat: `ParquetFileRepo::list_by_namespace_not_to_delete`

* feat: `ChunkAddr: Clone`

* test: ensure that querier keeps same partition objects

* test: improve `create_parquet_file` flexibility

* feat: sync chunks in querier

* test: improve `test_parquet_file`
2022-03-04 08:53:39 +00:00
Andrew Lamb d6afda0227
refactor: split `influxdb_ioxd`, `clap_blocks` and `serving_readiness` from influxdb_iox crate (#3908)
* refactor: split influxdb_ioxd, clap_blocks, and serving_readiness out of influxdb_iox

split out serving readiness, get compiling

* fix: hakari

* fix: hakari again

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-03-03 16:48:30 +00:00
kodiakhq[bot] 04a7a957fe
Merge branch 'main' into dom/rustls 2022-03-03 11:59:40 +00:00
Dom Dwyer da145ffbe4 feat: batch upsert of columns to catalog
Adds ColumnRepo::create_or_get_many() to upsert multiple columns in one
round trip to Postgres.
2022-03-03 11:17:30 +00:00
Dom Dwyer 46bb107be4 refactor: use rustls
Removes openssl as a dependency, switching to rustls[1] as the TLS
implementation throughout.

It is important to note that this change brings with it a significant
behavioural difference - rustls does not currently support IP SANs in
certificates (instead only supporting fully-qualified names / DNS) and
this will manifest as a failure to connect to IP endpoints over TLS.
This might be a blocker that prevents us using rustls exclusively, but
there's noe asy way to know without trying it. Fortunately the rustls
project has received funding to work on IP SAN support[2].

[1]: https://github.com/rustls/rustls
[2]: https://www.abetterinternet.org/post/preparing-rustls-for-wider-adoption/
2022-03-03 11:05:20 +00:00
Dom Dwyer 4785a7d028 refactor: bump Azure SDK
Update to the latest Azure SDK to pick up rustls support.
2022-03-03 11:02:20 +00:00
Raphael Taylor-Davies 02a2cfde54
feat: update rskafka (#3805) (#3910) 2022-03-03 10:53:47 +00:00
Carol (Nichols || Goulding) 3f2a58b47f
refactor: pub use data_types from data_types2
So it's clearer which parts of data_types the NG design is using, and
which types can be cleaned up eventually.
2022-03-02 13:55:31 -05:00
Carol (Nichols || Goulding) 2a90841715
refactor: Move IngesterQueryRequest to data_types2
So that querier doesn't need to depend on ingester.
2022-03-02 13:52:13 -05:00
Carol (Nichols || Goulding) 8f3e44bf76
refactor: Extract a crate for shared data types in the new design 2022-03-02 12:16:15 -05:00
Carol (Nichols || Goulding) 141a6087d0
feat: Querier able to send Flight queries to Ingester
Fixes #3773.
2022-03-02 11:50:45 -05:00
Raphael Taylor-Davies 44cfdc7aca
feat: update to latest rskafka logging (#3805) (#3896) 2022-03-02 11:05:03 +00:00
Marco Neumann af57664f53
feat: wire up flight into the querier (#3889) 2022-03-02 09:20:26 +00:00
Marco Neumann 936f51013d
test: querier shutdown and background task handling (#3881)
This is similar to what we already have for the ingester.
2022-03-02 08:59:50 +00:00
Raphael Taylor-Davies 143311f63f
feat: additional write buffer logging (#3805) (#3887)
* feat: additional write buffer logging (#3805)

* fix: assertion direction

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-03-01 17:08:57 +00:00
Nga Tran 0e0dc500f6
feat: prepare data to send to querier (#3825)
* feat: changes needed to apply tombstones correctly on the life-cycle ingest bacthes

* refactor: adjust the  design after discussing with Paul

* feat: apply the coming tombstone on all data but persiting one

* chore: fmt

* fix: build on buffer tombstone

* test: delete & write tests for a parition and some cleanup

* feat: No need add processed tombstones for newly created parquet file in the ingester becasue all deletes before that parquet file is created were applied

* chore: cleanup

* feat: intitial implementation for preparing data to send back to the Querier

* feat: full implementation of prepare_data_to_querier

* fix: apply filters for the batches

* chore: Apply suggestions from code review

Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>

* chore: cleanup

* fix: typos in comments

* fix: typos in comments

* fix: typos in comments

* test: create different scenarios and test them

* chore: fix typos

* test: add tests with deletes

* chore: make pub pub(crate)

* chore: Apply suggestions from code review

Co-authored-by: Jake Goulding <jake.goulding@integer32.com>

* refactor: address review comments

* fix: keep batches in their arrival order

* refactor: not assign unecessary values to enum

* refactor: use bitflags enum

* fix: use bitflags correctly

* chore: Apply suggestions from code review

Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>

* refactor: avoid using use at the end of the function

* chore: merge main to branch

* fix: fix downgrade versions

* refactor: address review comments

* chore: remove unnecessary comments

* refactor: Make the whole test_utils module test-only and bring paths into module scope

Co-authored-by: Paul Dix <paul@pauldix.net>
Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
Co-authored-by: Jake Goulding <jake.goulding@integer32.com>
Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
Co-authored-by: Carol (Nichols || Goulding) <carol.nichols@gmail.com>
2022-03-01 01:00:45 +00:00
Marco Neumann 6e2470bf5f
feat: create CatalogChunk in querier (#3862)
* feat: `NamespaceRepo::get_by_id`

* feat: create `CatalogChunk` in querier
2022-02-28 17:20:38 +00:00
Marco Neumann b213796c98
feat: sync namespaces in querier (#3865)
* feat: `NamespaceRepo::list`

* feat: sync namespaces in querier

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-02-28 15:01:28 +00:00
Marco Neumann 77a0c74830
chore: `cargo update` + less rustsec exceptions (#3873)
It seems that dependabot doesn't catch with the flood of updates, so
this is a manual update run:

```console
$ cargo update
    Updating crates.io index
    Updating git repository `https://github.com/Azure/azure-sdk-for-rust.git`
    Updating git repository `https://github.com/apache/arrow-datafusion.git`
    Updating git repository `https://github.com/influxdata/rskafka.git`
    Updating git repository `https://github.com/mkmik/heappy`
    Updating anyhow v1.0.53 -> v1.0.55
    Updating autocfg v1.0.1 -> v1.1.0
    Updating clang-sys v1.3.0 -> v1.3.1
    Updating clap_derive v3.1.0 -> v3.1.2
    Updating core-foundation v0.9.2 -> v0.9.3
    Updating crossbeam-epoch v0.9.6 -> v0.9.7
    Updating crossbeam-queue v0.3.3 -> v0.3.4
    Updating crossbeam-utils v0.8.6 -> v0.8.7
    Updating debugid v0.7.2 -> v0.7.3
    Updating fd-lock v3.0.3 -> v3.0.4
    Updating inferno v0.10.10 -> v0.10.12
    Updating io-lifetimes v0.4.4 -> v0.5.3
    Updating kube-client v0.69.0 -> v0.69.1
    Updating kube-core v0.69.0 -> v0.69.1
    Updating libm v0.2.1 -> v0.2.2
    Updating linux-raw-sys v0.0.37 -> v0.0.42
    Removing memmap v0.7.0
      Adding memmap2 v0.5.3
    Updating ntapi v0.3.6 -> v0.3.7
    Updating output_vt100 v0.1.2 -> v0.1.3
    Updating rgb v0.8.31 -> v0.8.32
    Updating rustix v0.32.1 -> v0.33.3
    Updating rustls v0.20.2 -> v0.20.4
    Updating security-framework v2.6.0 -> v2.6.1
    Updating security-framework-sys v2.6.0 -> v2.6.1
    Updating semver v1.0.4 -> v1.0.6
    Updating symbolic-common v8.5.0 -> v8.6.1
    Updating symbolic-demangle v8.5.0 -> v8.6.1
    Updating tower-http v0.2.2 -> v0.2.3
    Updating unicode-segmentation v1.8.0 -> v1.9.0
    Updating zeroize v1.5.2 -> v1.5.3
```

This also allows us to remove a rustsec exception.
2022-02-28 11:24:32 +00:00
dependabot[bot] a9d52aaef3
chore(deps): Bump mockito from 0.30.0 to 0.31.0 (#3871)
* chore(deps): Bump mockito from 0.30.0 to 0.31.0

Bumps [mockito](https://github.com/lipanski/mockito) from 0.30.0 to 0.31.0.
- [Release notes](https://github.com/lipanski/mockito/releases)
- [Commits](https://github.com/lipanski/mockito/compare/0.30.0...0.31.0)

---
updated-dependencies:
- dependency-name: mockito
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* chore: disallow `RUSTSEC-2020-0095`

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Marco Neumann <marco@crepererum.net>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-02-28 10:05:24 +00:00
dependabot[bot] c289467239
chore(deps): Bump clap from 3.1.1 to 3.1.2 (#3870)
Bumps [clap](https://github.com/clap-rs/clap) from 3.1.1 to 3.1.2.
- [Release notes](https://github.com/clap-rs/clap/releases)
- [Changelog](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md)
- [Commits](https://github.com/clap-rs/clap/compare/v3.1.1...v3.1.2)

---
updated-dependencies:
- dependency-name: clap
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-02-28 09:53:25 +00:00
Marco Neumann f966f4c7a4
feat: create `ParquetChunk` in querier (#3857)
Adds a small adapter that is able to produce `ParquetChunk`s for NG.

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-02-25 08:54:16 +00:00
Marco Neumann 49d1be30e7
feat: wire up `ParquetFilePath` for NG (#3853)
It's a bit of a duck-type hack, but if we wanna just `ParquetFileChunk`
in the new architecture, we somehow need it to accept new-gen paths.
Also path handling should be somewhat centralized since
ingester/compactor/querier all need to construct them. So having a
`ParquetFilePath` that supports both path styles seems to be a
not-to-bad solution. This should obviously be cleaned up in some
not-to-distant future.
2022-02-24 16:05:38 +00:00
Luke Bond 4731913c44
feat: skeleton of querier CLI (#3843)
* feat: skeleton of querier CLI

* chore: wrap metrics in opt&arc in querier to satisfy new api

* chore: derive debug in querier handler

* chore: add join handles and their shutdown to nascent querier server

* chore: querier server http unimpl -> 404

* fix: join/shutdown fix in querier; removed unused delegates
2022-02-24 15:42:56 +00:00
Marco Neumann 9079e6ddb0
feat: backoff retries in ingester (#3841)
* feat: add `backoff` crate

* feat: backoff retries in ingester

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-02-23 17:58:16 +00:00