Commit Graph

122 Commits (034d9b371d86d076078ea3aab1114011f9e93e70)

Author SHA1 Message Date
Marco Neumann 2b84e5c087
feat: measure "probably reloaded" cache loads (#4813)
To roughly gauge how much data we re-load into cached (i.e. data that
was already loaded but was later evicted due to LRU pressure or TTL
eviction) this change introduces a new metric that estimates if a cache
entry that is requested from the loader was already seen before (using a
probabilistic filter).
2022-06-13 13:51:45 +00:00
Andrew Lamb 3592aa52d8
chore: Update datafusion + `arrow`/`parquet`/`arrow-flight` to `15.0.0` (#4743)
* chore: Update datafusion + `arrow`/`parquet`/`arrow-flight` to `15.0.0`

* chore: Update APIs

* chore: Run cargo hakari tasks

* feat: normalize parquet file metadata

* chore: update size tests

* chore: add docs on metadata stripping

* chore: TEMP UPDATE TO DF BRANCH

* chore: Update for new API

* fix: Update to latest DF

* fix: cargo hakari

Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: Raphael Taylor-Davies <r.taylordavies@googlemail.com>
2022-06-03 10:32:26 +00:00
Marko Mikulicic cdbe546e50
fix: return gRPC error on panic (#4686) 2022-05-25 07:06:25 +00:00
Luke Bond b76a0080d5
chore: remove unused iox_gitops_adapter (#4675)
* chore: remove unused iox_gitops_adapter

* chore: Run cargo hakari tasks

Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
2022-05-24 10:28:43 +00:00
dependabot[bot] 6bc0c74c7d
chore(deps): Bump once_cell from 1.10.0 to 1.11.0 (#4646)
* chore(deps): Bump once_cell from 1.10.0 to 1.11.0

Bumps [once_cell](https://github.com/matklad/once_cell) from 1.10.0 to 1.11.0.
- [Release notes](https://github.com/matklad/once_cell/releases)
- [Changelog](https://github.com/matklad/once_cell/blob/master/CHANGELOG.md)
- [Commits](https://github.com/matklad/once_cell/compare/v1.10.0...v1.11.0)

---
updated-dependencies:
- dependency-name: once_cell
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* chore: Run cargo hakari tasks

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-05-20 07:40:38 +00:00
Andrew Lamb 3a33e806c7
chore: Update datafusion + `arrow`/`parquet`/`arrow-flight` to `14.0.0` (#4619)
* chore: Update datafusion deps

* chore: update arrow/parquet/arrow flight deps

* chore: Run cargo hakari tasks

* chore: Update location of utils

* chore: Update some more APIs

Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
2022-05-17 14:13:03 +00:00
CircleCI[bot] ca1edc3b4d chore: Run cargo hakari tasks 2022-05-16 01:38:11 +00:00
Raphael Taylor-Davies f2bb0fdf77
feat: update to crates.io object_store version (#4595)
* feat: update to crates.io object_store version

* chore: Run cargo hakari tasks

* fix: tests

* chore: remove object store integration test plumbing

Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
2022-05-13 16:26:07 +00:00
dependabot[bot] 2a15b525a9
chore(deps): Bump clap from 3.1.12 to 3.1.17 (#4537)
* chore(deps): Bump clap from 3.1.12 to 3.1.17

Bumps [clap](https://github.com/clap-rs/clap) from 3.1.12 to 3.1.17.
- [Release notes](https://github.com/clap-rs/clap/releases)
- [Changelog](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md)
- [Commits](https://github.com/clap-rs/clap/compare/v3.1.12...v3.1.17)

---
updated-dependencies:
- dependency-name: clap
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* chore: update lockfile

* chore: Run cargo hakari tasks

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
Co-authored-by: Raphael Taylor-Davies <r.taylordavies@googlemail.com>
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
2022-05-09 10:13:28 +00:00
CircleCI[bot] d2283c5186 chore: Run cargo hakari tasks 2022-05-06 19:59:27 +00:00
Andrew Lamb 02893e598c
chore: Update datafusion and upgrade arrow/parquet/arrow-flight to 13 (#4516)
* chore: Tool for automating arrow version update

* chore: Update datafusion and arrow/parquet/arrow-flight

* fix: update for changes in Arrow API

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-05-05 00:21:02 +00:00
Andrew Lamb 7b6754cc03
chore: Update dependencies via `cargo update` (#4460)
* chore: Update dependencies via `cargo update`

* chore: Run cargo hakari tasks

Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
2022-04-28 18:48:51 +00:00
Andrew Lamb e13d3433ae
feat: Use datafusion serialization code rather than our own copy of it (#4421)
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-04-28 13:03:34 +00:00
Andrew Lamb 73bed810da
chore: Update arrow, arrow-flight, parquet, tonic, prost, etc (#4357)
* chore: Update datafusion

* chore: Update arrow/arrow-flight/parquet to 12

* chore: update datafusion correctly

* chore: Update prost, tonic, and dependents

* fix: Fixup some api changes

* fix: Update test output in db

* fix: Update test output in parquet_file

* fix: remove old pbjson types

* fix: Add "--experimental_allow_proto3_optional" flag

* chore: Run cargo hakari tasks

* fix: compile error

* chore: Update heappy

Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-04-20 11:12:17 +00:00
dependabot[bot] 72a2a68eea
chore(deps): Bump sqlx from 0.5.11 to 0.5.12 (#4333)
* chore(deps): Bump sqlx from 0.5.11 to 0.5.12

Bumps [sqlx](https://github.com/launchbadge/sqlx) from 0.5.11 to 0.5.12.
- [Release notes](https://github.com/launchbadge/sqlx/releases)
- [Changelog](https://github.com/launchbadge/sqlx/blob/master/CHANGELOG.md)
- [Commits](https://github.com/launchbadge/sqlx/compare/v0.5.11...v0.5.12)

---
updated-dependencies:
- dependency-name: sqlx
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* chore: Run cargo hakari tasks

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-04-15 15:03:08 +00:00
CircleCI[bot] 0a253abd57 chore: Run cargo hakari tasks 2022-04-14 10:42:13 +00:00
CircleCI[bot] 3a78661ad7 chore: Run cargo hakari tasks 2022-04-13 15:39:26 +00:00
Dom Dwyer 63e2a522cf refactor: disable max_debug static tracing level
The tracing "release_max_level_*" feature flags are not additive, and
are hard to use - especially when the workspace-hack crate enables
features behind your back!
2022-04-12 17:49:50 +01:00
Dom Dwyer a4df93e604 chore: workspace-hack update 2022-04-12 13:05:08 +01:00
dependabot[bot] 3689e17704
chore(deps): Bump croaring from 0.5.1 to 0.5.2 (#4267)
* chore(deps): Bump croaring from 0.5.1 to 0.5.2

Bumps [croaring](https://github.com/saulius/croaring-rs) from 0.5.1 to 0.5.2.
- [Release notes](https://github.com/saulius/croaring-rs/releases)
- [Commits](https://github.com/saulius/croaring-rs/compare/0.5.1...0.5.2)

---
updated-dependencies:
- dependency-name: croaring
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* chore: Run cargo hakari tasks

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-04-11 08:22:43 +00:00
dependabot[bot] 17af5fcbd1
chore(deps): Bump tokio-util from 0.7.0 to 0.7.1 (#4154)
* chore(deps): Bump tokio-util from 0.7.0 to 0.7.1

Bumps [tokio-util](https://github.com/tokio-rs/tokio) from 0.7.0 to 0.7.1.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-util-0.7.0...tokio-util-0.7.1)

---
updated-dependencies:
- dependency-name: tokio-util
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* chore: Run cargo hakari tasks

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-03-29 08:39:02 +00:00
Andrew Lamb 5c69a3f43b
chore: Update deps: datafusion, arrow/arrow-flight/parquet to 11, zstd to 0.11 (#4119)
* chore: update datafusion

* chore(deps): Bump arrow from 10.0.0 to 11.0.0

Bumps [arrow](https://github.com/apache/arrow-rs) from 10.0.0 to 11.0.0.
- [Release notes](https://github.com/apache/arrow-rs/releases)
- [Changelog](https://github.com/apache/arrow-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/apache/arrow-rs/compare/10.0.0...11.0.0)

---
updated-dependencies:
- dependency-name: arrow
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

* chore(deps): Bump arrow-flight from 10.0.0 to 11.0.0

Bumps [arrow-flight](https://github.com/apache/arrow-rs) from 10.0.0 to 11.0.0.
- [Release notes](https://github.com/apache/arrow-rs/releases)
- [Changelog](https://github.com/apache/arrow-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/apache/arrow-rs/compare/10.0.0...11.0.0)

---
updated-dependencies:
- dependency-name: arrow-flight
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

* chore: update parquet to 11.0.0

* fix: error on create schema, test for same

* fix: upgrade zstd

* chore: Run cargo hakari tasks

* fix: fix logical merge conflict

* fix: hakari

* fix: hakari

* fix: update newly introduced dep

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-03-24 15:27:36 +00:00
Andrew Lamb fb75ef7b82
test: add end to end test for all in one mode, restructure fixture (#4114)
* test: add end to end test for all in one mode, restructure fixture

* docs: fix typos and clarify schema requrements
2022-03-24 12:53:25 +00:00
Andrew Lamb 29b89aaca7
refactor: extract influxrpc, flight and testing gRPC out of influxdb_ioxd (#4106)
* refactor: extract grpc service implementations out of influxdb_ioxd

* chore: Run cargo hakari tasks

* refactor: rename server_common to service_common

* refactor: rename server_grpc_influxrpc to service_grpc_influxrpc

* refactor: rename server_grpc_flight to service_grpc_flight

* refactor: rename server_grpc_testing to service_grpc_testing

* fix: Cargo.toml

Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
2022-03-23 20:14:45 +00:00
dependabot[bot] a23efce408
chore(deps): Bump kube-derive from 0.69.1 to 0.70.0 (#4073)
* chore(deps): Bump kube-derive from 0.69.1 to 0.70.0

Bumps [kube-derive](https://github.com/kube-rs/kube-rs) from 0.69.1 to 0.70.0.
- [Release notes](https://github.com/kube-rs/kube-rs/releases)
- [Changelog](https://github.com/kube-rs/kube-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/kube-rs/kube-rs/compare/0.69.1...0.70.0)

---
updated-dependencies:
- dependency-name: kube-derive
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* chore(deps): Bump kube-runtime from 0.69.1 to 0.70.0

Bumps [kube-runtime](https://github.com/kube-rs/kube-rs) from 0.69.1 to 0.70.0.
- [Release notes](https://github.com/kube-rs/kube-rs/releases)
- [Changelog](https://github.com/kube-rs/kube-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/kube-rs/kube-rs/compare/0.69.1...0.70.0)

---
updated-dependencies:
- dependency-name: kube-runtime
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* chore: upgrade kube to version 0.70

* chore: hakari

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Marco Neumann <marco@crepererum.net>
2022-03-21 15:32:45 +00:00
Andrew Lamb 2c3d30ca32
chore: Update datafusion, arrow, flight and parquet (#4000)
* chore: Update datafusion, arrow, flight and parquet

* fix: api change

* fix: fmt

* fix: update test metadata size

* fix: Update sizes in parquet test

* fix: more metadata size update
2022-03-10 12:24:47 +00:00
Dom Dwyer baae6910f2 build: remove openssl dependency
Switches a stray reqwest dependency to use rustls.
2022-03-07 13:47:39 +00:00
dependabot[bot] 48908054d1
chore(deps): Bump once_cell from 1.9.0 to 1.10.0 (#3955)
* chore(deps): Bump once_cell from 1.9.0 to 1.10.0

Bumps [once_cell](https://github.com/matklad/once_cell) from 1.9.0 to 1.10.0.
- [Release notes](https://github.com/matklad/once_cell/releases)
- [Changelog](https://github.com/matklad/once_cell/blob/master/CHANGELOG.md)
- [Commits](https://github.com/matklad/once_cell/compare/v1.9.0...v1.10.0)

---
updated-dependencies:
- dependency-name: once_cell
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* chore: Run cargo hakari tasks

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
2022-03-07 08:54:20 +00:00
Andrew Lamb e09f39d6a0
chore: Update datafusion (#3943)
* chore: Update datafusion

* refactor: update for new datafusion

* chore: Run cargo hakari tasks

Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
2022-03-04 19:37:46 +00:00
Andrew Lamb d6afda0227
refactor: split `influxdb_ioxd`, `clap_blocks` and `serving_readiness` from influxdb_iox crate (#3908)
* refactor: split influxdb_ioxd, clap_blocks, and serving_readiness out of influxdb_iox

split out serving readiness, get compiling

* fix: hakari

* fix: hakari again

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-03-03 16:48:30 +00:00
Dom Dwyer ab010f9d82 fix: regenerate workspace hack config 2022-03-03 11:55:00 +00:00
Dom Dwyer 46bb107be4 refactor: use rustls
Removes openssl as a dependency, switching to rustls[1] as the TLS
implementation throughout.

It is important to note that this change brings with it a significant
behavioural difference - rustls does not currently support IP SANs in
certificates (instead only supporting fully-qualified names / DNS) and
this will manifest as a failure to connect to IP endpoints over TLS.
This might be a blocker that prevents us using rustls exclusively, but
there's noe asy way to know without trying it. Fortunately the rustls
project has received funding to work on IP SAN support[2].

[1]: https://github.com/rustls/rustls
[2]: https://www.abetterinternet.org/post/preparing-rustls-for-wider-adoption/
2022-03-03 11:05:20 +00:00
Carol (Nichols || Goulding) d7bd46f086
fix: Update hakari files for cargo-hakari 0.9.12 (#3872) 2022-02-28 09:18:23 +00:00
dependabot[bot] 18c2ca1bb5
chore(deps): Bump tower from 0.4.11 to 0.4.12 (#3817)
* chore(deps): Bump tower from 0.4.11 to 0.4.12

Bumps [tower](https://github.com/tower-rs/tower) from 0.4.11 to 0.4.12.
- [Release notes](https://github.com/tower-rs/tower/releases)
- [Commits](https://github.com/tower-rs/tower/compare/tower-0.4.11...tower-0.4.12)

---
updated-dependencies:
- dependency-name: tower
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* chore: update workspace-hack

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Raphael Taylor-Davies <r.taylordavies@googlemail.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-02-23 11:14:15 +00:00
dependabot[bot] ad3868ed7c
chore(deps): Bump tokio from 1.16.1 to 1.17.0 (#3814)
* chore(deps): Bump tokio from 1.16.1 to 1.17.0

Bumps [tokio](https://github.com/tokio-rs/tokio) from 1.16.1 to 1.17.0.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.16.1...tokio-1.17.0)

---
updated-dependencies:
- dependency-name: tokio
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* build: update workspace-hack

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Dom Dwyer <dom@itsallbroken.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-02-22 16:27:43 +00:00
Luke Bond a66e29e5b3
chore: port sqlx-hotswap-pool over from conductor (#3750)
* chore: port sqlx-hotswap-pool over from conductor

Co-authored-by: Marko Mikulicic <mkm@influxdata.com>

* chore: workspace hack fixes

* fix: unique schema per test db connection

* fix: adjust search path in catalog pg tests to see if it fixes test schema issue

* fix: actually fixed sqlx hotswap pool test

Co-authored-by: Marko Mikulicic <mkm@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-02-15 16:18:36 +00:00
Andrew Lamb a30803e692
chore: Update datafusion, update `arrow`/`parquet`/`arrow-flight` to 9.0 (#3733)
* chore: Update datafusion

* chore: Update arrow

* fix: missing updates

* chore: Update cargo.lock

* fix: update for smaller parquet size

* fix: update test for smaller parquet files

* test: ensure parquet_file tests write multiple row groups

* fix: update callsite

* fix: Update for tests

* fix: harkari

* fix: use IoxObjectStore::existing

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-02-15 12:10:24 +00:00
dependabot[bot] 89105ccfab
chore(deps): Bump tokio-util from 0.6.9 to 0.7.0 (#3743)
Bumps [tokio-util](https://github.com/tokio-rs/tokio) from 0.6.9 to 0.7.0.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/commits)

---
updated-dependencies:
- dependency-name: tokio-util
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-02-15 11:33:41 +00:00
Luke Bond d39cf0158b
chore: update gitops adapter k8s deps (via dependabot) (#3736)
* chore: update gitops adapter k8s deps (via dependabot)

* chore: workspace hack updates

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-02-14 17:47:15 +00:00
dependabot[bot] 6c3b208438
chore(deps): bump cache_loader_async from 0.1.2 to 0.2.0 (#3665)
* chore(deps): bump cache_loader_async from 0.1.2 to 0.2.0

Bumps [cache_loader_async](https://github.com/ZeroTwo-Bot/cache-loader-async-rs) from 0.1.2 to 0.2.0.
- [Release notes](https://github.com/ZeroTwo-Bot/cache-loader-async-rs/releases)
- [Changelog](https://github.com/ZeroTwo-Bot/cache-loader-async-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/ZeroTwo-Bot/cache-loader-async-rs/commits/v0.2.0)

---
updated-dependencies:
- dependency-name: cache_loader_async
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* fix: breaking changes from `cache_loader_async`

* fix: Update workspace hack: tokio full is no longer needed

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Marco Neumann <marco@crepererum.net>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
Co-authored-by: Carol (Nichols || Goulding) <carol.nichols@gmail.com>
2022-02-10 17:01:11 +00:00
dependabot[bot] 9369152096
chore(deps): bump tracing-core from 0.1.21 to 0.1.22 (#3675)
Bumps [tracing-core](https://github.com/tokio-rs/tracing) from 0.1.21 to 0.1.22.
- [Release notes](https://github.com/tokio-rs/tracing/releases)
- [Commits](https://github.com/tokio-rs/tracing/compare/tracing-core-0.1.21...tracing-core-0.1.22)

---
updated-dependencies:
- dependency-name: tracing-core
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-02-08 15:12:39 +00:00
Luke Bond de2a013786
feat: gitops adapter (#3656)
* feat: scaffolding of gitops adapter bin crate

* chore: refactor gitops adapter; calls CLI now; status update fixes

* feat: gitops adapter now calls out to CLI once per topic; improved tests

* chore: add mock failure script for gitops adapter

* chore: update workspace-hack

* chore: refactor away unecessary to_string in gitops syncer

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-02-08 13:27:36 +00:00
Marco Neumann 50cff27b01
chore: remove rdkafka dependency (#3625)
All features are now covered by rskafka. This also removes the need to
specify a server ID for write buffer consumers. This was only used for
rdkafka since there we needed to specify a consumer group, even though
we did not use any transactions.
2022-02-03 13:33:56 +00:00
Marco Neumann bc4b7f8a5b
test: ensure that rskafka and rdkakfa work together (#3624)
* chore: upgrade rskafka + enable snappy support

* test: ensure that rskafka and rdkakfa work together

Before removing rdkafka ensure that:

- rskafka can consume existing messages produced by rdkafka so we do not
  need to drain existing topics
- rdkafka can consume new messages produced by rskafka so we can roll
  back

I ran the whole `write_buffer` test suite (including the newly added
tests) using Apache Kafka as well as Redpanda.

* test: ensure we handle consumer offset in error case correctly

* docs: explain test setup

Co-authored-by: Andrew Lamb <alamb@influxdata.com>

Co-authored-by: Andrew Lamb <alamb@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-02-03 12:52:42 +00:00
Marco Neumann 0a2cb36ddf
fix: workspace-hack (#3593) 2022-02-01 12:32:58 +00:00
Raphael Taylor-Davies 4101d16f71
chore: feature flag consistency (#3574)
* chore: feature flag consistency

* chore: add aarch64-apple-darwin to hakari

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-01-28 16:38:59 +00:00
Andrew Lamb 2062267d0f
chore: Update hashbrown (#3551)
* chore: Update hashbrown

* fix: hakari

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-01-27 15:34:10 +00:00
Andrew Lamb 5488c257d1
chore: Update datafusion, upgrade to arrow/parqet/arrow-flight 8.0.0 (#3517)
* chore: Update datafusion

* chore: update to arrow 8

* fix: update to use new DataFusion APIs

* fix: update case for sortedness

* fix: cargo hakari
2022-01-27 13:33:27 +00:00
Marco Neumann 2928254c0f
fix: test logging (#3536)
- Use a more standard way to setup the tracing subsystem (as described
  in tracing-subscriber docs)
- Also capture content from `log` crate
- Play nice w/ Rust's libtest message capture

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-01-26 10:28:51 +00:00
Paul Dix d825dab8e2 fix: hakari workspace hack 2022-01-19 14:48:00 -05:00
Paul Dix d4d4b4c087 fix: workspace hack after adding iox_catalog crate 2022-01-17 10:04:24 -05:00
Andrew Lamb dd23056efd
chore: update datafusion, arrow, prost, tonic, pbjson, etc (#3455)
* chore: update datafusion, arrow, prost, tonic, etc

* fix: update pprof as well

* chore: update hakari

* fix: update pbjson

* chore: update heappy

* fix: hakari

* fix: workaround https://github.com/influxdata/influxdb_iox/issues/3458

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-01-13 17:07:15 +00:00
Marco Neumann 37bb7f2120 chore: `cargo update`
dependabot currently doesn't work due to
https://github.com/dependabot/dependabot-core/issues/4574

Excluded `quote` due to
https://github.com/dtolnay/quote/issues/204
2022-01-11 14:57:51 +01:00
Andrew Lamb 527885f7f8
chore: Update datafusion (#3413)
* chore: Update datafusion and update code to handle timezone aware timestamps

* fix: cargo hakari

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-12-23 14:52:12 +00:00
Andrew Lamb ccba68fe3e
chore: Update datafusion and sqlparser, and arrow (#3385)
* chore: Update datafusion and sqlparser, and arrow

* fix: cargo hakari

* test: fix metadata size

* fix: update some metadata sizes

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-12-16 12:13:40 +00:00
Marko Mikulicic b0c6713bd7 fix: Don't swallow exit code when hakari test fails 2021-12-14 14:49:14 +01:00
Carol (Nichols || Goulding) 7499eac067
fix: Disable uuid serde feature; we're not actually serializing any UUIDs
Connects to #3117.
2021-12-06 09:37:31 -05:00
Carol (Nichols || Goulding) e8d9e35321
fix: influxdb_iox tests need the json feature of reqwest
I think this didn't fail before because influxdb_iox used to use
influxdb2_client, which turned on the json feature. Then the workspace
hack crate also turned on the json feature, so when influxdb_iox stopped
using influxdb2_client, it *still* worked. But excluding the object
store crates from hakari analysis that also turned on the json feature
meant nothing turned it on anymore, so now influxdb_iox needs to turn it
on itself.
2021-12-06 09:37:15 -05:00
Carol (Nichols || Goulding) 76909b07e4
fix: Exclude optional object-store implementation crates from cargo-hakari
Aha! cargo-hakari turns on all features, but usually in development all
the specific object storage implementation crates are off.

Excluding them in the hakari config reduces the workspace crate a bit.

Connects to #3117.
2021-12-06 09:37:15 -05:00
Carol (Nichols || Goulding) 1b0a2eb7c7
fix: Specify nom features in the same way cexpr does
This removes nom from the workspace-hack crate. I'm not sure what's
going on here as cexpr is setting default-features false but then
explicitly setting the features to be the same as nom's default
features, and cargo-hakari seems to be resolving that differently than
using nom's default features. I think this might be a bug in
cargo-hakari, going to investigate.

Connects to #3117.
2021-12-06 09:37:14 -05:00
Carol (Nichols || Goulding) 6b940166fe
fix: Remove a deprecated memchr feature from the workspace crate
This was made possible by upgrading nom, which was using the deprecated
feature name. This will save us from pain in the future when the
deprecated feature is removed.

Unfortunately, this adds nom to the workspace hack crate, but we'll see
if it needs to stay there :)

Connects to #3117.
2021-12-06 09:37:14 -05:00
Carol (Nichols || Goulding) bd0a2b36e3
fix: Remove futures-util proc-macro features and update to futures-* 0.3.18
The `futures-util` crate's `proc-macro-hack` and `proc-macro-nested`
features used to be part of `futures-util`'s default feature sets, so
the workspace-hack crate listed them.

`futures-util` 0.3.18 removed those features entirely.

Cargo won't update to versions of crates that don't have features that
have been specified, so the workspace-hack crate was blocking Cargo from
updating `futures-util` and a few other crates to 0.3.18.

Cargo's docs recommend against removing features in semver-compatible
releases, but nothing really prevents crate authors from doing this.

Connects to #3117.
2021-12-06 09:37:14 -05:00
Carol (Nichols || Goulding) ddc1504135
fix: Exclude influxdb2_client from the workspace hack crate
This enables the removal of the futures and futures-io crates.

Connects to #3117.
2021-12-06 09:37:14 -05:00
Carol (Nichols || Goulding) cf6286364e
fix: Exclude criterion from hakari analysis
Connects to #3117.

Criterion builds clap without default features, which causes clap to be
listed in the workspace hack crate with its default features. In
general, actually building criterion will be rare (and we should move
all benches to separate crates that can be excluded) so it should be
fine to rebuild clap when building benchmarks anyway.
2021-12-06 09:36:50 -05:00
Carol (Nichols || Goulding) 0fda9608d8
fix: Exclude optional pprof crate from workspace-hack 2021-12-06 09:36:32 -05:00
Carol (Nichols || Goulding) f3c814e091
fix: Exclude crates conductor depends on from workspace hack 2021-11-22 11:55:38 -05:00
Carol (Nichols || Goulding) fa36adc3e3
fix: Update the workspace hack crate that no longer needs cache_loader_async 2021-11-19 14:34:24 -05:00
Carol (Nichols || Goulding) a6908f1613
fix: Exclude tikv-jemalloc-sys from workspace hack as it's mutually exclusive with heappy 2021-11-19 13:46:17 -05:00
Carol (Nichols || Goulding) 2f452268af
fix: Exclude some bench/test crates from the workspace hack 2021-11-19 13:38:27 -05:00
Carol (Nichols || Goulding) 93d6c2becd
feat: Add CI jobs and documentation for the workspace-hack crate 2021-11-19 11:05:34 -05:00
Carol (Nichols || Goulding) d1c6f76a0e
feat: Results of running cargo hakari generate 2021-11-19 09:21:15 -05:00
Carol (Nichols || Goulding) f66cd681c2
feat: Results of using cargo hakari init workspace-hack 2021-11-19 09:20:35 -05:00