Commit Graph

1460 Commits (6b81d4c0beb25f58c097d46cc0a995766ef6fc1d)

Author SHA1 Message Date
dependabot[bot] 6b81d4c0be
chore(deps): Bump comfy-table from 6.0.0 to 6.1.0 (#5480)
Bumps [comfy-table](https://github.com/nukesor/comfy-table) from 6.0.0 to 6.1.0.
- [Release notes](https://github.com/nukesor/comfy-table/releases)
- [Changelog](https://github.com/Nukesor/comfy-table/blob/main/CHANGELOG.md)
- [Commits](https://github.com/nukesor/comfy-table/compare/v6.0.0...v6.1.0)

---
updated-dependencies:
- dependency-name: comfy-table
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-08-29 07:38:37 +00:00
Carol (Nichols || Goulding) 5c20d6248f
chore: Upgrade lz4-sys due to RUSTSEC-2022-0051 (#5479)
https://rustsec.org/advisories/RUSTSEC-2022-0051
2022-08-26 20:30:19 +00:00
Luke Bond 3950ca3a17
feat: upsert partition & update sort key for each day in bulk ingest (#5447)
* feat: upsert partition & update sort key for each day in bulk ingest

feat: import schema now supports earliest/latest time merging
chore: tests & tidying up for bulk ingest catalog update

* fix: always sort time last in PK in import schema update catalog

* chore: additional test for computing sort key in bulk ingest

* chore: bulk import catalog update gets sequencer from sharder service

chore: import update schema tests refactor using sharder svc mock

* chore: dead code fix

* chore: import schema sequencer lookup test

* chore: clarifying comment in import schema catalog update
2022-08-25 10:47:12 +00:00
Dom Dwyer abf26767c1 refactor: infallible JumpHash initialisation
This doesn't really need to be fallible but forces propagation of a ton
of error handling - no shards is always a sign of something being very
wrong, and can be caught in the caller if it's for some reason an
acceptable state / can be recovered from.
2022-08-24 13:18:57 +02:00
Marko Mikulicic 4beb721a9a
fix: Revert Bump dotenvy from 0.15.1 to 0.15.2 (#5450) (#5455)
This reverts commit 84acbd2fad.

Closes #5454

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-08-24 09:10:09 +00:00
dependabot[bot] 84acbd2fad
chore(deps): Bump dotenvy from 0.15.1 to 0.15.2 (#5450)
Bumps [dotenvy](https://github.com/allan2/dotenvy) from 0.15.1 to 0.15.2.
- [Release notes](https://github.com/allan2/dotenvy/releases)
- [Changelog](https://github.com/allan2/dotenvy/blob/master/CHANGELOG.md)
- [Commits](https://github.com/allan2/dotenvy/commits/v0.15.2)

---
updated-dependencies:
- dependency-name: dotenvy
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-08-23 11:24:42 +00:00
Marco Neumann 064606380b
feat: refresh policy for caches (#5431)
* feat: refresh policy for caches

For #5318 we want to have a policy that refreshes keys before they are
too old. I initially tried to fold both TTL and the refresh system into
a single policy but than decided that they will basically be two
policies in one with a harder-to-test interface. Semantically TTL and
refresh are also a bit different (but will usually be used together):

- **TTL:** Prevents that a users gets data that is too old. It is some kind
  of "soft correctness". In some sense this is related to the "remove
  if" policy where some part of the system knows for sure (or with
  reasonable likelyhood) that a cache entry is outdated. Note that TTL's
  primary job is NOT to clean up memory from old keys (even though it
  indirectly does that). There is no reason cached entries should be
  removed except for correctness (TTL and remove-if) or resource
  pressure -- and the latter is handled by the LRU policy.
- **Refresh:** While TTL is some kind of deadline, we often have good
  reason to refresh the key before we pull the plug, namely when an
  entry is used and a bit old (but not too old). The concrete mechanism
  to archive this is flexible. At the moment the policy is rather
  simple -- just start a refresh task if a key is old and we receive a
  GET request -- but can be extended in the future.

This also adds some integration tests for TTL+refresh. There will be
follow-up changes to test the interaction with LRU as well, althouh I am
pretty certain that there won't be any surprises due to the excessive
testing we have in place for the policy backend itself as well as all
the policies.

This change also does NOT integrate the refresh with the querier for the
sake of keeping the changeset "small" (i.e. it is already rather large).

* docs: improve

Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>

Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
2022-08-22 08:45:22 +00:00
Stuart Carnie 2d795bd604
chore: cargo update (#5439)
* pest 2.2.1 → 2.3.0
* serde 1.0.143 → 1.0.144
* serde-json 1.0.83 → 1.0.85

pest_meta and pest_generator 2.2.1 were yanked
2022-08-22 05:15:48 +00:00
dependabot[bot] ed38b01e91
chore(deps): Bump sqlparser from 0.20.0 to 0.21.0 (#5429)
Bumps [sqlparser](https://github.com/sqlparser-rs/sqlparser-rs) from 0.20.0 to 0.21.0.
- [Release notes](https://github.com/sqlparser-rs/sqlparser-rs/releases)
- [Changelog](https://github.com/sqlparser-rs/sqlparser-rs/blob/main/CHANGELOG.md)
- [Commits](https://github.com/sqlparser-rs/sqlparser-rs/compare/v0.20.0...v0.21.0)

---
updated-dependencies:
- dependency-name: sqlparser
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-08-19 10:41:38 +00:00
Marco Neumann 5ce0618b8f
chore: `cargo update` (#5432)
```
cpufeatures v0.2.2 -> v0.2.3
plotters v0.3.2 -> v0.3.3
plotters-svg v0.3.2 -> v0.3.3
```

`plotters v0.3.2` was yanked.
2022-08-19 09:56:23 +00:00
Stuart Carnie b4e5895d7a
feat: Add influxdb_influxql_parser crate (#5415)
* feat: Add crate; parse quoted identifiers

* chore: Run cargo hakari tasks

* chore: satisfy linter

* chore: Use `test_helpers::Result`

* feat: Add all InfluxQL keywords

* chore: Update influxdb_influxql_parser/src/lib.rs

Co-authored-by: Marco Neumann <marco@crepererum.net>

* chore: PR feedback

* chore: PR Feedback, remove Result<()>

* chore: Update Cargo.lock

Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: Marco Neumann <marco@crepererum.net>
2022-08-18 23:09:45 +00:00
Marco Neumann d75df2b610
chore: `cargo update` (#5426)
```
bumpalo v3.10.0 -> v3.11.0
either v1.7.0 -> v1.8.0
iana-time-zone v0.1.45 -> v0.1.46
rustix v0.35.8 -> v0.35.9
```

`rustix` is important because `0.35.8` was yanked.
2022-08-18 08:53:00 +00:00
Marco Neumann 3dca9c1b43
feat: async sleep with `TimeProvider` (#5417)
* feat: async sleep with `TimeProvider`

This is helpful to mock "ticked" loops or to control certain async time
periods.

Will be used to test the refresh policy developed in #5318.

* refactor: use a single `TimeProvider::sleep` impl

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-08-17 14:50:35 +00:00
Andrew Lamb 7f0ae53d6f
chore: Update to (almost) released object_store 0.4.0 (#5419)
* chore: update object_store

* chore: update hakari config

* chore: Run cargo hakari tasks

Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
2022-08-17 13:44:48 +00:00
Andrew Lamb bd4b708055
chore: Update datafusion pin + other dependencies (#5418)
* chore: Update datafusion pin

* chore: Update other depdecies

* fix: Update for changes in API
2022-08-17 10:42:37 +00:00
dependabot[bot] 2e638fe19c
chore(deps): Bump libc from 0.2.131 to 0.2.132 (#5414)
Bumps [libc](https://github.com/rust-lang/libc) from 0.2.131 to 0.2.132.
- [Release notes](https://github.com/rust-lang/libc/releases)
- [Commits](https://github.com/rust-lang/libc/compare/0.2.131...0.2.132)

---
updated-dependencies:
- dependency-name: libc
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-08-17 08:40:57 +00:00
dependabot[bot] 78665d3092
chore(deps): Bump once_cell from 1.13.0 to 1.13.1 (#5413)
Bumps [once_cell](https://github.com/matklad/once_cell) from 1.13.0 to 1.13.1.
- [Release notes](https://github.com/matklad/once_cell/releases)
- [Changelog](https://github.com/matklad/once_cell/blob/master/CHANGELOG.md)
- [Commits](https://github.com/matklad/once_cell/compare/v1.13.0...v1.13.1)

---
updated-dependencies:
- dependency-name: once_cell
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-08-17 08:31:46 +00:00
Dom Dwyer 180ff9f681 feat: table name in schema validation errors
Scopes all schema validation errors to include the table name in the
error output.
2022-08-16 19:00:44 +02:00
Marco Neumann 7e97620b37
chore: `cargo-update` (#5405)
```
anyhow v1.0.60 -> v1.0.61
bytemuck v1.11.0 -> v1.12.0
hdrhistogram v7.5.0 -> v7.5.1
iana-time-zone v0.1.44 -> v0.1.45
memmap2 v0.5.5 -> v0.5.7
os_str_bytes v6.2.0 -> v6.3.0
```

`iana-time-zone` is important because `0.1.44` was yanked.
2022-08-16 12:51:46 +00:00
Luke Bond 10fee5535a
feat: import schema updates iox catalog (#5385)
* feat: import schema updates iox catalog

- renamed import/schema module to aggregate_tsm_schema to not conflic
  with schema crate
- fetch schema from iox catalog, and validate/merge/create as needed

chore: add catalog dsn config to import schema command
chore: import schema command connects to catalog
chore: import schema merge validation errors return non-zero code
chore: simplified and tidies import update catalog code

chore: tests and refactoring of import schema catalog update

* chore: require retention on ns creation in import

* chore: fixed bad test in import schema validation

* chore: friendlier errors & more tests in import schema catalog update
2022-08-16 11:05:27 +00:00
dependabot[bot] 724759862f
chore(deps): Bump pin-project from 1.0.11 to 1.0.12 (#5402)
Bumps [pin-project](https://github.com/taiki-e/pin-project) from 1.0.11 to 1.0.12.
- [Release notes](https://github.com/taiki-e/pin-project/releases)
- [Changelog](https://github.com/taiki-e/pin-project/blob/main/CHANGELOG.md)
- [Commits](https://github.com/taiki-e/pin-project/compare/v1.0.11...v1.0.12)

---
updated-dependencies:
- dependency-name: pin-project
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-08-16 08:04:25 +00:00
Marco Neumann 0ccefa0d0c
refactor: port TTL backend to policy framework (#5396)
* refactor: port TTL backend to policy framework

Note that this is "just" a port, it does NOT change how TTL works. This
will be done in #5318.

Helps with #5320.

* fix: ensure inner backend is empty

* test: add some smoke test
2022-08-15 16:48:16 +00:00
dependabot[bot] 0a44a42999
chore(deps): Bump futures from 0.3.21 to 0.3.23 (#5394)
Bumps [futures](https://github.com/rust-lang/futures-rs) from 0.3.21 to 0.3.23.
- [Release notes](https://github.com/rust-lang/futures-rs/releases)
- [Changelog](https://github.com/rust-lang/futures-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/futures-rs/compare/0.3.21...0.3.23)

---
updated-dependencies:
- dependency-name: futures
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-08-15 12:19:17 +00:00
dependabot[bot] 69ccfafd14
chore(deps): Bump futures-task from 0.3.21 to 0.3.23 (#5393)
Bumps [futures-task](https://github.com/rust-lang/futures-rs) from 0.3.21 to 0.3.23.
- [Release notes](https://github.com/rust-lang/futures-rs/releases)
- [Changelog](https://github.com/rust-lang/futures-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/futures-rs/compare/0.3.21...0.3.23)

---
updated-dependencies:
- dependency-name: futures-task
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-08-15 10:27:22 +00:00
dependabot[bot] 2c934addf5
chore(deps): Bump futures-channel from 0.3.21 to 0.3.23 (#5388)
Bumps [futures-channel](https://github.com/rust-lang/futures-rs) from 0.3.21 to 0.3.23.
- [Release notes](https://github.com/rust-lang/futures-rs/releases)
- [Changelog](https://github.com/rust-lang/futures-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/futures-rs/compare/0.3.21...0.3.23)

---
updated-dependencies:
- dependency-name: futures-channel
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-08-15 09:42:53 +00:00
dependabot[bot] a14e161109
chore(deps): Bump futures-core from 0.3.21 to 0.3.23 (#5387)
Bumps [futures-core](https://github.com/rust-lang/futures-rs) from 0.3.21 to 0.3.23.
- [Release notes](https://github.com/rust-lang/futures-rs/releases)
- [Changelog](https://github.com/rust-lang/futures-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/futures-rs/compare/0.3.21...0.3.23)

---
updated-dependencies:
- dependency-name: futures-core
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-08-15 08:59:53 +00:00
dependabot[bot] 6a4425ae08
chore(deps): Bump chrono from 0.4.21 to 0.4.22 (#5392)
Bumps [chrono](https://github.com/chronotope/chrono) from 0.4.21 to 0.4.22.
- [Release notes](https://github.com/chronotope/chrono/releases)
- [Changelog](https://github.com/chronotope/chrono/blob/main/CHANGELOG.md)
- [Commits](https://github.com/chronotope/chrono/compare/v0.4.21...v0.4.22)

---
updated-dependencies:
- dependency-name: chrono
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-08-15 08:31:23 +00:00
dependabot[bot] 38277cf0fb
chore(deps): Bump nix from 0.24.2 to 0.25.0 (#5391)
* chore(deps): Bump nix from 0.24.2 to 0.25.0

Bumps [nix](https://github.com/nix-rust/nix) from 0.24.2 to 0.25.0.
- [Release notes](https://github.com/nix-rust/nix/releases)
- [Changelog](https://github.com/nix-rust/nix/blob/master/CHANGELOG.md)
- [Commits](https://github.com/nix-rust/nix/compare/v0.24.2...v0.25.0)

---
updated-dependencies:
- dependency-name: nix
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* chore: Run cargo hakari tasks

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-08-15 08:10:09 +00:00
dependabot[bot] 822ec18fcd
chore(deps): Bump clap from 3.2.16 to 3.2.17 (#5390)
Bumps [clap](https://github.com/clap-rs/clap) from 3.2.16 to 3.2.17.
- [Release notes](https://github.com/clap-rs/clap/releases)
- [Changelog](https://github.com/clap-rs/clap/blob/v3.2.17/CHANGELOG.md)
- [Commits](https://github.com/clap-rs/clap/compare/v3.2.16...v3.2.17)

---
updated-dependencies:
- dependency-name: clap
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-08-15 07:56:11 +00:00
Dom dbe6b4947c
Merge branch 'main' into dom/bump-rskafka 2022-08-12 09:20:37 +01:00
Dom Dwyer 7118334774 build: bump rskafka
Bump rskafka to pick up connection pre-warming:

    https://github.com/influxdata/rskafka/pull/166
2022-08-12 10:13:25 +02:00
dependabot[bot] f0ae8c093b
chore(deps): Bump libc from 0.2.129 to 0.2.131 (#5379)
Bumps [libc](https://github.com/rust-lang/libc) from 0.2.129 to 0.2.131.
- [Release notes](https://github.com/rust-lang/libc/releases)
- [Commits](https://github.com/rust-lang/libc/commits)

---
updated-dependencies:
- dependency-name: libc
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-08-12 07:53:06 +00:00
Dom Dwyer faa1db9a24 build: bump rskafka
Bump rskafka & fix minor breakage in order to pick up client
pre-warming:

    https://github.com/influxdata/rskafka/pull/165
2022-08-11 17:26:06 +02:00
Dom Dwyer 7174f38f3f build: bump rskafka
Bumps rskafka to HEAD to pick up:

    https://github.com/influxdata/rskafka/pull/164
2022-08-11 14:00:21 +02:00
dependabot[bot] ae6ac27960
chore(deps): Bump console-subscriber from 0.1.6 to 0.1.7 (#5374)
Bumps [console-subscriber](https://github.com/tokio-rs/console) from 0.1.6 to 0.1.7.
- [Release notes](https://github.com/tokio-rs/console/releases)
- [Commits](https://github.com/tokio-rs/console/compare/console-subscriber-v0.1.6...tokio-console-v0.1.7)

---
updated-dependencies:
- dependency-name: console-subscriber
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-08-11 11:08:10 +00:00
dependabot[bot] 692ec97c2f
chore(deps): Bump ahash from 0.7.6 to 0.8.0 (#5373)
* chore(deps): Bump ahash from 0.7.6 to 0.8.0

Bumps [ahash](https://github.com/tkaitchuck/ahash) from 0.7.6 to 0.8.0.
- [Release notes](https://github.com/tkaitchuck/ahash/releases)
- [Commits](https://github.com/tkaitchuck/ahash/compare/v0.7.6...v0.8.0)

---
updated-dependencies:
- dependency-name: ahash
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* chore: Run cargo hakari tasks

* fix: `ahash` features

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: Marco Neumann <marco@crepererum.net>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-08-11 08:37:31 +00:00
Jake Goulding 7787c51b57
feat: add new CLI command to run the compactor once 2022-08-10 11:28:51 -04:00
Jake Goulding ce908c8678
refactor: Use CompactorHandlerImpl::new_with_compactor in service 2022-08-10 11:28:51 -04:00
Marco Neumann 22037b2461
chore: update libm to 0.2.5 (#5371)
The former version (0.2.4) was yanked, so our CI is now failing.
2022-08-10 15:03:27 +00:00
Luke Bond 7e9918f067
chore: import validate merged schema (#5367)
* feat: import schema merge now outputs validation results

chore: refactor import crate

chore: renamed some structs for clarity in import crate

* chore: tests for import schema merge validation

* chore: Run cargo hakari tasks

* chore: clippy

* chore: make hashmap loop easier to read in import schema validation

Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-08-10 12:15:37 +00:00
dependabot[bot] 624b1b33af
chore(deps): Bump libc from 0.2.127 to 0.2.129 (#5363)
Bumps [libc](https://github.com/rust-lang/libc) from 0.2.127 to 0.2.129.
- [Release notes](https://github.com/rust-lang/libc/releases)
- [Commits](https://github.com/rust-lang/libc/compare/0.2.127...0.2.129)

---
updated-dependencies:
- dependency-name: libc
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-08-10 09:23:29 +00:00
Marco Neumann 4da124d862
feat: concurrent garbage collector deletes (#5364)
This should speed up the prod process a bit.

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-08-10 09:14:46 +00:00
Luke Bond c5f062bba0
feat: initial commit of schema merge bulk import tool (#5344)
* feat: initial commit of schema merge bulk import tool

* chore: use observability depds instead of tracing-*

* chore: removed debug printlns

* chore: fix feature decls for cloud providers for import crate

* chore: use println instead of info in import- no need for a simple CLI

* chore: tidy whitespace

* chore: remove unused dep in import

* chore: Run cargo hakari tasks

* chore: removed unimpld import job subcommand

* chore: clarifying comment about custom serialisation code

* chore: clarifying comment about schema merge code in import

* chore: fix wrong comment in import command

* chore: bump object store dep to get bugfix

* chore: rename import schema struct for clarity

* chore: run `cargo hakari generate`

Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-08-10 09:07:38 +00:00
dependabot[bot] 4bc16356bc
chore(deps): Bump serde from 1.0.142 to 1.0.143 (#5362)
Bumps [serde](https://github.com/serde-rs/serde) from 1.0.142 to 1.0.143.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.142...v1.0.143)

---
updated-dependencies:
- dependency-name: serde
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-08-10 08:37:06 +00:00
dependabot[bot] 7975d70632
chore(deps): Bump chrono from 0.4.20 to 0.4.21 (#5361)
* chore(deps): Bump chrono from 0.4.20 to 0.4.21

Bumps [chrono](https://github.com/chronotope/chrono) from 0.4.20 to 0.4.21.
- [Release notes](https://github.com/chronotope/chrono/releases)
- [Changelog](https://github.com/chronotope/chrono/blob/main/CHANGELOG.md)
- [Commits](https://github.com/chronotope/chrono/compare/v0.4.20...v0.4.21)

---
updated-dependencies:
- dependency-name: chrono
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* chore: Run cargo hakari tasks

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
2022-08-10 08:25:45 +00:00
Andrew Lamb 16ddc5efc6
chore: Update datafusion / arrow/parquet/arrow-flight and prost/tonic ecosystem (#5360)
* chore: Update datafusion and arrow

* chore: Update Cargo.lock

* chore: update to Decimal128

* chore: Update tonic/prost/pbjson/etc

* chore: Run cargo hakari tasks

* fix: doctest in generated types

Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
2022-08-09 17:30:44 +00:00
Raphael Taylor-Davies dadcc369b1
chore: update object_store to fix credentials client (#5359)
* chore: update object_store to fix credentials client

* chore: Run cargo hakari tasks

Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
2022-08-09 13:17:43 +00:00
Andrew Lamb b21799acae
chore: Update datafusion, get `date_bin` (#5340)
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-08-09 11:01:37 +00:00
Raphael Taylor-Davies ccb45d7bac
chore: update to rusoto-less object_store (#5342)
* chore: update to rusoto-less object_store

* chore: Run cargo hakari tasks

* chore: further fixes

* chore: document workaround

* chore: review feedback

Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
2022-08-09 09:06:03 +00:00
Dom Dwyer 284a3069ce feat: Kafka client produce() instrumentation
Adds a decorator over the underlying kafka client to capture the latency
distribution of the low-level kafka writes, independent of the
aggregation/DML batching framework that sits "above" this client.

The latency measurements include the serialisation overhead, protocol
overhead, and actual network I/O.
2022-08-08 15:24:35 +02:00