Influxdata
/
influxdb
mirror of https://github.com/influxdata/influxdb.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
8,499 Commits
187 Branches
453 Tags
379 MiB
Commit Graph

8499 Commits (b1756528a9ad8022a983334e0d967dfeb22f089e)

Author SHA1 Message Date
Andrew Charlton b1756528a9
Merge pull request #5149 from influxdata/ach-parse-tsm-key 
chore: Add org and bucket ID parsing to tsm_reader
 2022-07-20 10:12:41 +01:00
Andrew Charlton 4750b54ec6
Merge branch 'main' into ach-parse-tsm-key 2022-07-20 09:50:45 +01:00
Marko Mikulicic b8236e2b9d
fix: Fix SeriesKey sort order for special _measurement and _field (#5150) 
* fix: Fix SeriesKey sort order for special _measurement and _field

* fix: Update expected test output

* fix: Update more tests

* fix: Re-sort tag key when using binary encoding

Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
 2022-07-20 08:45:17 +00:00
Marco Neumann b8d9799a26
feat: wire span all the way to `QuerierTable::chunks` (#5134) 
* feat: pass context to `QueryDatabase::chunks`

* feat: wire span all the way to `QuerierTable::chunks`

This is required for #5129.
 2022-07-19 14:12:55 +00:00
Andrew Charlton 7accaaace5 chore: Add org and bucket ID parsing to tsm_reader 2022-07-19 14:39:52 +01:00
Andrew Lamb 449b4cf1d6
chore: Update datafusion pin (#5148) 
* chore: Update datafusion pin

* fix: Update for changes upstream
 2022-07-19 13:13:37 +00:00
Andrew Lamb 17231b4001
fix: warn if `--data-dir is specified but `--object_store` type is not file (#5147) 
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2022-07-19 01:00:54 +00:00
Jake Goulding f7a0fd43d2
feat: make object store garbage collector into a long-running service (#5135) 
* refactor: remove unused logging config

* chore: remove the object store garbage collector CLI tool

* refactor: accept an object store and catalog

* refactor: make Result type alias public like the error

* refactor: remove public modifier from modules

* refactor: allow shutting down the object store garbage collector

* feat: Introduce the object-store garbage collection server

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2022-07-18 21:27:38 +00:00
kodiakhq[bot] bceee0e2a0
Merge pull request #5136 from influxdata/cn/filter-files-to-compact 
feat: Select a subset of parquet files for a partition to compact
 2022-07-18 20:05:02 +00:00
Carol (Nichols || Goulding) 1eb4640931
fix: Stop when exactly equal to the max file size limit too 2022-07-18 15:41:17 -04:00
Carol (Nichols || Goulding) 154cd28928
fix: Clarify filter_parquet_files doc comment 2022-07-18 15:41:17 -04:00
Carol (Nichols || Goulding) 0a545bf325
fix: Clarify how metrics are recorded in the docs for the metric fields 2022-07-18 15:41:17 -04:00
Carol (Nichols || Goulding) 07e10852a8
feat: Add an input file count threshold to the compactor settings 2022-07-18 15:41:17 -04:00
Carol (Nichols || Goulding) 128833e7d9
fix: Change placeholder new_param to input_size_threshold_bytes 2022-07-18 15:16:43 -04:00
Carol (Nichols || Goulding) d62b1ed7ee
feat: Select a subset of parquet files for a partition to compact 
Fixes #5120.
 2022-07-18 15:14:22 -04:00
Carol (Nichols || Goulding) 4416f1ce37
fix: Remove max number of level 0 files configuration option 2022-07-18 15:08:16 -04:00
Carol (Nichols || Goulding) 57c70fcec5
fix: Remove redundant 'compaction' naming from CompactorConfig fields 2022-07-18 15:03:33 -04:00
Carol (Nichols || Goulding) 0828fb5376
fix: Use more accurate number of bytes for MB and GB 2022-07-18 15:01:41 -04:00
Nga Tran c8f4000f04
feat: Select compaction candidates (#5131) 
* feat: initial implementation for selecting compaction candidates

* feat: 2 catalog functions to choose the most thorughput partitions to compact and the selecting candidate function itself

* test: tests for the new 2 queries

* feat: more tests and metrics for chooing compaction candidates

* chore: Apply self suggestions from self review

* chore: cleanup

* chore: fix doc comment

* chore: Apply suggestions from code review

Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>

* refactor: address review comments

* fix: get the right time provider for the tests

* refactor: remove the left over compaction_

* fix: typos

* fix: make the param name and env name consistent

* refactor: make relevant iSomething to uSomething

* fix: typo

Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
 2022-07-18 18:05:13 +00:00
Andrew Lamb e2d871b00b
chore: Update datafusion and arrow/parquet/arrow-flight to `18.0.0` (#5079) 
* chore: Update datafusion to 10.0.0, arrow/parquet/arrow-flight to 18

* chore: Run cargo hakari tasks

* fix: update cargo pin

Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2022-07-18 15:01:03 +00:00
kodiakhq[bot] 7f648d8961
Merge pull request #5137 from influxdata/cn/level-up 
refactor: replace level_2 with level_1
 2022-07-18 14:52:55 +00:00
kodiakhq[bot] 7421d18ed5
Merge branch 'main' into cn/level-up 2022-07-18 14:45:36 +00:00
Marco Neumann 9c2b6cd96c
fix: always pass proper context to `InfluxRpcPlanner` (#5144) 
There were some instances were we forgot to pass context (and therefore
tracing) information to `InfluxRpcPlanner`. This removes the `Default`
implementation requires to always pass a context when creating
`InfluxRpcPlanner` to prevent this type of bug.

Ref #5129.

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2022-07-18 14:45:22 +00:00
jaymebrd fc0b705c8c
Merge pull request #5143 from jaymebrd/chore/remove-github-acs-action 
chore: remove github acs action trigger
 2022-07-18 12:54:02 +01:00
Jayme Bird b3257365ad chore: remove github acs action trigger 2022-07-18 12:42:19 +01:00
dependabot[bot] 1eeee9809c
chore(deps): Bump rustyline from 9.1.2 to 10.0.0 (#5139) 
Bumps [rustyline](https://github.com/kkawakam/rustyline) from 9.1.2 to 10.0.0.
- [Release notes](https://github.com/kkawakam/rustyline/releases)
- [Changelog](https://github.com/kkawakam/rustyline/blob/master/History.md)
- [Commits](https://github.com/kkawakam/rustyline/compare/v9.1.2...v10.0.0)

---
updated-dependencies:
- dependency-name: rustyline
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-07-18 08:10:22 +00:00
dependabot[bot] fd2d262862
chore(deps): Bump nix from 0.24.1 to 0.24.2 (#5140) 
Bumps [nix](https://github.com/nix-rust/nix) from 0.24.1 to 0.24.2.
- [Release notes](https://github.com/nix-rust/nix/releases)
- [Changelog](https://github.com/nix-rust/nix/blob/v0.24.2/CHANGELOG.md)
- [Commits](https://github.com/nix-rust/nix/compare/v0.24.1...v0.24.2)

---
updated-dependencies:
- dependency-name: nix
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-07-18 06:15:49 +00:00
dependabot[bot] 0b4afad08f
chore(deps): Bump hashbrown from 0.12.2 to 0.12.3 (#5141) 
Bumps [hashbrown](https://github.com/rust-lang/hashbrown) from 0.12.2 to 0.12.3.
- [Release notes](https://github.com/rust-lang/hashbrown/releases)
- [Changelog](https://github.com/rust-lang/hashbrown/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/hashbrown/compare/v0.12.2...v0.12.3)

---
updated-dependencies:
- dependency-name: hashbrown
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-07-18 06:06:24 +00:00
dependabot[bot] f870c8a887
chore(deps): Bump crypto-common from 0.1.5 to 0.1.6 (#5138) 
Bumps [crypto-common](https://github.com/RustCrypto/traits) from 0.1.5 to 0.1.6.
- [Release notes](https://github.com/RustCrypto/traits/releases)
- [Commits](https://github.com/RustCrypto/traits/compare/crypto-common-v0.1.5...crypto-common-v0.1.6)

---
updated-dependencies:
- dependency-name: crypto-common
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-07-18 05:36:20 +00:00
Jake Goulding 635f535e0e refactor: replace level_2 with level_1 2022-07-16 21:49:45 -04:00
dependabot[bot] c0ac01c67b
chore(deps): Bump handlebars from 4.3.1 to 4.3.2 (#5133) 
Bumps [handlebars](https://github.com/sunng87/handlebars-rust) from 4.3.1 to 4.3.2.
- [Release notes](https://github.com/sunng87/handlebars-rust/releases)
- [Changelog](https://github.com/sunng87/handlebars-rust/blob/master/CHANGELOG.md)
- [Commits](https://github.com/sunng87/handlebars-rust/compare/v4.3.1...v4.3.2)

---
updated-dependencies:
- dependency-name: handlebars
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2022-07-15 21:18:50 +00:00
dependabot[bot] 9359539acf
chore(deps): Bump clap from 3.2.11 to 3.2.12 (#5132) 
Bumps [clap](https://github.com/clap-rs/clap) from 3.2.11 to 3.2.12.
- [Release notes](https://github.com/clap-rs/clap/releases)
- [Changelog](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md)
- [Commits](https://github.com/clap-rs/clap/compare/v3.2.11...v3.2.12)

---
updated-dependencies:
- dependency-name: clap
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2022-07-15 21:10:51 +00:00
Andrew Lamb 5bebff0b06
Revert "feat: skip ingester buffering if INFLUXDB_IOX_INGESTER_SKIP_BUFFER is set" (#5116) 
This reverts commit ca6875f60bec935eb6079b684d6eaa0cbc8a5306.

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2022-07-15 13:22:45 +00:00
Marco Neumann f0bd278652
feat: add tracing to instrumented semaphores (#5130) 
This will allow us to easily see how much time we spend during query
processing waiting for the query semaphore.

Ref #5129.
 2022-07-15 07:50:28 +00:00
kodiakhq[bot] b6e10c1804
Merge pull request #5125 from influxdata/dependabot/cargo/clap-3.2.11 
chore(deps): Bump clap from 3.2.10 to 3.2.11
 2022-07-14 14:34:37 +00:00
kodiakhq[bot] 14472c4e66
Merge branch 'main' into dependabot/cargo/clap-3.2.11 2022-07-14 14:26:56 +00:00
kodiakhq[bot] 787759d152
Merge pull request #5126 from influxdata/dependabot/cargo/tokio-1.20.0 
chore(deps): Bump tokio from 1.19.2 to 1.20.0
 2022-07-14 14:26:34 +00:00
kodiakhq[bot] 18ffe581b5
Merge branch 'main' into dependabot/cargo/tokio-1.20.0 2022-07-14 14:18:51 +00:00
kodiakhq[bot] 52b8682aae
Merge pull request #5124 from influxdata/cn/gather-files-to-compact 
feat: Gather parquet files for a partition compaction operation
 2022-07-14 14:12:46 +00:00
kodiakhq[bot] b836c6bced
Merge branch 'main' into cn/gather-files-to-compact 2022-07-14 14:05:29 +00:00
Marco Neumann 512f9850ee
refactor: ingester seek log debug => info (#5127) 
This message will be printed once per partition on ingester startup and
shouldn't be too noisy, but is very helpful to judge "replay" /
"catch-up".
 2022-07-14 10:28:16 +00:00
CircleCI[bot] 42b234e7a3 chore: Run cargo hakari tasks 2022-07-14 01:22:37 +00:00
dependabot[bot] 9b67de2f43
chore(deps): Bump tokio from 1.19.2 to 1.20.0 
Bumps [tokio](https://github.com/tokio-rs/tokio) from 1.19.2 to 1.20.0.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.19.2...tokio-1.20.0)

---
updated-dependencies:
- dependency-name: tokio
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
 2022-07-14 01:21:43 +00:00
dependabot[bot] 8179f113b9
chore(deps): Bump clap from 3.2.10 to 3.2.11 
Bumps [clap](https://github.com/clap-rs/clap) from 3.2.10 to 3.2.11.
- [Release notes](https://github.com/clap-rs/clap/releases)
- [Changelog](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md)
- [Commits](https://github.com/clap-rs/clap/compare/v3.2.10...v3.2.11)

---
updated-dependencies:
- dependency-name: clap
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2022-07-14 01:21:09 +00:00
Carol (Nichols || Goulding) de74415cbe
feat: Gather parquet files for a partition compaction operation 
Fixes #5118.

Given a partition ID, look up the non-deleted Parquet files for that
partition. Separate them into level 0 and level 1, and sort the level 0
files by max sequence number.

This is not called anywhere yet.
 2022-07-13 16:53:21 -04:00
kodiakhq[bot] 45cd4eb504
Merge pull request #5122 from influxdata/cn/back-to-2-levels 
fix: Remove unused level 1 compaction; move level 2 to level 1
 2022-07-13 19:58:33 +00:00
Carol (Nichols || Goulding) d19c468b9d
fix: Remove unused level 1 compaction; move level 2 to level 1 
Fixes #5119.
 2022-07-13 15:05:09 -04:00
kodiakhq[bot] a8c5bd7ac9
Merge pull request #5093 from influxdata/cn/compactor-metrics 
feat: Record metric for number of files in a compaction by compaction level
 2022-07-13 15:54:43 +00:00
Carol (Nichols || Goulding) 61c023139b
refactor: Switch compaction levels to an enum with values rather than separate consts 
Bonuses:

- Type checking
- Validation
- Less casting
- Exhaustiveness checking
- Less use of the numerical value
 2022-07-13 11:30:36 -04:00
Carol (Nichols || Goulding) 34fcf6a584
fix: Line wrap to 100 columns 2022-07-13 11:29:13 -04:00