Commit Graph

5142 Commits (b0d8b679bed0c9064a2a3f24340a20ef0c3fccaa)

Author SHA1 Message Date
Marco Neumann 11413eae8d docs: mention that `Chunk{Id,Order}::next` might panic 2021-09-20 15:32:44 +02:00
kodiakhq[bot] c7e6fffaaa
Merge branch 'main' into ntran/delete_scan 2021-09-20 13:29:47 +00:00
Marco Neumann cef5aeee52 refactor: introduce `ChunkId` type 2021-09-20 13:10:41 +02:00
kodiakhq[bot] 96b1d29c1f
Merge pull request #2585 from influxdata/dependabot/cargo/tracing-subscriber-0.2.24
chore(deps): bump tracing-subscriber from 0.2.22 to 0.2.24
2021-09-20 09:52:13 +00:00
dependabot[bot] a51e9d8c19
chore(deps): bump tracing-subscriber from 0.2.22 to 0.2.24
Bumps [tracing-subscriber](https://github.com/tokio-rs/tracing) from 0.2.22 to 0.2.24.
- [Release notes](https://github.com/tokio-rs/tracing/releases)
- [Commits](https://github.com/tokio-rs/tracing/compare/tracing-subscriber-0.2.22...tracing-subscriber-0.2.24)

---
updated-dependencies:
- dependency-name: tracing-subscriber
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-09-20 09:27:21 +00:00
kodiakhq[bot] c77f1ef4e2
Merge pull request #2564 from influxdata/crepererum/issue2518a
feat: delete catalog pres. catalog <=> in-mem catalog API
2021-09-20 09:24:27 +00:00
kodiakhq[bot] 140c71eaf0
Merge branch 'main' into crepererum/issue2518a 2021-09-20 09:16:39 +00:00
Raphael Taylor-Davies e96aa49390
feat: disable prometheus protobuf (#2578)
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-09-20 09:16:04 +00:00
Raphael Taylor-Davies f62d0eab3c
feat: disable bytes serde (#2580)
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-09-20 09:07:12 +00:00
Raphael Taylor-Davies 9fba7b0ba0
feat: remove routerify (#2586)
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-09-20 08:57:19 +00:00
Marco Neumann acf698c366 fix: delete predicate sorting 2021-09-20 10:48:32 +02:00
Marco Neumann c900704e58 chore: upgrade datafusion 2021-09-20 10:48:32 +02:00
Marco Neumann 0c5ba3786b refactor: rename closure to make syntax a bit clearer 2021-09-20 10:48:32 +02:00
Marco Neumann 4c4fd59724 docs: extend comment about (not) cleanup up delete predicates 2021-09-20 10:48:32 +02:00
Marco Neumann 492d991f49 feat: delete catalog pres. catalog <=> in-mem catalog API
First step towards #2518. Creates the Rust API to communicate delete
predicates between the preserved catalog and the in-memory catalog and
adds tests ensuring that the in-mem catalog produces the wanted errors
as well as correct checkpoints (similar to how this is done for the
parquet file tracking already).

**This does NOT contain the actual preservation!**
2021-09-20 10:48:32 +02:00
kodiakhq[bot] 1752bf5563
Merge pull request #2584 from influxdata/dependabot/cargo/rand_distr-0.4.2
chore(deps): bump rand_distr from 0.4.1 to 0.4.2
2021-09-20 08:47:57 +00:00
dependabot[bot] 876bb10cf8
chore(deps): bump rand_distr from 0.4.1 to 0.4.2
Bumps [rand_distr](https://github.com/rust-random/rand) from 0.4.1 to 0.4.2.
- [Release notes](https://github.com/rust-random/rand/releases)
- [Changelog](https://github.com/rust-random/rand/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-random/rand/compare/rand_distr-0.4.1...rand_distr-0.4.2)

---
updated-dependencies:
- dependency-name: rand_distr
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-09-20 08:39:39 +00:00
kodiakhq[bot] 598179160a
Merge pull request #2583 from influxdata/dependabot/cargo/dirs-4.0.0
chore(deps): bump dirs from 3.0.2 to 4.0.0
2021-09-20 08:37:40 +00:00
dependabot[bot] 1112597c44
chore(deps): bump dirs from 3.0.2 to 4.0.0
Bumps [dirs](https://github.com/soc/dirs-rs) from 3.0.2 to 4.0.0.
- [Release notes](https://github.com/soc/dirs-rs/releases)
- [Commits](https://github.com/soc/dirs-rs/commits)

---
updated-dependencies:
- dependency-name: dirs
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-09-20 08:28:04 +00:00
kodiakhq[bot] 5182ae92b0
Merge pull request #2582 from influxdata/dependabot/cargo/tracing-0.1.28
chore(deps): bump tracing from 0.1.27 to 0.1.28
2021-09-20 08:26:03 +00:00
dependabot[bot] c926994491
chore(deps): bump tracing from 0.1.27 to 0.1.28
Bumps [tracing](https://github.com/tokio-rs/tracing) from 0.1.27 to 0.1.28.
- [Release notes](https://github.com/tokio-rs/tracing/releases)
- [Commits](https://github.com/tokio-rs/tracing/compare/tracing-0.1.27...tracing-0.1.28)

---
updated-dependencies:
- dependency-name: tracing
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-09-20 08:17:24 +00:00
kodiakhq[bot] 6f72028189
Merge pull request #2581 from influxdata/dependabot/cargo/hyper-0.14.13
chore(deps): bump hyper from 0.14.12 to 0.14.13
2021-09-20 08:13:33 +00:00
dependabot[bot] 82c3fb35e7
chore(deps): bump hyper from 0.14.12 to 0.14.13
Bumps [hyper](https://github.com/hyperium/hyper) from 0.14.12 to 0.14.13.
- [Release notes](https://github.com/hyperium/hyper/releases)
- [Changelog](https://github.com/hyperium/hyper/blob/master/CHANGELOG.md)
- [Commits](https://github.com/hyperium/hyper/compare/v0.14.12...v0.14.13)

---
updated-dependencies:
- dependency-name: hyper
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-09-20 07:53:18 +00:00
kodiakhq[bot] fa5c884aab
Merge pull request #2528 from influxdata/crepererum/compress_parquet_metadata
feat: compress encoded parquet metadata
2021-09-20 07:51:24 +00:00
Marco Neumann 831e55d79e refactor: make error messages more precise 2021-09-20 09:42:55 +02:00
Marco Neumann 0f5198c88d test: fix tests dealing w/ parquet metadata sizes
Sizes now depend on the actual content and therefore we need
deterministic timestamps.
2021-09-20 09:42:53 +02:00
Marco Neumann e15631002e test: allow test code to specify exact parquet creation timestamp
This is required for deterministic sizes since different timestamp lead
to different compression ratios.
2021-09-20 09:42:52 +02:00
Marco Neumann 9c80d32af5 refactor: use normal google timestamps in parquet metadata again
We changed from Google timestamp (which use variable-sized integers) to
our own fixed-sized integer timestamps so that the size of the parquet
metadata does not depend on the timestamp. However with the introduction
of compression this is the case anyways (since slightly different
timestamps lead to different compression results) and we need now
derministic timestamps for tests. So there is now point in using our own
timestamp type. Switching back to the variable-sized type also shrinks
the post-compression results a bit.
2021-09-20 09:34:03 +02:00
Marco Neumann afc507ae14 feat: compress encoded parquet metadata
Depending on the number of columns, this should safe between 60% and
75%.
2021-09-20 09:33:18 +02:00
Marco Neumann 7e804db0a3 fix: use btree map for some protobuf messages for deterministic outputs 2021-09-20 09:33:18 +02:00
Raphael Taylor-Davies 5298543321
feat: don't use opentelemetry async trace exporter (#2576)
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-09-19 13:20:13 +00:00
Raphael Taylor-Davies 4382bc0b71
feat: disable Snafu futures feature (#2573)
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-09-19 11:36:24 +00:00
Raphael Taylor-Davies 3ffb16daa6
feat: remove parse_duration (#2574)
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-09-19 11:27:24 +00:00
Raphael Taylor-Davies 62f302b31a
feat: remove human format (#2577) 2021-09-19 11:14:26 +00:00
Raphael Taylor-Davies 0ff2f9bb60
feat: alter http query endpoint (#2559)
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-09-18 09:24:27 +00:00
kodiakhq[bot] 63a1ed147f
Merge pull request #2566 from influxdata/cn/detailed-db-list
feat: Detailed database list command
2021-09-17 20:23:10 +00:00
Nga Tran 364d245eae feat: apply negated delete predicates during scan 2021-09-17 16:20:42 -04:00
Carol (Nichols || Goulding) ac9c25d33c fix: Use prettytable for detailed database output 2021-09-17 15:59:36 -04:00
Carol (Nichols || Goulding) 6520985b5d refactor: Make the test regexes more readable 2021-09-17 15:27:23 -04:00
Carol (Nichols || Goulding) 51a40b31bf feat: Add a --detailed option to the database list CLI
That will list both active and deleted databases with their generations.

Closes #2462.
2021-09-17 15:27:23 -04:00
Carol (Nichols || Goulding) 423a976744 refactor: Use iterator adaptors rather than for loops 2021-09-17 15:27:23 -04:00
Carol (Nichols || Goulding) 44a89cdf75 refactor: Change DeletedDatabase to DetailedDatabase
So this info can be reused for active databases in detailed database
lists.
2021-09-17 15:27:22 -04:00
kodiakhq[bot] cbf166c035
Merge pull request #2464 from influxdata/cn/restore
feat: Add an API for restoring a database that was marked deleted
2021-09-17 18:01:45 +00:00
kodiakhq[bot] 23cc980d9e
Merge branch 'main' into cn/restore 2021-09-17 17:52:56 +00:00
Nga Tran 60a866ddcb refactor: merge delete predicates into select predicate 2021-09-17 07:52:33 -04:00
kodiakhq[bot] 0168c043ad
Merge pull request #2565 from influxdata/ntran/delete_tests
test: tests for delete
2021-09-16 22:05:37 +00:00
Nga Tran 243cc1f88c fix: compile error after merge from main 2021-09-16 17:56:33 -04:00
Nga Tran 0444d1b4fd chore: merge main to branch 2021-09-16 17:28:37 -04:00
Nga Tran 6cfeeb352b refactor: address review comments 2021-09-16 17:21:06 -04:00
Raphael Taylor-Davies 37b615f301
feat: migrate operations CLI to use pbjson (#2562)
* feat: migrate operations CLI to use pbjson

* fix: reserve removed field

* chore: review feedback

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-09-16 19:15:24 +00:00