Commit Graph

5072 Commits (5c78b7d5ae637715b2117d2bf06b795f0aab29ac)

Author SHA1 Message Date
Nga Tran a06afb3932 feat: optimize scan for chunks without delete preidcates and without the need to sort output 2021-09-21 15:56:19 -04:00
kodiakhq[bot] 6d38eaaad7
Merge pull request #2604 from influxdata/ntran/delete_tests_on
test: turn all delete tests on
2021-09-21 19:32:49 +00:00
Nga Tran b4b33c378e test: turn all delete tests on 2021-09-21 15:23:41 -04:00
kodiakhq[bot] 4b0d6a8b08
Merge pull request #2601 from influxdata/ntran/delete_more_tests
fix: All chunks now are applied delete predicates during scan
2021-09-21 18:57:10 +00:00
kodiakhq[bot] 91ce46576c
Merge branch 'main' into ntran/delete_more_tests 2021-09-21 18:47:37 +00:00
Nga Tran 2f371b6f79 refactor: address review comments 2021-09-21 14:46:24 -04:00
Nga Tran 61ed67a5d9 refactor: cleanup 2021-09-21 12:28:18 -04:00
Nga Tran 93551bdd1e fix: all chunks now are applied delete predicates during scan 2021-09-21 12:17:59 -04:00
Andrew Lamb 823ff1029a
test: Add end to end tracing test with mock Jaeger agent (#2594) 2021-09-21 16:07:05 +00:00
kodiakhq[bot] 3e74859822
Merge pull request #2595 from influxdata/crepererum/issue2575
ci: add cargo audit
2021-09-21 15:36:56 +00:00
kodiakhq[bot] 0a52822eab
Merge branch 'main' into crepererum/issue2575 2021-09-21 15:25:40 +00:00
Marco Neumann fb7299a169
fix: bubble up write errors (#2598)
Fixes #2538.

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-09-21 11:08:37 -04:00
Marco Neumann a80f6dfc3e ci: deny cargo audit warnings 2021-09-21 15:59:04 +02:00
Marco Neumann 98900af703 ci: add cargo audit
For now we ignore warnings (e.g. "crate is yanked" or "crate is
unmaintained") because:

- internal crates w/ names of crates.io crates (e.g. `query`) are
  treated like crates.io-crates even though they shouldn't, see
  https://github.com/rustsec/rustsec/issues/232
- many crates are currently unmaintained and require a bit of upstream
  work (e.g. `chrono` is currently not very active but uses an old
  version of `time` which uses the unmaintained `stdweb`)

Closes #2575.
2021-09-21 15:59:04 +02:00
kodiakhq[bot] 2384ea72af
Merge pull request #2599 from influxdata/crepererum/fix_dependabot_cfg
chore: fix dependabot config
2021-09-21 13:56:04 +00:00
Marco Neumann 6c93d99483 chore: fix dependabot config
My mistake in #2597.
2021-09-21 15:45:18 +02:00
kodiakhq[bot] 15cbba4201
Merge pull request #2597 from influxdata/crepererum/upgrade_azure
chore: upgrade azure dependencies
2021-09-21 13:35:36 +00:00
Marco Neumann ef5ab67c77 chore: upgrade azure dependencies 2021-09-21 13:35:11 +02:00
kodiakhq[bot] b21440af39
Merge pull request #2596 from influxdata/crepererum/do_not_abuse_panic_hook
test: do not (ab)use the panic hook for replay tests
2021-09-21 10:58:32 +00:00
kodiakhq[bot] c46bc8722c
Merge branch 'main' into crepererum/do_not_abuse_panic_hook 2021-09-21 10:50:35 +00:00
Andrew Lamb cf39825770
chore: update DataFusion (#2592)
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-09-21 10:25:24 +00:00
Marco Neumann 015dfb3b16 test: do not (ab)use the panic hook for replay tests
The old construct uses a single assert-statement for both:

- "bubble-up" scenario, were a panic should fire
- a check, were a panic should not fire

That makes it easy to add new tests. However we need two rather
questionable things to make that work:

- catch panic: to convert an assertion to a check
- a custom panic hook: to make tests not overly verbose (aka caught
  panics should not show up on stdout)

Esp. the custom panic hook doesn't work too well w/ multi-threaded tests
since it might swallow error messages from unrelated tests and makes
debugging of CI failures hard.

So instead of using assertions for checks, we now implement a proper
assertion and a check for each test. That's a bit more code per check
but easier probably more stable.
2021-09-21 12:00:37 +02:00
kodiakhq[bot] afe0d11083
Merge pull request #2589 from influxdata/crepererum/issue2518b
feat: teach preserved catalog to handle delete predicates
2021-09-21 07:47:50 +00:00
kodiakhq[bot] fa30f2206d
Merge branch 'main' into crepererum/issue2518b 2021-09-21 07:39:57 +00:00
Nga Tran 85989cc8a3 test: add more delete tests and test scenarios 2021-09-20 18:18:08 -04:00
Raphael Taylor-Davies 46088a7ff1
feat: disable rustyline dirs-next dependency (#2579) 2021-09-20 14:42:07 +00:00
Marco Neumann 6682178d6f feat: teach preserved catalog to handle delete predicates 2021-09-20 15:51:14 +02:00
kodiakhq[bot] 680f70f1e9
Merge pull request #2588 from influxdata/crepererum/chunk_id
refactor: introduce `ChunkId` type
2021-09-20 13:47:11 +00:00
kodiakhq[bot] 77d84ca5ab
Merge branch 'main' into crepererum/chunk_id 2021-09-20 13:39:05 +00:00
kodiakhq[bot] 2c9b07264c
Merge pull request #2572 from influxdata/ntran/delete_scan
feat: apply negated delete predicates during scan
2021-09-20 13:38:30 +00:00
Marco Neumann 11413eae8d docs: mention that `Chunk{Id,Order}::next` might panic 2021-09-20 15:32:44 +02:00
kodiakhq[bot] c7e6fffaaa
Merge branch 'main' into ntran/delete_scan 2021-09-20 13:29:47 +00:00
Marco Neumann cef5aeee52 refactor: introduce `ChunkId` type 2021-09-20 13:10:41 +02:00
kodiakhq[bot] 96b1d29c1f
Merge pull request #2585 from influxdata/dependabot/cargo/tracing-subscriber-0.2.24
chore(deps): bump tracing-subscriber from 0.2.22 to 0.2.24
2021-09-20 09:52:13 +00:00
dependabot[bot] a51e9d8c19
chore(deps): bump tracing-subscriber from 0.2.22 to 0.2.24
Bumps [tracing-subscriber](https://github.com/tokio-rs/tracing) from 0.2.22 to 0.2.24.
- [Release notes](https://github.com/tokio-rs/tracing/releases)
- [Commits](https://github.com/tokio-rs/tracing/compare/tracing-subscriber-0.2.22...tracing-subscriber-0.2.24)

---
updated-dependencies:
- dependency-name: tracing-subscriber
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-09-20 09:27:21 +00:00
kodiakhq[bot] c77f1ef4e2
Merge pull request #2564 from influxdata/crepererum/issue2518a
feat: delete catalog pres. catalog <=> in-mem catalog API
2021-09-20 09:24:27 +00:00
kodiakhq[bot] 140c71eaf0
Merge branch 'main' into crepererum/issue2518a 2021-09-20 09:16:39 +00:00
Raphael Taylor-Davies e96aa49390
feat: disable prometheus protobuf (#2578)
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-09-20 09:16:04 +00:00
Raphael Taylor-Davies f62d0eab3c
feat: disable bytes serde (#2580)
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-09-20 09:07:12 +00:00
Raphael Taylor-Davies 9fba7b0ba0
feat: remove routerify (#2586)
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-09-20 08:57:19 +00:00
Marco Neumann acf698c366 fix: delete predicate sorting 2021-09-20 10:48:32 +02:00
Marco Neumann c900704e58 chore: upgrade datafusion 2021-09-20 10:48:32 +02:00
Marco Neumann 0c5ba3786b refactor: rename closure to make syntax a bit clearer 2021-09-20 10:48:32 +02:00
Marco Neumann 4c4fd59724 docs: extend comment about (not) cleanup up delete predicates 2021-09-20 10:48:32 +02:00
Marco Neumann 492d991f49 feat: delete catalog pres. catalog <=> in-mem catalog API
First step towards #2518. Creates the Rust API to communicate delete
predicates between the preserved catalog and the in-memory catalog and
adds tests ensuring that the in-mem catalog produces the wanted errors
as well as correct checkpoints (similar to how this is done for the
parquet file tracking already).

**This does NOT contain the actual preservation!**
2021-09-20 10:48:32 +02:00
kodiakhq[bot] 1752bf5563
Merge pull request #2584 from influxdata/dependabot/cargo/rand_distr-0.4.2
chore(deps): bump rand_distr from 0.4.1 to 0.4.2
2021-09-20 08:47:57 +00:00
dependabot[bot] 876bb10cf8
chore(deps): bump rand_distr from 0.4.1 to 0.4.2
Bumps [rand_distr](https://github.com/rust-random/rand) from 0.4.1 to 0.4.2.
- [Release notes](https://github.com/rust-random/rand/releases)
- [Changelog](https://github.com/rust-random/rand/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-random/rand/compare/rand_distr-0.4.1...rand_distr-0.4.2)

---
updated-dependencies:
- dependency-name: rand_distr
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-09-20 08:39:39 +00:00
kodiakhq[bot] 598179160a
Merge pull request #2583 from influxdata/dependabot/cargo/dirs-4.0.0
chore(deps): bump dirs from 3.0.2 to 4.0.0
2021-09-20 08:37:40 +00:00
dependabot[bot] 1112597c44
chore(deps): bump dirs from 3.0.2 to 4.0.0
Bumps [dirs](https://github.com/soc/dirs-rs) from 3.0.2 to 4.0.0.
- [Release notes](https://github.com/soc/dirs-rs/releases)
- [Commits](https://github.com/soc/dirs-rs/commits)

---
updated-dependencies:
- dependency-name: dirs
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-09-20 08:28:04 +00:00
kodiakhq[bot] 5182ae92b0
Merge pull request #2582 from influxdata/dependabot/cargo/tracing-0.1.28
chore(deps): bump tracing from 0.1.27 to 0.1.28
2021-09-20 08:26:03 +00:00