823ff1029a
test: Add end to end tracing test with mock Jaeger agent ( #2594 )
2021-09-21 16:07:05 +00:00
3e74859822
Merge pull request #2595 from influxdata/crepererum/issue2575
...
ci: add cargo audit
2021-09-21 15:36:56 +00:00
0a52822eab
Merge branch 'main' into crepererum/issue2575
2021-09-21 15:25:40 +00:00
fb7299a169
fix: bubble up write errors ( #2598 )
...
Fixes #2538 .
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-09-21 11:08:37 -04:00
a80f6dfc3e
ci: deny cargo audit warnings
2021-09-21 15:59:04 +02:00
98900af703
ci: add cargo audit
...
For now we ignore warnings (e.g. "crate is yanked" or "crate is
unmaintained") because:
- internal crates w/ names of crates.io crates (e.g. `query`) are
treated like crates.io-crates even though they shouldn't, see
https://github.com/rustsec/rustsec/issues/232
- many crates are currently unmaintained and require a bit of upstream
work (e.g. `chrono` is currently not very active but uses an old
version of `time` which uses the unmaintained `stdweb`)
Closes #2575 .
2021-09-21 15:59:04 +02:00
2384ea72af
Merge pull request #2599 from influxdata/crepererum/fix_dependabot_cfg
...
chore: fix dependabot config
2021-09-21 13:56:04 +00:00
6c93d99483
chore: fix dependabot config
...
My mistake in #2597 .
2021-09-21 15:45:18 +02:00
15cbba4201
Merge pull request #2597 from influxdata/crepererum/upgrade_azure
...
chore: upgrade azure dependencies
2021-09-21 13:35:36 +00:00
ef5ab67c77
chore: upgrade azure dependencies
2021-09-21 13:35:11 +02:00
b21440af39
Merge pull request #2596 from influxdata/crepererum/do_not_abuse_panic_hook
...
test: do not (ab)use the panic hook for replay tests
2021-09-21 10:58:32 +00:00
c46bc8722c
Merge branch 'main' into crepererum/do_not_abuse_panic_hook
2021-09-21 10:50:35 +00:00
cf39825770
chore: update DataFusion ( #2592 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-09-21 10:25:24 +00:00
015dfb3b16
test: do not (ab)use the panic hook for replay tests
...
The old construct uses a single assert-statement for both:
- "bubble-up" scenario, were a panic should fire
- a check, were a panic should not fire
That makes it easy to add new tests. However we need two rather
questionable things to make that work:
- catch panic: to convert an assertion to a check
- a custom panic hook: to make tests not overly verbose (aka caught
panics should not show up on stdout)
Esp. the custom panic hook doesn't work too well w/ multi-threaded tests
since it might swallow error messages from unrelated tests and makes
debugging of CI failures hard.
So instead of using assertions for checks, we now implement a proper
assertion and a check for each test. That's a bit more code per check
but easier probably more stable.
2021-09-21 12:00:37 +02:00
afe0d11083
Merge pull request #2589 from influxdata/crepererum/issue2518b
...
feat: teach preserved catalog to handle delete predicates
2021-09-21 07:47:50 +00:00
fa30f2206d
Merge branch 'main' into crepererum/issue2518b
2021-09-21 07:39:57 +00:00
46088a7ff1
feat: disable rustyline dirs-next dependency ( #2579 )
2021-09-20 14:42:07 +00:00
6682178d6f
feat: teach preserved catalog to handle delete predicates
2021-09-20 15:51:14 +02:00
680f70f1e9
Merge pull request #2588 from influxdata/crepererum/chunk_id
...
refactor: introduce `ChunkId` type
2021-09-20 13:47:11 +00:00
77d84ca5ab
Merge branch 'main' into crepererum/chunk_id
2021-09-20 13:39:05 +00:00
2c9b07264c
Merge pull request #2572 from influxdata/ntran/delete_scan
...
feat: apply negated delete predicates during scan
2021-09-20 13:38:30 +00:00
11413eae8d
docs: mention that `Chunk{Id,Order}::next` might panic
2021-09-20 15:32:44 +02:00
c7e6fffaaa
Merge branch 'main' into ntran/delete_scan
2021-09-20 13:29:47 +00:00
cef5aeee52
refactor: introduce `ChunkId` type
2021-09-20 13:10:41 +02:00
96b1d29c1f
Merge pull request #2585 from influxdata/dependabot/cargo/tracing-subscriber-0.2.24
...
chore(deps): bump tracing-subscriber from 0.2.22 to 0.2.24
2021-09-20 09:52:13 +00:00
a51e9d8c19
chore(deps): bump tracing-subscriber from 0.2.22 to 0.2.24
...
Bumps [tracing-subscriber](https://github.com/tokio-rs/tracing ) from 0.2.22 to 0.2.24.
- [Release notes](https://github.com/tokio-rs/tracing/releases )
- [Commits](https://github.com/tokio-rs/tracing/compare/tracing-subscriber-0.2.22...tracing-subscriber-0.2.24 )
---
updated-dependencies:
- dependency-name: tracing-subscriber
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2021-09-20 09:27:21 +00:00
c77f1ef4e2
Merge pull request #2564 from influxdata/crepererum/issue2518a
...
feat: delete catalog pres. catalog <=> in-mem catalog API
2021-09-20 09:24:27 +00:00
140c71eaf0
Merge branch 'main' into crepererum/issue2518a
2021-09-20 09:16:39 +00:00
e96aa49390
feat: disable prometheus protobuf ( #2578 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-09-20 09:16:04 +00:00
f62d0eab3c
feat: disable bytes serde ( #2580 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-09-20 09:07:12 +00:00
9fba7b0ba0
feat: remove routerify ( #2586 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-09-20 08:57:19 +00:00
acf698c366
fix: delete predicate sorting
2021-09-20 10:48:32 +02:00
c900704e58
chore: upgrade datafusion
2021-09-20 10:48:32 +02:00
0c5ba3786b
refactor: rename closure to make syntax a bit clearer
2021-09-20 10:48:32 +02:00
4c4fd59724
docs: extend comment about (not) cleanup up delete predicates
2021-09-20 10:48:32 +02:00
492d991f49
feat: delete catalog pres. catalog <=> in-mem catalog API
...
First step towards #2518 . Creates the Rust API to communicate delete
predicates between the preserved catalog and the in-memory catalog and
adds tests ensuring that the in-mem catalog produces the wanted errors
as well as correct checkpoints (similar to how this is done for the
parquet file tracking already).
**This does NOT contain the actual preservation!**
2021-09-20 10:48:32 +02:00
1752bf5563
Merge pull request #2584 from influxdata/dependabot/cargo/rand_distr-0.4.2
...
chore(deps): bump rand_distr from 0.4.1 to 0.4.2
2021-09-20 08:47:57 +00:00
876bb10cf8
chore(deps): bump rand_distr from 0.4.1 to 0.4.2
...
Bumps [rand_distr](https://github.com/rust-random/rand ) from 0.4.1 to 0.4.2.
- [Release notes](https://github.com/rust-random/rand/releases )
- [Changelog](https://github.com/rust-random/rand/blob/master/CHANGELOG.md )
- [Commits](https://github.com/rust-random/rand/compare/rand_distr-0.4.1...rand_distr-0.4.2 )
---
updated-dependencies:
- dependency-name: rand_distr
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2021-09-20 08:39:39 +00:00
598179160a
Merge pull request #2583 from influxdata/dependabot/cargo/dirs-4.0.0
...
chore(deps): bump dirs from 3.0.2 to 4.0.0
2021-09-20 08:37:40 +00:00
1112597c44
chore(deps): bump dirs from 3.0.2 to 4.0.0
...
Bumps [dirs](https://github.com/soc/dirs-rs ) from 3.0.2 to 4.0.0.
- [Release notes](https://github.com/soc/dirs-rs/releases )
- [Commits](https://github.com/soc/dirs-rs/commits )
---
updated-dependencies:
- dependency-name: dirs
dependency-type: direct:production
update-type: version-update:semver-major
...
Signed-off-by: dependabot[bot] <support@github.com>
2021-09-20 08:28:04 +00:00
5182ae92b0
Merge pull request #2582 from influxdata/dependabot/cargo/tracing-0.1.28
...
chore(deps): bump tracing from 0.1.27 to 0.1.28
2021-09-20 08:26:03 +00:00
c926994491
chore(deps): bump tracing from 0.1.27 to 0.1.28
...
Bumps [tracing](https://github.com/tokio-rs/tracing ) from 0.1.27 to 0.1.28.
- [Release notes](https://github.com/tokio-rs/tracing/releases )
- [Commits](https://github.com/tokio-rs/tracing/compare/tracing-0.1.27...tracing-0.1.28 )
---
updated-dependencies:
- dependency-name: tracing
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2021-09-20 08:17:24 +00:00
6f72028189
Merge pull request #2581 from influxdata/dependabot/cargo/hyper-0.14.13
...
chore(deps): bump hyper from 0.14.12 to 0.14.13
2021-09-20 08:13:33 +00:00
82c3fb35e7
chore(deps): bump hyper from 0.14.12 to 0.14.13
...
Bumps [hyper](https://github.com/hyperium/hyper ) from 0.14.12 to 0.14.13.
- [Release notes](https://github.com/hyperium/hyper/releases )
- [Changelog](https://github.com/hyperium/hyper/blob/master/CHANGELOG.md )
- [Commits](https://github.com/hyperium/hyper/compare/v0.14.12...v0.14.13 )
---
updated-dependencies:
- dependency-name: hyper
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2021-09-20 07:53:18 +00:00
fa5c884aab
Merge pull request #2528 from influxdata/crepererum/compress_parquet_metadata
...
feat: compress encoded parquet metadata
2021-09-20 07:51:24 +00:00
831e55d79e
refactor: make error messages more precise
2021-09-20 09:42:55 +02:00
0f5198c88d
test: fix tests dealing w/ parquet metadata sizes
...
Sizes now depend on the actual content and therefore we need
deterministic timestamps.
2021-09-20 09:42:53 +02:00
e15631002e
test: allow test code to specify exact parquet creation timestamp
...
This is required for deterministic sizes since different timestamp lead
to different compression ratios.
2021-09-20 09:42:52 +02:00
9c80d32af5
refactor: use normal google timestamps in parquet metadata again
...
We changed from Google timestamp (which use variable-sized integers) to
our own fixed-sized integer timestamps so that the size of the parquet
metadata does not depend on the timestamp. However with the introduction
of compression this is the case anyways (since slightly different
timestamps lead to different compression results) and we need now
derministic timestamps for tests. So there is now point in using our own
timestamp type. Switching back to the variable-sized type also shrinks
the post-compression results a bit.
2021-09-20 09:34:03 +02:00
afc507ae14
feat: compress encoded parquet metadata
...
Depending on the number of columns, this should safe between 60% and
75%.
2021-09-20 09:33:18 +02:00
Andrew Lamb
kodiakhq[bot]
Marco Neumann
Raphael Taylor-Davies
dependabot[bot]