a5c74f2798
feat: ability to inject mocked write buffers into server/database
2021-08-12 10:46:16 +02:00
7d105e9229
docs: fix warnings
2021-08-12 09:30:54 +02:00
3de6b44e23
build: use new rustdoc lint name ( #2261 )
...
* fix: nocache feature code rot
The MBChunk::snapshot code when using the "nocache" option no longer
compiles - this commit updates it to match the not(nocache) code.
* build: use updated broken_intra_doc_links name
The broken_intra_doc_links lint was renamed
rustdoc::broken_intra_doc_links
https://doc.rust-lang.org/rustdoc/lints.html
2021-08-11 19:48:51 +00:00
65b1ca2071
fix: also seed persistence windows when skipping replay
2021-08-11 10:27:52 +02:00
2082042626
test: do not wipe-on-error during tests
2021-08-11 10:27:51 +02:00
2eaf486eac
fix: always remember max seen sequ. numbers during replay
...
Do not forget max seen sequence numbers for partition-sequencer
combinations that can be skipped during replay.
Fixes #2215 .
2021-08-11 10:26:12 +02:00
2344c28f4e
feat: drain database jobs on shutdown ( #2239 )
...
* feat: drain database jobs on shutdown
* chore: fmt
* chore: review feedback
* chore: use join() not member directly
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-08-10 16:47:37 +00:00
29ac62c6f8
fix: reduce flakiness of lock_tracker_metrics test ( #2238 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-08-10 11:47:08 +00:00
4cf9244457
test: restore test assertions
2021-08-10 11:29:48 +02:00
cd414f28ef
fix: incorrect speculation of post-persist sequence ranges
...
This fixes an edge case where the speculated sequence ranges that can be
obtained from flush handles do not account for overlapping windows. The
symptom being that the resulting partition checkpoint marked sequence
numbers as unpersisted that where already persisted.
Fixes #2206 .
2021-08-10 11:29:48 +02:00
cd5f4e1755
feat: background worker panic handling ( #2091 ) ( #2234 )
...
* feat: worker panic handling (#2091 )
* chore: add test comments
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-08-10 09:17:56 +00:00
564819d24f
feat: Server own background worker ( #2232 )
...
* feat: Server own background worker
* chore: fix shutdown
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-08-09 18:01:48 +00:00
4dcee10d1e
refactor: do not construct replay plan when skipping replay
...
Up until now we only skipped the execution of the replay plan, not its
construction. The replay plan construction has some bugs left, so let's
move this part behind the toggle as well.
2021-08-09 15:23:39 +02:00
c11eb25d4e
feat: remove create_database_lock ( #2227 )
...
* feat: remove create_database_lock
* chore: review feedback
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-08-09 13:22:11 +00:00
bf15e50ce7
Merge branch 'main' into crepererum/fix_checkpoint_ordering3
2021-08-09 12:27:20 +00:00
54a8fff328
feat: database initialization logging ( #2228 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-08-09 12:13:33 +00:00
559db4529d
refactor: Move DatabaseStore out of query crate ( #2219 )
...
* refactor: Move DatabaseStore out of query crate
* fix: doc links
2021-08-09 12:06:25 +00:00
92334a3747
docs: explain test intend
2021-08-09 13:26:31 +02:00
ae93a1cb89
test: adjust replay tests
2021-08-09 10:54:23 +02:00
950286e5b7
feat: make replay planning work w/ unordered checkpoints
2021-08-09 10:54:23 +02:00
57bbae7e34
refactor: persistence windows row counts are non-zero
2021-08-09 10:33:24 +02:00
c957d8154f
feat: blocking Freezable ( #2224 )
...
* feat: blocking Freezable
* chore: test
2021-08-08 19:26:11 +00:00
1f450ef371
feat: add Database abstraction ( #2186 ) ( #2203 )
...
* feat: add Database abstraction
* chore: minor tweaks
* chore: remove redundant test fixture restart
* chore: review feedback
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-08-08 17:14:23 +00:00
d41b44d312
feat: use zstd compression when writing parquet files ( #2218 )
...
* feat: use ZSTD when writing parquet files
* fix: test
2021-08-06 18:45:55 +00:00
5d525cdc70
docs: Add note about what uses `ApplicationState` ( #2216 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-08-06 14:44:06 +00:00
4c79e3548e
test: do not rely on too many edge cases
2021-08-06 10:24:26 +02:00
882f89cecf
fix: only warn when partition ckpt and DB ckpt mins are out-of-sync
...
There are currently a few bugs and semi-understood edge cases that can
lead to this case. So instead of bailing out, just issue a warning.
2021-08-06 09:48:26 +02:00
4ffdb3d95d
test: drop-unpersisted is not required to trigger that bug
2021-08-06 09:48:26 +02:00
bde2b2b5df
refactor: `Tick` -> `MakeWritesPersistable`
2021-08-05 14:21:36 +02:00
548145a70e
docs: state that `background_worker_now_override` is for testing only
...
Co-authored-by: Andrew Lamb <alamb@influxdata.com>
2021-08-05 14:08:24 +02:00
015d858f88
test: add failing regression test for #2185
...
We need a partition that is partially persisted for this.
This requires some rework for the time handling in `Db` to make it
mockable.
The remaining bits are test framework extensions.
2021-08-05 11:44:44 +02:00
dd9beab166
feat: error database if no rules ( #2187 )
2021-08-04 11:58:59 +00:00
60aee3e70c
refactor: avoid copying a sequence
2021-08-04 13:23:30 +02:00
1b2e331ec1
test: extend replay tests a bit
2021-08-04 13:23:30 +02:00
af1edcdcbb
fix: docstrings
...
Co-authored-by: Andrew Lamb <alamb@influxdata.com>
2021-08-04 13:23:30 +02:00
39f30fd0b6
test: make test queries easier to understand
2021-08-04 13:23:30 +02:00
567ef7e991
test: expland replay tests a bit
2021-08-04 13:23:30 +02:00
b868cd160e
docs: fix code comment about sequence ranges
2021-08-04 13:23:30 +02:00
ed70b73fd8
test: determistic concurreny for `TestDb`
2021-08-04 13:23:30 +02:00
a2bc97b923
feat: prune sequence numbers during replay
...
This only prunes entire sequence numbers, it does not (yet!) prune
individual rows for sequence numbers that are partially persisted.
2021-08-04 13:23:30 +02:00
7a18087044
feat: Log messages during database initialization ( #2180 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-08-04 11:04:41 +00:00
65991270e4
refactor: rename handle and shutdown to link them to background worker
2021-08-04 12:04:47 +02:00
c2faf0876b
fix: fix typo and explain policy storage
2021-08-04 11:55:31 +02:00
42953b0561
fix: increase max wait time for compaction to 60s
2021-08-04 11:51:07 +02:00
164c6e3743
feat: improve hard buffer logging and use that as test assertions
2021-08-04 11:49:05 +02:00
657f469317
test: fix `seek_to_end_works`
2021-08-04 11:33:47 +02:00
6ce1984d75
test: improve hard buffer limit tests
2021-08-04 11:33:47 +02:00
3ac88ffc49
fix: hard buffer limits around write buffer consumption
...
- when reading entries from write buffer during normal playback, do not
throw away entries when hitting the hard buffer limit. instead wait
for compaction to sort it out
- during playback, wait for compaction
2021-08-04 11:33:47 +02:00
9ea04a42ff
refactor: start background worker before performing replay
...
This enables compaction during replay.
2021-08-04 11:33:47 +02:00
0fe8eda89e
refactor: move lifecycle policy into Db struct
2021-08-04 11:33:47 +02:00
Marco Neumann
Dom
Raphael Taylor-Davies
kodiakhq[bot]
Andrew Lamb