Commit Graph

11 Commits (2f1efcf517f7504d728f6faddbf296bdc0e5ea26)

Author SHA1 Message Date
Marco Neumann 1ef2bc1887 refactor: `Db::{write_chunk_to_object_store => Db::persist_partition}`
The previous method allowed to persist any chunk -- even ones that
should not be persisted yet and w/o any order of peristence. That will
break our persistence windows. So instead offer a sane higher-level
interface that can trigger persistence of a partition within the
boundaries of the lifecycle rules. This needs some adjustments for our
test suite.
2021-07-16 12:07:58 +02:00
Raphael Taylor-Davies cbeeb97cff
feat: flush open window on persist (#1985)
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-07-14 16:58:20 +00:00
Andrew Lamb 670826daf9
refactor: make object_store construction interface consistent (#1944)
* refactor: make object_store construction interface consistent

* fix: benchmarks

* fix: doc build

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-07-12 12:56:36 +00:00
Andrew Lamb 9e1723620c
refactor: rename load_chunk_to_read_buffer to move_chunk_to_read_buffer (#1857)
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-06-30 16:53:18 +00:00
Carol (Nichols || Goulding) 93881da016 feat: Make Write Buffer store_entry async
In preparation for the Kafka write buffer implementation needing to call
async functions.
2021-06-23 10:48:18 -04:00
Raphael Taylor-Davies ea04ce40dc
feat: transactional lifecycle API (#1753)
* feat: transactional lifecycle API

* chore: remove redundant upgrade

* feat: lifecycle error propagation

* chore: add usage doctest

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-06-21 13:09:53 +00:00
Andrew Lamb 856751deec
feat: Lifecycle manager unloads, rather than drop, chunks when soft limit is hit (#1701)
* feat: unload chunks from memory rather than dropping them

* docs: Update server/src/db/lifecycle.rs

Co-authored-by: Marco Neumann <marco@crepererum.net>

* docs: Update comment wording

Co-authored-by: Marco Neumann <marco@crepererum.net>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-06-14 13:14:39 +00:00
Marco Neumann f4693e36c0 refactor: `catalog_checkpoint_interval` => `catalog_transactions_until_checkpoint` 2021-06-14 10:34:32 +02:00
Marco Neumann 88ec1ef0cf test: enable checkpointing in catalog benchmark
This now creates a checkpoint every 10 transactions. To make it a bit
more fair increase the chunk count to 109, so we have some transactions
after the last checkpoint. With that we improve performance from 10.5s
to 1.2s (or even 0.3s if we would keep the chunk count at 100).
2021-06-14 10:08:32 +02:00
Marco Neumann 2e6f51cbfb fix: fix `server_benchmarks::benches::catalog_persistence` 2021-06-14 10:08:32 +02:00
Marco Neumann 876f642860 test: add benchmark for catalog loading 2021-06-10 15:42:21 +02:00