7059f16b9e
refactor: Turn mutable_linger_seconds into a non-optional ( #1917 )
2021-07-08 11:25:57 +02:00
0ec11eb907
fix: Handle None mutable_linger_seconds in can_move
...
This will be followed by a cleanup PR that will no longer make mutable_linger_seconds an option
and move the defaulting to the configuration layer.
Closes #1899
2021-07-08 11:02:09 +02:00
c63e2fe605
chore: Add debug logs to maybe_compact_chunks ( #1911 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-07-07 12:06:26 +00:00
17b82ebcd6
chore: Add some more tests for can_move
2021-07-07 13:03:38 +02:00
41f49db3c1
fix: Increase log level for important lifecycle events
...
This can possibly help us better understand #1899 .
The practical reason for increasing the log level for those
events is that we cannot currently ingest debug logs in our log aggregation system,
but it currently takes 7h to reproduce #1899 which means we don't have access to debug logs
from `kubectl logs` to help us troubleshoot.
That said, I do think that these logs are not debug logs but legit lifecycle information.
In particular if a log says "unexpected ..." I think it may be better regarded as an error/warning.
(If it becomes too verbose, it means its either a bug or not that unexpected).
2021-07-07 10:57:56 +02:00
4f5fe62428
feat: add DB name to lifecycle logs ( #1900 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-07-06 16:14:28 +00:00
f45679183c
chore: use saturating sub instead of simple sub
2021-07-06 16:41:36 +02:00
677314f52f
fix: `persist_row_threshold` limits the out chunk row count
...
`persist_row_threshold` should limit the rows of the post-compaction
output chunk (and hence the sum of rows over the input chunks), not
the number of rows of each individual input chunk.
Fixes #1874 .
2021-07-06 15:17:56 +02:00
404da38d6f
Merge branch 'main' into pd-remove-mb-size-limit-checks
2021-07-01 20:01:32 +00:00
5b00bc69e6
refactor: use Arc<Db> in lifecycle actions ( #1873 )
...
* refactor: use Arc<Db> in lifecycle actions
* chore: review feedback
2021-07-01 19:56:33 +00:00
61917c107f
chore: add test for can_move on row count
2021-07-01 15:49:44 -04:00
91f5478012
feat: remove MUB size threshold
...
Removes the MUB chunk close based on size. Also add a check in lifecycle policy to move if the MUB chunk crosses a default row count threshold.
2021-07-01 14:58:29 -04:00
f1a100c6ae
refactor: remove now unused chunk sort order ( #1854 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-07-01 16:39:45 +00:00
43cabac3ac
feat: don't compact more than row threshold ( #1868 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-07-01 16:31:50 +00:00
99a15cd452
refactor: single lifecycle error enumeration ( #1859 )
...
* refactor: single lifecycle error enumeration
* fix: fmt
Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
2021-06-30 18:35:57 +00:00
635e4d0a7d
refactor: plumbing to prevent compaction starving persistence ( #1852 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-06-30 13:43:25 +00:00
297fc12db8
feat: compact chunks ( #1776 )
...
* feat: compact chunks
* chore: review feedback
* chore: clippy lints
* chore: document sort key algorithm
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-06-24 16:49:10 +00:00
01b0fdabb7
feat: make lifecycle partition-aware ( #1767 )
...
* feat: make lifecycle partition-aware
* chore: further docs
* chore: rename to maybe_free_memory
* chore: fix logical conflicts
* chore: ensure only drops unpersisted chunks
* chore: clippy lints
* chore: fix doc
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-06-22 09:24:15 +00:00
7f188c3c94
chore: lint `lifecycle` crate
2021-06-22 11:05:55 +02:00
ea04ce40dc
feat: transactional lifecycle API ( #1753 )
...
* feat: transactional lifecycle API
* chore: remove redundant upgrade
* feat: lifecycle error propagation
* chore: add usage doctest
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-06-21 13:09:53 +00:00
bf54ab51f2
refactor: split lifecycle into separate crate ( #1730 )
2021-06-15 15:57:47 +00:00
Marko Mikulicic
Marco Neumann
kodiakhq[bot]
Raphael Taylor-Davies
Paul Dix