Commit Graph

6 Commits (e33a0781280ff49926bb466c2f073f3fde721db7)

Author SHA1 Message Date
Joe-Blount 99d0530a21
fix: compactor stuck looping with unproductive compactions (needs vertical split) (#8056)
* chore: adjust with_max_num_files_per_plan to more common setting

This significantly increases write amplification (see change in `written` at the conclusion of the cases)

* fix: compactor looping with unproductive compactions

* chore: formatting cleanup

* chore: fix typo in comment

* chore: add test case that compacts too many files at once

* fix: enforce max file count for compaction

* chore: insta churn from prior commit

---------

Co-authored-by: Dom <dom@itsallbroken.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-06-23 09:19:06 +00:00
Joe-Blount e0ccc2e345
Revert "fix: compactor stuck looping with unproductive compactions (needs vertical split) (#8039)" (#8041)
This reverts commit b219b4b003.
2023-06-21 23:14:35 +00:00
Joe-Blount b219b4b003
fix: compactor stuck looping with unproductive compactions (needs vertical split) (#8039)
* chore: adjust with_max_num_files_per_plan to more common setting

This significantly increases write amplification (see change in `written` at the conclusion of the cases)

* fix: compactor looping with unproductive compactions

* chore: formatting cleanup

* chore: fix typo in comment
2023-06-21 20:23:50 +00:00
Joe-Blount a21596f604
chore: add L1/L2 accumulated size test (#8012)
This adds 4 small test cases intending to test how compaction decisions made affect the final size of L1/L2 files.
The assumption is that when a steady stream of small L0 files is arriving, the compactor needs to be rewriting L1s so they grow to a reasonable size instead of getting left small.
2023-06-16 13:28:59 +00:00
Carol (Nichols || Goulding) 9229ce5668
fix: Rename compactor2_test_utils to compactor_test_utils 2023-05-09 11:02:11 +02:00
Carol (Nichols || Goulding) dd9c5d1b13
fix: Rename compactor2 to compactor 2023-05-09 10:58:55 +02:00