Commit Graph

7 Commits (69486994a9901433201cf51ff4dddcea627a8790)

Author SHA1 Message Date
XuanYang-cn a1386bae7f
fix: Skip to submit l0 tasks when scheduler full (#31270)
See also: #31242

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-03-15 10:21:12 +08:00
XuanYang-cn def72947c7
fix: Trigger l0 compaction when l0 views don't change (#30729)
Trigger l0 compaction when l0 views don't change

So that leftover l0 segments would be compacted in the end.

1. Refresh LevelZero plans in comactionPlanHandler, remove the meta
dependency
of compaction trigger v2
2. Add ForceTrigger method for CompactionView interface
3. rename mu to taskGuard
4. Add a new TriggerTypeLevelZeroViewIDLE
5. Add an idleTicker for compaction view manager

See also: #30098, #30556

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

---------

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-03-05 16:37:00 +08:00
XuanYang-cn 0b6beb7e0f
fix: Fill in info in CompactionSegmentBinlogs (#30279)
After #28873, PartitionID and CollectionID should be filled in
CompactionSegmentBinlog so that DataNode can compose
the correct logPath. However There're some places left forgotten to fill
in the information, causing Datanode downloading `xxx/0/0/xxxx/xxxx`
binlogs during compaction

See also: #30213

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-01-26 10:51:02 +08:00
smellthemoon e52ce370b6
enhance:don't store logPath in meta to reduce memory (#28873)
don't store logPath in meta to reduce memory, when service get
segmentinfo, generate logpath from logid.
#28885

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-01-18 22:06:31 +08:00
XuanYang-cn 2c62bcbbf6
fix: Fix some left comments (#29027)
See also: #28814

---------

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2023-12-13 18:48:39 +08:00
XuanYang-cn 5bac7f7897
fix: Fix L0 compaction in datacoord (#28814)
See also: #27606

---------

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2023-12-05 18:44:37 +08:00
XuanYang-cn 9b371067d2
feat: Add Compaction views and triggers (#27906)
- Add Compaction l0 views
- Add Compaction scheduler
- Add Compaction triggerv2
- Add Compaction view manager

See also: #27606

---------

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2023-11-23 17:30:25 +08:00