milvus

Commit Graph

Author	SHA1	Message	Date
XuanYang-cn	4926021c02	fix: Skip mark compaction timeout for mix and l0 compaction (#37118 ) Timeout is a bad design for long running tasks, especially using a static timeout config. We should monitor execution progress and fail the task if the progress has been stale for a long time. This pr is a small patch to stop DC from marking compaction tasks timeout, while still waiting for DN to finish. The design is self-conflicted. After this pr, mix and L0 compaction are no longer controlled by DC timeout, but clustering is still under timeout control. The compaction queue capacity grows larger for priority calc, hence timeout compactions appears more often, and when timeout, the queuing tasks will be timeout too, no compaction will success after. See also: #37108, #37015 --------- Signed-off-by: yangxuan <xuan.yang@zilliz.com>	2024-10-28 14:33:29 +08:00
wayblink	49b562207c	fix: Refine compactionTask to avoid data race (#36936 ) issue: #36897 Signed-off-by: wayblink <anyang.wang@zilliz.com>	2024-10-24 09:55:28 +08:00
wayblink	277c50fd31	fix:add log in mixCompactionTask and set fail/timeout task to clean (#35970 ) issue: #35966 Signed-off-by: wayblink <anyang.wang@zilliz.com>	2024-10-17 18:05:23 +08:00
XuanYang-cn	323400c190	enhance: Enable to write multiple segments in mix compactor (#35705 ) Prevent segments to be written larger than maxSize * expansionRate See also: #35584 Signed-off-by: yangxuan <xuan.yang@zilliz.com> --------- Signed-off-by: yangxuan <xuan.yang@zilliz.com>	2024-08-30 11:29:01 +08:00
congqixia	c992a61a23	enhance: Separate allocator pkg in datacoord (#35622 ) Related to #28861 Move allocator interface and implementation into separate package. Also update some unittest logic. Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>	2024-08-22 10:06:56 +08:00
yihao.dai	ca758c36cc	enhance: Pre-allocate ids for compaction (#34187 ) This PR removes the dependency of compaction on the ID allocator by pre-allocating the logID and segmentID. issue: https://github.com/milvus-io/milvus/issues/33957 --------- Signed-off-by: bigsheeper <yihao.dai@zilliz.com>	2024-07-17 13:23:42 +08:00
zhenshan.cao	ac4f3997ce	enhance: Reconstructing Compaction to possess persistence capability (#33265 ) issue #33586 Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>	2024-06-05 10:17:50 +08:00

7 Commits (f813fb45637633f61ba73ace5d1a0acabd815897)