milvus

Commit Graph

Author	SHA1	Message	Date
wei liu	75463725b3	fix: skip loading non-existent L0 segments to prevent load blocking (#43576 ) issue: #43557 In 2.5 branch, L0 segments must be loaded before other segments. If an L0 segment has been garbage collected but is still in the target list, the load operation would keep failing, preventing other segments from being loaded. This patch adds a segment existence check for L0 segments in getSealedSegmentDiff. Only L0 segments that actually exist will be included in the load list. Changes: - Add checkSegmentExist function parameter to SegmentChecker constructor - Filter L0 segments by existence check in getSealedSegmentDiff - Add unit tests using mockey to verify the fix behavior Signed-off-by: Wei Liu <wei.liu@zilliz.com>	2025-07-31 14:33:38 +08:00
congqixia	3d58b2ecee	fix: [2.5] Make controller wait checker worker quit (#42704 ) (#42726 ) Cherry-pick from master pr: #42704 Related to #42702 This patch add wait logic for `CheckerController` Nil check already exists due to code branching Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>	2025-06-16 15:14:38 +08:00
Xianhui Lin	a4eb2ce224	fix: [2.5]Revert qc statschecker for json key stats (#40125 ) Revert qc statschecker for json key stats issue:https://github.com/milvus-io/milvus/issues/36995 pr:https://github.com/milvus-io/milvus/pull/39876 Signed-off-by: Xianhui.Lin <xianhui.lin@zilliz.com>	2025-02-24 13:31:55 +08:00
congqixia	709594f158	enhance: [2.5] Use v2 package name for pkg module (#40117 ) Cherry-pick from master pr: #39990 Related to #39095 https://go.dev/doc/modules/version-numbers Update pkg version according to golang dep version convention Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>	2025-02-23 00:46:01 +08:00
Xianhui Lin	f0964f769d	enhance: [2.5]Add json key inverted index in stats for optimization (#39876 ) Add json key inverted index in stats for optimization issue: https://github.com/milvus-io/milvus/issues/36995 pr: https://github.com/milvus-io/milvus/pull/38039 --------- Signed-off-by: Xianhui.Lin <xianhui.lin@zilliz.com> Co-authored-by: luzhang <luzhang@zilliz.com>	2025-02-16 20:12:15 +08:00
wei liu	f7ecafe77d	enhance: Skip update index for L0 segment (#34099 ) try to update index for l0 segment, will failed by `index not found` This PR skip update index for l0 segment Signed-off-by: Wei Liu <wei.liu@zilliz.com>	2024-07-01 10:26:06 +08:00
Chun Han	f7af323d1e	fix: sync partitiion stats blocking balance task(#33741 ) (#33742 ) related: #33741 Signed-off-by: MrPresent-Han <chun.han@zilliz.com>	2024-06-11 14:21:56 +08:00
wayblink	a1232fafda	feat: Major compaction (#33620 ) #30633 Signed-off-by: wayblink <anyang.wang@zilliz.com> Co-authored-by: MrPresent-Han <chun.han@zilliz.com>	2024-06-10 21:34:08 +08:00
wei liu	2013d97243	enhance: Enable to dynamic update balancer policy in querycoord (#33037 ) issue: #33036 This PR enable to dynamic update balancer policy without restart querycoord. --------- Signed-off-by: Wei Liu <wei.liu@zilliz.com>	2024-05-21 14:29:39 +08:00
wei liu	6dd7297178	fix: Skip generate balance task when target not ready (#30724 ) issue: #30723 This PR skip generate balance task when collection's target isn't ready. also refine the check stale logic in query coord's scheduler, if channel exist in current or next target, task won't be canceled. --------- Signed-off-by: Wei Liu <wei.liu@zilliz.com>	2024-02-23 10:32:53 +08:00
aoiasd	f84d9a589a	fix: channel checker reduce balancing channels. (#30087 ) Ignore leader unavailable when channel checker judge repeat channel to avoid channel checker remove channels balancing. relate: https://github.com/milvus-io/milvus/issues/29841 https://github.com/milvus-io/milvus/issues/29838 Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>	2024-01-26 10:59:00 +08:00
wei liu	f8695aef9d	fix: Trigger leader checker too frequency (#29991 ) issue: #29841 This PR fix leader checker use wrong check interval, which causes leader checker trigger too frequency Signed-off-by: Wei Liu <wei.liu@zilliz.com>	2024-01-17 19:40:53 +08:00
wei liu	e98c62abbb	enhance: refactor leader_observer to leader_checker (#29454 ) issue: #29453 sync distribution by rpc will also call loadSegment/releaseSegment, which may cause all kinds of concurrent case on same segment, such as concurrent load and release on one segment. This PR add leader_checker which generate load/release task to correct the leader view, instead of calling sync distribution by rpc --------- Signed-off-by: Wei Liu <wei.liu@zilliz.com>	2024-01-05 15:54:55 +08:00
Bingyi Sun	45e6801ce4	feat: Add checker activation service interfaces (#28850 ) issue: #28610 --------- Signed-off-by: sunby <sunbingyi1992@gmail.com>	2023-12-04 17:38:37 +08:00
Bingyi Sun	8514a39d1a	feat: Add checker activation (#28611 ) issue: https://github.com/milvus-io/milvus/issues/28610 --------- Signed-off-by: sunby <sunbingyi1992@gmail.com>	2023-11-24 18:08:24 +08:00
wei liu	86ec6f4832	fix load index for stopping node (#28047 ) Signed-off-by: Wei Liu <wei.liu@zilliz.com>	2023-11-03 07:58:18 +08:00
congqixia	852be152de	Change task sourceID to stringer interface (#27965 ) Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>	2023-10-27 01:08:12 +08:00
congqixia	cd5f03f80c	Add var-name sub linter in revive (#27424 ) Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>	2023-10-07 10:09:31 +08:00
yah01	a8ce1b6686	Refine QueryCoord stopping (#27371 ) Signed-off-by: yah01 <yah2er0ne@outlook.com>	2023-09-27 16:27:27 +08:00
SimFG	26f06dd732	Format the code (#27275 ) Signed-off-by: SimFG <bang.fu@zilliz.com>	2023-09-21 09:45:27 +08:00
MrPresent-Han	2101f2d289	fix unstable checker id due to go map iteration(#26943 ) (#26944 ) Signed-off-by: MrPresent-Han <chun.han@zilliz.com>	2023-09-10 10:11:16 +08:00
congqixia	758aad705d	Fix checker using default interval after manual check (#26953 ) Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>	2023-09-09 08:29:16 +08:00
congqixia	1045c88102	Support replace indexed field in QueryCoord (#25747 ) Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>	2023-07-19 21:22:58 +08:00
wei liu	6534396b3d	enable config different interval for different checker (#25514 ) Signed-off-by: Wei Liu <wei.liu@zilliz.com>	2023-07-19 16:50:57 +08:00
congqixia	ed81eaa963	Make CollectionObserver trigger checker more frequently during load procedure (#23928 ) Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>	2023-05-08 14:06:41 +08:00
MrPresent-Han	b517bc9e6a	refine balance mechanism including:(#23454 ) (#23763 ) (#23791 ) 1. balance granuity to replica to avoid influence unrelated replicas 2. avoid balance back and forth Signed-off-by: MrPresent-Han <jamesharden11122@gmail.com>	2023-05-04 12:22:40 +08:00
jaime	c9d0c157ec	Move some modules from internal to public package (#22572 ) Signed-off-by: jaime <yun.zhang@zilliz.com>	2023-04-06 19:14:32 +08:00
MrPresent-Han	afd874b736	enhance segment balance by considering global rowCount(##22914) (#23056 ) Signed-off-by: MrPresent-Han <jamesharden11122@gmail.com> Co-authored-by: xiaofan-luan <xiaofan.luan@zilliz.com>	2023-04-03 14:16:25 +08:00
yah01	3d8f0156c7	Refine scheduler & executor of QueryCoord (#22761 ) Signed-off-by: yah01 <yang.cen@zilliz.com>	2023-03-16 17:43:55 +08:00
Enwei Jiao	89b810a4db	Refactor all params into ParamItem (#20987 ) Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com> Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>	2022-12-07 18:01:19 +08:00
SimFG	f8cff79804	Support the graceful stop for the query node (#20851 ) Signed-off-by: SimFG <bang.fu@zilliz.com> Signed-off-by: SimFG <bang.fu@zilliz.com>	2022-12-06 22:59:19 +08:00
yah01	a82c235599	Scheduler should returns err if the queue is full (#20465 ) Signed-off-by: yah01 <yang.cen@zilliz.com> Signed-off-by: yah01 <yang.cen@zilliz.com>	2022-11-11 17:55:05 +08:00
yah01	8bfa55e560	Fix memory & goroutine leak (#20152 ) Signed-off-by: yah01 <yang.cen@zilliz.com> Signed-off-by: yah01 <yang.cen@zilliz.com>	2022-10-28 14:55:32 +08:00
yah01	c61a4d1243	Fix panic due to double stop (#19849 ) Signed-off-by: yah01 <yang.cen@zilliz.com> Signed-off-by: yah01 <yang.cen@zilliz.com>	2022-10-19 12:13:28 +08:00
yah01	1c71844b8d	Add license header (#19678 ) Signed-off-by: yah01 <yang.cen@zilliz.com> Signed-off-by: yah01 <yang.cen@zilliz.com>	2022-10-11 11:39:22 +08:00
yah01	9dcee37e1c	Skip canceled tasks (#19513 ) Signed-off-by: yah01 <yang.cen@zilliz.com> Signed-off-by: yah01 <yang.cen@zilliz.com>	2022-09-28 19:58:54 +08:00
Bingyi Sun	626854cf0c	Refactor QueryCoord (#18836 ) Signed-off-by: sunby <bingyi.sun@zilliz.com> Co-authored-by: yah01 <yang.cen@zilliz.com> Co-authored-by: Wei Liu <wei.liu@zilliz.com> Co-authored-by: Congqi Xia <congqi.xia@zilliz.com> Signed-off-by: sunby <bingyi.sun@zilliz.com> Co-authored-by: sunby <bingyi.sun@zilliz.com> Co-authored-by: yah01 <yang.cen@zilliz.com> Co-authored-by: Wei Liu <wei.liu@zilliz.com> Co-authored-by: Congqi Xia <congqi.xia@zilliz.com>	2022-09-15 18:48:32 +08:00

37 Commits (2.5)