4aa8a12ce8
fix: [2.4] Check partition in current target when observing partition load status ( #34282 ) ( #34305 )
...
Cherry-pick from master
pr: #34282
See also #34234
`LoadPartitions` does not guarantee the current target has loading
partitions if there are some partitions already loaded before.
This PR check current target contains the partition to load when
advancing loading percentage to 100.
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-07-02 15:48:10 +08:00
f664b51ebe
enhance: [2.4] try to speed up the loading of small collections ( #33746 )
...
- issue: #33569
- pr: #33570
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-06-11 15:07:55 +08:00
b044e5503e
enhance:Improve load speed ( #32898 )
...
fix #32897
add memory check when load collection
Signed-off-by: xiaofanluan <xiaofan.luan@zilliz.com>
2024-05-11 10:29:31 +08:00
4cdf6c3c41
fix: Check partition nil before observe load progress ( #32659 )
...
See also #32441 #32615
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-04-28 16:29:25 +08:00
d7ff1bbe5c
enhance: Make querycoordv2 collection observer task driven ( #32441 )
...
See also #32440
- Add loadTask in collection observer
- For load collection/partitions, load task shall timeout as a whole
- Change related constructor to load jobs
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-04-22 10:39:22 +08:00
c4806b69c4
enhance: Refactor leader view manager interface ( #31133 )
...
issue: #31091
This PR add GetByFilter interface in leader view manager, instead of all
kind of get func
---------
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-04-10 15:13:36 +08:00
797847904c
enhance: Change some frequency log to rated level ( #29720 )
...
This PR change some frequency log to rated level
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-01-11 16:30:50 +08:00
e98c62abbb
enhance: refactor leader_observer to leader_checker ( #29454 )
...
issue: #29453
sync distribution by rpc will also call loadSegment/releaseSegment,
which may cause all kinds of concurrent case on same segment, such as
concurrent load and release on one segment.
This PR add leader_checker which generate load/release task to correct
the leader view, instead of calling sync distribution by rpc
---------
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-01-05 15:54:55 +08:00
514e279f3a
enhance: Remove useless log in collection observer ( #29554 )
...
This PR removed the useless log in collection observer
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-12-28 17:16:47 +08:00
1b90630633
Fix the target updated before version updated to cause data missing ( #28250 )
...
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-11-08 11:36:22 +08:00
e0222b2ce3
refine target manager code style ( #27883 )
...
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-10-25 00:44:12 +08:00
93a877f55e
Make qcv2 target&leader observer execute in parallel ( #27844 )
...
- Add `taskDispatcher` to submit and run task async safely
- Change `LeaderObeserver` and `TargetObserver` schedule and manual check action to submitting task into dispatcher
- Fix logic problem in collection observer when manual check return false
See also #27494
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-10-24 10:14:11 +08:00
42c475a0e0
remove useless log in querycoord ( #27362 )
...
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-10-11 10:13:34 +08:00
eca79d149c
Add ctx control for observer manual check methods ( #27531 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-10-09 11:07:33 +08:00
a8ce1b6686
Refine QueryCoord stopping ( #27371 )
...
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-09-27 16:27:27 +08:00
941a383019
Fix failed to load collection with more than 128 partitions ( #26763 )
...
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-09-02 00:09:01 +08:00
949c320185
remove pull target from qc recover ( #26775 )
...
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-09-01 11:17:01 +08:00
6f89620a43
remove pull target rpc from lock ( #26054 )
...
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-08-04 10:31:06 +08:00
0bc03ede0d
Add eventlog pkg and support grpc streaming event observation ( #25812 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-07-25 17:23:01 +08:00
2bed5bc461
Fix false load success for partition load timeout but collection remains ( #25379 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-07-06 19:10:25 +08:00
68ae199a9f
load segment with target version, avoid read redundant segment ( #24929 )
...
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-06-27 11:48:45 +08:00
ed81eaa963
Make CollectionObserver trigger checker more frequently during load procedure ( #23928 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-05-08 14:06:41 +08:00
c9d0c157ec
Move some modules from internal to public package ( #22572 )
...
Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-04-06 19:14:32 +08:00
74da53c027
fix update load percentage ( #23054 )
...
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-03-30 10:48:23 +08:00
1f718118e9
Dynamic load/release partitions ( #22655 )
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-03-20 14:55:57 +08:00
4005508880
Fix load collection blocks when memory is limited. ( #21990 )
...
Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@zilliz.com>
2023-02-06 16:07:53 +08:00
b0b0929c6c
Change CreateAt to UpdatedAt when checking timeout ( #21769 )
...
Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@zilliz.com>
2023-01-18 17:29:43 +08:00
defb7660c8
Add refresh option for LoadCollection/LoadPartitions interfaces ( #21776 )
...
/kind improvement
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2023-01-18 16:41:44 +08:00
b91bb5a729
Fix load timeout after next target updates ( #21759 )
...
Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@zilliz.com>
2023-01-17 15:15:46 +08:00
c8f89907b6
Fix current target may be updated to an invalid target ( #21742 )
...
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-01-17 11:41:51 +08:00
b04eb17088
Fix load timeout ( #21156 )
...
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2022-12-12 18:43:23 +08:00
27219ce9c0
Fix load progress bug ( #21069 )
...
issue: #21023
Signed-off-by: sunby <bingyi.sun@zilliz.com>
Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@zilliz.com>
2022-12-08 18:41:20 +08:00
89b810a4db
Refactor all params into ParamItem ( #20987 )
...
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2022-12-07 18:01:19 +08:00
0297ab1a46
Make progress if any channel/segment was loaded on node ( #20775 )
...
Signed-off-by: yah01 <yang.cen@zilliz.com>
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-12-06 18:29:19 +08:00
c49f20ea94
Load should base on the accurate loaded number ( #20999 )
...
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2022-12-06 17:13:18 +08:00
d37ebe538b
Record load latency for loaded collection ( #20538 )
...
Signed-off-by: yah01 <yang.cen@zilliz.com>
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-11-15 10:11:07 +08:00
c5cd92d36e
update target ( #19296 )
...
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2022-11-07 19:37:04 +08:00
c61a4d1243
Fix panic due to double stop ( #19849 )
...
Signed-off-by: yah01 <yang.cen@zilliz.com>
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-10-19 12:13:28 +08:00
1c71844b8d
Add license header ( #19678 )
...
Signed-off-by: yah01 <yang.cen@zilliz.com>
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-10-11 11:39:22 +08:00
e6c7286bee
Remove task timeout ( #19583 )
...
Signed-off-by: yah01 <yang.cen@zilliz.com>
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-09-30 19:48:55 +08:00
626854cf0c
Refactor QueryCoord ( #18836 )
...
Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: yah01 <yang.cen@zilliz.com>
Co-authored-by: Wei Liu <wei.liu@zilliz.com>
Co-authored-by: Congqi Xia <congqi.xia@zilliz.com>
Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: yah01 <yang.cen@zilliz.com>
Co-authored-by: Wei Liu <wei.liu@zilliz.com>
Co-authored-by: Congqi Xia <congqi.xia@zilliz.com>
2022-09-15 18:48:32 +08:00
congqixia
SimFG
Xiaofan
wei liu
yah01
jaime
yihao.dai
Bingyi Sun
Ten Thousand Leaves
yah01
smellthemoon
Enwei Jiao