c28f1a24f8
Disable cgo pool in QueryNode ( #19501 )
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-09-28 16:51:50 +08:00
e5a67682ef
Fix Cgo lock os thread failed ( #19148 ) ( #19487 )
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
Co-authored-by: xiaofan-luan <xiaofan.luan@zilliz.com>
Co-authored-by: Xiaofan <83447078+xiaofan-luan@users.noreply.github.com>
Co-authored-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-09-28 10:58:54 +08:00
ec5c2c03f5
Set cgopool's ExpiryDuration to infinite duration ( #19465 )
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-09-27 19:42:56 +08:00
fe2e3dcf4f
Cherry Pick of SyncSegments ( #19312 ) ( #19476 )
...
Add a SyncSegments to sync meta between DN and DC
Add SyncSegments into proto (#19312 )
See also: #19072
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-09-27 18:16:54 +08:00
a67e2c071b
Use segment statslog instead of separate statslog ( #18991 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-09-05 19:19:11 +08:00
8286afa5db
Add InvalidNodeID logic to remove segments from offline segments ( #18975 ) ( #18990 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-09-02 16:06:59 +08:00
0733aece2d
fix remove offline nodes in watchDmChannel ( #18906 )
...
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2022-08-30 19:34:57 +08:00
2c4e20a886
Fix load segment hangs forever ( #18844 )
...
Caused if the context is timeout
Signed-off-by: yah01 <yang.cen@zilliz.com>
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-08-29 19:44:56 +08:00
9953072d58
Add unit test for case of stream closed ( #18858 )
...
Signed-off-by: yah01 <yang.cen@zilliz.com>
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-08-29 18:16:56 +08:00
6929e94065
Fix watchDmChannel may be out-of-date after compaction issue and add context ( #18790 )
...
Signed-off-by: wayblink <anyang.wang@zilliz.com>
Signed-off-by: wayblink <anyang.wang@zilliz.com>
2022-08-23 17:44:56 +08:00
588d165819
fix load segment hangs ( #18783 )
...
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2022-08-23 16:42:52 +08:00
bc62ca1fab
Release collection resources when all partition released ( #18569 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-08-09 18:32:37 +08:00
d62381afc9
Add cgo worker pool for querynode ( #18542 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-08-09 18:30:42 +08:00
702e3253d0
Close Node/Segment detector when close ShardCluster ( #18476 ) ( #18568 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-08-09 18:18:36 +08:00
9899a5be98
Remove Load Delta Channel Request ( #18394 )
...
Fix #18393
remove watch delta channel task related logic
Co-authored-by: xiaofan-luan <xiaofan.luan@zilliz.com>
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
Co-authored-by: Congqi Xia <congqi.xia@zilliz.com>
2022-08-04 17:50:37 +08:00
dcd6c9e5a9
Limit the concurrency level for single load request ( #18410 )
...
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-07-29 11:36:32 +08:00
1e038a75ef
Use errgroup to load DeltaData in parallel ( #18384 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-07-23 21:36:29 +08:00
f132197ae9
fix sync problem during reblance ( #18344 )
...
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2022-07-20 15:58:28 +08:00
8fcd6aa56b
Fix addSegment error, avoid the risk of mem leaks ( #18308 )
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-07-19 20:16:28 +08:00
85117efdc8
Fix query too slow when insert multi repeated pk data ( #18189 )
...
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-07-11 10:26:24 +08:00
507518f869
Get search field id from search plan and log if loaded index when search segments ( #18184 )
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-07-08 21:58:21 +08:00
997e0668c8
Fix bug in metrics ( #18144 )
...
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-07-08 14:10:22 +08:00
e02dbd5c35
Fix handoff segment unit test uses wrong node id ( #18137 ) ( #18142 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-07-07 17:04:21 +08:00
2dee6534a1
Fix data race in ShardCluster.SyncReplicas ( #18054 ) ( #18059 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-07-05 18:30:20 +08:00
dba779e8c5
Return NotShardLeader error code when dml channel is not watched ( #18055 ) ( #18058 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-07-05 18:28:26 +08:00
e6cfbe8ed1
Improve flow graph node name and log msgID in QueryCoord ( #18044 )
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-07-05 16:46:23 +08:00
e9cf804b85
Log querynode's runtime cpu num ( #18039 )
...
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-07-04 19:12:20 +08:00
c4d3677956
Count growing segment number to estimate cpu usage ( #17820 )
...
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-07-01 22:28:23 +08:00
54c8b762da
Make querynode handle segmentChangeInfo with multiple vchannel ( #18010 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-07-01 19:04:18 +08:00
ad2526a039
Return malloc_trim result and log if malloc_trim successfully ( #17960 )
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-07-01 10:40:19 +08:00
49d9e5facb
Remove growing according to offline as well ( #17968 ) ( #17974 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-06-30 22:32:19 +08:00
ade1f45c34
Fix bugs in monitoring metrics ( #17975 )
...
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-06-30 21:32:19 +08:00
baf1d728c5
Fix bug: get correct number of cpu ( #17973 )
...
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-06-30 21:12:17 +08:00
a875e75557
Wait last version search to release growing safely ( #17713 ) ( #17808 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-06-30 20:22:18 +08:00
0318f150af
Print log after segment search done ( #17877 )
...
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-06-29 20:08:18 +08:00
2e63738884
Log segment type as much as possible in QueryNode ( #17869 )
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-06-28 17:00:18 +08:00
2dd1dc092b
Fix handoff isn't processed correctly ( #17738 ) ( #17848 )
...
Signed-off-by: yah01 <yang.cen@zilliz.com>
Co-authored-by: Congqi Xia <congqi.xia@zilliz.com>
Co-authored-by: Congqi Xia <congqi.xia@zilliz.com>
Co-authored-by: Congqi Xia <congqi.xia@zilliz.com>
2022-06-27 23:18:17 +08:00
ce67ad3524
Fix: Miss channel info when load growing segment ( #17789 )
...
Signed-off-by: zhagnlu <lu.zhang@zilliz.com>
Co-authored-by: zhagnlu <lu.zhang@zilliz.com>
2022-06-25 11:38:15 +08:00
1fb2b9fed8
Refine logs to reduce print ( #17759 )
...
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-06-24 11:56:17 +08:00
294f7e4408
fix: log datapb.SegmentInfo is too large #17685 ( #17727 )
...
Signed-off-by: kejiang <ke.jiang@zilliz.com>
Co-authored-by: kejiang <ke.jiang@zilliz.com>
2022-06-23 14:36:13 +08:00
216e2f80aa
Add msg id to log in query path ( #17677 )
...
/kind improvement
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2022-06-23 10:46:13 +08:00
6cb00aa5bb
fix: log datapb.SegmentInfo is too large #17685 ( #17693 )
...
Signed-off-by: kejiang <ke.jiang@zilliz.com>
Co-authored-by: kejiang <ke.jiang@zilliz.com>
2022-06-22 21:46:13 +08:00
8cf54137cf
Add nil check in reduce and add collection rwmutex when release segments ( #17698 )
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-06-22 16:12:14 +08:00
0282f69eb4
Add msg id to log in search path. ( #17654 )
...
issue: #17513
Add msg id to log in search path,
which should help us to trace a detailed request more easier.
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2022-06-21 13:30:12 +08:00
aa4b42c74b
Remove wait before release collection/partition ( #17623 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-06-17 19:00:11 +08:00
92d06b2e30
Purge memory by the memory state and try to purge after each search ( #17565 )
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-06-17 17:46:10 +08:00
785a5a757f
Use segment version instead of ref cnt ( #17609 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-06-17 17:38:12 +08:00
be4077421e
Filter duplicated sealed segments which are both online and offline ( #17593 )
...
Signed-off-by: Letian Jiang <letian.jiang@zilliz.com>
2022-06-16 17:42:11 +08:00
eb5b0b7fc8
Move SegmentInfo out of VchannelInfo, leave Id only to decrease kv size. Get complete SegmentInfo through RPC ( #17441 )
...
Resolves : #17233 #16047
Signed-off-by: wayblink <anyang.wang@zilliz.com>
2022-06-16 12:00:10 +08:00
875b6f88b0
Add channel information to flow graph node ( #17349 )
...
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2022-06-15 21:04:10 +08:00
bigsheeper
XuanYang-cn
congqixia
wei liu
yah01
wayblink
Xiaofan
xige-16
zhenshan.cao
zhagnlu
codeman
Jiquan Long
Letian Jiang