b251c3a682
enhance: add ctx for HandleCStatus and callers ( #29517 )
...
See also #29516
Make `HandleCStatus` print trace id for better logging
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-12-27 16:10:47 +08:00
02bc0d0dd5
fix: Add scope limit for querynode DeleteRequest ( #29474 )
...
See also #27515
When Delegator processes delete data, it forwards delete data with only
segment id specified. When two segments has same segment id but one is
growing and the other is sealed, the delete will be applied to both
segments which causes delete data out of order when concurrent load
segment occurs.
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-12-26 14:28:47 +08:00
ac95c52171
enhance: change protection to RLock for loadStreamDelete ( #29450 )
...
See also: #29332
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-12-25 23:27:02 +08:00
1eacdc591b
fix: delegator may mark segment offline by mistake ( #29343 )
...
See also #29332
The segment may be released before or during the request when delegator
tries to forward delete request to yet. Currently, these two situation
returns different error code.
In this particular case, `ErrSegmentNotLoaded` and `ErrSegmentNotFound`
shall both be ignored preventing return search service unavailable by
mistake.
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-12-20 21:22:43 +08:00
9b3e06ae86
enhance: add more metrics for level zero segments ( #29029 )
...
- Add SegmentNum metric for level zero segments
- Add level zero segments size metirc
---------
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-12-07 14:34:35 +08:00
cc952e0486
enhance: optimize forwarding level0 deletions by respecting partition ( #28456 )
...
- Cache the level 0 deletions after loading level0 segments
- Divide the level 0 deletions by partition
related: #27349
---------
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-11-21 18:24:22 +08:00
d20ea061d6
Fix panic while forwarding empty deletions to growing segment ( #28213 )
...
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-11-08 16:42:21 +08:00
ece592a42f
Deliver L0 segments delete records ( #27722 )
...
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-11-07 01:44:18 +08:00
ecec5dfcfd
fix retry on offline node ( #28079 )
...
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-11-03 10:14:16 +08:00
e4fdf5e68e
Refine offline segments logic in shard delegator ( #28073 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-11-01 20:04:24 +08:00
8041fc3c75
avoid add empty growing segment to delegator distribution ( #27930 )
...
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-10-26 10:10:10 +08:00
d9431266bb
Fix panic at loadSegment during rolling upgrade ( #27671 )
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-10-13 16:31:35 +08:00
26f06dd732
Format the code ( #27275 )
...
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-09-21 09:45:27 +08:00
cc9974979f
Add staticcheck linter and fix existing problems ( #27174 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-09-19 10:05:22 +08:00
e51cb739eb
Remove some logs in querynode delegator ( #27156 )
...
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2023-09-18 10:57:20 +08:00
2a5d574a0d
Fix querynodev2 concurrent load logic ( #26959 )
...
Fix logic error from #26926
function `waitSegmentLoadDone` shall return error when context is done
Make delegator control concurrency for each same segment
Related to #26908
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-09-10 07:41:18 +08:00
7f1ae35e72
Add timeout in dispatcher, AsConsumer and Seek ( #26686 )
...
See also: #25309
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2023-09-08 09:51:17 +08:00
4b58c71908
Add ctx parameter for organizeTask and GetWorker method ( #26835 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-09-05 10:05:48 +08:00
3349db4aa7
Refine errors to remove changes breaking design ( #26521 )
...
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-09-04 09:57:09 +08:00
89fc9aad82
Improve sync target version logic ( #26630 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-08-29 23:12:27 +08:00
7af0f7d90c
avoid concurrent sub/unsub on same channel ( #26454 )
...
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-08-23 10:10:22 +08:00
eea9197306
Perform alter distribution under mutex protection ( #26229 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-08-09 13:05:15 +08:00
54c0e64059
Fix search on empty segments set bug ( #26136 )
...
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2023-08-08 11:17:08 +08:00
a132122370
fix time unit in insert/delete cost metrics value ( #25893 )
...
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-07-25 15:07:01 +08:00
b533c68632
Forbid update checkpoint without msgID ( #25694 )
...
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2023-07-25 10:43:04 +08:00
fc19b85a40
fix count(*）retrieve redundant growing segment ( #25825 )
...
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-07-24 14:09:00 +08:00
32827f538a
add metrics for delegator insert/delete cost ( #25733 )
...
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-07-21 15:30:59 +08:00
cc71e065e3
Support load index after segment loaded ( #25567 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-07-18 10:51:19 +08:00
a7023ab3cc
Narrow the scope of deleteMut lock in loadStreamDelete ( #25436 )
...
Signed-off-by: Ted Yu <yuzhihong@gmail.com>
2023-07-17 09:30:34 +08:00
b62c82af22
fix set target version in loading sealed segment ( #25603 )
...
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-07-14 20:00:31 +08:00
205a7c430a
Fix QueryNode panic while inserting to released segment ( #25495 )
...
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-07-12 19:48:29 +08:00
f9e2d00f91
Prevent `exclusive consumer` exception in pulsar ( #25376 )
...
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-07-12 17:26:30 +08:00
68ae199a9f
load segment with target version, avoid read redundant segment ( #24929 )
...
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-06-27 11:48:45 +08:00
41af0a98fa
Use go-api/v2 for milvus-proto ( #24770 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-06-09 01:28:37 +08:00
5f84923bfa
Fix search/query unloaded data ( #24368 )
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-05-26 15:01:26 +08:00
400364483d
Improve predict the resource usage ( #23968 )
...
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-05-11 15:33:24 +08:00
7e0d1492c7
Load delete from channel checkpoint ( #23961 )
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-05-09 19:10:41 +08:00
3ad9ff7a9a
fix release segment failed ( #23462 )
...
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-04-17 18:46:30 +08:00
6cd73d4e2a
Wait safe signal before release growing ( #23358 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-04-11 17:56:31 +08:00
ba02e70301
fix bug when syncing distribution without schema and remove unused logic( #23085 ) ( #23216 )
...
Signed-off-by: MrPresent-Han <jamesharden11122@gmail.com>
2023-04-07 19:32:29 +08:00
c9d0c157ec
Move some modules from internal to public package ( #22572 )
...
Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-04-06 19:14:32 +08:00
e2096965c7
fix leader view ( #23038 )
...
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-03-29 14:06:02 +08:00
081572d31c
Refactor QueryNode ( #21625 )
...
Signed-off-by: yah01 <yang.cen@zilliz.com>
Co-authored-by: Congqi Xia <congqi.xia@zilliz.com>
Co-authored-by: aoiasd <zhicheng.yue@zilliz.com>
2023-03-27 00:42:00 +08:00
congqixia
yah01
wei liu
yihao.dai
SimFG
Xiaofan
XuanYang-cn
Bingyi Sun
aoiasd
Zhihong Yu
yah01
MrPresent-Han
jaime