a8b7629315
fix: exclude insertData before growing checkpoint ( #29558 )
...
Resolves : #29556
Refine exclude segment function signature
Add exclude growing before checkpoint logic
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-12-28 18:18:54 +08:00
6597c72992
fix: compose exclude info from flushed segment id ( #29548 )
...
See also #29526
Previous PR removed flushed segment info from request, which causes
pipeline failing to exclude flushed segment info
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-12-28 14:02:54 +08:00
b251c3a682
enhance: add ctx for HandleCStatus and callers ( #29517 )
...
See also #29516
Make `HandleCStatus` print trace id for better logging
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-12-27 16:10:47 +08:00
02bc0d0dd5
fix: Add scope limit for querynode DeleteRequest ( #29474 )
...
See also #27515
When Delegator processes delete data, it forwards delete data with only
segment id specified. When two segments has same segment id but one is
growing and the other is sealed, the delete will be applied to both
segments which causes delete data out of order when concurrent load
segment occurs.
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-12-26 14:28:47 +08:00
b4af6f8c40
fix: sync action load segment with lack collection index info list ( #28788 )
...
relate: https://github.com/milvus-io/milvus/issues/28779
https://github.com/milvus-io/milvus/issues/28637
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2023-12-04 18:14:34 +08:00
bf633bb5d7
enhance: refine the retry error ( #28573 )
...
return the last error but not combining all errors, to improve
readability and erorr handling
resolve : #28572
---------
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-11-30 18:34:32 +08:00
4b8bc2798e
enhance: Print nq ( #28507 )
...
Log nq in search path.
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-11-30 10:42:27 +08:00
911a915798
feat: enable balance based on growing segment row count ( #28623 )
...
issue: #28622
query node with delegator will has more rows than other query node due
to delgator loads all growing rows.
This PR enable the balance segment which based on the num of growing
rows in leader view.
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-11-27 14:58:26 +08:00
098c1c926d
fix: Add IndexList check for load segment request ( #28601 )
...
See also #28022 #28034
The load segment may reaches before watch dml channel, so the index meta
may be empty as well
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-11-23 19:42:30 +08:00
3e77365de5
fix: correct autoindex segment num ( #28387 )
...
Fix #28386
Current code snippet
```
// get delegator
sd, ok := node.delegators.Get(channel)
if !ok {
err := merr.WrapErrChannelNotFound(channel)
log.Warn("Query failed, failed to get shard delegator for search", zap.Error(err))
return nil, err
}
req, err = node.optimizeSearchParams(ctx, req, sd)
if err != nil {
log.Warn("failed to optimize search params", zap.Error(err))
return nil, err
}
// do search
results, err := sd.Search(searchCtx, req)
```
We could move these into `ShardDelegator`, and directly use sealed
segment num in `Search` methods, also segment num got outside could be
wrong when we specify partitions.
Signed-off-by: chasingegg <chao.gao@zilliz.com>
2023-11-22 11:12:22 +08:00
73f2bab454
enhance:add some log when create client and get component states ( #28160 )
...
/kind improvement
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2023-11-22 09:12:22 +08:00
d20ea061d6
Fix panic while forwarding empty deletions to growing segment ( #28213 )
...
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-11-08 16:42:21 +08:00
ece592a42f
Deliver L0 segments delete records ( #27722 )
...
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-11-07 01:44:18 +08:00
7485eeb689
fix sync distribution with wrong version ( #28130 )
...
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-11-03 19:02:18 +08:00
dc89730a50
Support collection-level mmap control ( #26901 )
...
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-11-02 23:52:16 +08:00
9b737b77a7
fix fromShardLeader typo ( #28061 )
...
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-11-02 15:50:16 +08:00
1b9b8a31a8
Fix QueryNode panic while upgrading ( #28034 )
...
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-10-31 15:50:13 +08:00
13877a07ff
Add ctx parameter for tsafe pkg & NewDelegator method ( #27877 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-10-26 19:14:10 +08:00
be980fbc38
Refine state check ( #27541 )
...
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-10-11 21:01:35 +08:00
0695c88844
refine_log_of_update_target_version ( #27572 )
...
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-10-11 06:51:33 +08:00
2d6a968233
Refine QueryNode errors ( #27380 )
...
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-09-26 22:09:28 +08:00
6539a5ae2c
Refine DataCoord status ( #27262 )
...
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-09-26 17:15:27 +08:00
7f7c71ea7d
Decoupling client and server API in types interface ( #27186 )
...
Co-authored-by:: aoiasd <zhicheng.yue@zilliz.com>
Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-09-26 09:57:25 +08:00
26f06dd732
Format the code ( #27275 )
...
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-09-21 09:45:27 +08:00
168e82ee10
Fix panic while handling with the nil status ( #27040 )
...
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-09-15 10:09:21 +08:00
394c898b4c
Discard SyncDistribution set action from legacy querycoord ( #27027 )
...
Since Milvus in lower version (< 2.3.0), there is no load info in set action
which may corrupt data integrity and cause panicking
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-09-13 09:35:18 +08:00
00c65fa0d7
Refine QueryNode errors ( #27013 )
...
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-09-12 16:07:18 +08:00
e107d0794c
support complex delete expression ( #25752 )
...
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2023-09-12 10:19:17 +08:00
528948559f
fix false load failure for long unserviable period( #26813 ) ( #26818 )
...
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
2023-09-06 12:57:15 +08:00
3349db4aa7
Refine errors to remove changes breaking design ( #26521 )
...
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-09-04 09:57:09 +08:00
e8f1b1736e
Remove log.Error(err.error())-style log ( #26783 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-09-01 13:09:01 +08:00
8330c18dc9
add log for loading segment( #26564 ) ( #26640 )
...
/kind improvement
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
2023-08-31 12:03:00 +08:00
7af0f7d90c
avoid concurrent sub/unsub on same channel ( #26454 )
...
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-08-23 10:10:22 +08:00
123ad921e0
Fix querynodev2 search/query segments return unexpected grpc err ( #26341 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-08-17 18:24:18 +08:00
7d61355ab0
Refactor log for Query ( #26310 )
...
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-08-14 18:57:32 +08:00
9ca8f28005
Fix missing totalChannelNum bug ( #26295 )
...
Signed-off-by: chasingegg <chao.gao@zilliz.com>
2023-08-11 18:35:29 +08:00
ca1349708b
Remove time travel ralted testcase ( #26119 )
...
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-08-10 18:53:17 +08:00
302897f866
refine look aside balance logic ( #25837 )
...
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-07-25 18:51:01 +08:00
fc19b85a40
fix count(*）retrieve redundant growing segment ( #25825 )
...
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-07-24 14:09:00 +08:00
cc71e065e3
Support load index after segment loaded ( #25567 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-07-18 10:51:19 +08:00
49655d2f13
fix panic due to lack of indexInfo on querynode( #25362 ) ( #25613 )
...
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
2023-07-17 19:23:19 +08:00
d216f9abda
Clear collection meta after all channels/segments released ( #25486 )
...
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-07-14 10:28:30 +08:00
b72809db15
fix release sealed segment on unwatch channel ( #25555 )
...
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-07-13 14:30:29 +08:00
205a7c430a
Fix QueryNode panic while inserting to released segment ( #25495 )
...
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-07-12 19:48:29 +08:00
bcd3abf644
Separate segments/shards reduce latency metrics ( #25475 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-07-11 18:18:28 +08:00
5aec6036dc
Add index info in GetDataDistribution response ( #25444 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-07-11 11:22:29 +08:00
cc730c1ee5
fix consume growing from dml after release ( #25421 )
...
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-07-10 17:36:28 +08:00
a47fc28023
fix querynode panic in get distribution ( #25405 )
...
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-07-07 18:26:25 +08:00
d5ec7dda80
fix metric label in search/query count ( #25377 )
...
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-07-06 17:48:26 +08:00
3f9384b515
Fix exclude segment info dml position for flushed segments ( #25200 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-07-06 17:30:26 +08:00
congqixia
aoiasd
yah01
yihao.dai
wei liu
Gao
smellthemoon
jaime
SimFG
MrPresent-Han
Enwei Jiao
yah01