ac95c52171
enhance: change protection to RLock for loadStreamDelete ( #29450 )
...
See also: #29332
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-12-25 23:27:02 +08:00
3e77365de5
fix: correct autoindex segment num ( #28387 )
...
Fix #28386
Current code snippet
```
// get delegator
sd, ok := node.delegators.Get(channel)
if !ok {
err := merr.WrapErrChannelNotFound(channel)
log.Warn("Query failed, failed to get shard delegator for search", zap.Error(err))
return nil, err
}
req, err = node.optimizeSearchParams(ctx, req, sd)
if err != nil {
log.Warn("failed to optimize search params", zap.Error(err))
return nil, err
}
// do search
results, err := sd.Search(searchCtx, req)
```
We could move these into `ShardDelegator`, and directly use sealed
segment num in `Search` methods, also segment num got outside could be
wrong when we specify partitions.
Signed-off-by: chasingegg <chao.gao@zilliz.com>
2023-11-22 11:12:22 +08:00
cc952e0486
enhance: optimize forwarding level0 deletions by respecting partition ( #28456 )
...
- Cache the level 0 deletions after loading level0 segments
- Divide the level 0 deletions by partition
related: #27349
---------
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-11-21 18:24:22 +08:00
e4fdf5e68e
Refine offline segments logic in shard delegator ( #28073 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-11-01 20:04:24 +08:00
13877a07ff
Add ctx parameter for tsafe pkg & NewDelegator method ( #27877 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-10-26 19:14:10 +08:00
9ba79a97e4
Add some log for delete task when it need query ( #27317 )
...
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2023-10-24 14:54:12 +08:00
be980fbc38
Refine state check ( #27541 )
...
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-10-11 21:01:35 +08:00
258e1ccd66
Refine querynode scheduler lifetime ( #26915 )
...
This PR refines scheduler lifetime control:
- Move private tri-state into lifetime package
- Make scheduler block incoming "Add" task
- Make scheduler Stop wait until all previously accepted task done
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-09-28 10:21:26 +08:00
26f06dd732
Format the code ( #27275 )
...
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-09-21 09:45:27 +08:00
e107d0794c
support complex delete expression ( #25752 )
...
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2023-09-12 10:19:17 +08:00
2a5d574a0d
Fix querynodev2 concurrent load logic ( #26959 )
...
Fix logic error from #26926
function `waitSegmentLoadDone` shall return error when context is done
Make delegator control concurrency for each same segment
Related to #26908
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-09-10 07:41:18 +08:00
af5c01082b
Refine delegator lifetime control ( #26881 )
...
- Add SafeChan interface in lifetime package
- Embed SafeChan into interface
- Replace private lifetime struct in delegator package with
- Refine delegator on-going task lifetime control and wait all accepted task done
- Fix potential goroutine leakage from if delegator closed concurrently
/kind improvement
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-09-07 10:11:15 +08:00
528948559f
fix false load failure for long unserviable period( #26813 ) ( #26818 )
...
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
2023-09-06 12:57:15 +08:00
4b58c71908
Add ctx parameter for organizeTask and GetWorker method ( #26835 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-09-05 10:05:48 +08:00
7af0f7d90c
avoid concurrent sub/unsub on same channel ( #26454 )
...
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-08-23 10:10:22 +08:00
855a04539a
Set delegator lastUpdateTs to startTs ( #26416 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-08-17 12:24:18 +08:00
99c633954a
Downgrade the log level in search path ( #26188 )
...
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-08-08 18:35:14 +08:00
0e26486abc
fix segment num in log ( #25876 )
...
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-07-25 09:13:00 +08:00
fc19b85a40
fix count(*）retrieve redundant growing segment ( #25825 )
...
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-07-24 14:09:00 +08:00
8d193a3e56
Skip unnecessary reduce phase during search ( #25166 )
...
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2023-06-27 19:22:45 +08:00
68ae199a9f
load segment with target version, avoid read redundant segment ( #24929 )
...
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-06-27 11:48:45 +08:00
e40d95e093
add search/query request cost metrics on qn ( #24413 )
...
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-06-15 14:24:38 +08:00
e81eb56b92
Add metric of waiting for tSafe latency ( #24765 )
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-06-09 11:26:35 +08:00
41af0a98fa
Use go-api/v2 for milvus-proto ( #24770 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-06-09 01:28:37 +08:00
5f84923bfa
Fix search/query unloaded data ( #24368 )
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-05-26 15:01:26 +08:00
ce03248b1a
simplify logic circle in search ( #24110 )
...
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-05-23 16:01:26 +08:00
c9d0c157ec
Move some modules from internal to public package ( #22572 )
...
Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-04-06 19:14:32 +08:00
e3e3ac1525
Use peek segment info in querynodev2 ( #23183 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-04-03 16:10:24 +08:00
b6f69fe7f2
Check if loaded before delegator search/query ( #23162 )
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-04-03 14:26:23 +08:00
081572d31c
Refactor QueryNode ( #21625 )
...
Signed-off-by: yah01 <yang.cen@zilliz.com>
Co-authored-by: Congqi Xia <congqi.xia@zilliz.com>
Co-authored-by: aoiasd <zhicheng.yue@zilliz.com>
2023-03-27 00:42:00 +08:00
congqixia
Gao
yah01
aoiasd
SimFG
MrPresent-Han
wei liu
Jiquan Long
yihao.dai
jaime
yah01