c6665c2a4c
test: support multiple data/querynodes in integration test ( #30618 )
...
issue: https://github.com/milvus-io/milvus/issues/29507
Signed-off-by: yiwangdr <yiwangdr@gmail.com>
2024-02-21 11:54:53 +08:00
8fc4ebfa11
fix: empty MetricType ( #30216 )
...
issue: #30102 #30225
we should read MetricType from SearchResult,
because query node never
1. read metricType from LoadMeta
2. store to collection
3. set SearchRequest.MetricType
Signed-off-by: PowderLi <min.li@zilliz.com>
2024-01-28 15:33:02 +08:00
f2c0ead51a
enhance: Add trace span for scheduling read tasks in QueryNode ( #30265 )
...
This PR adds a trace span for search/query task scheduling duration
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-01-25 16:59:00 +08:00
0d4e781f69
fix: the system rejects all queries and never recovers if enabled read rate limit ( #30061 )
...
fix #30060
Signed-off-by: yah01 <yang.cen@zilliz.com>
2024-01-17 23:30:55 +08:00
60e88fb833
fix: Restore the MVCC functionality. ( #29749 )
...
When the TimeTravel functionality was previously removed, it
inadvertently affected the MVCC functionality within the system. This PR
aims to reintroduce the internal MVCC functionality as follows:
1. Add MvccTimestamp to the requests of Search/Query and the results of
Search internally.
2. When the delegator receives a Query/Search request and there is no
MVCC timestamp set in the request, set the delegator's current tsafe as
the MVCC timestamp of the request. If the request already has an MVCC
timestamp, do not modify it.
3. When the Proxy handles Search and triggers the second phase ReQuery,
divide the ReQuery into different shards and pass the MVCC timestamp to
the corresponding Query requests.
issue: #29656
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2024-01-09 11:38:48 +08:00
9e0163e12f
enhance: use GPU pool for gpu tasks ( #29678 )
...
- this much improve the performance for GPU index
Signed-off-by: yah01 <yang.cen@zilliz.com>
2024-01-04 17:50:46 +08:00
b251c3a682
enhance: add ctx for HandleCStatus and callers ( #29517 )
...
See also #29516
Make `HandleCStatus` print trace id for better logging
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-12-27 16:10:47 +08:00
be980fbc38
Refine state check ( #27541 )
...
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-10-11 21:01:35 +08:00
6ff4ffa6c2
Fix scheduler may panic during close ( #27431 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-09-28 16:53:26 +08:00
258e1ccd66
Refine querynode scheduler lifetime ( #26915 )
...
This PR refines scheduler lifetime control:
- Move private tri-state into lifetime package
- Make scheduler block incoming "Add" task
- Make scheduler Stop wait until all previously accepted task done
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-09-28 10:21:26 +08:00
6539a5ae2c
Refine DataCoord status ( #27262 )
...
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-09-26 17:15:27 +08:00
26f06dd732
Format the code ( #27275 )
...
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-09-21 09:45:27 +08:00
33e3e78937
add metrics for query reduce ( #27201 )
...
/kind improvement
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
2023-09-19 14:53:22 +08:00
e107d0794c
support complex delete expression ( #25752 )
...
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2023-09-12 10:19:17 +08:00
3349db4aa7
Refine errors to remove changes breaking design ( #26521 )
...
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-09-04 09:57:09 +08:00
5dae6a654a
Protect segment from being released while query/search ( #26322 )
...
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-08-16 18:38:17 +08:00
ca1349708b
Remove time travel ralted testcase ( #26119 )
...
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-08-10 18:53:17 +08:00
f33a89387f
enable config shard level cost metrics whether contains worker's cost ( #26132 )
...
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-08-09 11:37:15 +08:00
302897f866
refine look aside balance logic ( #25837 )
...
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-07-25 18:51:01 +08:00
bcd3abf644
Separate segments/shards reduce latency metrics ( #25475 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-07-11 18:18:28 +08:00
dc1cb54404
[Fixup] unexpected read_task_ready_len metric ( #25349 )
...
Signed-off-by: chyezh <ye.zhen@zilliz.com>
2023-07-06 20:02:29 +08:00
d7d61f529c
[Feature|Pick] enable scheduler policy and add user-task-polling policy ( #24839 )
...
Signed-off-by: chyezh <ye.zhen@zilliz.com>
2023-07-03 18:24:25 +08:00
f7ecbff986
Make merged task channel buffered ( #25242 )
...
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-06-30 14:58:22 +08:00
9b91519c88
fix search cost metrics on replica selection ( #25059 )
...
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-06-27 19:02:44 +08:00
9250423288
Fix QueryNode metrics ( #24916 )
...
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-06-16 16:02:39 +08:00
75adf1df98
fix group nq metrics ( #24918 )
...
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-06-15 19:50:39 +08:00
e40d95e093
add search/query request cost metrics on qn ( #24413 )
...
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-06-15 14:24:38 +08:00
41af0a98fa
Use go-api/v2 for milvus-proto ( #24770 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-06-09 01:28:37 +08:00
44e5daae3a
Fix querynode read quota ( #24412 )
...
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2023-06-08 17:24:36 +08:00
31880ab427
Use conc.PoolOption instead of ants.Option ( #24585 )
...
- Add conc.PoolOption to setup conc.Pool
- Change panic default behavior
- Make future has error when job panicks
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-06-01 15:52:31 +08:00
1c64d0618c
Revert "Consume threads of the number of nq ( #24410 )" ( #24554 )
...
This reverts commit 71a7fef5c5
2023-05-31 16:41:33 +08:00
71a7fef5c5
Consume threads of the number of nq ( #24410 )
...
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-05-26 17:33:27 +08:00
eab94489ba
Refine the merge algorithm ( #23767 )
...
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-04-27 18:26:35 +08:00
f0ababb420
Use merr in querynodev2 ( #23652 )
...
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2023-04-26 17:06:34 +08:00
f8ff97fe29
Fix travel timestamp set unique leads to disable task merge ( #23605 )
...
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-04-25 19:04:34 +08:00
4fe363c4b2
Add KNN cgo pool ( #23526 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-04-19 23:56:31 +08:00
c7c89511b6
Fix data race of timerecord ( #23524 )
...
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-04-19 14:18:30 +08:00
7c4cafc83c
Fix task merge doesn't work ( #23405 )
...
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-04-14 18:18:29 +08:00
0527d2706c
Fix offline set always nil ( #23340 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-04-11 14:42:30 +08:00
c9d0c157ec
Move some modules from internal to public package ( #22572 )
...
Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-04-06 19:14:32 +08:00
2a760c108c
Add metrics for QueryNodeV2 ( #23163 )
...
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-04-03 15:24:24 +08:00
081572d31c
Refactor QueryNode ( #21625 )
...
Signed-off-by: yah01 <yang.cen@zilliz.com>
Co-authored-by: Congqi Xia <congqi.xia@zilliz.com>
Co-authored-by: aoiasd <zhicheng.yue@zilliz.com>
2023-03-27 00:42:00 +08:00
yiwangdr
PowderLi
congqixia
yah01
zhenshan.cao
yah01
SimFG
MrPresent-Han
aoiasd
Enwei Jiao
wei liu
chyezh
smellthemoon
jaime