f3eeecf146
Reduce the number of metrics ( #16030 )
...
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-03-15 21:51:21 +08:00
c984c4ea73
Set load meta info in load balance task ( #16060 )
...
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2022-03-15 20:37:21 +08:00
6a66faa2fb
Explictly specify Load operation meta ( #16027 )
...
Fix #15996
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-03-14 09:50:01 +08:00
7124e91d53
Refactor assigning loading segments task in QueryCoord ( #15895 )
...
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-03-07 19:23:58 +08:00
3e862ecdfb
Merge msgChannelConfig and knowhereConfig into commonConfig ( #15843 )
...
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-03-04 11:17:56 +08:00
88527fc0a9
Fix metrics label conflict ( #15839 )
...
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2022-03-03 16:05:57 +08:00
e1bfd14db1
Add Prometheus metrics for QueryCoord ( #15606 )
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
Co-authored-by: jingkl <34296482+jingkl@users.noreply.github.com>
2022-02-28 16:51:55 +08:00
1e8e3eb912
Fix wrong flushSegmentInfos in watchdmChannel request ( #15740 )
...
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-02-24 19:23:52 +08:00
55cdc5aa35
Get indexInfo and segment size by queryCoord ( #14207 )
...
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-02-08 21:57:46 +08:00
4bccc85534
Use MsgChannelConfig in GlobalParams for all components ( #15163 )
...
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-02-02 00:35:43 +08:00
6336e2326c
Set triggerCondition when queryCoord reload loadbalanceTask meta ( #15380 )
...
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-01-25 17:26:13 +08:00
dfc6670f0f
Unsubscribe channel after query node down ( #15230 )
...
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-01-17 17:37:37 +08:00
384d3169d3
Don't remove nodeInfo when querycoord panic at loadBalanceTask ( #15189 )
...
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-01-13 17:59:34 +08:00
4e956ee10c
Panic when recover querynode failed ( #15161 )
...
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-01-12 17:43:37 +08:00
a304684b55
Delete useless var in watchDeletaChannelTask ( #14641 )
...
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-01-04 19:21:31 +08:00
d4ac010165
Fix not load delete log when handoff segment ( #14758 )
...
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-01-04 14:27:18 +08:00
95f0e9a4a6
Fix proxy send query before queryNode watchQueryChannel and search timeout ( #14640 )
...
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-12-30 22:51:21 +08:00
b9730cb564
Fix not load delete log when handoff segment ( #14613 )
...
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-12-30 19:45:20 +08:00
b063bfda8f
Add index info in loadSegmentRequest ( #14524 )
...
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-12-30 19:09:33 +08:00
903e67b0ec
Fix recovery failed after query node down ( #14449 )
...
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-12-29 12:15:21 +08:00
e3771bee3d
Fix data loss after balance task rescheduled ( #14096 )
...
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-12-24 10:59:31 +08:00
891c202b73
Use GlobalParamTable for all components ( #13885 )
...
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-12-23 18:39:11 +08:00
d39a4a3f2d
Fix recover multi collection after query node down ( #13952 )
...
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-12-22 16:29:06 +08:00
4e4ff54d98
Watch query channel to near the latest position ( #13682 )
...
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-12-21 13:50:54 +08:00
11c8461c11
Refactor dmChannelWatchInfo to queryCoord meta ( #13543 )
...
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-12-21 11:57:39 +08:00
1ca08d9138
Simplify flow graph in QueryNode ( #13613 )
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-12-17 20:12:42 +08:00
42ac5f76ed
Add constraints to the combination of load and release ( #13110 )
...
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-12-15 22:11:09 +08:00
97d5231a0a
Reorganize querycoord proto ( #13181 )
...
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-12-15 16:53:12 +08:00
322f0c5c5c
Remove unused variables ( #13380 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-12-15 11:11:24 +08:00
6f8ea8cc7b
Fix querycoord panic at getRecoveryInfo ( #12983 )
...
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-12-13 10:29:27 +08:00
874c24862b
[skip ci] Fix a grammar mistake in querycoord task ( #13127 )
...
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
Co-authored-by: yun.zhang <yun.zhang@zilliz.com>
2021-12-10 10:13:44 +08:00
af22977cd3
Fixbug: grpc connection is closed by mistake ( #12307 )
...
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-12-03 15:15:32 +08:00
b45d7baf97
Add logs for add delta channels ( #12347 )
...
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-12-01 16:39:31 +08:00
48bdeb56d8
Merge watch delta channel positions ( #12309 )
...
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-11-26 22:47:17 +08:00
ad83090925
Merge delta channels when load partition ( #12274 )
...
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-11-25 18:49:16 +08:00
88012acb86
Clear useless segments info. ( #12262 )
...
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-11-25 17:37:15 +08:00
d7cc1c1e36
Check load segments length before watch delta channel ( #12200 )
...
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-11-23 11:21:15 +08:00
d666c2a190
Fix txn limit error when querycoord remove task from etcd ( #12084 )
...
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-11-18 19:35:15 +08:00
3acb73ebe6
[skip ci]Update license for qc task ( #12016 )
...
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-11-17 19:49:32 +08:00
55d162356b
Fix wrong set of segmentLoadInfo.enableIndex ( #11964 )
...
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-11-17 14:37:11 +08:00
8a0ee27799
Estimate segment size according index and binlog file header ( #11875 )
...
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-11-17 09:47:12 +08:00
4cd7680f87
Reschedule load segments ( #11391 )
...
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-11-13 08:49:08 +08:00
d857577a7b
Postpone the execution of handoff until index creation is complete ( #11648 )
...
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-11-12 18:49:10 +08:00
dba0ae4421
Refactor segment allocate policy on querycoord ( #11181 )
...
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-11-11 12:56:42 +08:00
fcbe46ba75
Remove sealed segemnt from query node after compaction ( #11430 )
...
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-11-08 21:00:02 +08:00
82c199227c
Add execute logic for balancing sealed segments ( #11343 )
...
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-11-06 15:22:56 +08:00
09f8fa1040
Add query coord watch delta channel ( #11218 )
...
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-11-05 14:47:19 +08:00
6d43ac4a6e
Fix load segment request too large ( #10812 )
...
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-10-28 17:26:28 +08:00
2f53a57814
Fix too large request when load collection ( #10768 )
...
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-10-27 19:32:21 +08:00
a414d16c06
Fix remove segment when querynode receive segmentChangeInfo ( #10619 )
...
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-10-26 13:04:18 +08:00
9ebabdac79
Add handoffTask ( #10330 )
...
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-10-24 22:39:09 +08:00
40750f03c7
Enhance querycoord log ( #10523 )
...
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-10-24 15:16:00 +08:00
8a046a6495
BroadCast sealedSegmentChangeInfo to queryChannel ( #10084 )
...
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-10-22 19:07:15 +08:00
d6bf0c928f
Get stats delta path from datacoord ( #10362 )
...
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-10-22 14:31:13 +08:00
f9408259b4
Add comment for shuffleSegmentsToQueryNode ( #10336 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-10-21 11:08:37 +08:00
ef6de3e122
Comment MaxSendSizeToEtcd ( #10165 )
...
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-10-21 10:53:09 +08:00
1103ca8fca
Fix golint error in condition of queryCoord ( #10230 )
...
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-10-21 10:51:14 +08:00
c5bba3ec2d
Fix assign load segment req error ( #10310 )
...
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-10-20 22:32:36 +08:00
e8a7c482c8
Use proto Size to calculate req size ( #10294 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-10-20 19:27:27 +08:00
f467260208
Split large loadSegmentReq to multi small reqs ( #10125 )
...
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-10-19 10:40:35 +08:00
d6f95f49e8
Fix golint error in querycoord ( #10127 )
...
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-10-18 21:34:47 +08:00
759603671c
[skip ci] Add comment for loadPartitionTask ( #10028 )
...
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-10-16 22:01:15 +08:00
59764d17bf
[skip ci] Add comment for ReleasePartitionTask ( #10027 )
...
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-10-16 21:55:24 +08:00
1b50695a40
[skip ci] Add comment for ReleaseCollectionTask ( #10026 )
...
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-10-16 21:43:32 +08:00
da5195aab8
[skip ci] Add comment for LoadCollectionTask ( #9989 )
...
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-10-15 20:29:15 +08:00
56466508cc
[skip ci] Add comment for maxRetryNum ( #9988 )
...
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-10-15 20:25:08 +08:00
e80fbd1f19
Fix golint error in the task of querycoord ( #9866 )
...
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-10-14 20:18:33 +08:00
609cc24309
Fix golint error in querycoord ( #9836 )
...
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-10-13 21:46:32 +08:00
e481a2775a
[skip ci] Add comment for set child task ( #9837 )
...
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-10-13 21:26:33 +08:00
c84600a496
[skip ci] Add comment for set taskID ( #9835 )
...
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-10-13 21:24:41 +08:00
ddf0cd2174
[skip ci] Add comment for taskID ( #9833 )
...
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-10-13 21:22:47 +08:00
c2731dddff
[skip ci] Add comment for task state ( #9757 )
...
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-10-12 23:40:36 +08:00
235d736a49
Add max number of retries for interTask in querycoord ( #8215 )
...
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-10-11 09:54:37 +08:00
937dbf4279
Add flushed segments ( #9476 )
...
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-10-08 19:09:12 +08:00
9400694867
Add ut for querycoord's meta ( #7968 )
...
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-09-29 09:56:04 +08:00
85c2b4d0f6
Add log for releasePartion task ( #8222 )
...
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-09-23 21:06:02 +08:00
e75693e1a3
Add log for release collection task ( #8226 )
...
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-09-18 18:45:51 +08:00
c63086fefa
Add ut for querycoord's cluster ( #7894 )
...
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-09-15 20:40:07 +08:00
b7103357fd
Load failed if collection beyonds the memory limit ( #7434 )
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-09-07 11:35:18 +08:00
c8a1f780c1
Use proto marshal instead of MarshalTextString in querycoord ( #6958 )
...
* use proto marshal instead of marshalToText
Signed-off-by: xige-16 <xi.ge@zilliz.com>
* log error
Signed-off-by: xige-16 <xi.ge@zilliz.com>
* don't retry after init meta/cluster/scheduler failed
Signed-off-by: xige-16 <xi.ge@zilliz.com>
* fix return err
Signed-off-by: xige-16 <xi.ge@zilliz.com>
* log inconsistent task info
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-08-03 22:03:25 +08:00
87c5a49d9a
Refactor showCollection/showPartition to get load state in memory ( #6935 )
...
* refactor showCollection/showPartition to get load state in memory
Signed-off-by: xige-16 <xi.ge@zilliz.com>
* update proto
Signed-off-by: xige-16 <xi.ge@zilliz.com>
* update proto
Signed-off-by: xige-16 <xi.ge@zilliz.com>
* update milvus.proto
Signed-off-by: xige-16 <xi.ge@zilliz.com>
* add show result log to debug
Signed-off-by: xige-16 <xi.ge@zilliz.com>
* fix wrong err message
Signed-off-by: xige-16 <xi.ge@zilliz.com>
* update pymilvus-orm version
Signed-off-by: xige-16 <xi.ge@zilliz.com>
* fix repeated load error
Signed-off-by: xige-16 <xi.ge@zilliz.com>
* update proto
Signed-off-by: xige-16 <xi.ge@zilliz.com>
* fix status nil error
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-08-02 22:39:25 +08:00
2b720fd2f7
Fix load hang when restart querynode many time in short time ( #6351 )
...
* fix load hang when restart querynode many time in short time
Signed-off-by: xige-16 <xi.ge@zilliz.com>
* add multi queryNode ut
Signed-off-by: xige-16 <xi.ge@zilliz.com>
* add ut for restart querynode
Signed-off-by: xige-16 <xi.ge@zilliz.com>
* set queryCoord contex to load collection
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-07-13 14:16:00 +08:00
ddb027fb9e
clear proxy sarchStream when releaseCollection ( #6258 )
...
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-07-02 10:40:13 +08:00
2a07124885
Change default grpc retry config ( #6233 )
...
* Change default grpc retry config
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
* fix error
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
* change timeout
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
* fix error
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
* add retry code
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
* change time
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-07-01 15:24:17 +08:00
a7a9eeeffe
Add time record for build index and search process ( #6231 )
...
* add timerecord for build index process
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* fix build error
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* add timerecord for query process
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-06-30 19:46:14 +08:00
f146d3825f
Fix search hang after querynode restart ( #6212 )
...
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-06-30 17:48:19 +08:00
49a0528d53
Fix trace ( #6218 )
...
* Fix trace
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
* fix go fmt
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-06-30 16:18:13 +08:00
6b65529086
Fix search error after releasePartition ( #6135 )
...
* fix search error after releasePartition
Signed-off-by: xige-16 <xi.ge@zilliz.com>
* code format
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-06-26 16:08:11 +08:00
cea83dde5a
fix multi loadCollectition not work ( #6097 )
...
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-06-24 21:10:13 +08:00
5417fee45f
fix releasePartition not work after loadCollection ( #6025 )
...
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-06-23 17:44:12 +08:00
2af1c27811
Rename queryservice to querycoordinator ( #5931 )
...
* Rename queryservice to querycoordinator
Signed-off-by: xige-16 <xi.ge@zilliz.com>
* fix param table
Signed-off-by: xige-16 <xi.ge@zilliz.com>
* code format
Signed-off-by: xige-16 <xi.ge@zilliz.com>
* fix unittest
Signed-off-by: xige-16 <xi.ge@zilliz.com>
* service to coordinator
Signed-off-by: xige-16 <xi.ge@zilliz.com>
* code format
Signed-off-by: xige-16 <xi.ge@zilliz.com>
* set querycoord in ci
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-06-22 16:44:09 +08:00
zhenshan.cao
Jiquan Long
congqixia
yah01
Cai Yudong
cai.zhang
bigsheeper
xige-16
jaime
godchen
XuanYang-cn