Commit Graph

75 Commits (8f1ba6a0bb0121a32cd5e1f4dba4a0747d4513ec)

Author SHA1 Message Date
yah01 4def7047b4
Fix the scheduler enqueues concurrent load tasks (#17950)
Signed-off-by: yah01 <yang.cen@zilliz.com>

related to #17850
2022-07-01 19:54:19 +08:00
yah01 ef6859a4d6
Refactor handoff handler (#17806)
Signed-off-by: yah01 <yang.cen@zilliz.com>
Co-authored-by: xiaofan-luan <xiaofan.luan@zilliz.com>

Co-authored-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-06-29 23:26:19 +08:00
wayblink 46e0e2658b
Move datacoord related methods from meta to globalMetaBroker (#17812)
Signed-off-by: wayblink <anyang.wang@zilliz.com>
2022-06-27 21:04:17 +08:00
yah01 8388478ef3
SyncReplicaSegments syncs all segments (#17774)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-06-24 23:24:15 +08:00
wayblink 1a336bbbf1
Fix: WatchDmChannelsRequest can be too large to save in etcd (#17722)
Signed-off-by: wayblink <anyang.wang@zilliz.com>
2022-06-24 10:54:15 +08:00
yah01 b4f21259ef
Fix SegmentChangeInfo is not set properly (#17701)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-06-22 21:08:13 +08:00
yah01 a077bad84a
Make updating replica atomic, balance plan idempotent (#17686)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-06-22 13:40:13 +08:00
congqixia f9553970f9
Add BindContext function for querycoord task scheduler (#17531)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-06-14 17:12:09 +08:00
Xiaofan 66f26943f8
Add Err code when task rollback (#17472)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-06-09 21:08:07 +08:00
yah01 a2d2ad88bd
Make assigning segments faster (#17377)
This improve the Load performance,
and let the LoadBalance fails fast, which allows us to retry it timely

Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-06-06 16:52:05 +08:00
yah01 cc69c5cdd3
Make Cluster interface's methods called outside public (#17315)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-06-02 13:16:05 +08:00
cai.zhang bcf3b7426a
Add distributed lock for segment refgerence (#16782)
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
2022-05-31 16:36:03 +08:00
congqixia c88514bc49
Remove not used QueryChannel in Proxy and Query Cluster (#16856)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-05-30 19:50:04 +08:00
yah01 de0ba6d495
Fix GetQuerySegmentInfo() returns incorrect result after LoadBalance (#17190)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-05-25 15:17:59 +08:00
yah01 7746a5b742
Add NodeIds field for QuerySegmentInfo (#17121)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-05-20 18:03:58 +08:00
Bingyi Sun 59bc0a7000
Add some log. (#16978)
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
2022-05-16 17:15:55 +08:00
yah01 2d0f908dba
Fix updating segments' NodeIds correctly after LoadBalance (#16854)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-05-10 15:47:53 +08:00
congqixia 8c004fa981
Fix replicaID is allow zero from watch channel task (#16788)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-05-05 18:11:51 +08:00
yah01 20936f78da
Fix LoadBalance failed to sync segments to shard leader when the leader is offline (#16770)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-05-05 16:25:50 +08:00
yah01 93416d5524
Fix LoadBalance loads the same segment multiple times (#16734)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-04-29 15:19:47 +08:00
Xiaofan c1ff0cec8f
Change QueryCoord Log level (#16590)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-04-26 11:29:54 +08:00
congqixia 69252f812d
Implement memory replica in Proxy, QueryNode and QueryCoord (#16470)
Related to #16298 #16291 #16154
Co-authored-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: yangxuan <xuan.yang@zilliz.com>
Co-authored-by: yah01 <yang.cen@zilliz.com>
Co-authored-by: Letian Jiang <letian.jiang@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-04-20 16:15:41 +08:00
bigsheeper e1bfd14db1
Add Prometheus metrics for QueryCoord (#15606)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Co-authored-by: jingkl <34296482+jingkl@users.noreply.github.com>
2022-02-28 16:51:55 +08:00
xige-16 55cdc5aa35
Get indexInfo and segment size by queryCoord (#14207)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-02-08 21:57:46 +08:00
xige-16 6336e2326c
Set triggerCondition when queryCoord reload loadbalanceTask meta (#15380)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-01-25 17:26:13 +08:00
Xieql 1e7c51859a
[skip e2e] Improve annotation (#14799)
Signed-off-by: Xieql <qianglong.xie@zilliz.com>
2022-01-05 10:33:30 +08:00
xige-16 a304684b55
Delete useless var in watchDeletaChannelTask (#14641)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-01-04 19:21:31 +08:00
Xieql c7d47cefcd
[skip e2e] Improve annotation (#14717)
Signed-off-by: Xieql <qianglong.xie@zilliz.com>
2022-01-04 19:15:42 +08:00
xige-16 903e67b0ec
Fix recovery failed after query node down (#14449)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-12-29 12:15:21 +08:00
xige-16 8e2e9c3735
Fix wrong meta when balance's watchQueryChannel task failed (#14235)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-12-25 23:12:50 +08:00
xige-16 4e4ff54d98
Watch query channel to near the latest position (#13682)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-12-21 13:50:54 +08:00
xige-16 11c8461c11
Refactor dmChannelWatchInfo to queryCoord meta (#13543)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-12-21 11:57:39 +08:00
xige-16 97d5231a0a
Reorganize querycoord proto (#13181)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-12-15 16:53:12 +08:00
Xiangyu Wang 405b3cd932
Use the same lint rules with golint in revive (#13324)
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-12-14 15:31:07 +08:00
dragondriver 255be8f4a8
Fix querycoord panic when close before init scheduler (#13051)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-12-11 10:15:13 +08:00
JackLCL 554b7918e9
[skip ci]Fix error log output format in task_scheduler.go (#13129)
Signed-off-by: JackLCL <chenglong.li@zilliz.com>
2021-12-10 10:16:03 +08:00
jaime 4e538f9929
[skip ci] Fix a grammar mistake in querycoord task_scheduler (#13126)
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Co-authored-by: yun.zhang <yun.zhang@zilliz.com>
2021-12-10 10:11:59 +08:00
xige-16 99397535be
Fix update rollBackTask after triggerTask fail (#12228)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-11-24 10:19:15 +08:00
xige-16 d666c2a190
Fix txn limit error when querycoord remove task from etcd (#12084)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-11-18 19:35:15 +08:00
XuanYang-cn 3acb73ebe6
[skip ci]Update license for qc task (#12016)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-11-17 19:49:32 +08:00
Xiaofan f654621650
Remove the etcd deletion of segment data path on querynode (#11863)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2021-11-17 12:13:10 +08:00
xige-16 8a0ee27799
Estimate segment size according index and binlog file header (#11875)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-11-17 09:47:12 +08:00
xige-16 fcbe46ba75
Remove sealed segemnt from query node after compaction (#11430)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-11-08 21:00:02 +08:00
godchen 09f8fa1040
Add query coord watch delta channel (#11218)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-11-05 14:47:19 +08:00
xige-16 373722d1ca
Load segment before watch dmCahnnel when load collection (#10831)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-10-29 18:36:40 +08:00
xige-16 6d43ac4a6e
Fix load segment request too large (#10812)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-10-28 17:26:28 +08:00
xige-16 0cedd9d47e
Compaction multiSegmentChangeInfo to a single info (#10587)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-10-26 15:18:22 +08:00
xige-16 9ebabdac79
Add handoffTask (#10330)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-10-24 22:39:09 +08:00
xige-16 8a046a6495
BroadCast sealedSegmentChangeInfo to queryChannel (#10084)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-10-22 19:07:15 +08:00
xige-16 f68a3358dd
Fix multiSave childTask failed to etcd (#10340)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-10-21 12:18:36 +08:00