6fd83464cd
Fix watchDmChannel may be out-of-date after compaction issue and add context ( #18681 )
...
Signed-off-by: wayblink <anyang.wang@zilliz.com>
Signed-off-by: wayblink <anyang.wang@zilliz.com>
2022-08-23 17:44:56 +08:00
b427450593
Remove Load Delta Channel Request ( #18497 )
...
Fix #18393
remove watch delta channel task related logic
Co-authored-by: xiaofan-luan <xiaofan.luan@zilliz.com>
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
Co-authored-by: Congqi Xia <congqi.xia@zilliz.com>
2022-08-04 17:50:37 +08:00
a79c5e2366
fix sync problem during reblance ( #18332 )
...
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2022-07-20 15:02:30 +08:00
c97563590c
fix missing handling channels while a node down ( #18250 )
...
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-07-18 13:06:28 +08:00
d278213e4f
Check load request both in impl and task ( #18276 )
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-07-15 10:24:28 +08:00
ca88e37c98
fix receive messages before waitToFinish ( #18101 )
...
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2022-07-07 10:00:21 +08:00
7d6624fcad
Improve flow graph node name and log msgID in QueryCoord ( #18043 )
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-07-05 16:46:23 +08:00
39db27865f
Refine log, remove huge logs ( #18036 )
...
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-07-04 16:34:20 +08:00
4def7047b4
Fix the scheduler enqueues concurrent load tasks ( #17950 )
...
Signed-off-by: yah01 <yang.cen@zilliz.com>
related to #17850
2022-07-01 19:54:19 +08:00
66a04450ad
Fix LoadBalance Sync all segments to shard leaders ( #18002 )
...
Signed-off-by: yah <yang.cen@zilliz.com>
2022-07-01 19:18:24 +08:00
27eca2881a
Clean segments as releasing collection ( #17932 )
...
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-07-01 19:16:25 +08:00
ef6859a4d6
Refactor handoff handler ( #17806 )
...
Signed-off-by: yah01 <yang.cen@zilliz.com>
Co-authored-by: xiaofan-luan <xiaofan.luan@zilliz.com>
Co-authored-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-06-29 23:26:19 +08:00
46e0e2658b
Move datacoord related methods from meta to globalMetaBroker ( #17812 )
...
Signed-off-by: wayblink <anyang.wang@zilliz.com>
2022-06-27 21:04:17 +08:00
6840486efc
Fix load progress complete before syncReplica ( #17828 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-06-27 20:52:16 +08:00
8388478ef3
SyncReplicaSegments syncs all segments ( #17774 )
...
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-06-24 23:24:15 +08:00
94a5122044
Fix LoadBalance removes source nodes from DmChannel ( #17744 )
...
Only NodeDown LoadBalance should remove source nodes
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-06-24 12:40:14 +08:00
1a336bbbf1
Fix: WatchDmChannelsRequest can be too large to save in etcd ( #17722 )
...
Signed-off-by: wayblink <anyang.wang@zilliz.com>
2022-06-24 10:54:15 +08:00
b4f21259ef
Fix SegmentChangeInfo is not set properly ( #17701 )
...
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-06-22 21:08:13 +08:00
a077bad84a
Make updating replica atomic, balance plan idempotent ( #17686 )
...
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-06-22 13:40:13 +08:00
f5fa93aa0b
Return err if failed to assign segments/channels to nodes ( #17616 )
...
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-06-17 12:42:11 +08:00
0f87763682
Fix LoadBalance may not update the shard leader ( #17608 )
...
This happens probably with concurrent updating replicas,
some goroutines modify the nodes list of replicas,
and the others modify the shard leaders of replicas
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-06-17 08:58:10 +08:00
3f42f5f345
Set the task state to TaskFailed if error occurs ( #17598 )
...
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-06-16 19:18:11 +08:00
7d5c8c5f38
Fix bug not remove offline node ( #17560 )
...
The LoadBalance task won't remove the offline node if the node never load/watch any segment/dmchannel
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-06-16 16:00:10 +08:00
eb5b0b7fc8
Move SegmentInfo out of VchannelInfo, leave Id only to decrease kv size. Get complete SegmentInfo through RPC ( #17441 )
...
Resolves : #17233 #16047
Signed-off-by: wayblink <anyang.wang@zilliz.com>
2022-06-16 12:00:10 +08:00
ea5041aec2
Acquiring the segment reference lock on task level ( #17544 )
...
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
2022-06-15 21:38:10 +08:00
f9553970f9
Add BindContext function for querycoord task scheduler ( #17531 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-06-14 17:12:09 +08:00
a5b008acec
ignore getReplica's error when handle rebalanceTask ( #17469 )
...
Signed-off-by: Enwei Jiao <jiaoew2011@gmail.com>
2022-06-10 14:50:08 +08:00
cc69c5cdd3
Make Cluster interface's methods called outside public ( #17315 )
...
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-06-02 13:16:05 +08:00
f5bd519e49
Add retry mechanism for NodeDown LoadBalance ( #17306 )
...
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-06-01 20:00:03 +08:00
bcf3b7426a
Add distributed lock for segment refgerence ( #16782 )
...
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
2022-05-31 16:36:03 +08:00
c88514bc49
Remove not used QueryChannel in Proxy and Query Cluster ( #16856 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-05-30 19:50:04 +08:00
5872c5afb6
Fix updating shard leaders may lost some modifications ( #17218 )
...
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-05-26 17:14:02 +08:00
de0ba6d495
Fix GetQuerySegmentInfo() returns incorrect result after LoadBalance ( #17190 )
...
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-05-25 15:17:59 +08:00
75ca64f8c7
Refine task type logs to string ( #17196 )
...
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2022-05-25 03:08:00 +08:00
ffaead6ad9
Add load meta in hand off task. ( #17179 )
...
issue: #16842
Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@zilliz.com>
2022-05-24 18:24:00 +08:00
86728490a2
Fix partition not found ( #17132 )
...
issue: #16842
Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@zilliz.com>
2022-05-20 19:37:57 +08:00
7746a5b742
Add NodeIds field for QuerySegmentInfo ( #17121 )
...
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-05-20 18:03:58 +08:00
dcfe472586
Fix LoadBalance doesn't save the modification to replicas' shards ( #17064 )
...
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-05-19 16:51:57 +08:00
9eeec4a2d5
Add collection load cache and InvalidateCollMetaCache by collID ( #16882 )
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-05-19 10:13:56 +08:00
33c855dcd2
Fix LoadBalance doesn't remove the source nodes from segment ( #17051 )
...
If the triggerCondition isn't NodeDown, the removing won't happen.
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-05-18 11:55:56 +08:00
59bc0a7000
Add some log. ( #16978 )
...
Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@zilliz.com>
2022-05-16 17:15:55 +08:00
e38c6f6c44
Fix load the same segments multiple times for manual LoadBalance ( #16921 )
...
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-05-16 15:41:56 +08:00
2d0f908dba
Fix updating segments' NodeIds correctly after LoadBalance ( #16854 )
...
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-05-10 15:47:53 +08:00
c82e2453eb
Modify the replicas' shard info after load balance ( #16785 )
...
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-05-05 21:15:50 +08:00
20936f78da
Fix LoadBalance failed to sync segments to shard leader when the leader is offline ( #16770 )
...
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-05-05 16:25:50 +08:00
93416d5524
Fix LoadBalance loads the same segment multiple times ( #16734 )
...
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-04-29 15:19:47 +08:00
206b554d49
Sync the distribution of segments to shard leaders ( #16685 )
...
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-04-28 16:01:48 +08:00
60f7fef3b3
Remove offline nodes from replica info after LoadBalanceTask done ( #16653 )
...
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-04-26 18:23:46 +08:00
c1ff0cec8f
Change QueryCoord Log level ( #16590 )
...
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-04-26 11:29:54 +08:00
69252f812d
Implement memory replica in Proxy, QueryNode and QueryCoord ( #16470 )
...
Related to #16298 #16291 #16154
Co-authored-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: yangxuan <xuan.yang@zilliz.com>
Co-authored-by: yah01 <yang.cen@zilliz.com>
Co-authored-by: Letian Jiang <letian.jiang@zilliz.com>
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-04-20 16:15:41 +08:00
wayblink
Xiaofan
wei liu
bigsheeper
Enwei Jiao
yah01
congqixia
cai.zhang
Enwei Jiao
Jiquan Long
Bingyi Sun