milvus-io
/
milvus
mirror of https://github.com/milvus-io/milvus.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
15,186 Commits
140 Branches
117 Tags
594 MiB
Commit Graph

439 Commits (68661ddc8db3b33d3f01fb647b5aa828d7007a69)

Author SHA1 Message Date
bigsheeper 1da970af18
Remove useless statsService in QueryNode (#17395) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-07-11 15:12:25 +08:00
yah01 08612ac56c
Fix QueryCoord unit test stucks (#18197) 
Signed-off-by: yah <yang.cen@zilliz.com>
 2022-07-11 12:44:25 +08:00
wayblink 43ae6791e6
Make VChannelInfo read compatible with 2.0.2 (#17995) 
Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2022-07-07 21:58:22 +08:00
cai.zhang 44e59d6318
Fix bug for clean etcd in QueryNCoord (#18145) 
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
 2022-07-07 18:32:21 +08:00
Enwei Jiao ca88e37c98
fix receive messages before waitToFinish (#18101) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2022-07-07 10:00:21 +08:00
Xiaofan a9b1d71a8f
change the etcd compareAndSwap interface (#18068) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-07-06 13:54:21 +08:00
bigsheeper 7d6624fcad
Improve flow graph node name and log msgID in QueryCoord (#18043) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-07-05 16:46:23 +08:00
yah01 39db27865f
Refine log, remove huge logs (#18036) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-07-04 16:34:20 +08:00
yah01 4def7047b4
Fix the scheduler enqueues concurrent load tasks (#17950) 
Signed-off-by: yah01 <yang.cen@zilliz.com>

related to #17850
 2022-07-01 19:54:19 +08:00
yah01 66a04450ad
Fix LoadBalance Sync all segments to shard leaders (#18002) 
Signed-off-by: yah <yang.cen@zilliz.com>
 2022-07-01 19:18:24 +08:00
yah01 27eca2881a
Clean segments as releasing collection (#17932) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-07-01 19:16:25 +08:00
Jiquan Long 47c9e77155
Fix querycoord ut data race (#17980) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2022-07-01 10:58:18 +08:00
cai.zhang 01fc411566
Handle etcd compacted error (#17886) 
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
 2022-06-30 18:54:19 +08:00
yah01 ef6859a4d6
Refactor handoff handler (#17806) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
Co-authored-by: xiaofan-luan <xiaofan.luan@zilliz.com>

Co-authored-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-06-29 23:26:19 +08:00
congqixia 106f19a7b8
Fix querycoord cannot update deltaChannel info (#17901) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-06-29 15:38:17 +08:00
yah01 797218a8ad
Fix handoff isn't processed correctly (#17738) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
Co-authored-by: Congqi Xia <congqi.xia@zilliz.com>

Co-authored-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-06-27 22:08:17 +08:00
cai.zhang ef0fc37ad7
Fixed the bug that IndexCoord lost some meta information (#17818) 
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
 2022-06-27 21:52:17 +08:00
wayblink 46e0e2658b
Move datacoord related methods from meta to globalMetaBroker (#17812) 
Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2022-06-27 21:04:17 +08:00
congqixia 6840486efc
Fix load progress complete before syncReplica (#17828) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-06-27 20:52:16 +08:00
Bingyi Sun 9d2e5aacbc
Fix missing dml nodeIDs when recovering from old version (#17800) 
issue: #17416
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
 2022-06-26 18:08:16 +08:00
yah01 8388478ef3
SyncReplicaSegments syncs all segments (#17774) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-06-24 23:24:15 +08:00
yah01 94a5122044
Fix LoadBalance removes source nodes from DmChannel (#17744) 
Only NodeDown LoadBalance should remove source nodes

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-06-24 12:40:14 +08:00
wayblink 1a336bbbf1
Fix: WatchDmChannelsRequest can be too large to save in etcd (#17722) 
Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2022-06-24 10:54:15 +08:00
yah01 b4f21259ef
Fix SegmentChangeInfo is not set properly (#17701) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-06-22 21:08:13 +08:00
yah01 a077bad84a
Make updating replica atomic, balance plan idempotent (#17686) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-06-22 13:40:13 +08:00
yah01 1e87b54542
Add the offlines nodes previously crashed to offline nodes channel (#17663) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-06-21 16:08:13 +08:00
wayblink f2bd910df5
[skip e2e]Fix log mistake: WatchDmChannels -> WatchDeltaChannels (#17643) 
Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2022-06-20 15:10:12 +08:00
congqixia 50cecc65ed
Fix mock querynode server session not revoked (#17229) 
Revoke mock querynode server session when it's stopped
This PR reduces the running time of TestLoadBalanceIndexedSegmentsAfterNodeDown from 60+ seconds to less than 1+ seconds
Also related to #17212 #17215

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-06-18 18:22:12 +08:00
wayblink 074ec3060a
Support return dropped segments info in GetSegmentInfo rpc (#17617) 
Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2022-06-17 18:24:12 +08:00
congqixia 785a5a757f
Use segment version instead of ref cnt (#17609) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-06-17 17:38:12 +08:00
congqixia cc3ecc4bd5
Make querycoord channel allocator respect context (#17552) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-06-17 16:02:12 +08:00
yah01 f5fa93aa0b
Return err if failed to assign segments/channels to nodes (#17616) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-06-17 12:42:11 +08:00
yah01 0f87763682
Fix LoadBalance may not update the shard leader (#17608) 
This happens probably with concurrent updating replicas,
some goroutines modify the nodes list of replicas,
and the others modify the shard leaders of replicas

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-06-17 08:58:10 +08:00
yah01 3f42f5f345
Set the task state to TaskFailed if error occurs (#17598) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-06-16 19:18:11 +08:00
Xiaofan 1f6fbf91b2
Fix pulsar unsubsribe issue (#17562) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-06-16 17:28:11 +08:00
yah01 7d5c8c5f38
Fix bug not remove offline node (#17560) 
The LoadBalance task won't remove the offline node if the node never load/watch any segment/dmchannel

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-06-16 16:00:10 +08:00
wayblink eb5b0b7fc8
Move SegmentInfo out of VchannelInfo, leave Id only to decrease kv size. Get complete SegmentInfo through RPC (#17441) 
Resolves: #17233 #16047

Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2022-06-16 12:00:10 +08:00
cai.zhang ea5041aec2
Acquiring the segment reference lock on task level (#17544) 
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
 2022-06-15 21:38:10 +08:00
congqixia f9553970f9
Add BindContext function for querycoord task scheduler (#17531) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-06-14 17:12:09 +08:00
Enwei Jiao a5b008acec
ignore getReplica's error when handle rebalanceTask (#17469) 
Signed-off-by: Enwei Jiao <jiaoew2011@gmail.com>
 2022-06-10 14:50:08 +08:00
Xiaofan 66f26943f8
Add Err code when task rollback (#17472) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-06-09 21:08:07 +08:00
congqixia 97a871cc82
Make querycoord segment allocator respect context (#17452) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-06-09 20:14:06 +08:00
Letian Jiang dfaed5acdd
Add QueryServiceAvailable field in ShowCollections (#17456) 
Signed-off-by: Letian Jiang <letian.jiang@zilliz.com>
 2022-06-09 18:20:07 +08:00
yah01 a2d2ad88bd
Make assigning segments faster (#17377) 
This improve the Load performance,
and let the LoadBalance fails fast, which allows us to retry it timely

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-06-06 16:52:05 +08:00
xige-16 8c69790383
Fix lost delete msg caused by loadSegment after watchDeltaChannel (#17308) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-06-02 16:56:04 +08:00
yah01 cc69c5cdd3
Make Cluster interface's methods called outside public (#17315) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-06-02 13:16:05 +08:00
yah01 f5bd519e49
Add retry mechanism for NodeDown LoadBalance (#17306) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-06-01 20:00:03 +08:00
cai.zhang bcf3b7426a
Add distributed lock for segment refgerence (#16782) 
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
 2022-05-31 16:36:03 +08:00
congqixia c88514bc49
Remove not used QueryChannel in Proxy and Query Cluster (#16856) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-05-30 19:50:04 +08:00
yah01 b09359b12f
Remove useless collection ID in error message (#17269) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-05-30 14:40:01 +08:00