milvus-io
/
milvus
mirror of https://github.com/milvus-io/milvus.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
14,745 Commits
140 Branches
117 Tags
580 MiB
Commit Graph

5270 Commits (7746a5b74262a88a9a07bf045568e58c381fbb60)

Author SHA1 Message Date
yah01 7746a5b742
Add NodeIds field for QuerySegmentInfo (#17121) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-05-20 18:03:58 +08:00
congqixia 90ee23df22
Fix shardLeader cache concurrent access (#17120) 
Fix  write map without mutex control
Also GetShards returns a copy of leader list instead of original one

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-05-20 14:33:57 +08:00
ownbylichaobao 2f8a7e7793
Support tls/ssl connection to etcd (#7703) (#17012) 
Signed-off-by: lichaobao <1527563274@qq.com>
 2022-05-20 12:29:19 +08:00
groot d525e95550
Bulkload not allow duplicate numpy file (#17105) 
Signed-off-by: groot <yihua.mo@zilliz.com>
 2022-05-20 10:27:56 +08:00
Letian Jiang 70825a35cf
Make querynode LoadSegments interface idempotent (#17109) 
This PR makes the following changes:
* separate LoadSegmentsTask into two phases: PreExecute and Execute
* filters out segments that are already loaded in PreExecute phase

Signed-off-by: Letian Jiang <letian.jiang@zilliz.com>
 2022-05-20 09:15:57 +08:00
congqixia 599763d9bf
Fix replicas info is not removed after release (#17111) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-05-19 20:05:57 +08:00
cai.zhang e05cfe194e
Fix data race for indexcoord test (#17057) 
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
 2022-05-19 17:25:56 +08:00
Jiquan Long 276ba69819
Replace plan parser with antlr (#17090) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2022-05-19 17:15:57 +08:00
yah01 dcfe472586
Fix LoadBalance doesn't save the modification to replicas' shards (#17064) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-05-19 16:51:57 +08:00
congqixia ae2864639e
Fix replica freeAll order (#17106) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-05-19 14:53:56 +08:00
congqixia 7eb6fcfb34
Limit the query node timetick log rate (#17065) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-05-19 10:19:56 +08:00
bigsheeper 9eeec4a2d5
Add collection load cache and InvalidateCollMetaCache by collID (#16882) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-05-19 10:13:56 +08:00
congqixia b95681770b
Add deferred clean up for ShardCluster (#17094) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-05-18 21:51:56 +08:00
SimFG cf08b5aa11
Fix 'TestChannelManager_StateTransfer' ut when etcd is slow (#17082) 
Signed-off-by: SimFG <bang.fu@zilliz.com>
 2022-05-18 17:19:57 +08:00
Jiquan Long 99b4042f8c
Disable segment small index (#16935) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2022-05-18 15:27:56 +08:00
yah01 33c855dcd2
Fix LoadBalance doesn't remove the source nodes from segment (#17051) 
If the triggerCondition isn't NodeDown, the removing won't happen.

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-05-18 11:55:56 +08:00
XuanYang-cn 127dd34b37
Make proxy use roundrobin to choose replica (#17063) 
Fixes: #17055

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2022-05-17 22:35:57 +08:00
cai.zhang b37b87eb97
The GetIndexFilePaths interface judges whether the index is complete (#17052) 
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
 2022-05-17 20:41:56 +08:00
Jiquan Long 77b0f290a8
Fix prefix match on sealed segment (#17050) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2022-05-17 20:15:55 +08:00
congqixia 5282397c1b
Fix VectorChunkManager cache enable parameter (#17058) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-05-17 17:37:55 +08:00
cai.zhang d6bb86dd46
Return buildID when creating index if the index alreay exists (#17045) 
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
 2022-05-17 15:21:57 +08:00
xige-16 c2f1d65fa1
Print err info when insert msg not aligned (#16305) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-05-17 13:01:57 +08:00
yah01 960d35e517
Add MockCluster, make unit tests reliable (#17032) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-05-17 12:57:56 +08:00
XuanYang-cn a1480aaf76
Clear shard leaders cache when releasing (#16942) 
See also: #16926

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2022-05-17 11:11:56 +08:00
Xiaofan 5355153805
Minio Error is not Handled gracefully (#17003) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-05-16 19:25:55 +08:00
Xiaofan 327bd6abbe
Support Txn interface in rockskv (#17027) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-05-16 19:17:55 +08:00
xige-16 935b729a0c
Fix multiple deletes not taking effect (#17033) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-05-16 18:39:56 +08:00
congqixia ffff291018
Fix Handoff log in shardCluster service (#17039) 
Fix #17035

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-05-16 18:29:56 +08:00
congqixia 94f0951f0d
Fix query lock logic in query shard (#17034) 
Previously query shard locks the querylock in collectionReplica before any search/query
The lock range is too large and easy to cause dead lock

This PR makes following changes:
- Rename collectionReplica to metaReplica which is more reasonable
- Make release collection operation cancels waiting search/query request
- Reduce the queryLock to collection level
- Add some unit tests for timeout & released case

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-05-16 18:23:55 +08:00
Bingyi Sun 59bc0a7000
Add some log. (#16978) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
 2022-05-16 17:15:55 +08:00
yah01 e38c6f6c44
Fix load the same segments multiple times for manual LoadBalance (#16921) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-05-16 15:41:56 +08:00
Xiaofan 338edcd3cc
Load binlog for different field in parallel (#17005) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-05-15 16:11:54 +08:00
SimFG ea2c395a38
Fix the comment info in task_test.go (#16982) 
Signed-off-by: SimFG <bang.fu@zilliz.com>
 2022-05-15 09:03:54 +08:00
jaime 328ab7ecf6
Fix kafka client hang after killing broker pod of parition leader (#16983) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
 2022-05-14 23:05:54 +08:00
congqixia 23f7ef931c
Remove not used QueryService (#16973) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-05-13 20:27:54 +08:00
congqixia ec66ac69ac
Simplify chunk manager in QueryShard (#16976) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-05-13 19:55:54 +08:00
yah01 a382133a8a
Add new node into the replica which has the most of offline nodes (#16907) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-05-13 18:31:54 +08:00
Xiaofan 52fc9a558c
Fix valid error message (#16953) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-05-13 18:19:53 +08:00
Jiquan Long ca3ecec5b5
Fix ut data race (#16939) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2022-05-13 17:43:53 +08:00
groot 168935f4cf
Fix bulkload bug for string primary key (#16958) 
Signed-off-by: groot <yihua.mo@zilliz.com>
 2022-05-13 16:07:54 +08:00
Letian Jiang f0dee539bb
Improve query shard log (#16950) 
Signed-off-by: Letian Jiang <letian.jiang@zilliz.com>
 2022-05-13 13:45:53 +08:00
XuanYang-cn 43b36b4e5f
Make proxy update cache with NotShardLeader status code (#16940) 
See also: #16926

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2022-05-13 11:35:53 +08:00
Jiquan Long c5149c7e85
Replace text codec of pb with binary codec (#16955) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2022-05-13 09:53:53 +08:00
Ten Thousand Leaves 63385b7c12
Make data queryable and data indexed states as independent flags (#16837) 
/kind improvement

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-05-12 19:23:53 +08:00
congqixia ae717bf991
Fix channelUnsubscribe data race and logic (#16946) 
- Add a RWMutex for container/list which is not goroutine-safe
- Fix the element in list is never removed

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-05-12 18:09:53 +08:00
xige-16 08ad77c71b
Delete all repeated primary keys (#16863) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-05-12 14:09:53 +08:00
Letian Jiang b3eb2b1d0d
Support deltaLog loading on growing segment (#16903) 
Signed-off-by: Letian Jiang <letian.jiang@zilliz.com>
 2022-05-12 11:59:53 +08:00
congqixia a8b81e215b
Fix deadlock in release replica (#16925) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-05-11 21:57:53 +08:00
congqixia a0614a3ff8
Use NodeIds instead of NodeID in segment info (#16912) 
The NodeID field is deprecated in current design
Query Node segment detector shall read NodeIds instead of NodeID

Fix #16911

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-05-11 17:39:53 +08:00
xige-16 9e6d9fb7d8
Print collectionName when load failed (#16878) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-05-11 09:47:53 +08:00