Commit Graph

14474 Commits (4cb7c28bfac26b07ad87025018c13fed42be8159)

Author SHA1 Message Date
congqixia 4cb7c28bfa
Fix ReplicaInfo package name ()
Fix 
PR 16373 breaks the build after ReplicaInfo moved from querypb to milvuspb
Also fix  problem in Makefile
Limit action/setup-go version to ~1.17.1(>=1.17.1 <1.18.0)

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-04-06 19:47:31 +08:00
edward.zeng 086e8968ef
Upgrade minio version to RELEASE.2022-03-17T06-34-49Z ()
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
2022-04-06 15:47:32 +08:00
congqixia 18d76261a0
Add ShardNodeDetector implementation ()
Use etcd as a ShardNodeDetector
Unit tests is based on embedded etcd

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-04-06 15:43:31 +08:00
Ten Thousand Leaves 1e6f6bd072
Implement set segment state call from RootCoord to DataCoord. ()
/kind feature

issue: 
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-04-06 15:33:32 +08:00
XuanYang-cn 78200009a3
Impl GetReplicas in Proxy ()
See also: 

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-04-06 14:57:31 +08:00
bigsheeper 2b27fa4609
Print numTimestamps instead of Timestamps ()
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-04-06 14:47:31 +08:00
cai.zhang 4267dc2efc
Handle error and oragnize log ()
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2022-04-06 14:33:30 +08:00
bigsheeper 7161a7760c
Fix segIDAssigner spelling mistake ()
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-04-06 14:11:31 +08:00
bigsheeper 4f4a417c8f
Unify in log print ()
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-04-06 14:09:35 +08:00
cai.zhang e73139deeb
Compressed IndexNode log ()
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2022-04-06 11:33:31 +08:00
cai.zhang 04076393c5
Fix log error ()
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2022-04-06 10:13:32 +08:00
cai.zhang f9c6b3980f
Compressed log ()
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2022-04-06 10:11:36 +08:00
Cai Yudong 543ec4a209
Rename log to milvus_log ()
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-04-05 22:55:30 +08:00
Cai Yudong 7323180861
Move bench to directory unittest ()
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-04-05 12:43:30 +08:00
Cai Yudong 0c53f06a33
Fix build error on mac ()
Signed-off-by: Cai Yudong <yudong.cai@zilliz.com>
2022-04-04 21:09:32 +08:00
Letian Jiang 09b90fd8bf
Fix QueryCollection retrieve unit test ()
Signed-off-by: Letian Jiang <letian.jiang@zilliz.com>
2022-04-04 21:07:30 +08:00
Ten Thousand Leaves 2bd0a9526a
Implement task expiring logic and make all deadlines configurable. ()
/kind feature

issue: 
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-04-03 11:37:29 +08:00
groot be8d9a8b6b
Support import numpy file ()
Signed-off-by: groot <yihua.mo@zilliz.com>
2022-04-03 11:27:29 +08:00
sre-ci-robot 6682d1b635 Update all contributors
Signed-off-by: sre-ci-robot <sre-ci-robot@zilliz.com>
2022-04-02 12:00:51 +00:00
xige-16 99984b88e1
Support delete varChar value ()
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-04-02 17:43:29 +08:00
congqixia aa1c26de77
Add ShardCluster implementation ()
ShardCluster maintains shard replica meta information
It watches node & segment change events
Provides shard replica search/query services

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-04-02 17:35:29 +08:00
edward.zeng 573eed5bd3
Bump milvus version to v2.0.2 ()
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
2022-04-02 16:00:31 +08:00
Letian Jiang 028c5cb882
Modify grpc interface for replica Search/Query in QueryNode ()
Signed-off-by: Letian Jiang <letian.jiang@zilliz.com>
2022-04-02 14:15:31 +08:00
Xiaofan 7f7379d55f
Fix DML channel AsProducer after Close ()
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-04-02 13:09:29 +08:00
XuanYang-cn 5494f3c318
Remove offLineNodes from watch states check ()
This PR also
- adds more information in log
- makes Session able to logged by zap.Any/zap.String

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-04-01 22:33:29 +08:00
Jenny Li 7303036c19
[skip e2e]Update issue example into #<xyx> ()
Use real issue example will combine too many pr metioned make the pr author annoying

Signed-off-by: Jenny Li <jing.li@zilliz.com>
2022-04-01 21:33:29 +08:00
congqixia 6111dcde52
Fix ticker leakage in datanode flowgraph ()
Using  in case ticker need to be closed may cause leakage
Use time.NewTicker with deferred Stop instead

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-04-01 19:35:29 +08:00
XuanYang-cn e9090a62ab
Rearrange search/queryTask for readability ()
`searchTask` has 683 lines of code, `queryTask` has 485 lines of code.
`task.go` contains 4k+ codes including `searchTask` and `queryTask`.

It was so much pain navigating codes of searchTask and queryTask though task.go,
task.go and task_test.go are literaly unreadable.

This PR moves
1. 650+ lines of code of `searchTask` from `task.go` to `task_search.go`.
2. 1.4k+ lines of test code of `searchTask` from `task_test.go` to
   `task_search_test.go`.
3. 450+ lines of code of `queryTask` from `task.go` to `task_query.go`.
4. 200+ lines of test code of `queryTask from `task_test.go to
   `task_query_test.go`.

This PR also rearrange methods positions of `searchTask` and
`queryTask`:
-  Putting the most important methods `PreExecute`, `Execute`, and
   `PosExecute` at the beginning of the file.
-  Moves interface methods `ID`, `SetID`, `Type`, `BeginTs`, etc.
   that nobody cares about to the bottom of the file.

See also: 

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-04-01 18:59:29 +08:00
congqixia 7a44fff8cd
Use fmt.Errorf instead of string concat in tsafe_replica ()
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-04-01 18:37:28 +08:00
yah01 c005f07ccc
Modify proto to add GetReplicas() interface ()
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-04-01 16:15:29 +08:00
Jiquan Long fd589baca7
Integrates marisa trie index ()
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2022-04-01 15:31:29 +08:00
Jenny Li 8b0f260b05
[skip e2e] Add commit-id after date ()
Signed-off-by: Jenny Li <jing.li@zilliz.com>
2022-04-01 14:25:29 +08:00
Letian Jiang cb65860e22
Add implementation for QueryShardService ()
Signed-off-by: Letian Jiang <letian.jiang@zilliz.com>
2022-04-01 14:19:29 +08:00
zhuwenxing 0c2fc2721e
[skip e2e]Update deploy test github action ()
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2022-04-01 14:09:29 +08:00
Ten Thousand Leaves 45be3deb3f
Implement RC, DC, DN calling path for import. ()
/kind feature

issue: 
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-04-01 11:33:28 +08:00
congqixia 376a7f6567
Fix Datacoord unsubAttempt using pchannel ()
Fix the channel name used when Datacoord unsub for DataNode
This channel shall always be pChannel instead of vChannel

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-04-01 11:25:28 +08:00
groot bd241cd28b
Add performance test case for import ()
Signed-off-by: groot <yihua.mo@zilliz.com>
2022-04-01 10:07:28 +08:00
zhenshan.cao b03da87df5
Accelerate query speed in sealed segment ()
Fix delete duplicate key

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-03-31 17:33:28 +08:00
XuanYang-cn 58addbf9f4
Add GetShardLeaders rpc in querycoord proto ()
See also: 

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-03-31 16:39:29 +08:00
Jenny Li e20385c817
[skip e2e] Update mergify rule for adding centos7 ()
Signed-off-by: Jenny Li <jing.li@zilliz.com>
2022-03-31 15:19:28 +08:00
congqixia f529f8c1fc
Use fmt.Errorf instead of str concat in func.go ()
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-03-31 15:03:28 +08:00
Jenny Li 56c0ea9b1d
[skip e2e] Add centos build ()
Signed-off-by: Jenny Li <jing.li@zilliz.com>
2022-03-31 14:49:28 +08:00
Letian Jiang ad9a2217ae
Add interface for QueryShardService and QueryShard ()
Signed-off-by: Letian Jiang <letian.jiang@zilliz.com>
2022-03-31 14:05:28 +08:00
Ten Thousand Leaves 0d54697d42
Implement ReportImport related logic. ()
/kind feature

issue: 
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-03-31 13:51:28 +08:00
Bingyi Sun baa745b83e
Improve log ()
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
2022-03-31 11:33:28 +08:00
aakejiang 30be20e357
correct DEVELOPMENT.md ()
Signed-off-by: kejiang <ke.jiang@zilliz.com>

Co-authored-by: kejiang <ke.jiang@zilliz.com>
2022-03-31 09:21:28 +08:00
zhuwenxing fd3ef9b11b
[skip e2e]Update deploy test script ()
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2022-03-31 09:19:32 +08:00
Bingyi Sun 40fab91448
improve log ()
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
2022-03-30 23:05:28 +08:00
XuanYang-cn a0bb758be5
Refine log in proxy searchTask ()
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-03-30 22:09:28 +08:00
congqixia 899638810a
Add error log when tsafe register watcher more than once ()
Add some test case to cover the error case
Add log in error branch

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-03-30 22:07:28 +08:00