milvus-io
/
milvus
mirror of https://github.com/milvus-io/milvus.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
14,450 Commits
133 Branches
115 Tags
559 MiB
Commit Graph

5069 Commits (5494f3c318ac9c443bca2e457f6e4bcbbe378f41)

Author SHA1 Message Date
XuanYang-cn 5494f3c318
Remove offLineNodes from watch states check (#16347) 
This PR also
- adds more information in log
- makes Session able to logged by zap.Any/zap.String

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2022-04-01 22:33:29 +08:00
congqixia 6111dcde52
Fix ticker leakage in datanode flowgraph (#16346) 
Using  in case ticker need to be closed may cause leakage
Use time.NewTicker with deferred Stop instead

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-04-01 19:35:29 +08:00
XuanYang-cn e9090a62ab
Rearrange search/queryTask for readability (#16325) 
`searchTask` has 683 lines of code, `queryTask` has 485 lines of code.
`task.go` contains 4k+ codes including `searchTask` and `queryTask`.

It was so much pain navigating codes of searchTask and queryTask though task.go,
task.go and task_test.go are literaly unreadable.

This PR moves
1. 650+ lines of code of `searchTask` from `task.go` to `task_search.go`.
2. 1.4k+ lines of test code of `searchTask` from `task_test.go` to
   `task_search_test.go`.
3. 450+ lines of code of `queryTask` from `task.go` to `task_query.go`.
4. 200+ lines of test code of `queryTask from `task_test.go to
   `task_query_test.go`.

This PR also rearrange methods positions of `searchTask` and
`queryTask`:
-  Putting the most important methods `PreExecute`, `Execute`, and
   `PosExecute` at the beginning of the file.
-  Moves interface methods `ID`, `SetID`, `Type`, `BeginTs`, etc.
   that nobody cares about to the bottom of the file.

See also: #16298

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2022-04-01 18:59:29 +08:00
congqixia 7a44fff8cd
Use fmt.Errorf instead of string concat in tsafe_replica (#16344) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-04-01 18:37:28 +08:00
yah01 c005f07ccc
Modify proto to add GetReplicas() interface (#16328) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-04-01 16:15:29 +08:00
Jiquan Long fd589baca7
Integrates marisa trie index (#16192) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2022-04-01 15:31:29 +08:00
Letian Jiang cb65860e22
Add implementation for QueryShardService (#16317) 
Signed-off-by: Letian Jiang <letian.jiang@zilliz.com>
 2022-04-01 14:19:29 +08:00
Ten Thousand Leaves 45be3deb3f
Implement RC, DC, DN calling path for import. (#16321) 
/kind feature

issue: #15604
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-04-01 11:33:28 +08:00
congqixia 376a7f6567
Fix Datacoord unsubAttempt using pchannel (#16337) 
Fix the channel name used when Datacoord unsub for DataNode
This channel shall always be pChannel instead of vChannel

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-04-01 11:25:28 +08:00
groot bd241cd28b
Add performance test case for import (#16296) 
Signed-off-by: groot <yihua.mo@zilliz.com>
 2022-04-01 10:07:28 +08:00
zhenshan.cao b03da87df5
Accelerate query speed in sealed segment (#16287) 
Fix delete duplicate key

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2022-03-31 17:33:28 +08:00
XuanYang-cn 58addbf9f4
Add GetShardLeaders rpc in querycoord proto (#16299) 
See also: #16298

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2022-03-31 16:39:29 +08:00
congqixia f529f8c1fc
Use fmt.Errorf instead of str concat in func.go (#16315) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-03-31 15:03:28 +08:00
Letian Jiang ad9a2217ae
Add interface for QueryShardService and QueryShard (#16309) 
Signed-off-by: Letian Jiang <letian.jiang@zilliz.com>
 2022-03-31 14:05:28 +08:00
Ten Thousand Leaves 0d54697d42
Implement ReportImport related logic. (#16312) 
/kind feature

issue: #15604
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-03-31 13:51:28 +08:00
Bingyi Sun baa745b83e
Improve log (#16307) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
 2022-03-31 11:33:28 +08:00
Bingyi Sun 40fab91448
improve log (#16236) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
 2022-03-30 23:05:28 +08:00
XuanYang-cn a0bb758be5
Refine log in proxy searchTask (#16263) 
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2022-03-30 22:09:28 +08:00
congqixia 899638810a
Add error log when tsafe register watcher more than once (#16294) 
Add some test case to cover the error case
Add log in error branch

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-03-30 22:07:28 +08:00
Jiquan Long ba37531456
Add support for loading multiple indexes (#16138) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2022-03-30 21:11:28 +08:00
yah01 0f53989f17
Modify proto to support replicas (#16289) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-03-30 17:57:28 +08:00
Cai Yudong 5a9e1af3bb
Update create plan CGO interface (#16271) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-03-30 17:13:27 +08:00
groot ffa06c77b6
Import util functions (#16237) 
Signed-off-by: groot <yihua.mo@zilliz.com>
 2022-03-30 16:25:30 +08:00
Xiaofan 801eeffbcc
Replace cgo parquet reader to go parquet reader (#16199) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-03-30 15:21:28 +08:00
congqixia 18a3e9f265
Add grpc interface for replica Search/Query in QueryNode (#16197) 
Resolves #16195
Add Search and Query grpc interface in query proto and types/types.go

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-03-30 12:03:27 +08:00
Xiaofan b6b3c9863c
Fix incompatibility of stats log (#16277) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-03-30 10:15:28 +08:00
Cai Yudong 0baa8c60a5
Fix MinGW build error (#16267) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-03-30 00:15:27 +08:00
Jiquan Long cd0f3cbcd3
Refine log (#16269) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2022-03-29 22:43:28 +08:00
cai.zhang e2e345079f
Add indexMeta version for IndexCoord log (#16266) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2022-03-29 21:03:27 +08:00
jaime d5d6e115bd
Refine delete segment data log (#16241) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
 2022-03-29 18:41:27 +08:00
Jiquan Long 3219b4e392
Skip dimension check for scalar index (#16257) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2022-03-29 15:35:27 +08:00
Cai Yudong 1160693e4c
Fix centos build error (#16251) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-03-29 15:13:30 +08:00
aakejiang de70dfe0c3
refine log (#16256) 
Signed-off-by: kejiang <ke.jiang@zilliz.com>

Co-authored-by: kejiang <ke.jiang@zilliz.com>
 2022-03-29 15:05:27 +08:00
congqixia 91d5436807
Improve log level and param function in QueryCollection (#16254) 
Use debug level for all debug information
Use specific parameter function for appliable ones

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-03-29 12:31:27 +08:00
zhenshan.cao 11cea5aad0
Refine proxy log (#16246) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2022-03-29 02:29:27 +08:00
Cai Yudong 2ac53c89e9
Revert #16242 (#16247) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-03-29 00:53:27 +08:00
XuanYang-cn 23b3a9ed8a
Log state in datanode handleWatchInfo (#16231) 
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2022-03-29 00:15:28 +08:00
Cai Yudong 9a6b2b8806
Fix centos build error (#16242) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-03-28 23:23:27 +08:00
bigsheeper 9d673006e0
Improve primary keys print (#16232) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-03-28 23:15:27 +08:00
xige-16 d2424fc50a
Print msgID instead of loadInfo when load segment (#16238) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-03-28 23:13:27 +08:00
congqixia 2c03287e79
Improve log level in proxy QueryTask (#16234) 
Use warn instead of debug in error case

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-03-28 23:11:26 +08:00
XuanYang-cn a77dd1034c
Adopt the extended state in DataCoord (#16200) 
ChannelWatchInfo with ToWatch, ToRelease will trigger a timer.
ChannelManager now reacts to different ChannelWatch states.

- WatchSuccess > log this info

- WatchFailure/WatchTimeout > ToRelease

- ReleaseSuccess > Delete, reassign if not from DropCollection

- ReleaseFailure/ReleaseTimeout > Cleanup subscription and Delete,
  reassgin if not from DropCollection.

Some Notes:
1. Reassignment will add this channel to buffer if there's only one node.

See also: #15846

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2022-03-28 22:33:27 +08:00
Jiquan Long 973ea406ca
Improve log of show collections (#16233) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2022-03-28 21:47:27 +08:00
cai.zhang c8688a6275
Organize log for GetIndexFilePaths (#16230) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2022-03-28 21:27:27 +08:00
Ji Bin 4f08f6d8c6
Workaround for knowhere under windows (#16214) 
issue #16205

Signed-off-by: Ji Bin <matrixji@live.com>
 2022-03-28 17:41:27 +08:00
yah01 178eb8cce1
Modify proto to support replicas (#16218) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-03-28 16:43:27 +08:00
Ten Thousand Leaves 40c703dacd
Support check index state in root coord, for bulk load feature. (#16198) 
issue: #15604

/kind feature

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-03-28 16:41:28 +08:00
Bingyi Sun e0ffe54547
Add interface to get replications by collection ID (#16213) 
issue: #16158
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
 2022-03-28 14:35:27 +08:00
Cai Yudong 3e3756d3dc
Upgrade to knowhere v1.1.1 to support Mac M1 build (#16210) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-03-28 14:33:26 +08:00
godchen 478f6ca11e
Add cache for vector chunk (#15912) 
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
 2022-03-26 22:05:26 +08:00