Commit Graph

2049 Commits (939afc866bfd63ca26fe46ee882641b5874e360d)

Author SHA1 Message Date
bigsheeper 0e81c62031
Check collection and plan before calling cgo functions (#7733)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-09-10 18:12:01 +08:00
groot c8f860dcfb
Add unittest for proxy_client_manager (#7729)
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2021-09-10 18:10:00 +08:00
yukun d60ea8bc21
Change prefix length to FixedChannelNameLen + 1 (#7717)
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
2021-09-10 17:42:10 +08:00
bigsheeper cb5cd1cf12
Add index unittests for query node (#7674)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-09-10 17:11:09 +08:00
groot 020f109dd8
Add unittest for rootcoord util (#7688)
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2021-09-10 16:41:48 +08:00
sunby db01040b53
Fix query request lost bug (#7715)
Issue: #7633
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-09-10 16:34:44 +08:00
XuanYang-cn ce33855652
[skip-ci]Fix datanode comment typo (#7689)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-09-10 14:46:00 +08:00
Xiangyu Wang 494600c37c
Add unit tests for msg.go (#7691)
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-09-10 14:24:02 +08:00
cai.zhang be9427e8b9
Increase covd coverage for paramtable (#7583)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2021-09-10 14:14:12 +08:00
Cai Yudong 4e23ed9a18
Revert apply physical channels from pool (d5f7e358 and 3b960969) (#7687)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-09-10 13:00:00 +08:00
congqixia d3be316bc2
Add Comment for datacoord cluster (#7676)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-09-10 11:26:00 +08:00
Xiangyu Wang 6b86a18a7e
Comments unused msg from msg.go (#7686)
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-09-10 11:12:05 +08:00
groot 16a6afa9bd
Fix build version typo (#7639)
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2021-09-10 11:10:00 +08:00
yukun 5a303e7672
Fix rocksmq load with prefix (#7678)
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
2021-09-10 10:22:01 +08:00
yukun 34228eb74b
Add topic lock for DestroyConsumerGroup (#7636)
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
2021-09-10 10:20:00 +08:00
nameczz a392f1005c
improve CreateCollectionRequest comment in milvus.proto (#7590)
Signed-off-by: Gitea <zizhao.chen@zilliz.com>
2021-09-10 10:06:00 +08:00
bigsheeper 8216478b3d
Add replica and insertNode unittests for query node (#7638)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-09-10 09:33:43 +08:00
Xiangyu Wang 42aafdc407
Remove unused msg from msg.go (#7612)
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-09-09 19:56:00 +08:00
sunby 49b6c3101d
Remove datacoord useless code (#7673)
Issue: #7624
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-09-09 19:08:03 +08:00
Xiangyu Wang 624ea2f676
Add unit tests for msg.go (#7659)
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-09-09 19:03:40 +08:00
dragondriver 596357dc85
Add more data definition test cases (#7623)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-09-09 19:02:08 +08:00
Cai Yudong d5f7e358aa
Let dml tolerant physical channels generated by old version (#7644)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-09-09 18:08:02 +08:00
zwd1208 02d5397c6f
[skip ci] Remove useless commets (#7662)
Signed-off-by: Weida Zhu <weida.zhu@zilliz.com>
2021-09-09 18:00:19 +08:00
zhenshan.cao 29e5138274
Fix unittest: timeout caused by excessive use of resources (#7653)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-09-09 17:39:30 +08:00
bigsheeper 1a5e5989c8
Add segment unittests for query node (#7632)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-09-09 17:36:02 +08:00
groot 314eed2172
Update storage unittest (#7637)
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2021-09-09 17:12:55 +08:00
congqixia 8e814a2972
Make flowgraph Close only once (#7631)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-09-09 17:12:38 +08:00
groot a3666c843f
Add unittest for chunk manager (#7628)
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2021-09-09 17:12:15 +08:00
cai.zhang f6621dc847
Increase code coverage for indexnode component (#7520)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>

Co-authored-by: edward.zeng <jie.zeng@zilliz.com>
2021-09-09 17:09:08 +08:00
sunby 06cee85a39
Add data node unit tests (#7622)
Issue: #6357
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-09-09 16:40:00 +08:00
XuanYang-cn 64aad49959
Add ut in datasync service to 88% (#7615)
See also: #6357

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-09-09 15:36:01 +08:00
congqixia 69794fd32d
Refactor insertBufferNode and add unit tests (#7621)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-09-09 15:00:00 +08:00
groot dbd2a02462
Add unittest for VectorChunkManager (#7610)
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2021-09-09 14:34:00 +08:00
zhenshan.cao 89358ad91e
Deep copy rocksdb data to byte slice (#7605)
Add unittest

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-09-09 10:56:00 +08:00
congqixia c96b19a640
Add datacoord metricsinfo unit test (#7595)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-09-09 10:16:00 +08:00
XuanYang-cn 762060e670
Add ut for segmentReplica to 100% (#7589)
See also: #6357
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-09-09 10:14:00 +08:00
bigsheeper 5906551f41
Add task unittest for query node (#7599)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-09-09 10:10:00 +08:00
dragondriver 9b21b3b5fd
Initialize test framework of proxy (#7563)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-09-09 10:06:29 +08:00
godchen 3c2821e5e2
Add rmq consumer close (#7596)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-09-08 21:49:59 +08:00
bigsheeper d3a18a66b5
Add tSafeReplica and task scheduler unittests (#7587)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-09-08 20:50:00 +08:00
XuanYang-cn 074b610811
Add ut for dmstream_input_node to 100% (#7593)
See also: #6357

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-09-08 19:12:00 +08:00
groot 159d3702a0
Rename unittest for indexparamcheck (#7580)
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2021-09-08 19:10:03 +08:00
bigsheeper 50c677f72b
Remove useless message and add service time node unittest (#7565)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-09-08 17:10:15 +08:00
Xiangyu Wang 2224099f22
Add unit tests for msg.go (#7577)
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-09-08 16:54:00 +08:00
groot ae3a43cf37
Rename unittest for funcutil (#7581)
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2021-09-08 16:53:30 +08:00
groot 618fef0a43
Add unittest for mqclient (#7578)
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2021-09-08 16:52:35 +08:00
sunby 79516dacd2
Add log unit tests (#7561)
Issue: #7491
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-09-08 15:03:59 +08:00
zhenshan.cao b0524adbd8
Support specified shard number when create collection (#7482)
Add shards_num to CollectionInfo

Modify dmlChannelNum config

Compatible with old meta

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-09-08 15:00:00 +08:00
cai.zhang e1081b6783
Increase indexcoord component code coverage (#7558)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2021-09-08 14:23:59 +08:00
yukun a560aefc36
Fix search result is wrong when the term content in expression is not in order (#7557)
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
2021-09-08 11:49:59 +08:00
congqixia 664aeb09c6
Add ut and comments for datacoord meta (#7560)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-09-08 11:35:59 +08:00
groot 58700dedbc
Update unittest name of distance (#7537)
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2021-09-08 11:28:25 +08:00
yukun fbc352263c
Improve rocksmq client code coverage (#7540)
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
2021-09-08 11:03:59 +08:00
XuanYang-cn 4785e452be
Fix Arrow unable to find Boost error (#7552)
Resolve: #7549

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-09-08 10:59:59 +08:00
sunby bbb6051cdc
Remove time-consuming test (#7551)
issue: #7491
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-09-08 10:59:52 +08:00
XuanYang-cn a5767e05c6
Add ut for flowgraph_delete_node to 100% (#7550)
See also: #6357
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-09-08 10:41:59 +08:00
congqixia 76f5264e79
Fix MqMsgStream Seek logic (#7536)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-09-07 16:35:59 +08:00
sunby 62f4cb5e1b
Remove stats watcher (#7501)
issue: #7500
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-09-07 16:11:07 +08:00
sunby 668d10d1da
Add log unit tests (#7524)
issue: #7491
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-09-07 16:07:58 +08:00
bigsheeper bcccd767ac
Add unittests for querynode (#7510)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-09-07 15:45:59 +08:00
XuanYang-cn 29756c6ce8
Add ut for flow_graph_dd_node to 100% (#7511)
This PR:
- Change mutex to sync.Map
- Add ut for flow_graph_dd_node.go to 100%

See also: #6357

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-09-07 15:41:59 +08:00
groot 89815a64d7
Update unittest name of paramtable (#7526)
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2021-09-07 15:20:40 +08:00
xige-16 468bef4478
Fix dataset's memory leak in knowWhere when search (#7527)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-09-07 14:25:59 +08:00
congqixia 5b22697ec4
Add segment manager unit tests (#7525)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-09-07 13:59:58 +08:00
bigsheeper b7103357fd
Load failed if collection beyonds the memory limit (#7434)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-09-07 11:35:18 +08:00
Cai Yudong d15d3fa0ac
Improve rootcoord codecov (#7518)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-09-07 11:16:37 +08:00
groot 1401ed516d
Add unittest for flowgraph (#7512) 2021-09-07 09:39:47 +08:00
dragondriver 42b687bf48
Add unittest for task scheduler (#7508)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-09-06 20:49:04 +08:00
yukun 7025a6e925
Improve rocksmq code coverage (#7483)
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
2021-09-06 19:36:42 +08:00
cai.zhang e6e03fc93f
Add unittest for index component (#7475)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2021-09-06 17:54:41 +08:00
congqixia fb4e23bc79
Add datacoord server unit tests (#7499)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-09-06 17:02:41 +08:00
sunby 5e0d724738
Add log unit tests (#7492)
issue: #7491
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-09-06 16:50:42 +08:00
Xiangyu Wang 79a0a5482f
Add unit test for mq_factory (#7477)
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-09-06 14:59:21 +08:00
zhenshan.cao 86afde4598
Fix unittest for rootcoord, should filter out timetick msg first (#7490)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-09-06 11:52:41 +08:00
groot af7b7537f0
Update unittest for calc_distance (#7488) 2021-09-06 11:35:02 +08:00
congqixia 1f590d6c26
Add datacoord services related unit tests (#7467)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-09-06 11:12:42 +08:00
groot ccfb350bd1
Unittest for funcutil (#7473)
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2021-09-04 12:13:25 +08:00
Cai Yudong 5a16236768
Improve codecov for paramtable and typeutil (#7471)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-09-04 10:22:41 +08:00
dragondriver 960741ee16
Support more detailed metrics information (#7472)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-09-03 19:35:47 +08:00
dragondriver 1db573731b
Cache the system information metrics (#7436)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-09-03 17:15:26 +08:00
yukun 94272bba87
Support query by expression (#7386)
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
2021-09-03 17:12:55 +08:00
godchen db8067d61c
Add trace ut (#7445)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-09-03 15:41:25 +08:00
groot 229965a9bb
Unittest for indexparamcheck (#7464)
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2021-09-03 14:22:17 +08:00
Xiangyu Wang acf4354b83
Add logs in rootcoord (#7457)
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-09-03 14:10:10 +08:00
dragondriver 9b87da1ee8
Fix deadlock of channels time ticker (#7444)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-09-03 11:53:29 +08:00
godchen 8b1ae98fa9
Add id allocator ut (#7440)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-09-03 09:44:10 +08:00
Cai Yudong 7edd96989f
Improve codecov for paramtable and typeutil (#7451)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-09-02 22:18:10 +08:00
yukun d5d73833b2
Add some logs in rocksmq retention (#7403)
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
2021-09-02 20:04:10 +08:00
congqixia d3027c0d28
Fix possible deadlock (#7428)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-09-02 14:56:10 +08:00
Cai Yudong bd3056fcbe
Improve tso codecov (#7407)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-09-01 18:29:59 +08:00
dragondriver 3721cbfea1
Proxy metrics integrates with other coordinators (#7400)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-09-01 17:35:00 +08:00
yukun 71aa300499
Fix rocksmq retention panic when delete by message size (#7393)
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
2021-09-01 12:03:58 +08:00
dragondriver 89748ac639
Expose metrics of data cluster (#7177)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-09-01 10:13:15 +08:00
Cai Yudong 3b9609692b
Apply for msgstream from pool when creating collection (#7377)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-08-31 18:35:58 +08:00
godchen 08a31010e9
Change log (#7366)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-08-31 12:07:58 +08:00
dragondriver f0dbdd7f13
Expose metrics of root coordinator (#7228)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-08-31 11:45:59 +08:00
sunby 8f660ae4e6
Add log print in proxy (#7369)
issue: #7316
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-08-31 11:17:59 +08:00
Cai Yudong f4b53df727
Revert "Apply for msgstream from pool when creating collection (#7323)" (#7376)
This reverts commit e6de86a433.
2021-08-31 11:07:58 +08:00
dragondriver 5b354313df
Add unittest for timestamp allocator (#7367)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-08-31 10:33:58 +08:00
dragondriver 84bddd0a03
Add unittest cases for proxy (#7364)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-08-31 10:25:59 +08:00
Cai Yudong e6de86a433
Apply for msgstream from pool when creating collection (#7323)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-08-31 09:47:58 +08:00
congqixia 1c88c8b0a5
Clean fmt.Println in util/flowgraph (#7355)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-08-30 15:29:57 +08:00
congqixia 65f168eac1
Fix datanode corner cases (#7336)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-08-30 10:03:58 +08:00
xige-16 af574d17d5
Fix querycoord panic when stop nil conn of querynoClient (#7346)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-08-28 12:51:57 +08:00
zhenshan.cao cc99bae250
Add unittest for proxy (#7342)
Fix unittest data race

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-08-28 10:21:57 +08:00
cai.zhang 880aa540e6
Fix bug for expression boundary (#7339)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2021-08-28 10:13:57 +08:00
godchen c333af0dcf
Add query node segment filter (#7303)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-08-28 10:12:00 +08:00
godchen 2dc2cb1a28
Fix collection nil when add query collection (#7311)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-08-27 17:51:56 +08:00
bigsheeper 14391fe9a2
Release flowgraphs properly (#7266)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-08-27 17:03:56 +08:00
bigsheeper 1e60cccaf9
Set inMemoryPercentage to 0 when releasing partition (#7310)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-08-27 16:51:11 +08:00
congqixia 0281bd8d22
Change context to correct package (#7331)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-08-27 15:37:56 +08:00
zhenshan.cao 7c60e6aa00
Avoid port conflicts between embedded etcd and external etcd (#7320)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-08-27 10:13:57 +08:00
sunby 920b09cf7d
Print insert log once receive a request (#7318)
issue: #7316
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-08-27 10:08:47 +08:00
Xiangyu Wang c62f23e201
[skip ci]Remove neza2017 from reviewers (#7324)
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-08-27 00:27:36 +08:00
xige-16 055d94ede1
Add ut for queryCoord (#7212)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-08-26 14:17:54 +08:00
groot 8701c477e2
Add delete api into proto (#7304)
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2021-08-26 12:15:52 +08:00
congqixia 4d9f89df09
Add unit test for 7150 fix (#7276)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-08-25 14:41:52 +08:00
godchen b4c55f6b3c
Add delete node (#7261)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-08-25 11:41:52 +08:00
congqixia daae05641a
Construct bloom filter when inserting (#7268)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-08-25 10:29:52 +08:00
congqixia d83bc7fb54
Fix 0 ts when build index (#7272)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-08-24 21:15:52 +08:00
sunby 4d56d53c0b
Add a tool to move binlog path to SegmentInfo (#7243)
issue: #7214
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-08-24 15:51:51 +08:00
bigsheeper 653ec6cedb
Remove service time check in streaming search (#7253)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-08-24 15:07:52 +08:00
Cai Yudong 0ab70271a6
correct empty search result handle (#7244)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-08-24 11:13:52 +08:00
Xiaofan 58b2089692
Import Embedded Etcd Kv Part 1(#7112) (#7174)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2021-08-24 09:45:51 +08:00
wt 780a4d2c8e
Update script of run_go_codecov (#7248)
Signed-off-by: wangting0128 <ting.wang@zilliz.com>
2021-08-23 22:01:51 +08:00
congqixia f5451d12f3
Refactor datacoord allocator (#7226)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-08-23 17:59:51 +08:00
godchen db25550fd0
Add segment filter interface (#7231)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-08-23 17:49:50 +08:00
Cai Yudong 8405d90f5e
proxy add illegal check for search result (#7227)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-08-23 17:03:51 +08:00
godchen 33583525c5
Add connection manager (#6867)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-08-23 10:00:12 +08:00
godchen bf2c427609
Fix connection timeout (#7203)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-08-20 17:52:11 +08:00
congqixia be5c492e5a
Add datacoord unit tests (#7197)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-08-20 17:50:12 +08:00
congqixia 881d0af1fc
Add sync logic in TestWatchChannel (#7188)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-08-20 17:48:13 +08:00
XuanYang-cn 9e71daa759
Fix rocksdb not installed problem (#7200) 2021-08-20 17:46:48 +08:00
groot 1c6f5104f7
Improper error returned by calc_distance (#7175)
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2021-08-20 16:19:42 +08:00
sunby 6e34f4c7f1
Add a seal policy which restrict the lifetime of a segment (#7172)
issue: #7164
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-08-20 15:42:12 +08:00
紫晴 7233b62dad
Enable 70% of delta condition judgment for codecov (#7160)
Signed-off-by: wangting0128 <ting.wang@zilliz.com>
2021-08-19 15:50:12 +08:00
congqixia ec9ccd8bd1
Fix DropCollection not processed by datanode (#7151)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-08-19 15:06:12 +08:00
cai.zhang 171088409b
Increase the timeout for grpc connection when create index (#7163)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2021-08-19 14:32:11 +08:00
xige-16 76066b5659
Fix IVF_FLAT index's wrong meta_data (#7167)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-08-19 14:26:11 +08:00
sunby 3cc210da26
Add flush interval (#6996)
Time tick message is sent about per 0.2 seconds. DataCoord will call
`FlushSegment` too frequently. We add a min interval for segment flush. A
segment will called `FlushSegment` only once between the interval.

issue: #6995
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-08-19 14:08:10 +08:00
sunby 4f7c3e7d3a
Save binlog path in SegmentInfo (#6590)
issue: #6589
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-08-19 13:00:12 +08:00
dragondriver 648d22ee40
Expose metrics of IndexCoord and IndexNodes (#7157)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-08-19 10:28:10 +08:00
godchen 8cc24e09c9
Make users get vectors without using local storage (#7134)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-08-18 16:30:11 +08:00
紫晴 09525f5723
Update file of codecov (#7144)
Signed-off-by: wangting0128 <ting.wang@zilliz.com>
2021-08-18 14:48:11 +08:00
Cai Yudong 305fa9c17e
Fix timestamp go back issue for timeticksync (#7123)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-08-18 14:36:10 +08:00
groot b4eb20f2c0
Fix calc_distance TANIMOTO bug (#7140)
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2021-08-18 11:16:10 +08:00
dragondriver e45cb6c644
Expose system topology information by proxy (#7137)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-08-18 10:12:10 +08:00
Cai Yudong 21117241ec
Let system panic if kv operation fail (#7143)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-08-17 23:06:10 +08:00
cai.zhang 3d83e99a99
Adding indexnode should be processed asynchronously (#7141)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2021-08-17 20:26:11 +08:00
cai.zhang c2a4bd3e9c
Lock when writing meta instead of trying to connect to indexnode (#7136)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2021-08-17 17:54:10 +08:00
紫晴 1b4231b4e8
Add yaml file of codecov (#7124)
* Add yaml file of codecov

Signed-off-by: wangting0128 <ting.wang@zilliz.com>

* Update code of codecov

Signed-off-by: wangting0128 <ting.wang@zilliz.com>
2021-08-17 11:56:11 +08:00
dragondriver 7eb8a665b2
Expose metrics of QueryCoord and QueryNodes (#7113)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-08-17 10:06:11 +08:00
congqixia fb22d122e5
Add EarliestMessageID patch logic (#7119)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-08-16 19:08:09 +08:00
yukun 61bda45e35
Add ConfAdapter to check search (#6971)
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
2021-08-16 18:54:10 +08:00
yukun 847586eb95
Add rocksmq retention (#6617)
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
2021-08-16 18:46:10 +08:00
yukun 671b2737d2
Remove RetrieveRequest and RetrieveTask (#6848)
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
2021-08-16 17:06:10 +08:00
xige-16 b035ec5c7b
Enable search when querynode only load segment (#7104)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-08-16 12:24:09 +08:00
sunby 9551dc3b7e
Fix segment auto flush bug (#7085)
issue: #7084
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-08-16 11:00:09 +08:00
Cai Yudong e771bda92f
optimize retrieve output vector code structure (#7102)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-08-14 11:18:10 +08:00
cai.zhang 382fa6f274
IndexNode dose not need to wait for IndexCoord to start to comlete (#7074)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2021-08-13 18:12:08 +08:00
XuanYang-cn 86ca96a02e
Fix master branch unittest error (#7083)
Resolves: #7080

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-08-13 14:00:14 +08:00
XuanYang-cn 6c5be83ea1
Refactor NewEtcdKV API (#7075)
This PR mainly did the following 3 things:

- Refactor NewEtcdKV API
- Add unittests for `kv/etcd/`, raise the coverage
up to 94%
- Correct some APIs from version to revision

The old NewEtcdKV() API took `clientv3.Client` as
one of the input, making everyone who using this API
had to create a new `clientv3.Client`. So there're a lot
of repeated codes everywhere.

Meanwhile, someplace didn't even check whether the
`clientv3.New()` returned error or not, and the old API
also didn't check whether the clientv3.Client was Nil.
This caused issue#6955.

Ectd's version and revision have nothing alike, they
represent different concepts, but our APIs kind of
mixed them all up. So I correct those APIs' names.

Resolves: #6955

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-08-13 11:04:09 +08:00
congqixia e1b3273d1c
Add Datanode watch etcd channel (#6965)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-08-13 10:50:09 +08:00
sunby 5c383d1d02
Add a checker to listen for the timetick. (#7061)
issue: #7060
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-08-12 19:58:08 +08:00
Xiangyu Wang 5e78fd6a97
[skip ci]Update approvers (#7063)
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-08-12 19:36:09 +08:00
Cai Yudong 6c75301c70
optimize search reduce logic (#7066)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-08-12 18:00:11 +08:00
dragondriver b3f10ae5cc
Fix 6990, enhance the log when parsing grpc recv/send parameters (#6993)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-08-12 16:16:09 +08:00
congqixia 3f21e3d062
Fix querynode tt producer leak (#7059)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-08-12 14:16:08 +08:00
bigsheeper b9d3897deb
Return error when any failed detected in loadSegment (#7005)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-08-12 14:14:08 +08:00
Cai Yudong 85ea051ddc
change growing segment default value to -1 (#7054)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-08-12 12:32:07 +08:00
sunby 31182e52a9
Fix expire allocation bug (#7053)
issue: #7052
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-08-12 10:48:08 +08:00
godchen 7fc7623b92
Add vector data memroy existence check (#7044)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-08-11 19:50:09 +08:00
XuanYang-cn 9d81a90402
Fix mem leak of flush cache in DataNode (#6862)
Resolves: #6858

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-08-11 14:24:09 +08:00
groot bdb8396e74
Fix CalcDistance wrong result when fetting vectors from collection (#6976)
* Fix CalcDistance wrong result when fetting vectors from collection

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* Fix CalcDistance wrong result when fetting vectors from collection

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* preset capacity

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* typo

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* error check

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* code lint

Signed-off-by: yhmo <yihua.mo@zilliz.com>
2021-08-10 11:59:28 +08:00
godchen 7557616fea
Change client timeout (#6982)
* Change client timeout

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* change timeout

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* fix error

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* change timeout time

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* fix conflicts

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* fix ut

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* remove ut

Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-08-06 14:09:26 +08:00
congqixia 9eb35996b5
Fix pulsar consumer goroutine leakage (#7004)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-08-06 09:45:24 +08:00
sunby 454620dfaa
Add unit test in datacoord (#6985)
issue: #6984
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-08-04 16:59:25 +08:00
dragondriver ad77a6e0d4
Fix #6859, increase the MaxCallRecvMsgSize and MaxCallSendMsgSize of grpc client (#6861)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-08-04 13:03:24 +08:00
bigsheeper 07cc449fbf
Fix data race in flow graph (#6946)
* Fix data race in flow graph

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

* add cancel func to flowgraph

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-08-03 22:43:25 +08:00
xige-16 c8a1f780c1
Use proto marshal instead of MarshalTextString in querycoord (#6958)
* use proto marshal instead of marshalToText

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* log error

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* don't retry after init meta/cluster/scheduler failed

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* fix return err

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* log inconsistent task info

Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-08-03 22:03:25 +08:00
bigsheeper 4123deef9e
Return error message in grpc.LoadSegment in query node (#6908)
* return load error

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

* init streaming meta in loadSegment

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

* fix excluded init

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

* improve impl

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-08-03 22:01:27 +08:00
xige-16 09199170a4
sync call querycoord releaseXXXX (#6978)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-08-03 21:47:24 +08:00
Xiaofan aba234fa3f
Fix Msgstream exponential retry (#6807)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2021-08-03 10:39:24 +08:00
dragondriver d2767f920c
Fix index parameters check of RHNSWPQ (#6950)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-08-03 10:25:25 +08:00
XuanYang-cn 47df4a2e57
Fix DataNode BackGroudGC problem (#6551)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-08-03 09:59:24 +08:00
xige-16 87c5a49d9a
Refactor showCollection/showPartition to get load state in memory (#6935)
* refactor showCollection/showPartition to get load state in memory

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* update proto

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* update proto

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* update milvus.proto

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* add show result log to debug

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* fix wrong err message

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* update pymilvus-orm version

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* fix repeated load error

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* update proto

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* fix status nil error

Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-08-02 22:39:25 +08:00
congqixia 4d5eeb2890
Fix minio object leaking goroutine (#6944)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-08-02 15:25:24 +08:00
Cai Yudong fe50f97726
Use id to tell search result validation in proxy reduce stage (#6905)
* use id to tell search result validation

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* enable test_search_binary_hamming_flat_index

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* code optimize

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix merge retrieve result issue

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-08-02 10:25:49 +08:00
bigsheeper e8ea8b51d5
improve query node log (#6897)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-07-31 10:47:22 +08:00
bigsheeper 7307332fab
Add nil check (#6832)
* add nil check

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

* use grpc functions

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

* use grpc functions

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

* fix indexCoord crash

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-07-29 20:11:23 +08:00
bigsheeper eee0f1e077
Add mutex for segment of segCore in querynode (#6870)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-07-29 16:03:22 +08:00
cai.zhang d51fec6678
Enhance log when create index failed (#6871)
* Enhance log when create index failed

Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>

* Fix bug for repeatedly drop index

Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>
2021-07-29 14:47:22 +08:00
godchen 61786f769c
Add file exist check (#6840)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-07-28 16:31:22 +08:00
sunby 23bffe399d
Enhance log print in data coordinator (#6681)
issue: #6680
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-07-28 11:43:22 +08:00
zhenshan.cao 8a1a841011
FixBug: Incorrect handling of time synchronization (#6814)
* FixBug: proxy send wrong sync timetick and DataCoord did not filter channel

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-07-27 14:34:48 +08:00
dragondriver 3cc8ee298e
Filter NaN when reducing search results (#6806)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-07-27 10:19:21 +08:00
xaxys e43b43e7d8
Refactor `RangeExpr` & `CompareExpr` (#6786)
* Changed `RangeExpr` proto to `UnaryRangeExpr` & `BinaryRangeExpr`
Several unit test unpassed.

Signed-off-by: xaxys <tpnnghd@163.com>

* Fix bugs to pass unit test. Fix format.

Signed-off-by: xaxys <tpnnghd@163.com>

* Remove debug information.

Signed-off-by: xaxys <tpnnghd@163.com>

* Fix format.
Remove debug information.
Unify variable name.
Add error information.
Remove `CompareExpr` test in `test_c_api.cpp`.

Signed-off-by: xaxys <tpnnghd@163.com>

* Fix code format.

Signed-off-by: xaxys <tpnnghd@163.com>

* Update `Plan.cpp`.

Signed-off-by: xaxys <tpnnghd@163.com>
2021-07-25 10:23:19 +08:00
zhenshan.cao 2962d31a5f
Fixbug: segment allocate but return value incorrect (#6788)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-07-24 20:49:20 +08:00
godchen db94d7771f
Read vector from disk (#6707)
* Read vector from disk

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* go fmt

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* fix git action error

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* fix error

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* fix test error

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* fix action error

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* fix caculate error

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* change var name

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* remove unused method

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* remove unused method

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* fix error

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* fix len error

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* remove unused code

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* change bytes to float method

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* change float to bytes method

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* fix action error

Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-07-24 09:25:22 +08:00
sunby aa20d88f6f
Fix datacoord stop bug (#6642)
issue: #6641
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-07-24 09:25:14 +08:00
sunby fc74a0f578
Auto create new segments when allocating rows more than (#6665)
max number of rows per segment

If user insert too much rows in a request. Now we will return a failed
response. Maybe auto creating new segments to hold that much rows is a
better way.

issue: #6664
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-07-23 21:58:33 +08:00
sunby 63387c83ef
Remove collection check when saving binlog (#6658)
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-07-23 21:23:22 +08:00
dragondriver 2821c08560
Add unittests for metric_type: tanimoto (#6773)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-07-23 20:59:33 +08:00
dragondriver cf8e9d52b1
Reduce the log of proxy (#6766)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-07-23 20:33:34 +08:00
bigsheeper 21bc5810c0
remove redundant log of query node (#6768)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-07-23 20:09:33 +08:00
bigsheeper 2f44fc40a3
fix historical unittest (#6705)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-07-23 19:55:07 +08:00
FluorineDog aba21baf82
refactor expr executor to use single bitset (#6667)
Signed-off-by: fluorinedog <fluorinedog@gmail.com>
2021-07-23 15:30:12 +08:00
neza2017 5adb69161f
Set CollectionInfo.CreateTime to the timestamp when save the metadata into etcd, (#6763)
not the timestamp when RootCoord receive the CreateCollection Request

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
2021-07-23 14:36:12 +08:00
cai.zhang e1b52b0583
Add unittest for IndexCoord and IndexNode (#6698)
* Add unittest for IndexCoord and IndexNode

Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>

* Fix verifiers

Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>
2021-07-23 10:44:12 +08:00
cai.zhang 9aa56cd7b8
Add log for IndexCoord and IndexNode (#6597)
* Add log for IndexCoord and IndexNode

Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>

* Make verifier

Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>

* Delete unused script

Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>
2021-07-22 11:40:11 +08:00
cai.zhang a3c662c7bf
Add watchWithVersion interface for etcdKV (#6663)
* Add watchWithVersion interface for etcdKV

Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>

* Add unittest

Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>

* Add LoadWithVersion interface

Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>
2021-07-21 18:20:11 +08:00
dragondriver 67c1fdd2c0
Show/Describe collections/partitions with created timestamp (#6700)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-07-21 18:00:14 +08:00
Cai Yudong cbb4052fdb
Support search and query output fields using wildcard (#6671)
* update wildcard polocy for search and query

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* mark xfail cases

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* mark xfail cases

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* optimize debug log

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix static-check

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-07-21 14:52:12 +08:00
dragondriver 26f890b5f5
Remove too frequent timetick log (#6682)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-07-21 11:26:11 +08:00
congqixia 9b1708f6e5
Fix proxy crashes when retrieve task is called (#6656)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-07-20 22:33:09 +08:00
Cai Yudong 744f2c7f51
optimize test_c_api.cpp (#6637)
* optimize test_c_api.cpp

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* clean redundant file

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* change DIM to const

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* optimize test_c_api.cpp

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update reduce testcase

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-07-20 17:13:21 +08:00
FluorineDog 234954931f
remove deprecated (#6623)
* remove deprecated

Signed-off-by: fluorinedog <fluorinedog@gmail.com>

* fix timeout

Signed-off-by: fluorinedog <fluorinedog@gmail.com>
2021-07-20 15:18:08 +08:00
godchen b9f5949680
Add vector chunk manager (#6613)
* add file manager

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* add file manager

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* add vector file manager

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* rename

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* go fmt

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* fix ut error

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* fix action error

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* fix memory error

Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-07-20 15:06:09 +08:00
dragondriver 99249a0224
Fix bug: override the compare function of SearchResultPair (#6628)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-07-20 14:38:11 +08:00
sunby ab5a7cbf44
Remove timesync package (#6619)
issue: #6618
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-07-19 18:53:09 +08:00
sunby b98b30b96f
Fix datacoord_test.go failed (#6558)
Create a mock rootcoord when testing datacoord to hide the network layer
issue: #6557
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-07-19 10:41:15 +08:00
dragondriver 0e2329c7ad
Refactor snapshot of timestamp statistics (#6600)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-07-17 16:31:29 +08:00
Cai Yudong a7b27db63e
update fillRetrieveResults parameter (#6598)
* update fillRetrieveResults parameter

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* optimize fillVectorFieldData process

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-07-17 15:17:30 +08:00
dragondriver 2d9361e0ba
Fix bug: remove duplicated primary field id when retrieve entities (#6577)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-07-17 10:53:30 +08:00
Cai Yudong a992dcf6a8
Support query return vector output field (#6570)
* improve code readibility

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add offset in RetrieveResults

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add VectorFieldInfo into Segment struct

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add new interface for query vector

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update load vector field logic

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update load vector field logic

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fill in field name in query result

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add FieldId into FieldData

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add fillVectorOutputFieldsIfNeeded

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update data_codec_test.go

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add DeserializeFieldData

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* realize query return vector output field

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix static-check

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* disable query vector case

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-07-16 17:19:55 +08:00
紫晴 4c7b57ffc5
Upload codecov report in GitHub Actions (#6565)
* Upload codecov report in GitHub Actions

Signed-off-by: wangting0128 <ting.wang@zilliz.com>
2021-07-16 14:52:21 +08:00
xige-16 2a42244ff6
fix watchQueryChannelTask's info can't be deleted from etcd (#6568)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-07-16 10:21:55 +08:00
dragondriver 84b9791221
Fix snapshot of timestmap statistics (#6543)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-07-15 17:47:55 +08:00
sunby ee38d14f5d
Fix channel lost after data node crash (#6545)
If we start up 2 data nodes and one of them crashes. We expect that all
channels of crashed node will be allcoated to the alive node. But now we
discover that these channels are lost after data node crash. The reason
is we pass a NodeInfo with empty channel info. We fix it and improve log
print.

issue: #6501
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-07-15 16:38:31 +08:00
bigsheeper 7b9f0d398f
update flowgraph unittest (#6537)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-07-15 14:51:55 +08:00
groot 6b001fd76a
fix debug error in IDE (#6539)
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2021-07-15 14:27:54 +08:00
XuanYang-cn dca6c6afde
Add unittest for BackGroundGC in DataNode (#6522)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-07-15 10:05:55 +08:00
dragondriver 5016e509be
Support wildcard match on search/query output fields (#6510)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-07-14 18:51:54 +08:00
neza2017 fac350cc58
fix tso (#6500)
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
2021-07-14 17:11:54 +08:00
dragondriver 7f99d9d714
Loose the limit of dimension when searching on a binary vector field (#6503)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-07-14 14:33:54 +08:00
cai.zhang f469a315d6
Refactor the logic of assign tasks in IndexCoord (#6328)
* Refactor the logic of assign tasks in IndexCoord

Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>

* Fix bugs

Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>

* Fix bug for unittest

Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>

* Add lock for map

Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>

* Improve code

Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>

* Fix unittest bug

Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>

* Reduce duriation for timetick

Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>

* Update orm version

Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>

* Reset sdk version

Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>

* Fix bug

Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>

* Reset orm version

Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>

* Reset test ip

Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>

* Fix bug

Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>

* Fix bug for unissued

Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>

* Rename some variables

Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>

* Fix bug

Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>

* Use break instead of continue in select::case

Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>
2021-07-14 14:15:55 +08:00
Cai Yudong 1c29b42434
change output_fields to out_fields_id for RetrieveRequest (#6506)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-07-14 13:11:54 +08:00
bigsheeper e50d39dfe1
Query result get global sealed segments from ETCD (#6377)
* get global sealed segments from ETCD

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

* fix race error

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-07-14 11:15:54 +08:00
cai.zhang f3a5b46288
Fix bug for repeatedly create bucket in minio (#6477)
* Fix bug for repeatedly create bucket in minio

Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>

* Simplify code

Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>

* Add log

Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>
2021-07-14 09:55:46 +08:00
Cai Yudong 724f10b9a0
Unify the usage of query and search (#6467)
Unify the usage of query and search

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-07-13 22:20:33 +08:00
neza2017 b66157358c
add more log for create collection (#6478)
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
2021-07-13 22:19:55 +08:00
dragondriver d896e3119e
Fix 6419, check if num_rows is greater than zero in proxy (#6439)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-07-13 16:33:55 +08:00
dragondriver 25b8c37383
Fix 6421, split raw vectors to serveral smaller one (#6449)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-07-13 16:05:36 +08:00
congqixia bc15b74244
Handle datasorter edge cases (#6465)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-07-13 14:37:54 +08:00
xaxys e5f8a77a2d
Extend grammar of boolean expression (#6460)
* Extend grammar of boolean expression

Signed-off-by: xaxys <tpnnghd@163.com>
2021-07-13 14:37:03 +08:00
xige-16 2b720fd2f7
Fix load hang when restart querynode many time in short time (#6351)
* fix load hang when restart querynode many time in short time

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* add multi queryNode ut

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* add ut for restart querynode

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* set queryCoord contex to load collection

Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-07-13 14:16:00 +08:00
neza2017 d4239f0cd8
masterservice -> rootcoord (#6447)
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
2021-07-13 09:37:40 +08:00
sunby 83be910baa
Refactor segment manager (#6441)
issue: #6440
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-07-12 17:24:25 +08:00
sunby a8e5fd2024
Refactor cluster in dataservice (#6356)
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-07-12 11:03:52 +08:00
dragondriver bd317a5461
Check if field data is empty (#6404)
* Check if field data is empty

Signed-off-by: dragondriver <jiquan.long@zilliz.com>

* FixBug: fast fail when output_fields contain fields that are not in schema

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>

Co-authored-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-07-11 19:39:52 +08:00
neza2017 8b89744c6b
search with exp (#6409)
* process float vector and binary vector field
on the search result

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* add more debug info on etcd

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* add debug info when grpc failed

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
2021-07-10 10:21:52 +08:00
FluorineDog 99ed122d11
Remove Dead Code, use signed type (#6398)
* make type signed

Signed-off-by: fluorinedog <fluorinedog@gmail.com>

* remove dead code

Signed-off-by: fluorinedog <fluorinedog@gmail.com>

* remove code

Signed-off-by: fluorinedog <fluorinedog@gmail.com>
2021-07-09 18:25:11 +08:00
FluorineDog 2ff072ea7b
Use larger dim for binary test to avoid false positive test failed (#6416)
Signed-off-by: fluorinedog <fluorinedog@gmail.com>
2021-07-09 18:12:40 +08:00
groot 19fc42c9eb
calc_distance typo (#6380)
* calc_distance typo

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* calc_distance typo

Signed-off-by: yhmo <yihua.mo@zilliz.com>
2021-07-09 14:53:52 +08:00
godchen 346e9cba4a
Add minio FGet method (#6386)
* Add minio FGet method

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* go fmt

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* change el append

Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-07-09 11:44:40 +08:00
zhenshan.cao 6b4b71bf03
Fixbug: remove wrong check code (#6372)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-07-08 20:17:43 +08:00
zhenshan.cao b53699b846
Fixbug: distinguish the name of the subscription based on ProxyID (#6333)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-07-08 19:52:39 +08:00
XuanYang-cn f0a68704e2
Add unittests for getChannelName in DataNode (#6346)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-07-08 15:52:40 +08:00
FluorineDog d972a0361d
Weaken debug message (#6324)
Signed-off-by: fluorinedog <fluorinedog@gmail.com>
2021-07-08 15:50:57 +08:00
quicksilver 7d920e9c9c
Fix compile error on CentOS (#6359)
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
2021-07-08 15:43:45 +08:00
yukun 33162ca40f
Fix query error message (#6344)
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
2021-07-08 15:05:52 +08:00
Cai Yudong 3387b07dfd
Optimize code under storage (#6335)
* rename AddOneStringToPayload/GetOneStringFromPayload to AddStringToPayload/GetStringFromPayload

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* code optimize

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename print_binglog_test to print_binlog_test

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update chap08_binlog.md

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix unittest

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* use SetEventTimestamp() to replace SetStartTimestamp() and SetEndTimestamp()

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* code optimize

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename AddStringToPayload/GetStringFromPayload to AddOneStringToPayload/GetOneStringFromPayload

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-07-07 19:10:07 +08:00
sunby 017484e093
Refactor meta in data coord (#6300)
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-07-07 14:02:01 +08:00
shengjun.li a1ba9e9dcd
Fix comile on CentOS (#6334)
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2021-07-07 12:34:02 +08:00
dragondriver 166548447b
Fix 6249, use more clear error message (#6316)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-07-06 14:40:04 +08:00
FluorineDog 97b1426666
fix benchmark, add support for primary key in schemaHelper (#6281)
* fix benchmark

Signed-off-by: fluorinedog <fluorinedog@gmail.com>

* fix tests

Signed-off-by: fluorinedog <fluorinedog@gmail.com>
2021-07-06 09:50:01 +08:00
sunby 836a45ec26
Remove sending newSegmentMsg to rootcoord (#6301)
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-07-06 09:24:05 +08:00
Cai Yudong 21a9bcd844
Create default partition when receive CreateCollectionMsg (#6305)
* create default partition when create collection

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix unittest

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* adjust sleep time for unittest

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-07-06 09:16:03 +08:00
shengjun.li 1f42b4ec9d
fix the case that knng is not full (#6308)
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2021-07-05 17:26:07 +08:00
neza2017 ca4a562a72
simplify AddIndex, remove collID,PartID from parameters (#6291)
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
2021-07-05 10:08:02 +08:00
Cai Yudong bc8392eb1e
Add collectionID and partitionID into SegmentIndexInfo (#6289)
* add collectionID and partitionID into SegmentIndexInfo

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add partID2SegID into meta table

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update DropIndex

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-07-03 20:00:40 +08:00
neza2017 f8fa025dcc
start background go routine to check if all the segments have indexed (#6290)
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
2021-07-03 17:54:25 +08:00
neza2017 111a24a49f
remove msgstream of segment info (#6280)
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
2021-07-03 14:36:18 +08:00
dragondriver 8dd2051a9a
Fix 5045, decrease the nb and nlist in unittest (#6282)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-07-02 18:20:14 +08:00
congqixia 76fd9ad339
Fix restart node logic (#6265)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-07-02 16:48:30 +08:00
sunby 07b0989628
Add GetFlushedSegments in data coordinator (#6253)
Send segment flush message by grpc and add a new interface
`GetFlushedSegments`

Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-07-02 11:16:20 +08:00
xige-16 ddb027fb9e
clear proxy sarchStream when releaseCollection (#6258)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-07-02 10:40:13 +08:00
groot a364d7303b
Implement calculate vectors distance in server (#6219)
* calc distance stage 3

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* calc distance stage 5

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* calc distance stage 6

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* code style

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* typo

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* typo

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* typo

Signed-off-by: yhmo <yihua.mo@zilliz.com>
2021-07-01 18:56:17 +08:00
godchen 2a07124885
Change default grpc retry config (#6233)
* Change default grpc retry config

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* fix error

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* change timeout

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* fix error

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* add retry code

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* change time

Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-07-01 15:24:17 +08:00
neza2017 6019c193fc
add grpc interface to process NewSegment And FlushSegmentCompleted (#6247)
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
2021-07-01 14:58:17 +08:00
congqixia 580e3a57cf
stop add new node into cluster before validated (#6241)
* stop add new new into cluster

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

* Fix startup & test cases

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-07-01 14:32:16 +08:00
neza2017 a518e408b1
add log when seek in msgstream (#6244)
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
2021-07-01 13:46:16 +08:00
FluorineDog bf8b2be4a7
Deprecate num_groups to simplify search API (#6230)
Signed-off-by: fluorinedog <fluorinedog@gmail.com>
2021-07-01 10:32:15 +08:00
zhenshan.cao ff4c6c7ff4
Fix travel timestamp and guarantee timestamp (#6234)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-06-30 21:02:13 +08:00
Cai Yudong a7a9eeeffe
Add time record for build index and search process (#6231)
* add timerecord for build index process

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix build error

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add timerecord for query process

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-06-30 19:46:14 +08:00
yukun 95aa3c85b6
Add index_name check for drop_index (#6217)
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
2021-06-30 17:56:12 +08:00
xige-16 ff93d1611f
merge retrive and search code in query node (#6227)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-06-30 17:50:15 +08:00
xige-16 f146d3825f
Fix search hang after querynode restart (#6212)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-06-30 17:48:19 +08:00
congqixia 3119ce2a6c
Stop create session in watch op (#6222)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-06-30 16:50:20 +08:00
godchen 49a0528d53
Fix trace (#6218)
* Fix trace

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* fix go fmt

Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-06-30 16:18:13 +08:00
Cai Yudong 6436f596de
add GetIndexStates (#6213)
* add GetIndexStates

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename indexservice to index_coord

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-06-30 14:32:19 +08:00
neza2017 4e4ab4adc6
check if etcd is health before register session (#6215)
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
2021-06-30 14:12:15 +08:00
XuanYang-cn 357af89d55
Add unittest in segment_replica (#6197)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-06-30 10:26:12 +08:00
congqixia ae072b4f1d
Add candidate management for datacoord dn cluster (#6196)
* Add candidate management for datacoord dn cluster

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

* Fix test cases

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-06-30 10:20:15 +08:00
FluorineDog b90b4f2058
renaming (#6192)
Signed-off-by: fluorinedog <fluorinedog@gmail.com>
2021-06-30 10:18:26 +08:00
XuanYang-cn 769cbb292a
Update doc in datanode (#6194)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-06-29 17:34:13 +08:00
sunby dd3ba9b2b3
Add rev by one when start watch event (#6152)
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-06-29 14:52:14 +08:00
godchen 1c1ffc8b32
Add connect err log (#6186)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-06-29 14:30:17 +08:00
Cai Yudong 95135c1681
fix blob file sort error (#6184)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-06-29 13:24:15 +08:00
congqixia c49ce4ddc2
Add datacoord Server comment and improve para (#6182)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-06-29 10:46:13 +08:00
congqixia 3a202cad2b
Reduce connect timeout time (#6181)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-06-29 09:30:17 +08:00
yukun b87baa108a
Add topks in searchResult (#6168)
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
2021-06-28 15:28:11 +08:00
yukun 7df4bf8d93
Change strange expression err hint (#6128)
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
2021-06-28 15:18:12 +08:00
neza2017 95ad498822
check proxy if exist before connect to it (#6165)
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
2021-06-28 15:08:12 +08:00
dragondriver d3c503f3aa
Calculate the real topk in proxy (#6132)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-06-28 14:30:14 +08:00
congqixia b27e6b52bf
Force cluster refresh for each dn change event (#6161)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-06-28 13:28:14 +08:00
yukun ae66446f16
Fix issue-5992 (#6158)
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
2021-06-28 12:16:13 +08:00
Cai Yudong a9a806ace0
optimize WatchProxy (#6153)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-06-28 11:04:14 +08:00
yukun effcbc7cb3
Fix issue-6143 (#6145)
* Fix issue-6143

Signed-off-by: kun yu <kun.yu@zilliz.com>

* Return error when output_fields contains vector field

Signed-off-by: kun yu <kun.yu@zilliz.com>

* Fix when output_fields contain primarykey

Signed-off-by: kun yu <kun.yu@zilliz.com>

* Add primarykey

Signed-off-by: kun yu <kun.yu@zilliz.com>
2021-06-28 10:12:13 +08:00
xige-16 f973456c10
filter delEvent when querynode register fail (#6150)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-06-27 12:16:09 +08:00
xige-16 ca4cba152c
return loaded collection when ShowCollectionsType_InMemory (#6149)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-06-27 12:10:08 +08:00
xige-16 6036ef2c7d
Fix standalone can't start after down (#6148)
* mutil go routine do registerNode and  deleteNode

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* code format

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* stop retry create querynode client

Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-06-26 22:28:10 +08:00
neza2017 b22ab71222
fix rmq (#6147)
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
2021-06-26 22:26:20 +08:00
neza2017 6d7edc6015
fix typo (#6142)
* fix typo

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* add log

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
2021-06-26 18:04:12 +08:00
xige-16 6b65529086
Fix search error after releasePartition (#6135)
* fix search error after releasePartition

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* code format

Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-06-26 16:08:11 +08:00
neza2017 2e37fca214
add log in session (#6136)
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
2021-06-26 14:14:11 +08:00
zhenshan.cao 42be531f9c
Add InsertCnt (#6131)
* Add InsertCnt

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>

* Enhance log

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-06-26 14:02:12 +08:00
yukun e94a27856e
Mkdir rocksmq_path if not exist (#6134)
Signed-off-by: kun yu <kun.yu@zilliz.com>
2021-06-26 12:28:07 +08:00
xige-16 8db95a63f6
Return error when search limit equal 0 or grater than 16384 (#6115)
* return error when search limit equal 0 or grater than 16384

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* code format

Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-06-26 11:44:11 +08:00
neza2017 a1d0e5c980
remove task scheduler (#6123)
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
2021-06-26 09:22:11 +08:00
bigsheeper 0d0c1901dd
Fix service time check of search in streaming (#6068)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-06-25 20:18:09 +08:00
yukun 39614aa8eb
Add rocksmq_path in config and paramtable (#6099)
* Add rocksmq_path in config and paramtable

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Add rocksdbPath in NewRmsFactory

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Change rdb default path to /vat/lib/milvus/rdb_data

Signed-off-by: fishpenguin <kun.yu@zilliz.com>
2021-06-25 19:44:11 +08:00
congqixia 384f440dbf
Fix panic when BuildIndex 0 while err = nil (#6125)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-06-25 17:58:09 +08:00
yukun eb818d05ab
Remove vector fields from output fields (#6112)
* Remove vector fields from output fields

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Add empty_collection error_code

Signed-off-by: fishpenguin <kun.yu@zilliz.com>
2021-06-25 16:52:11 +08:00
neza2017 da12e55c6c
rootcoord connect to other components in go routines (#6117)
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
2021-06-25 16:48:10 +08:00
godchen 73996aba91
Change client init timeout (#6107)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-06-25 15:50:23 +08:00
Cai Yudong 64d1fa2de2
Banner use UTC time (#6102)
* banner use UTC time

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename master to rootcoord

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-06-25 10:24:11 +08:00
xige-16 cea83dde5a
fix multi loadCollectition not work (#6097)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-06-24 21:10:13 +08:00
yukun 9b549e0fa4
Add primaryKey to output fields (#6094)
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
2021-06-24 21:08:14 +08:00
Cai Yudong f633d48ccc
fix rootcoord unittest (#6090)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-06-24 19:44:08 +08:00
groot a807dca132
Fix #6082 Unstable issue for standalone mode in CI (#6084)
* Fix #6082 Unstable issue for standalone mode in CI

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* Fix #6082 Unstable issue for standalone mode in CI

Signed-off-by: yhmo <yihua.mo@zilliz.com>
2021-06-24 19:28:07 +08:00
bigsheeper e87372761a
Remove tSafe in release partition task and return max timestamp (#6078)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-06-24 19:06:34 +08:00
sunby 25d3fead9f
Change error type to error functions in data coord (#6076)
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-06-24 19:05:42 +08:00
godchen 3785bb6330
Change retry times (#6083)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-06-24 19:05:06 +08:00
congqixia 10e5fc3382
Fix query node crash when retrieved 0 result (#6087)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-06-24 19:04:34 +08:00
yukun 0f409da136
Fix standalone seek hang after restart (#6073)
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
2021-06-24 16:44:07 +08:00
xige-16 710e2ca185
return error before queryCoord ready (#6067)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-06-24 16:00:15 +08:00
dragondriver 133814be8e
Fix pq params check when creating index (#6052)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-06-24 15:40:10 +08:00
godchen 064f8f76cd
change_coord_connect_retry (#6064)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-06-24 15:20:08 +08:00
neza2017 bf068d84ad
fix index coord (#6042)
* fix index coord

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* remove memstream from RmsFactory

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
2021-06-24 14:36:08 +08:00
congqixia f6761bbc33
Refactor datacoord segment status (#6039)
* Refactor datacoord segment status

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

* Remove lastExpireTime

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-06-24 14:20:10 +08:00
yukun c8bf9078ae
Change pchannels to vchannel in retrieveResults (#6054)
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
2021-06-24 13:40:03 +08:00
yukun b72e4c6372
Merge search_collection and retrieve_colletion into query_collection (#6037)
* Merge search_collection and retrieve_colletion into query_collection

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix static-check

Signed-off-by: fishpenguin <kun.yu@zilliz.com>
2021-06-23 20:26:10 +08:00
bigsheeper dca24f313a
fix search timeout error (#6027)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-06-23 18:04:22 +08:00
bigsheeper 0f89f40488
Use virtual channel in search result and retrieve result (#6030)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-06-23 18:00:37 +08:00
xige-16 5417fee45f
fix releasePartition not work after loadCollection (#6025)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-06-23 17:44:12 +08:00
dragondriver 1f524bdf3e
Fix id allocator (#6024)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-06-23 17:34:05 +08:00
bigsheeper 47d89abaa0
get channels by static converting (#6015)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-06-23 17:26:09 +08:00
godchen 17303bf4d3
Change session doc (#6022)
* change session doc

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* fix error

Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-06-23 17:14:10 +08:00
sunby 46f86a0354
Return sealed segments when calling flush (#6011)
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-06-23 16:56:11 +08:00
Cai Yudong a09a3a1905
rename service to coord (#6020)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-06-23 16:14:08 +08:00
neza2017 9ebca59099
fix time tick (#6017)
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
2021-06-23 15:28:09 +08:00
congqixia d6259d2dd6
Refactor policy (#6003)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-06-23 14:28:08 +08:00
XuanYang-cn b640627a38
Fix datanode restart bug (#5976) (#6004)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-06-23 12:26:10 +08:00
sunby a529410b38
Fix data race in internal/datacoord/meta.go (#6005)
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-06-23 12:20:06 +08:00
sunby fe8432016d
Refactor data coordinator (#6008)
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-06-23 12:10:12 +08:00
dragondriver a493151f70
Fix #5951 (#6010)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-06-23 12:06:10 +08:00
godchen fd07a44dba
Fix error in retry refactor (#6000)
* Fix error in retry refactor

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* fix error

Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-06-23 11:48:06 +08:00
Cai Yudong ba5aacd3f2
Remove GetChanNameByVirtualChan (#5999)
* remove redundant ctx

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* remove GetChanNameByVirtualChan

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-06-23 11:18:05 +08:00
godchen 99be4c09a5
change retry (#5996)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-06-23 09:24:10 +08:00
Cai Yudong 7dbf33b788
Update component names in config files (#5989)
* rename master/indexService in .yaml

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename proxyNode to proxy in config files

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* move master.yaml to root_coord.yaml, move proxy_node.yaml to proxy.yaml

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* move proto/query_service.proto to proto/query_coord.proto

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* move proxy_node_manager.go to proxy_manager.go

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add query_coord.pb.go

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update ci

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-06-22 19:08:03 +08:00
yukun 792dba4ae3
Fix id field insert failed when auto_id is true (#5984)
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
2021-06-22 18:46:04 +08:00
FluorineDog b51893d3f1
Enrich retrieve task error message (#5975)
Signed-off-by: fluorinedog <fluorinedog@gmail.com>
2021-06-22 18:42:03 +08:00
dragondriver fbaa847641
Add metrics for proxy (#5987)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-06-22 18:32:08 +08:00
sunby 0978b93a21
Refactor data coordinator (#5982)
Rename variable name and make error msg more clear

Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-06-22 18:24:08 +08:00
zhenshan.cao 0f05622c09
Modify error msg when index is not created (#5978)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-06-22 18:04:07 +08:00
congqixia 7a2546c24a
Fix proxy panic if quit before init (#5981)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-06-22 17:40:07 +08:00
bigsheeper b31bae351d
fix empty search result (#5979)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-06-22 16:52:08 +08:00
xige-16 2af1c27811
Rename queryservice to querycoordinator (#5931)
* Rename queryservice to querycoordinator

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* fix param table

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* code format

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* fix unittest

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* service to coordinator

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* code format

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* set querycoord in ci

Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-06-22 16:44:09 +08:00
yukun 282d2624e0
Not remove rocksdb folder when close and init (#5974)
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
2021-06-22 16:20:10 +08:00
sunby d2c8462dac
Rename params in data_coord.yaml (#5968)
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-06-22 16:18:21 +08:00
Cai Yudong d5f4ee6f44
Rename proto master/index_service/proxy_service (#5973)
* rename proto proxy_service to proxy

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename proto index_service to index_coord

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename proto master to root_coord

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-06-22 16:14:09 +08:00
neza2017 80bb418136
Release partition (#5971)
* add debug info

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* notify query coord to release partitions after drop partition

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
2021-06-22 16:08:08 +08:00
dragondriver fd57554d32
Add index params check in proxy (#5958)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-06-22 15:28:04 +08:00
Cai Yudong 2926a78968
Rename proxynode to proxy (#5961)
* rename package name from proxynode to proxy

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* move distributed/proxynode to distributed/proxy

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* move internal/proxynode to internal/proxy

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename proxynode to proxy

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-06-22 14:40:07 +08:00
xige-16 7039fb7c82
Fix the problem of stuck after loadBalance and loadFieldData error (#5960)
* delete nodeInfo after nodeDown

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* fix load balance can's stop

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* fix load field data error

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* contiue task loop after error in queryService

Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-06-22 14:10:09 +08:00
neza2017 5cdc1af1af
fix proxy node (#5965)
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
2021-06-22 14:04:08 +08:00
sunby c863a193de
Rename filename from dataservice to datacoordinator (#5946)
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-06-22 10:42:07 +08:00
congqixia c8420cfa10
Add proxy healthy check for working services (#5954)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-06-22 10:36:08 +08:00
godchen 8643127e99
Change grpc retry (#5925)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-06-22 09:46:06 +08:00
zhenshan.cao cdaadca40c
FixBug: do not query IndexState for segments which are not enabled index (#5942)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-06-21 20:34:16 +08:00
yukun b2eb596c23
Change search result to columnBased and return target_entry (#5896)
* Change search result to columnBased and return target_entry

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

Co-authored-by: fluorinedog <fluorinedog@gmail.com>

* synchronize the latest pymilvus-orm version

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Change pymilvus-orm version to 40

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

Co-authored-by: fluorinedog <fluorinedog@gmail.com>
2021-06-21 20:18:13 +08:00
bigsheeper 3ccfd84195
update meta sync in query service (#5937)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-06-21 19:20:31 +08:00
bigsheeper 2e615907ca
fix test_search_after_index_different_metric_type (#5921)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-06-21 18:46:15 +08:00
sunby 7f2d7c1da3
Rename dataservice to datacoordinator (#5912)
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-06-21 18:22:13 +08:00
XuanYang-cn 688a587f4f
Add metrics in datanode (#5926)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-06-21 18:08:15 +08:00
sunby 0300b682f8
Remove deprecated interfaces in data coordinator (#5929)
* Remove derecated interfaces in data coordinator

Signed-off-by: sunby <bingyi.sun@zilliz.com>

* Remove RegisterNode in data coordinator

Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-06-21 17:40:05 +08:00
Cai Yudong 92e429d812
Rename IndexService to IndexCoord (#5932)
* rename package indexservice to indexcoord

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename indexservice to indexcoord

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix queryservice static-check

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* move distributed/indexservice to distributed/indexcoord

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* move internal/indexservice to internal/indexcoord

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename indexservice to indexcoord

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename MasterComponent to RootCoordComponent

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename master to rootcoord for queryservice

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename master to rootcoord for dataservice

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename master to rootcoord for datanode

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename master to rootcoord for proxynode

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename master to rootcoord for querynode

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename master to rootcoord

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename IndexService to IndexCoord

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename IndexService to IndexCoord

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix rebase issue

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-06-21 17:28:03 +08:00
XuanYang-cn 441300140d
Tidy replica (#5920)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-06-21 16:00:22 +08:00
xige-16 c310c70c00
Assign values to msgPack's startPosition in the form of deepCopy (#5900)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-06-21 14:16:03 +08:00
godchen fbcd754e52
Add new retry (#5911)
* Add new retry

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* fix gofmt

Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-06-21 13:52:05 +08:00
dragondriver 2642415bc3
Redefine the auto_id and primary_key (#5903)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-06-21 11:42:18 +08:00
sunby da1f9f9241
Rename dataservice to datacoordinator (#5904)
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-06-21 11:40:15 +08:00
zhenshan.cao 958280ebb8
Add support for time travel (#5897)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-06-21 10:42:10 +08:00
yukun d6e14c83a4
Fix retrieve result reduce (#5898)
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
2021-06-21 09:42:07 +08:00
bigsheeper bbd8a7e13a
Add released partitions and fix search error in empty partition (#5893)
* rename flowgraphType to loadType

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

* add load type and released partitions

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

* filter released partition

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-06-19 18:38:07 +08:00
dragondriver 82053fbbf7
Fix timesync bug of dealing with default timestmap (#5895)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-06-19 17:40:04 +08:00
FluorineDog 255e3959af
support time travel (#5894)
* support time travel

Signed-off-by: fluorinedog <fluorinedog@gmail.com>

* lint

Signed-off-by: fluorinedog <fluorinedog@gmail.com>
2021-06-19 17:38:11 +08:00
sunby 3cc0ab1345
Remove panic in NewDataSyncService (#5888)
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-06-19 15:18:06 +08:00
neza2017 143b54143c
get pchannel name from vchannel name (#5892)
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
2021-06-19 14:18:08 +08:00
XuanYang-cn ab7f642740
Add alias in paramtable (#5878)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-06-19 12:38:06 +08:00
xige-16 e4c51aae36
Add recovery logic for querynode and queryservice (#5843)
* merge milvus/recovery2

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* add recovery logic in queryservice

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* debug smoke case

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* add etcd to querynode

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* fix release partition error

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* fix load balance error

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* debug querynode down and recovery

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* add log

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* fix showCollection

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* skip smoke test search without insert

Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-06-19 11:45:09 +08:00
neza2017 26b8d5966f
add debug info (#5887)
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
2021-06-19 11:34:08 +08:00
Cai Yudong 5a1f7e4feb
rename masterservice to rootcoord (#5875)
* rename masterservice to rootcoord

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix build error

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename service to coord

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename package

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename masterservice to rootcoord under distributed

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update scripts

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* enable more unittests

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-06-18 21:30:08 +08:00
bigsheeper 710ac1a4f1
remove query node id env dependency and fix start script (#5874)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-06-18 19:00:15 +08:00
dragondriver ad23c47ff5
Check if collection was loaded before search (#5853)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-06-18 16:32:07 +08:00
congqixia 2f8bccda8d
Improve help message (#5870)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-06-18 16:16:06 +08:00
congqixia 7ad33c34b8
Fix lock not applied (#5856)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-06-18 16:14:16 +08:00
sunby 33f5225968
Use physical channel to init msg stream (#5815)
* Use physical channel to init msg stream

Signed-off-by: sunby <bingyi.sun@zilliz.com>

* fix search collection error check (#1)

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Co-authored-by: bigsheeper <954206947@qq.com>
2021-06-18 16:02:05 +08:00
Cai Yudong 3b0951e3e6
Combine standalone binary into milvus (#5866)
* rename service to coord for cmd

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update docs

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update variable name

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* optimize roles.go

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* support milvus run standalone

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* remove cmd/standalone

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix static-check

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename proxynode to proxy

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename service to coord for compoments

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add comments for codacy check

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* set helm chart branch

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-06-18 15:20:08 +08:00
dragondriver 2875b10dc5
Add default timestamp when send statistics of pchan (#5854)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-06-18 15:14:08 +08:00
dragondriver 676a30e418
Use channelsMgr to manage search message stream (#5819)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-06-18 10:33:58 +08:00
neza2017 b2908c4780
Release collection (#5839)
* relese collection

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* gen-proto

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* release collection

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* release collection

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
2021-06-17 17:45:56 +08:00
bigsheeper 46151d203f
Fix tSafe cannot update correctly (#5820)
* Fix tSafe cannot update correctly

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

* skip test_load_partitions_release_collection

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

* skip test_release_collection_during_searching

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-06-17 16:56:04 +08:00
Cai Yudong ae3daff5e4
Rename Master to RootCoord (#5830)
* rename master_service.go to root_coordinator.go

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename service to coordinator under cmd

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename service to coord under cmd

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename service to coord for metrics

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename service to coord for masterservice

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix metrics unittest

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* roll back cmd

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-06-17 16:47:57 +08:00
Xiangyu Wang c660ef6403
Change partition tag to partition name (#5834)
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-06-17 16:08:01 +08:00
neza2017 4e1932e0b7
add default time tick (#5828)
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
2021-06-17 15:54:07 +08:00
neza2017 abc0faeb49
add default time tick (#5827)
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
2021-06-17 14:57:57 +08:00
godchen b7bf26b486
Update grpc trace (#5817)
* Update grpc trace

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* go fmt

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* remove useless code in mod

Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-06-17 14:17:56 +08:00
yukun e9a8d1c404
Add vChannels in proxy for query results (#5802)
* Fix proxynode for new retrieve logic

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Remove querynodenum from proxynode and querynode

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Remove QueryNodeIDList from proxy

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

Co-authored-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-06-16 20:15:59 +08:00
XuanYang-cn 18eb27aa7a
Tidy replica in datanode (#5805)
- Remove not using functions
- Remove `initNode` logic in ParamTable
- Delete collection.go and collection_test.go

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-06-16 19:03:57 +08:00
congqixia 5ce06de3d1
Add context usage and fix defer issue (#5796)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-06-16 18:35:59 +08:00
Cai Yudong f28403b99b
clean proxy service (#5812)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-06-16 18:19:57 +08:00
neza2017 71ed21f9bf
Pass smoke test (#5797)
* pass smoke test

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
2021-06-16 16:59:50 +08:00
bigsheeper d6f7fbf19f
release collection synchronously (#5798)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-06-16 15:31:57 +08:00
bigsheeper bf39644457
Fix channels in query node and fix search (#5795)
* fix add channel error

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

* fix tSafe vChannel key

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-06-16 12:05:56 +08:00
godchen 1c6786f85c
Add blob info (#5792)
* Add blob info

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* fix error

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* fix error

Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-06-16 12:03:57 +08:00
xige-16 97049ce293
use dataservice's getRecoveryInfo to get load and watch info (#5790)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-06-16 11:09:56 +08:00
Cai Yudong 587ccc0557
Optimize dml_channels (#5783)
* update timetickSync::UpdateTimeTick

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update dml_channels.go

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix unittest

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* remove ProduceAll and BroadcastAll

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-06-15 21:11:58 +08:00
yukun 5c18138a6b
Fix retrieve_collection return nil vchannel (#5787)
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
2021-06-15 20:07:30 +08:00
bigsheeper 58bf92a407
Add vChannel,pChannel in query node and get vChannel from master in query service (#5784)
* add vChannel,pChannel in query node and get vChannel from master in query service

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-06-15 20:06:10 +08:00
sunby d997dd20af
Modify GetRecoveryInfo logic (#5785)
To satisfy QueryNode's request, GetRecoveryInfo interface should return
unflushed segments with start position. Because of using the same code
for getting seek position in QueryNode and DataNode before, we add a
flag to differentiate.

Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-06-15 19:23:55 +08:00
XuanYang-cn 75f2b3c2ba
Add Unittest for allocator (#5781)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-06-15 19:11:55 +08:00
godchen c2ecce61c0
Make pulsar client singleton (#5782)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-06-15 19:03:56 +08:00
neza2017 96cb24e566
fix session util (#5780)
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
2021-06-15 18:53:55 +08:00
sunby 324f5d1df7
Fix memory leak in indexservice (#5779)
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-06-15 18:25:55 +08:00
zhenshan.cao ebe5073749 Update proto
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-06-15 16:56:09 +08:00
sunby 9246c663fc Add start position in segment (#5753)
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-06-15 16:39:01 +08:00
cai.zhang 1d4ff7c8d7 Add waitgroup for starting server (#5705)
* Add waitgroup for starting server

Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>

* [skip ci] Just for skip ci

Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>
2021-06-15 16:39:01 +08:00
cai.zhang cf8953a347 Add log for indexservice (#5754)
Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>
2021-06-15 16:39:01 +08:00
yukun fceba63a33 Add segment and channel msgs in retrieve results (#5750)
* Fix query hang bug (#5709)

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Add segment and channel msgs in retrieve results

Signed-off-by: fishpenguin <kun.yu@zilliz.com>
2021-06-15 16:38:59 +08:00
zhenshan.cao 5c69c6902d Refactor proxy channelTimeTick and fix search timeout (#5760)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-06-15 16:36:36 +08:00
Cai Yudong 0f4bd50ce3 Support ETCD endpoints array (#5755) 2021-06-15 16:36:36 +08:00
XuanYang-cn 5bcb94294c Set start positions in datanode (#5757)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-06-15 16:36:36 +08:00
congqixia 7beab6eddc Fix new miniokv retry logic (#5749)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-06-15 16:36:36 +08:00
XuanYang-cn dea0a4e522 Idempotent of FlushSegment (#5746)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-06-15 16:36:36 +08:00
cai.zhang 81d3546b7c Remove indexnode in indexservice when indexnode crashed (#5719)
* Remove indexnode in indexservice when indexnode crashed

Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>
2021-06-15 16:36:36 +08:00
cai.zhang 561897c3d0 Load nodes when indexservices started (#5740)
Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>
2021-06-15 16:36:36 +08:00
Cai Yudong 17b19233dc add BroadcastMany, send DdMsg to collection channels only (#5734)
* add BroadcastMany, send DdMsg to collection channels only

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix master unittest

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix time tick of masterservice

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* fix time tick unittest of masterservice

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* skip dataservice ut

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix format issue

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update timeticksync

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update timeticksync

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update SendDdXXXReq interface

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update master unittest

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* roll back unecessary change

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* enable ut

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

Co-authored-by: yefu.chen <yefu.chen@zilliz.com>
2021-06-15 16:36:36 +08:00
sunby ba9952e780 Fix unit test in dataservice (#5745)
After a unit test, the msg stream will be closed. But in pulsar go
client, the connection between client and server will be reconnected.
Then we rebuild the msgstream immediately and produce a message, this
message will be sent to the previous connection which cause timeout of
test. We append the channelName wil a random value.

Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-06-15 16:36:36 +08:00
zhenshan.cao 89c0acd8b8 Fix bug: describe_collection not return fail reason (#5742)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-06-15 16:36:36 +08:00
zhenshan.cao f74ea3beee [skip ci] Remove ProxyService (#5738)
* [skip ci] Remove ProxyService

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>

* Change helm branch

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>

* Fix bug: paramstable crashed

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-06-15 16:36:35 +08:00
sunby 8a0335be44 Fix dd node filter bug (#5736)
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-06-15 16:24:49 +08:00
zhenshan.cao 33a43f48c8 [skip ci] Fix bug: Search timeout and refactor client init logic (#5735)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-06-15 16:24:48 +08:00
XuanYang-cn ab2fd34a2f fix double messages bug in datanode recovery (#5733)
* fix double messages bug in datanode recovery

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* remove debug log

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-06-15 16:14:29 +08:00
neza2017 ac9dde7352 fix seek on pulsar msgstream (#5726)
* fix seek on pulsar msgstream

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* test seek on pulsar msgstream

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* msg-seek test test

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* add pure pulsar client seek test

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
2021-06-15 16:14:29 +08:00
Cai Yudong fd07d544c3 [skip ci]add timerecord for debug (#5729)
* add timerecord for debug

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update comments

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-06-15 16:14:29 +08:00
zhenshan.cao 6ec2074da4 [skip ci] Fix bug: concurrent map read and map write (#5713)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-06-15 16:14:29 +08:00
xige-16 f2e551e9a0 Add asConsumer before seek (#5706)
* add asConsumer before seek

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* get dmChannel from master

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* use pChannel instead in queryservice

Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-06-15 16:14:27 +08:00
zhenshan.cao 0ddd13aeb3 [skip ci] Fix bug: tick timestamp always zero (#5711)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-06-15 16:06:13 +08:00
congqixia 7d9f299ff6 Change default unregister policy (#5710)
* Change default unassign policy

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

* Fix gofmt

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-06-15 16:06:13 +08:00
congqixia a3788a99c5 Fix unregister node info has no channel (#5699)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-06-15 16:06:13 +08:00
congqixia a3679d5540 Add vchannel buffer for cluster (#5691)
* Add channel buffer for cluster

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

* Change default register policy

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-06-15 16:06:13 +08:00
XuanYang-cn 46699c4c2b fix datanode dropcollection error (#5690)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-06-15 16:06:13 +08:00
neza2017 351d87055a fix ttmsgstream (#5689)
* fix msgstream

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* fix asconsumer

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
2021-06-15 16:06:13 +08:00
congqixia c172af78bd Add dataservice state check (#5687)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-06-15 16:06:13 +08:00
neza2017 1727edb5d3 fix proxy node (#5686)
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
2021-06-15 16:06:13 +08:00
sunby 189ac881f3 Fix bugs (#5676)
* Remove redundant session startup

Signed-off-by: sunby <bingyi.sun@zilliz.com>

* Register datanode after start success

Signed-off-by: sunby <bingyi.sun@zilliz.com>

* fix meta snap shot

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* fix datanode message stream channel

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* Fix bugs when drop empty collection

Signed-off-by: sunby <bingyi.sun@zilliz.com>

* Fix bug of getting pchan statistics from task scheduler

Signed-off-by: dragondriver <jiquan.long@zilliz.com>

* Fix i/dist/dataservice test code

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

* Fix epoch lifetime not applied

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

* fix datanode flowgraph dd node

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* Fix handle datanode timetick bug

Signed-off-by: sunby <bingyi.sun@zilliz.com>

* Remove repack function of dml stream

Signed-off-by: dragondriver <jiquan.long@zilliz.com>

* fix proxynode

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* Apply extended seal policy

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

* add check for time tick

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* fix check

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* Fix the repack function of dml stream

Signed-off-by: dragondriver <jiquan.long@zilliz.com>

* Fix the bug when send statistics of pchan

Signed-off-by: dragondriver <jiquan.long@zilliz.com>

* Fix the repack function when craete dml stream

Signed-off-by: dragondriver <jiquan.long@zilliz.com>

* fix bugs

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* fix describe collection

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* Fix bug when send timestamp statistics

Signed-off-by: dragondriver <jiquan.long@zilliz.com>

* fix data node

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* Add length check before flush request

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

* add log for data node

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* Fix SaveBinlog bugs

Signed-off-by: sunby <bingyi.sun@zilliz.com>

* Add more log in datanode

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* Put SegmentState.Flushing as the last one in enum to fit the client

Signed-off-by: sunby <bingyi.sun@zilliz.com>

* Fix params in GetInsertBinlogPaths

Signed-off-by: sunby <bingyi.sun@zilliz.com>

* Rename policy

Signed-off-by: sunby <bingyi.sun@zilliz.com>

* Remove unused ddl functions and fields

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

* Remove pchan when drop collection

Signed-off-by: dragondriver <jiquan.long@zilliz.com>

* Add balanced assignment policy

Signed-off-by: sunby <bingyi.sun@zilliz.com>

* fix master ut

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* Add lock in session manager

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

* add log for debug

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* Fix some logic bug and typo

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

* Fix recover bugs

Signed-off-by: sunby <bingyi.sun@zilliz.com>

* Get collection scheme of a specific timestamp

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* Change CheckPoint to SegmentInfo in VchannelInfo

Signed-off-by: sunby <bingyi.sun@zilliz.com>

* Recover Unflushed segment numOfRows

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* Fix dataservice unit tests

Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: yefu.chen <yefu.chen@zilliz.com>
Co-authored-by: yangxuan <xuan.yang@zilliz.com>
Co-authored-by: dragondriver <jiquan.long@zilliz.com>
Co-authored-by: Congqi Xia <congqi.xia@zilliz.com>
2021-06-15 16:06:11 +08:00
XuanYang-cn e7fd3aa926 Flowgraph recovery (#5640)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-06-15 16:04:48 +08:00
congqixia f794ceb0cc Add segment info rand logic in test case (#5639)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-06-15 16:04:48 +08:00
sunby e9b282c0be Use ElementsMatch to test slice equality (#5641)
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-06-15 16:04:48 +08:00
congqixia e57e2f77de Add flush monitor and unit test (#5622)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-06-15 16:04:48 +08:00
XuanYang-cn ac19711d74 Destroy DataNode when drop collections (#5638)
* Destroy DataNode when drop collections

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* golanci-lint

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

Co-authored-by: yefu.chen <yefu.chen@zilliz.com>
2021-06-15 16:04:48 +08:00
sunby f054fc9be2 Add GetRecoveryInfo in dataservice (#5629)
When loading a partition, QueryNode fetches the binlogs need to be
loaded and the channels needed to be watched by GetRecoveryInfo

Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-06-15 16:04:48 +08:00
neza2017 74c73fa75f fix datanode (#5635)
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
2021-06-15 16:04:48 +08:00
neza2017 7064937dc3 fix insertBuffer node (#5632)
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
2021-06-15 16:04:48 +08:00
neza2017 0c362a8831 fix bug and add unit test for insert buffer (#5630)
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
2021-06-15 16:04:48 +08:00
neza2017 3c70675313 data node, save binlog to minIO, and let data service save these meta (#5618)
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
2021-06-15 16:04:48 +08:00
godchen 91ef35bad4 Change receive msg logic (#5605)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-06-15 16:04:48 +08:00
neza2017 0a82c6381f add auto flush for data node (#5609)
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
2021-06-15 16:04:48 +08:00
sunby 3a5c8c4d3a Add seek position to WatchDmChannels response (#5601)
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-06-15 16:04:48 +08:00
Cai Yudong a948b71e06 Support to send dd msg to all dml channels (#5597)
* send dd msg to dml channel

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add dml_channels.go

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix race

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-06-15 16:04:48 +08:00
sunby e7521afed5 Change SaveBinlogPath (#5576)
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-06-15 16:04:48 +08:00
congqixia d0d845805b Change WatchDmChannelsRequest proto (#5577)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-06-15 16:04:48 +08:00
sunby aa8a038305 Rename SegmentAllocator to SegmentManager (#5559)
Add numRows to segmentStatus and Rename SegmentAllocator to
SegmentManager. Remove SegmentAllocStats.

Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-06-15 16:04:48 +08:00
godchen 275881dbdb Remove address param (#5556)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-06-15 16:04:46 +08:00
congqixia 41794ec3a6 Change SaveBinLogPath proto & func (#5551)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-06-15 15:59:04 +08:00
neza2017 cb6ae3678a fix master service ut (#5555)
* fix master unitttest

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* tt

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* fix master ut

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
2021-06-15 15:59:04 +08:00
XuanYang-cn e96d39bf6a refactor datanode flowgraph (#5538)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-06-15 15:59:04 +08:00
sunby f1ccbb8f9a Fix init session in dataservice (#5522)
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-06-15 15:59:04 +08:00
neza2017 5966076880 fix data service (#5535)
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
2021-06-15 15:59:04 +08:00
dragondriver 31b400a9f7 Refactor the workflow of receiving search result from query node (#5527)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-06-15 15:59:04 +08:00
godchen 54ab03e28f Fix queryservice connect error (#5526)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-06-15 15:59:04 +08:00
dragondriver fca3eb7126 Add implementation of getMinTsStatistics function (#5524)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-06-15 15:59:04 +08:00
congqixia 62eaa0390a Add dataservice metrics with grpc prometheus middleware (#5523)
* Add dataservice metrics with grpc prometheus middleware

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

* register server metrics

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-06-15 15:59:04 +08:00
sunby 38b1f7dabe Change the logic of getting vchannel position (#5502)
Datanode send `SaveBinlogPath` request after every segment flush finish
with the binlog paths and dml/ddl position. But the flush of segments is
not sorted. So we sort the segments according to segment id and find the
largest segment id with not nil dml position which is the position of
the msgstream to recover.

Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-06-15 15:59:04 +08:00
congqixia 31c4a4f5ed Simplify dd buffer & add lock (#5513)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-06-15 15:59:04 +08:00
dragondriver 6ea11f67cb Dock channelsTimeTicker, channelsMgr with master service (#5509)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-06-15 15:59:04 +08:00
Cai Yudong 2cc330b117 Remove proxy service from master (#5510)
* remove proxy service

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* remove time sync

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* remove proxy service

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* remove proxy service

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* remove proxy service

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* remove proxy service from distributed/masterservice

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix unittest

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix static-check

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

Co-authored-by: yefu.chen <yefu.chen@zilliz.com>
2021-06-15 15:59:04 +08:00
congqixia 68bf983f21 Add Collection ID in ddNode and filter logic (#5496)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-06-15 15:59:04 +08:00
dragondriver 1f3c9d5dc7 Add statistics function to task scheduler (#5500)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-06-15 15:59:04 +08:00
XuanYang-cn 6c02ae4eb1 Remove Old msgposition logic (#5487)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-06-15 15:59:04 +08:00
dragondriver c213dc468a Add dqlChannelsMgr (#5488)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-06-15 15:59:04 +08:00
sunby 9a54f1f72d Watch channel if needed when allocating segmnets (#5482)
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-06-15 15:59:04 +08:00
dragondriver 0b41031ce4 Add channelsTimeTicker to proxy (#5462)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-06-15 15:59:04 +08:00
congqixia a2dd16472d Implement random reassign and add test case (#5474)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-06-15 15:59:04 +08:00
congqixia 8657251f41 DataNode stores segment msgpack positions (#5472)
* DataNode stores segment msgpack positions

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

* Fix typo caused deadlock

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

* Use MsgPack EndPositions for flush pos

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-06-15 15:59:04 +08:00
sunby 43534ef63f Add unit tests in dataservice (#5478)
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-06-15 15:59:04 +08:00
XuanYang-cn 7a8b34ee9f Remove not using code in DataNode (#5476)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-06-15 15:59:04 +08:00
sunby a14c35274f Fix bugs in dataservice (#5457)
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-06-15 15:59:04 +08:00
XuanYang-cn 55eac3ee7f Complete Flush procedure in DataNode (#5454)
* Complete Flush proceduer in DataNode

Resolves: #5220

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* fix unittest

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-06-15 15:59:04 +08:00
dragondriver 59a3d83f97 Replace insertChannelsMap with channelsMgr (#5453)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-06-15 15:59:04 +08:00
congqixia 607a8ffa1f Update WatchDmChannels signature (#5447)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-06-15 15:59:04 +08:00
sunby cfdd98b70f Fix recovery branch after rebasing master (#5444)
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-06-15 15:59:04 +08:00
sunby 07c6a4a669 Add dataservice register and discovery (#5435)
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-06-15 15:59:04 +08:00
congqixia e3956ad13f Change SaveBinLogPaths field2Path to repeated (#8)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-06-15 15:59:04 +08:00
yangxuan ef563fda8f Updata flushe procedure
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-06-15 15:59:04 +08:00
sunby c53afee616 DataNode scales flowgraph
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-06-15 15:59:04 +08:00
sunby b9cf4c1095 Fix recovery (#5417)
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-06-15 15:59:04 +08:00
sunby 81ac20143b Add persistency of datanode cluster (#5387)
We save channels registered in datanode in etcd and restore the cluster info after
restarting.

Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-06-15 15:59:04 +08:00
sunby 3662b3f0e5 Refactor dataservice (#5400)
Pick up grpc calls to grpc_handler.go and refactor code format

Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-06-15 15:59:01 +08:00
congqixia ef7339ceda DataService stores flush msgstream pos at VChannel granularity (#5397)
SaveBinLogPaths saves stream positions at VChannel granularity

resolves #5396

Signed-off-by: Congqi Xia congqi.xia@zilliz.com
2021-06-15 15:54:57 +08:00
FluorineDog bec9f2c182
Split segcore and plan proto for future feature (#5767)
* Split segcore and plan proto for future feature

Signed-off-by: fluorinedog <fluorinedog@gmail.com>

* lint

Signed-off-by: fluorinedog <fluorinedog@gmail.com>
2021-06-15 14:43:57 +08:00
bigsheeper cdbc6d2c94
Refactor query node and query service (#5751)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Co-authored-by: xige-16 <xi.ge@zilliz.com>
Co-authored-by: yudong.cai <yudong.cai@zilliz.com>
2021-06-15 12:41:40 +08:00
FluorineDog 4165b761c9
fix not in bug (#5741)
Signed-off-by: fluorinedog <fluorinedog@gmail.com>
2021-06-11 17:08:25 +08:00
FluorineDog 006dae35c3
fix retrieve bug (#5727)
Signed-off-by: fluorinedog <fluorinedog@gmail.com>
2021-06-11 00:06:55 +08:00
yukun e97936ba1c
Fix query hang bug (#5709)
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
2021-06-10 14:33:49 +08:00
bigsheeper 8aae0f7cc9
Refactor flow graph and load/watchDML in query node (#5682)
* Refactor flow graph and load/watchDML in query node

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-06-09 11:37:55 +08:00
yukun 313658767d
Add field_name and type in retrieve result (#5665)
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
2021-06-08 16:26:36 +08:00
Xiangyu Wang 68f703f983
Add Query impl in proxynode (#5644)
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-06-07 17:21:36 +08:00
sunby 50b3d39370
Add GetRecoveryInfo interface and proto (#5598)
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-06-07 14:16:36 +08:00
zhenshan.cao bc46e4780d
Fixbug:index is not loaded in some cases (#5633)
* Fixbug:index is not created in some cases

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>

* Delete unecessary print

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>

* fix create index

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

Co-authored-by: yefu.chen <yefu.chen@zilliz.com>
2021-06-06 09:41:35 +08:00
shengjun.li 8ead67ef95
IVF_SQ8 and IVF_PQ cannot be built on multiple GPUs (#5624)
* IVF_SQ8 and IVF_PQ cannot be built on multiple GPUs

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>

* index created by std::make_shared

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2021-06-05 17:19:34 +08:00
zhenshan.cao 7dac20c35c
Add log for poxynode transform data and querynode segment insert (#5627)
* Add log for poxynode transform data and querynode segment insert

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>

* Decrease print times

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-06-05 16:21:36 +08:00
cai.zhang 5037646cbd
Check dsl whether is None (#5616)
* Check dsl whether is None

Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>

* Format code

Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>
2021-06-05 14:57:34 +08:00
yukun 45e7d26b01
Fix retrieve when output_fields is nil (#5623)
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
2021-06-05 10:57:34 +08:00
FluorineDog 1ff4764ba6
Fix wrong primary key (#5604)
Signed-off-by: fluorinedog <fluorinedog@gmail.com>
2021-06-04 18:35:36 +08:00
zhenshan.cao 03e3b4beb4
Add log to startup process (#5594)
* Add log to startup process

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>

* Fixbug: avoid ctx exceed deadline

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>

* Fmt code

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>

* Fix bug: wrong time unit

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>

* Fixbug: datanservice client reconnect use time ctx

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-06-04 16:29:35 +08:00
Cai Yudong 2ba93deb5e
Support master get segment info channel name from config file (#5606)
* optimize msgstream unittest

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* get data service segment info channel name from config file directly

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-06-04 16:28:34 +08:00
Xiangyu Wang afa5da7df0
Add query rpc into proto (#5592)
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-06-04 12:02:34 +08:00
FluorineDog 9a90313390
Support GetEntityByIDs in CGo, fix segcore bugs (#5563)
Signed-off-by: fluorinedog <fluorinedog@gmail.com>
2021-06-04 10:38:34 +08:00
Xiangyu Wang e80f06c030
Add retrieve process log (#5561)
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-06-03 20:56:33 +08:00
Xiangyu Wang f791ccb246 Update OWNERS file
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-06-03 20:01:24 +08:00
yukun 57ece4a771
Fix retrieve_collection bugs (#5575)
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
2021-06-03 19:41:33 +08:00
dragondriver 99699b2021
Support to show collections which are loaded into memory (#5567)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-06-03 19:09:33 +08:00
godchen 0610ceb3dc
Fix ut error (#5573)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-06-03 18:44:33 +08:00
dragondriver e76830ab09
Support column-based insert request in proxy (#5560)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-06-03 15:03:34 +08:00
zhenshan.cao b0b8f58192
Fix bug: search timeout (#5557)
* Fix bug: search timeout

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>

* Add log and fix unittest bug

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-06-03 14:58:34 +08:00
Cai Yudong 4018dfc5b7
Support data segment channel seek to the break point when system start (#5540)
* add DecodeDdOperation

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* reload MsgPosition from etcd and seek when system start

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update testcase

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add unittest

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename const string

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix unittest

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* optimize master unittest

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add msg into dataSegmentChannel to test seek

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix seek test

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* roll back distributed/masterservice/masterservice_test.go

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-06-02 22:36:41 +08:00
yukun dcb4161c9f
Fix retrieve_service (#5531)
* Fix retrieve_service

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix retrieve_collection

Signed-off-by: fishpenguin <kun.yu@zilliz.com>
2021-06-02 19:18:33 +08:00