milvus-io
/
milvus
mirror of https://github.com/milvus-io/milvus.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
7,861 Commits
133 Branches
115 Tags
559 MiB
Commit Graph

1521 Commits (f6621dc8471f4ad846ea3cef319fd228003d5bd3)

Author SHA1 Message Date
sunby ee38d14f5d
Fix channel lost after data node crash (#6545) 
If we start up 2 data nodes and one of them crashes. We expect that all
channels of crashed node will be allcoated to the alive node. But now we
discover that these channels are lost after data node crash. The reason
is we pass a NodeInfo with empty channel info. We fix it and improve log
print.

issue: #6501
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-07-15 16:38:31 +08:00
bigsheeper 7b9f0d398f
update flowgraph unittest (#6537) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-07-15 14:51:55 +08:00
groot 6b001fd76a
fix debug error in IDE (#6539) 
Signed-off-by: yhmo <yihua.mo@zilliz.com>
 2021-07-15 14:27:54 +08:00
XuanYang-cn dca6c6afde
Add unittest for BackGroundGC in DataNode (#6522) 
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2021-07-15 10:05:55 +08:00
dragondriver 5016e509be
Support wildcard match on search/query output fields (#6510) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-07-14 18:51:54 +08:00
neza2017 fac350cc58
fix tso (#6500) 
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
 2021-07-14 17:11:54 +08:00
dragondriver 7f99d9d714
Loose the limit of dimension when searching on a binary vector field (#6503) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-07-14 14:33:54 +08:00
cai.zhang f469a315d6
Refactor the logic of assign tasks in IndexCoord (#6328) 
* Refactor the logic of assign tasks in IndexCoord

Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>

* Fix bugs

Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>

* Fix bug for unittest

Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>

* Add lock for map

Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>

* Improve code

Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>

* Fix unittest bug

Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>

* Reduce duriation for timetick

Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>

* Update orm version

Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>

* Reset sdk version

Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>

* Fix bug

Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>

* Reset orm version

Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>

* Reset test ip

Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>

* Fix bug

Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>

* Fix bug for unissued

Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>

* Rename some variables

Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>

* Fix bug

Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>

* Use break instead of continue in select::case

Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>
 2021-07-14 14:15:55 +08:00
Cai Yudong 1c29b42434
change output_fields to out_fields_id for RetrieveRequest (#6506) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-07-14 13:11:54 +08:00
bigsheeper e50d39dfe1
Query result get global sealed segments from ETCD (#6377) 
* get global sealed segments from ETCD

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

* fix race error

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-07-14 11:15:54 +08:00
cai.zhang f3a5b46288
Fix bug for repeatedly create bucket in minio (#6477) 
* Fix bug for repeatedly create bucket in minio

Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>

* Simplify code

Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>

* Add log

Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>
 2021-07-14 09:55:46 +08:00
Cai Yudong 724f10b9a0
Unify the usage of query and search (#6467) 
Unify the usage of query and search

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-07-13 22:20:33 +08:00
neza2017 b66157358c
add more log for create collection (#6478) 
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
 2021-07-13 22:19:55 +08:00
dragondriver d896e3119e
Fix 6419, check if num_rows is greater than zero in proxy (#6439) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-07-13 16:33:55 +08:00
dragondriver 25b8c37383
Fix 6421, split raw vectors to serveral smaller one (#6449) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-07-13 16:05:36 +08:00
congqixia bc15b74244
Handle datasorter edge cases (#6465) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-07-13 14:37:54 +08:00
xaxys e5f8a77a2d
Extend grammar of boolean expression (#6460) 
* Extend grammar of boolean expression

Signed-off-by: xaxys <tpnnghd@163.com>
 2021-07-13 14:37:03 +08:00
xige-16 2b720fd2f7
Fix load hang when restart querynode many time in short time (#6351) 
* fix load hang when restart querynode many time in short time

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* add multi queryNode ut

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* add ut for restart querynode

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* set queryCoord contex to load collection

Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-07-13 14:16:00 +08:00
neza2017 d4239f0cd8
masterservice -> rootcoord (#6447) 
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
 2021-07-13 09:37:40 +08:00
sunby 83be910baa
Refactor segment manager (#6441) 
issue: #6440
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-07-12 17:24:25 +08:00
sunby a8e5fd2024
Refactor cluster in dataservice (#6356) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-07-12 11:03:52 +08:00
dragondriver bd317a5461
Check if field data is empty (#6404) 
* Check if field data is empty

Signed-off-by: dragondriver <jiquan.long@zilliz.com>

* FixBug: fast fail when output_fields contain fields that are not in schema

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>

Co-authored-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-07-11 19:39:52 +08:00
neza2017 8b89744c6b
search with exp (#6409) 
* process float vector and binary vector field
on the search result

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* add more debug info on etcd

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* add debug info when grpc failed

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
 2021-07-10 10:21:52 +08:00
FluorineDog 99ed122d11
Remove Dead Code, use signed type (#6398) 
* make type signed

Signed-off-by: fluorinedog <fluorinedog@gmail.com>

* remove dead code

Signed-off-by: fluorinedog <fluorinedog@gmail.com>

* remove code

Signed-off-by: fluorinedog <fluorinedog@gmail.com>
 2021-07-09 18:25:11 +08:00
FluorineDog 2ff072ea7b
Use larger dim for binary test to avoid false positive test failed (#6416) 
Signed-off-by: fluorinedog <fluorinedog@gmail.com>
 2021-07-09 18:12:40 +08:00
groot 19fc42c9eb
calc_distance typo (#6380) 
* calc_distance typo

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* calc_distance typo

Signed-off-by: yhmo <yihua.mo@zilliz.com>
 2021-07-09 14:53:52 +08:00
godchen 346e9cba4a
Add minio FGet method (#6386) 
* Add minio FGet method

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* go fmt

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* change el append

Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-07-09 11:44:40 +08:00
zhenshan.cao 6b4b71bf03
Fixbug: remove wrong check code (#6372) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-07-08 20:17:43 +08:00
zhenshan.cao b53699b846
Fixbug: distinguish the name of the subscription based on ProxyID (#6333) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-07-08 19:52:39 +08:00
XuanYang-cn f0a68704e2
Add unittests for getChannelName in DataNode (#6346) 
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2021-07-08 15:52:40 +08:00
FluorineDog d972a0361d
Weaken debug message (#6324) 
Signed-off-by: fluorinedog <fluorinedog@gmail.com>
 2021-07-08 15:50:57 +08:00
quicksilver 7d920e9c9c
Fix compile error on CentOS (#6359) 
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
 2021-07-08 15:43:45 +08:00
yukun 33162ca40f
Fix query error message (#6344) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-07-08 15:05:52 +08:00
Cai Yudong 3387b07dfd
Optimize code under storage (#6335) 
* rename AddOneStringToPayload/GetOneStringFromPayload to AddStringToPayload/GetStringFromPayload

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* code optimize

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename print_binglog_test to print_binlog_test

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update chap08_binlog.md

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix unittest

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* use SetEventTimestamp() to replace SetStartTimestamp() and SetEndTimestamp()

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* code optimize

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename AddStringToPayload/GetStringFromPayload to AddOneStringToPayload/GetOneStringFromPayload

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-07-07 19:10:07 +08:00
sunby 017484e093
Refactor meta in data coord (#6300) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-07-07 14:02:01 +08:00
shengjun.li a1ba9e9dcd
Fix comile on CentOS (#6334) 
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
 2021-07-07 12:34:02 +08:00
dragondriver 166548447b
Fix 6249, use more clear error message (#6316) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-07-06 14:40:04 +08:00
FluorineDog 97b1426666
fix benchmark, add support for primary key in schemaHelper (#6281) 
* fix benchmark

Signed-off-by: fluorinedog <fluorinedog@gmail.com>

* fix tests

Signed-off-by: fluorinedog <fluorinedog@gmail.com>
 2021-07-06 09:50:01 +08:00
sunby 836a45ec26
Remove sending newSegmentMsg to rootcoord (#6301) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-07-06 09:24:05 +08:00
Cai Yudong 21a9bcd844
Create default partition when receive CreateCollectionMsg (#6305) 
* create default partition when create collection

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix unittest

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* adjust sleep time for unittest

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-07-06 09:16:03 +08:00
shengjun.li 1f42b4ec9d
fix the case that knng is not full (#6308) 
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
 2021-07-05 17:26:07 +08:00
neza2017 ca4a562a72
simplify AddIndex, remove collID,PartID from parameters (#6291) 
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
 2021-07-05 10:08:02 +08:00
Cai Yudong bc8392eb1e
Add collectionID and partitionID into SegmentIndexInfo (#6289) 
* add collectionID and partitionID into SegmentIndexInfo

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add partID2SegID into meta table

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update DropIndex

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-07-03 20:00:40 +08:00
neza2017 f8fa025dcc
start background go routine to check if all the segments have indexed (#6290) 
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
 2021-07-03 17:54:25 +08:00
neza2017 111a24a49f
remove msgstream of segment info (#6280) 
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
 2021-07-03 14:36:18 +08:00
dragondriver 8dd2051a9a
Fix 5045, decrease the nb and nlist in unittest (#6282) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-07-02 18:20:14 +08:00
congqixia 76fd9ad339
Fix restart node logic (#6265) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-07-02 16:48:30 +08:00
sunby 07b0989628
Add GetFlushedSegments in data coordinator (#6253) 
Send segment flush message by grpc and add a new interface
`GetFlushedSegments`

Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-07-02 11:16:20 +08:00
xige-16 ddb027fb9e
clear proxy sarchStream when releaseCollection (#6258) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-07-02 10:40:13 +08:00
groot a364d7303b
Implement calculate vectors distance in server (#6219) 
* calc distance stage 3

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* calc distance stage 5

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* calc distance stage 6

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* code style

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* typo

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* typo

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* typo

Signed-off-by: yhmo <yihua.mo@zilliz.com>
 2021-07-01 18:56:17 +08:00
godchen 2a07124885
Change default grpc retry config (#6233) 
* Change default grpc retry config

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* fix error

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* change timeout

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* fix error

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* add retry code

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* change time

Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-07-01 15:24:17 +08:00
neza2017 6019c193fc
add grpc interface to process NewSegment And FlushSegmentCompleted (#6247) 
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
 2021-07-01 14:58:17 +08:00
congqixia 580e3a57cf
stop add new node into cluster before validated (#6241) 
* stop add new new into cluster

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

* Fix startup & test cases

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-07-01 14:32:16 +08:00
neza2017 a518e408b1
add log when seek in msgstream (#6244) 
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
 2021-07-01 13:46:16 +08:00
FluorineDog bf8b2be4a7
Deprecate num_groups to simplify search API (#6230) 
Signed-off-by: fluorinedog <fluorinedog@gmail.com>
 2021-07-01 10:32:15 +08:00
zhenshan.cao ff4c6c7ff4
Fix travel timestamp and guarantee timestamp (#6234) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-06-30 21:02:13 +08:00
Cai Yudong a7a9eeeffe
Add time record for build index and search process (#6231) 
* add timerecord for build index process

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix build error

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add timerecord for query process

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-06-30 19:46:14 +08:00
yukun 95aa3c85b6
Add index_name check for drop_index (#6217) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-06-30 17:56:12 +08:00
xige-16 ff93d1611f
merge retrive and search code in query node (#6227) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-06-30 17:50:15 +08:00
xige-16 f146d3825f
Fix search hang after querynode restart (#6212) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-06-30 17:48:19 +08:00
congqixia 3119ce2a6c
Stop create session in watch op (#6222) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-06-30 16:50:20 +08:00
godchen 49a0528d53
Fix trace (#6218) 
* Fix trace

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* fix go fmt

Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-06-30 16:18:13 +08:00
Cai Yudong 6436f596de
add GetIndexStates (#6213) 
* add GetIndexStates

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename indexservice to index_coord

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-06-30 14:32:19 +08:00
neza2017 4e4ab4adc6
check if etcd is health before register session (#6215) 
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
 2021-06-30 14:12:15 +08:00
XuanYang-cn 357af89d55
Add unittest in segment_replica (#6197) 
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2021-06-30 10:26:12 +08:00
congqixia ae072b4f1d
Add candidate management for datacoord dn cluster (#6196) 
* Add candidate management for datacoord dn cluster

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

* Fix test cases

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-06-30 10:20:15 +08:00
FluorineDog b90b4f2058
renaming (#6192) 
Signed-off-by: fluorinedog <fluorinedog@gmail.com>
 2021-06-30 10:18:26 +08:00
XuanYang-cn 769cbb292a
Update doc in datanode (#6194) 
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2021-06-29 17:34:13 +08:00
sunby dd3ba9b2b3
Add rev by one when start watch event (#6152) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-06-29 14:52:14 +08:00
godchen 1c1ffc8b32
Add connect err log (#6186) 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-06-29 14:30:17 +08:00
Cai Yudong 95135c1681
fix blob file sort error (#6184) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-06-29 13:24:15 +08:00
congqixia c49ce4ddc2
Add datacoord Server comment and improve para (#6182) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-06-29 10:46:13 +08:00
congqixia 3a202cad2b
Reduce connect timeout time (#6181) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-06-29 09:30:17 +08:00
yukun b87baa108a
Add topks in searchResult (#6168) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-06-28 15:28:11 +08:00
yukun 7df4bf8d93
Change strange expression err hint (#6128) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-06-28 15:18:12 +08:00
neza2017 95ad498822
check proxy if exist before connect to it (#6165) 
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
 2021-06-28 15:08:12 +08:00
dragondriver d3c503f3aa
Calculate the real topk in proxy (#6132) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-06-28 14:30:14 +08:00
congqixia b27e6b52bf
Force cluster refresh for each dn change event (#6161) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-06-28 13:28:14 +08:00
yukun ae66446f16
Fix issue-5992 (#6158) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-06-28 12:16:13 +08:00
Cai Yudong a9a806ace0
optimize WatchProxy (#6153) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-06-28 11:04:14 +08:00
yukun effcbc7cb3
Fix issue-6143 (#6145) 
* Fix issue-6143

Signed-off-by: kun yu <kun.yu@zilliz.com>

* Return error when output_fields contains vector field

Signed-off-by: kun yu <kun.yu@zilliz.com>

* Fix when output_fields contain primarykey

Signed-off-by: kun yu <kun.yu@zilliz.com>

* Add primarykey

Signed-off-by: kun yu <kun.yu@zilliz.com>
 2021-06-28 10:12:13 +08:00
xige-16 f973456c10
filter delEvent when querynode register fail (#6150) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-06-27 12:16:09 +08:00
xige-16 ca4cba152c
return loaded collection when ShowCollectionsType_InMemory (#6149) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-06-27 12:10:08 +08:00
xige-16 6036ef2c7d
Fix standalone can't start after down (#6148) 
* mutil go routine do registerNode and  deleteNode

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* code format

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* stop retry create querynode client

Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-06-26 22:28:10 +08:00
neza2017 b22ab71222
fix rmq (#6147) 
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
 2021-06-26 22:26:20 +08:00
neza2017 6d7edc6015
fix typo (#6142) 
* fix typo

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* add log

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
 2021-06-26 18:04:12 +08:00
xige-16 6b65529086
Fix search error after releasePartition (#6135) 
* fix search error after releasePartition

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* code format

Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-06-26 16:08:11 +08:00
neza2017 2e37fca214
add log in session (#6136) 
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
 2021-06-26 14:14:11 +08:00
zhenshan.cao 42be531f9c
Add InsertCnt (#6131) 
* Add InsertCnt

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>

* Enhance log

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-06-26 14:02:12 +08:00
yukun e94a27856e
Mkdir rocksmq_path if not exist (#6134) 
Signed-off-by: kun yu <kun.yu@zilliz.com>
 2021-06-26 12:28:07 +08:00
xige-16 8db95a63f6
Return error when search limit equal 0 or grater than 16384 (#6115) 
* return error when search limit equal 0 or grater than 16384

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* code format

Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-06-26 11:44:11 +08:00
neza2017 a1d0e5c980
remove task scheduler (#6123) 
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
 2021-06-26 09:22:11 +08:00
bigsheeper 0d0c1901dd
Fix service time check of search in streaming (#6068) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-06-25 20:18:09 +08:00
yukun 39614aa8eb
Add rocksmq_path in config and paramtable (#6099) 
* Add rocksmq_path in config and paramtable

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Add rocksdbPath in NewRmsFactory

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Change rdb default path to /vat/lib/milvus/rdb_data

Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-06-25 19:44:11 +08:00
congqixia 384f440dbf
Fix panic when BuildIndex 0 while err = nil (#6125) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-06-25 17:58:09 +08:00
yukun eb818d05ab
Remove vector fields from output fields (#6112) 
* Remove vector fields from output fields

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Add empty_collection error_code

Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-06-25 16:52:11 +08:00
neza2017 da12e55c6c
rootcoord connect to other components in go routines (#6117) 
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
 2021-06-25 16:48:10 +08:00
godchen 73996aba91
Change client init timeout (#6107) 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-06-25 15:50:23 +08:00
Cai Yudong 64d1fa2de2
Banner use UTC time (#6102) 
* banner use UTC time

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename master to rootcoord

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-06-25 10:24:11 +08:00
xige-16 cea83dde5a
fix multi loadCollectition not work (#6097) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-06-24 21:10:13 +08:00
yukun 9b549e0fa4
Add primaryKey to output fields (#6094) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-06-24 21:08:14 +08:00
Cai Yudong f633d48ccc
fix rootcoord unittest (#6090) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-06-24 19:44:08 +08:00
groot a807dca132
Fix #6082 Unstable issue for standalone mode in CI (#6084) 
* Fix #6082 Unstable issue for standalone mode in CI

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* Fix #6082 Unstable issue for standalone mode in CI

Signed-off-by: yhmo <yihua.mo@zilliz.com>
 2021-06-24 19:28:07 +08:00
bigsheeper e87372761a
Remove tSafe in release partition task and return max timestamp (#6078) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-06-24 19:06:34 +08:00
sunby 25d3fead9f
Change error type to error functions in data coord (#6076) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-06-24 19:05:42 +08:00
godchen 3785bb6330
Change retry times (#6083) 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-06-24 19:05:06 +08:00
congqixia 10e5fc3382
Fix query node crash when retrieved 0 result (#6087) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-06-24 19:04:34 +08:00
yukun 0f409da136
Fix standalone seek hang after restart (#6073) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-06-24 16:44:07 +08:00
xige-16 710e2ca185
return error before queryCoord ready (#6067) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-06-24 16:00:15 +08:00
dragondriver 133814be8e
Fix pq params check when creating index (#6052) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-06-24 15:40:10 +08:00
godchen 064f8f76cd
change_coord_connect_retry (#6064) 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-06-24 15:20:08 +08:00
neza2017 bf068d84ad
fix index coord (#6042) 
* fix index coord

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* remove memstream from RmsFactory

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
 2021-06-24 14:36:08 +08:00
congqixia f6761bbc33
Refactor datacoord segment status (#6039) 
* Refactor datacoord segment status

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

* Remove lastExpireTime

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-06-24 14:20:10 +08:00
yukun c8bf9078ae
Change pchannels to vchannel in retrieveResults (#6054) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-06-24 13:40:03 +08:00
yukun b72e4c6372
Merge search_collection and retrieve_colletion into query_collection (#6037) 
* Merge search_collection and retrieve_colletion into query_collection

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix static-check

Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-06-23 20:26:10 +08:00
bigsheeper dca24f313a
fix search timeout error (#6027) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-06-23 18:04:22 +08:00
bigsheeper 0f89f40488
Use virtual channel in search result and retrieve result (#6030) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-06-23 18:00:37 +08:00
xige-16 5417fee45f
fix releasePartition not work after loadCollection (#6025) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-06-23 17:44:12 +08:00
dragondriver 1f524bdf3e
Fix id allocator (#6024) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-06-23 17:34:05 +08:00
bigsheeper 47d89abaa0
get channels by static converting (#6015) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-06-23 17:26:09 +08:00
godchen 17303bf4d3
Change session doc (#6022) 
* change session doc

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* fix error

Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-06-23 17:14:10 +08:00
sunby 46f86a0354
Return sealed segments when calling flush (#6011) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-06-23 16:56:11 +08:00
Cai Yudong a09a3a1905
rename service to coord (#6020) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-06-23 16:14:08 +08:00
neza2017 9ebca59099
fix time tick (#6017) 
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
 2021-06-23 15:28:09 +08:00
congqixia d6259d2dd6
Refactor policy (#6003) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-06-23 14:28:08 +08:00
XuanYang-cn b640627a38
Fix datanode restart bug (#5976) (#6004) 
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2021-06-23 12:26:10 +08:00
sunby a529410b38
Fix data race in internal/datacoord/meta.go (#6005) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-06-23 12:20:06 +08:00
sunby fe8432016d
Refactor data coordinator (#6008) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-06-23 12:10:12 +08:00
dragondriver a493151f70
Fix #5951 (#6010) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-06-23 12:06:10 +08:00
godchen fd07a44dba
Fix error in retry refactor (#6000) 
* Fix error in retry refactor

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* fix error

Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-06-23 11:48:06 +08:00
Cai Yudong ba5aacd3f2
Remove GetChanNameByVirtualChan (#5999) 
* remove redundant ctx

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* remove GetChanNameByVirtualChan

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-06-23 11:18:05 +08:00
godchen 99be4c09a5
change retry (#5996) 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-06-23 09:24:10 +08:00
Cai Yudong 7dbf33b788
Update component names in config files (#5989) 
* rename master/indexService in .yaml

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename proxyNode to proxy in config files

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* move master.yaml to root_coord.yaml, move proxy_node.yaml to proxy.yaml

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* move proto/query_service.proto to proto/query_coord.proto

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* move proxy_node_manager.go to proxy_manager.go

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add query_coord.pb.go

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update ci

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-06-22 19:08:03 +08:00
yukun 792dba4ae3
Fix id field insert failed when auto_id is true (#5984) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-06-22 18:46:04 +08:00
FluorineDog b51893d3f1
Enrich retrieve task error message (#5975) 
Signed-off-by: fluorinedog <fluorinedog@gmail.com>
 2021-06-22 18:42:03 +08:00
dragondriver fbaa847641
Add metrics for proxy (#5987) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-06-22 18:32:08 +08:00
sunby 0978b93a21
Refactor data coordinator (#5982) 
Rename variable name and make error msg more clear

Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-06-22 18:24:08 +08:00
zhenshan.cao 0f05622c09
Modify error msg when index is not created (#5978) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-06-22 18:04:07 +08:00
congqixia 7a2546c24a
Fix proxy panic if quit before init (#5981) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-06-22 17:40:07 +08:00
bigsheeper b31bae351d
fix empty search result (#5979) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-06-22 16:52:08 +08:00
xige-16 2af1c27811
Rename queryservice to querycoordinator (#5931) 
* Rename queryservice to querycoordinator

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* fix param table

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* code format

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* fix unittest

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* service to coordinator

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* code format

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* set querycoord in ci

Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-06-22 16:44:09 +08:00
yukun 282d2624e0
Not remove rocksdb folder when close and init (#5974) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-06-22 16:20:10 +08:00
sunby d2c8462dac
Rename params in data_coord.yaml (#5968) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-06-22 16:18:21 +08:00
Cai Yudong d5f4ee6f44
Rename proto master/index_service/proxy_service (#5973) 
* rename proto proxy_service to proxy

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename proto index_service to index_coord

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename proto master to root_coord

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-06-22 16:14:09 +08:00
neza2017 80bb418136
Release partition (#5971) 
* add debug info

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* notify query coord to release partitions after drop partition

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
 2021-06-22 16:08:08 +08:00
dragondriver fd57554d32
Add index params check in proxy (#5958) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-06-22 15:28:04 +08:00
Cai Yudong 2926a78968
Rename proxynode to proxy (#5961) 
* rename package name from proxynode to proxy

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* move distributed/proxynode to distributed/proxy

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* move internal/proxynode to internal/proxy

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename proxynode to proxy

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-06-22 14:40:07 +08:00
xige-16 7039fb7c82
Fix the problem of stuck after loadBalance and loadFieldData error (#5960) 
* delete nodeInfo after nodeDown

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* fix load balance can's stop

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* fix load field data error

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* contiue task loop after error in queryService

Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-06-22 14:10:09 +08:00
neza2017 5cdc1af1af
fix proxy node (#5965) 
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
 2021-06-22 14:04:08 +08:00
sunby c863a193de
Rename filename from dataservice to datacoordinator (#5946) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-06-22 10:42:07 +08:00
congqixia c8420cfa10
Add proxy healthy check for working services (#5954) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-06-22 10:36:08 +08:00
godchen 8643127e99
Change grpc retry (#5925) 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-06-22 09:46:06 +08:00
zhenshan.cao cdaadca40c
FixBug: do not query IndexState for segments which are not enabled index (#5942) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-06-21 20:34:16 +08:00
yukun b2eb596c23
Change search result to columnBased and return target_entry (#5896) 
* Change search result to columnBased and return target_entry

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

Co-authored-by: fluorinedog <fluorinedog@gmail.com>

* synchronize the latest pymilvus-orm version

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Change pymilvus-orm version to 40

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

Co-authored-by: fluorinedog <fluorinedog@gmail.com>
 2021-06-21 20:18:13 +08:00
bigsheeper 3ccfd84195
update meta sync in query service (#5937) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-06-21 19:20:31 +08:00
bigsheeper 2e615907ca
fix test_search_after_index_different_metric_type (#5921) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-06-21 18:46:15 +08:00
sunby 7f2d7c1da3
Rename dataservice to datacoordinator (#5912) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-06-21 18:22:13 +08:00
XuanYang-cn 688a587f4f
Add metrics in datanode (#5926) 
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2021-06-21 18:08:15 +08:00
sunby 0300b682f8
Remove deprecated interfaces in data coordinator (#5929) 
* Remove derecated interfaces in data coordinator

Signed-off-by: sunby <bingyi.sun@zilliz.com>

* Remove RegisterNode in data coordinator

Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-06-21 17:40:05 +08:00
Cai Yudong 92e429d812
Rename IndexService to IndexCoord (#5932) 
* rename package indexservice to indexcoord

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename indexservice to indexcoord

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix queryservice static-check

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* move distributed/indexservice to distributed/indexcoord

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* move internal/indexservice to internal/indexcoord

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename indexservice to indexcoord

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename MasterComponent to RootCoordComponent

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename master to rootcoord for queryservice

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename master to rootcoord for dataservice

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename master to rootcoord for datanode

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename master to rootcoord for proxynode

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename master to rootcoord for querynode

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename master to rootcoord

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename IndexService to IndexCoord

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename IndexService to IndexCoord

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix rebase issue

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-06-21 17:28:03 +08:00
XuanYang-cn 441300140d
Tidy replica (#5920) 
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2021-06-21 16:00:22 +08:00
xige-16 c310c70c00
Assign values to msgPack's startPosition in the form of deepCopy (#5900) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-06-21 14:16:03 +08:00
godchen fbcd754e52
Add new retry (#5911) 
* Add new retry

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* fix gofmt

Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-06-21 13:52:05 +08:00
dragondriver 2642415bc3
Redefine the auto_id and primary_key (#5903) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-06-21 11:42:18 +08:00
sunby da1f9f9241
Rename dataservice to datacoordinator (#5904) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-06-21 11:40:15 +08:00
zhenshan.cao 958280ebb8
Add support for time travel (#5897) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-06-21 10:42:10 +08:00
yukun d6e14c83a4
Fix retrieve result reduce (#5898) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-06-21 09:42:07 +08:00
bigsheeper bbd8a7e13a
Add released partitions and fix search error in empty partition (#5893) 
* rename flowgraphType to loadType

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

* add load type and released partitions

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

* filter released partition

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-06-19 18:38:07 +08:00
dragondriver 82053fbbf7
Fix timesync bug of dealing with default timestmap (#5895) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-06-19 17:40:04 +08:00
FluorineDog 255e3959af
support time travel (#5894) 
* support time travel

Signed-off-by: fluorinedog <fluorinedog@gmail.com>

* lint

Signed-off-by: fluorinedog <fluorinedog@gmail.com>
 2021-06-19 17:38:11 +08:00
sunby 3cc0ab1345
Remove panic in NewDataSyncService (#5888) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-06-19 15:18:06 +08:00
neza2017 143b54143c
get pchannel name from vchannel name (#5892) 
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
 2021-06-19 14:18:08 +08:00
XuanYang-cn ab7f642740
Add alias in paramtable (#5878) 
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2021-06-19 12:38:06 +08:00
xige-16 e4c51aae36
Add recovery logic for querynode and queryservice (#5843) 
* merge milvus/recovery2

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* add recovery logic in queryservice

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* debug smoke case

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* add etcd to querynode

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* fix release partition error

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* fix load balance error

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* debug querynode down and recovery

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* add log

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* fix showCollection

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* skip smoke test search without insert

Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-06-19 11:45:09 +08:00
neza2017 26b8d5966f
add debug info (#5887) 
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
 2021-06-19 11:34:08 +08:00
Cai Yudong 5a1f7e4feb
rename masterservice to rootcoord (#5875) 
* rename masterservice to rootcoord

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix build error

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename service to coord

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename package

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename masterservice to rootcoord under distributed

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update scripts

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* enable more unittests

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-06-18 21:30:08 +08:00
bigsheeper 710ac1a4f1
remove query node id env dependency and fix start script (#5874) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-06-18 19:00:15 +08:00
dragondriver ad23c47ff5
Check if collection was loaded before search (#5853) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-06-18 16:32:07 +08:00
congqixia 2f8bccda8d
Improve help message (#5870) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-06-18 16:16:06 +08:00
congqixia 7ad33c34b8
Fix lock not applied (#5856) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-06-18 16:14:16 +08:00
sunby 33f5225968
Use physical channel to init msg stream (#5815) 
* Use physical channel to init msg stream

Signed-off-by: sunby <bingyi.sun@zilliz.com>

* fix search collection error check (#1)

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Co-authored-by: bigsheeper <954206947@qq.com>
 2021-06-18 16:02:05 +08:00
Cai Yudong 3b0951e3e6
Combine standalone binary into milvus (#5866) 
* rename service to coord for cmd

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update docs

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update variable name

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* optimize roles.go

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* support milvus run standalone

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* remove cmd/standalone

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix static-check

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename proxynode to proxy

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename service to coord for compoments

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add comments for codacy check

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* set helm chart branch

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-06-18 15:20:08 +08:00
dragondriver 2875b10dc5
Add default timestamp when send statistics of pchan (#5854) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-06-18 15:14:08 +08:00
dragondriver 676a30e418
Use channelsMgr to manage search message stream (#5819) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-06-18 10:33:58 +08:00
neza2017 b2908c4780
Release collection (#5839) 
* relese collection

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* gen-proto

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* release collection

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* release collection

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
 2021-06-17 17:45:56 +08:00
bigsheeper 46151d203f
Fix tSafe cannot update correctly (#5820) 
* Fix tSafe cannot update correctly

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

* skip test_load_partitions_release_collection

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

* skip test_release_collection_during_searching

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-06-17 16:56:04 +08:00
Cai Yudong ae3daff5e4
Rename Master to RootCoord (#5830) 
* rename master_service.go to root_coordinator.go

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename service to coordinator under cmd

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename service to coord under cmd

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename service to coord for metrics

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename service to coord for masterservice

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix metrics unittest

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* roll back cmd

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-06-17 16:47:57 +08:00
Xiangyu Wang c660ef6403
Change partition tag to partition name (#5834) 
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-06-17 16:08:01 +08:00
neza2017 4e1932e0b7
add default time tick (#5828) 
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
 2021-06-17 15:54:07 +08:00
neza2017 abc0faeb49
add default time tick (#5827) 
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
 2021-06-17 14:57:57 +08:00
godchen b7bf26b486
Update grpc trace (#5817) 
* Update grpc trace

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* go fmt

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* remove useless code in mod

Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-06-17 14:17:56 +08:00
yukun e9a8d1c404
Add vChannels in proxy for query results (#5802) 
* Fix proxynode for new retrieve logic

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Remove querynodenum from proxynode and querynode

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Remove QueryNodeIDList from proxy

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

Co-authored-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-06-16 20:15:59 +08:00
XuanYang-cn 18eb27aa7a
Tidy replica in datanode (#5805) 
- Remove not using functions
- Remove `initNode` logic in ParamTable
- Delete collection.go and collection_test.go

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2021-06-16 19:03:57 +08:00
congqixia 5ce06de3d1
Add context usage and fix defer issue (#5796) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-06-16 18:35:59 +08:00
Cai Yudong f28403b99b
clean proxy service (#5812) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-06-16 18:19:57 +08:00
neza2017 71ed21f9bf
Pass smoke test (#5797) 
* pass smoke test

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
 2021-06-16 16:59:50 +08:00
bigsheeper d6f7fbf19f
release collection synchronously (#5798) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-06-16 15:31:57 +08:00
bigsheeper bf39644457
Fix channels in query node and fix search (#5795) 
* fix add channel error

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

* fix tSafe vChannel key

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-06-16 12:05:56 +08:00
godchen 1c6786f85c
Add blob info (#5792) 
* Add blob info

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* fix error

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* fix error

Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-06-16 12:03:57 +08:00
xige-16 97049ce293
use dataservice's getRecoveryInfo to get load and watch info (#5790) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-06-16 11:09:56 +08:00
Cai Yudong 587ccc0557
Optimize dml_channels (#5783) 
* update timetickSync::UpdateTimeTick

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update dml_channels.go

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix unittest

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* remove ProduceAll and BroadcastAll

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-06-15 21:11:58 +08:00
yukun 5c18138a6b
Fix retrieve_collection return nil vchannel (#5787) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-06-15 20:07:30 +08:00
bigsheeper 58bf92a407
Add vChannel,pChannel in query node and get vChannel from master in query service (#5784) 
* add vChannel,pChannel in query node and get vChannel from master in query service

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-06-15 20:06:10 +08:00
sunby d997dd20af
Modify GetRecoveryInfo logic (#5785) 
To satisfy QueryNode's request, GetRecoveryInfo interface should return
unflushed segments with start position. Because of using the same code
for getting seek position in QueryNode and DataNode before, we add a
flag to differentiate.

Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-06-15 19:23:55 +08:00
XuanYang-cn 75f2b3c2ba
Add Unittest for allocator (#5781) 
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2021-06-15 19:11:55 +08:00
godchen c2ecce61c0
Make pulsar client singleton (#5782) 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-06-15 19:03:56 +08:00
neza2017 96cb24e566
fix session util (#5780) 
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
 2021-06-15 18:53:55 +08:00
sunby 324f5d1df7
Fix memory leak in indexservice (#5779) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-06-15 18:25:55 +08:00
zhenshan.cao ebe5073749 Update proto 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-06-15 16:56:09 +08:00
sunby 9246c663fc Add start position in segment (#5753) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-06-15 16:39:01 +08:00
cai.zhang 1d4ff7c8d7 Add waitgroup for starting server (#5705) 
* Add waitgroup for starting server

Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>

* [skip ci] Just for skip ci

Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>
 2021-06-15 16:39:01 +08:00
cai.zhang cf8953a347 Add log for indexservice (#5754) 
Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>
 2021-06-15 16:39:01 +08:00
yukun fceba63a33 Add segment and channel msgs in retrieve results (#5750) 
* Fix query hang bug (#5709)

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Add segment and channel msgs in retrieve results

Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-06-15 16:38:59 +08:00
zhenshan.cao 5c69c6902d Refactor proxy channelTimeTick and fix search timeout (#5760) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-06-15 16:36:36 +08:00
Cai Yudong 0f4bd50ce3 Support ETCD endpoints array (#5755) 2021-06-15 16:36:36 +08:00
XuanYang-cn 5bcb94294c Set start positions in datanode (#5757) 
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2021-06-15 16:36:36 +08:00
congqixia 7beab6eddc Fix new miniokv retry logic (#5749) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-06-15 16:36:36 +08:00
XuanYang-cn dea0a4e522 Idempotent of FlushSegment (#5746) 
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2021-06-15 16:36:36 +08:00
cai.zhang 81d3546b7c Remove indexnode in indexservice when indexnode crashed (#5719) 
* Remove indexnode in indexservice when indexnode crashed

Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>
 2021-06-15 16:36:36 +08:00
cai.zhang 561897c3d0 Load nodes when indexservices started (#5740) 
Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>
 2021-06-15 16:36:36 +08:00
Cai Yudong 17b19233dc add BroadcastMany, send DdMsg to collection channels only (#5734) 
* add BroadcastMany, send DdMsg to collection channels only

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix master unittest

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix time tick of masterservice

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* fix time tick unittest of masterservice

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* skip dataservice ut

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix format issue

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update timeticksync

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update timeticksync

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update SendDdXXXReq interface

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update master unittest

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* roll back unecessary change

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* enable ut

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

Co-authored-by: yefu.chen <yefu.chen@zilliz.com>
 2021-06-15 16:36:36 +08:00
sunby ba9952e780 Fix unit test in dataservice (#5745) 
After a unit test, the msg stream will be closed. But in pulsar go
client, the connection between client and server will be reconnected.
Then we rebuild the msgstream immediately and produce a message, this
message will be sent to the previous connection which cause timeout of
test. We append the channelName wil a random value.

Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-06-15 16:36:36 +08:00
zhenshan.cao 89c0acd8b8 Fix bug: describe_collection not return fail reason (#5742) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-06-15 16:36:36 +08:00
zhenshan.cao f74ea3beee [skip ci] Remove ProxyService (#5738) 
* [skip ci] Remove ProxyService

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>

* Change helm branch

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>

* Fix bug: paramstable crashed

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-06-15 16:36:35 +08:00
sunby 8a0335be44 Fix dd node filter bug (#5736) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-06-15 16:24:49 +08:00
zhenshan.cao 33a43f48c8 [skip ci] Fix bug: Search timeout and refactor client init logic (#5735) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-06-15 16:24:48 +08:00
XuanYang-cn ab2fd34a2f fix double messages bug in datanode recovery (#5733) 
* fix double messages bug in datanode recovery

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* remove debug log

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2021-06-15 16:14:29 +08:00
neza2017 ac9dde7352 fix seek on pulsar msgstream (#5726) 
* fix seek on pulsar msgstream

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* test seek on pulsar msgstream

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* msg-seek test test

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* add pure pulsar client seek test

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
 2021-06-15 16:14:29 +08:00
Cai Yudong fd07d544c3 [skip ci]add timerecord for debug (#5729) 
* add timerecord for debug

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update comments

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-06-15 16:14:29 +08:00
zhenshan.cao 6ec2074da4 [skip ci] Fix bug: concurrent map read and map write (#5713) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-06-15 16:14:29 +08:00
xige-16 f2e551e9a0 Add asConsumer before seek (#5706) 
* add asConsumer before seek

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* get dmChannel from master

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* use pChannel instead in queryservice

Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-06-15 16:14:27 +08:00
zhenshan.cao 0ddd13aeb3 [skip ci] Fix bug: tick timestamp always zero (#5711) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-06-15 16:06:13 +08:00
congqixia 7d9f299ff6 Change default unregister policy (#5710) 
* Change default unassign policy

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

* Fix gofmt

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-06-15 16:06:13 +08:00
congqixia a3788a99c5 Fix unregister node info has no channel (#5699) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-06-15 16:06:13 +08:00
congqixia a3679d5540 Add vchannel buffer for cluster (#5691) 
* Add channel buffer for cluster

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

* Change default register policy

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-06-15 16:06:13 +08:00
XuanYang-cn 46699c4c2b fix datanode dropcollection error (#5690) 
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2021-06-15 16:06:13 +08:00
neza2017 351d87055a fix ttmsgstream (#5689) 
* fix msgstream

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* fix asconsumer

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
 2021-06-15 16:06:13 +08:00
congqixia c172af78bd Add dataservice state check (#5687) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-06-15 16:06:13 +08:00
neza2017 1727edb5d3 fix proxy node (#5686) 
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
 2021-06-15 16:06:13 +08:00
sunby 189ac881f3 Fix bugs (#5676) 
* Remove redundant session startup

Signed-off-by: sunby <bingyi.sun@zilliz.com>

* Register datanode after start success

Signed-off-by: sunby <bingyi.sun@zilliz.com>

* fix meta snap shot

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* fix datanode message stream channel

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* Fix bugs when drop empty collection

Signed-off-by: sunby <bingyi.sun@zilliz.com>

* Fix bug of getting pchan statistics from task scheduler

Signed-off-by: dragondriver <jiquan.long@zilliz.com>

* Fix i/dist/dataservice test code

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

* Fix epoch lifetime not applied

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

* fix datanode flowgraph dd node

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* Fix handle datanode timetick bug

Signed-off-by: sunby <bingyi.sun@zilliz.com>

* Remove repack function of dml stream

Signed-off-by: dragondriver <jiquan.long@zilliz.com>

* fix proxynode

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* Apply extended seal policy

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

* add check for time tick

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* fix check

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* Fix the repack function of dml stream

Signed-off-by: dragondriver <jiquan.long@zilliz.com>

* Fix the bug when send statistics of pchan

Signed-off-by: dragondriver <jiquan.long@zilliz.com>

* Fix the repack function when craete dml stream

Signed-off-by: dragondriver <jiquan.long@zilliz.com>

* fix bugs

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* fix describe collection

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* Fix bug when send timestamp statistics

Signed-off-by: dragondriver <jiquan.long@zilliz.com>

* fix data node

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* Add length check before flush request

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

* add log for data node

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* Fix SaveBinlog bugs

Signed-off-by: sunby <bingyi.sun@zilliz.com>

* Add more log in datanode

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* Put SegmentState.Flushing as the last one in enum to fit the client

Signed-off-by: sunby <bingyi.sun@zilliz.com>

* Fix params in GetInsertBinlogPaths

Signed-off-by: sunby <bingyi.sun@zilliz.com>

* Rename policy

Signed-off-by: sunby <bingyi.sun@zilliz.com>

* Remove unused ddl functions and fields

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

* Remove pchan when drop collection

Signed-off-by: dragondriver <jiquan.long@zilliz.com>

* Add balanced assignment policy

Signed-off-by: sunby <bingyi.sun@zilliz.com>

* fix master ut

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* Add lock in session manager

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

* add log for debug

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* Fix some logic bug and typo

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

* Fix recover bugs

Signed-off-by: sunby <bingyi.sun@zilliz.com>

* Get collection scheme of a specific timestamp

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* Change CheckPoint to SegmentInfo in VchannelInfo

Signed-off-by: sunby <bingyi.sun@zilliz.com>

* Recover Unflushed segment numOfRows

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* Fix dataservice unit tests

Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: yefu.chen <yefu.chen@zilliz.com>
Co-authored-by: yangxuan <xuan.yang@zilliz.com>
Co-authored-by: dragondriver <jiquan.long@zilliz.com>
Co-authored-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-06-15 16:06:11 +08:00
XuanYang-cn e7fd3aa926 Flowgraph recovery (#5640) 
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2021-06-15 16:04:48 +08:00
congqixia f794ceb0cc Add segment info rand logic in test case (#5639) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-06-15 16:04:48 +08:00
sunby e9b282c0be Use ElementsMatch to test slice equality (#5641) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-06-15 16:04:48 +08:00
congqixia e57e2f77de Add flush monitor and unit test (#5622) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-06-15 16:04:48 +08:00
XuanYang-cn ac19711d74 Destroy DataNode when drop collections (#5638) 
* Destroy DataNode when drop collections

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* golanci-lint

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

Co-authored-by: yefu.chen <yefu.chen@zilliz.com>
 2021-06-15 16:04:48 +08:00
sunby f054fc9be2 Add GetRecoveryInfo in dataservice (#5629) 
When loading a partition, QueryNode fetches the binlogs need to be
loaded and the channels needed to be watched by GetRecoveryInfo

Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-06-15 16:04:48 +08:00
neza2017 74c73fa75f fix datanode (#5635) 
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
 2021-06-15 16:04:48 +08:00
neza2017 7064937dc3 fix insertBuffer node (#5632) 
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
 2021-06-15 16:04:48 +08:00
neza2017 0c362a8831 fix bug and add unit test for insert buffer (#5630) 
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
 2021-06-15 16:04:48 +08:00
neza2017 3c70675313 data node, save binlog to minIO, and let data service save these meta (#5618) 
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
 2021-06-15 16:04:48 +08:00