41794ec3a6
Change SaveBinLogPath proto & func ( #5551 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-06-15 15:59:04 +08:00
cb6ae3678a
fix master service ut ( #5555 )
...
* fix master unitttest
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
* tt
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
* fix master ut
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
2021-06-15 15:59:04 +08:00
339a71aa0a
Add design for datanode flowgraph recovery ( #5562 )
...
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-06-15 15:59:04 +08:00
e96d39bf6a
refactor datanode flowgraph ( #5538 )
...
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-06-15 15:59:04 +08:00
f1ccbb8f9a
Fix init session in dataservice ( #5522 )
...
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-06-15 15:59:04 +08:00
5966076880
fix data service ( #5535 )
...
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
2021-06-15 15:59:04 +08:00
31b400a9f7
Refactor the workflow of receiving search result from query node ( #5527 )
...
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-06-15 15:59:04 +08:00
54ab03e28f
Fix queryservice connect error ( #5526 )
...
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-06-15 15:59:04 +08:00
fca3eb7126
Add implementation of getMinTsStatistics function ( #5524 )
...
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-06-15 15:59:04 +08:00
62eaa0390a
Add dataservice metrics with grpc prometheus middleware ( #5523 )
...
* Add dataservice metrics with grpc prometheus middleware
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
* register server metrics
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-06-15 15:59:04 +08:00
38b1f7dabe
Change the logic of getting vchannel position ( #5502 )
...
Datanode send `SaveBinlogPath` request after every segment flush finish
with the binlog paths and dml/ddl position. But the flush of segments is
not sorted. So we sort the segments according to segment id and find the
largest segment id with not nil dml position which is the position of
the msgstream to recover.
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-06-15 15:59:04 +08:00
31c4a4f5ed
Simplify dd buffer & add lock ( #5513 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-06-15 15:59:04 +08:00
6ea11f67cb
Dock channelsTimeTicker, channelsMgr with master service ( #5509 )
...
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-06-15 15:59:04 +08:00
2cc330b117
Remove proxy service from master ( #5510 )
...
* remove proxy service
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
* remove time sync
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
* remove proxy service
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
* remove proxy service
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
* remove proxy service
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
* remove proxy service from distributed/masterservice
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* fix unittest
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* fix static-check
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
Co-authored-by: yefu.chen <yefu.chen@zilliz.com>
2021-06-15 15:59:04 +08:00
68bf983f21
Add Collection ID in ddNode and filter logic ( #5496 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-06-15 15:59:04 +08:00
1f3c9d5dc7
Add statistics function to task scheduler ( #5500 )
...
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-06-15 15:59:04 +08:00
6c02ae4eb1
Remove Old msgposition logic ( #5487 )
...
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-06-15 15:59:04 +08:00
c213dc468a
Add dqlChannelsMgr ( #5488 )
...
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-06-15 15:59:04 +08:00
9a54f1f72d
Watch channel if needed when allocating segmnets ( #5482 )
...
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-06-15 15:59:04 +08:00
0b41031ce4
Add channelsTimeTicker to proxy ( #5462 )
...
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-06-15 15:59:04 +08:00
a2dd16472d
Implement random reassign and add test case ( #5474 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-06-15 15:59:04 +08:00
8657251f41
DataNode stores segment msgpack positions ( #5472 )
...
* DataNode stores segment msgpack positions
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
* Fix typo caused deadlock
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
* Use MsgPack EndPositions for flush pos
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-06-15 15:59:04 +08:00
43534ef63f
Add unit tests in dataservice ( #5478 )
...
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-06-15 15:59:04 +08:00
7a8b34ee9f
Remove not using code in DataNode ( #5476 )
...
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-06-15 15:59:04 +08:00
e7368c9447
Add OWNERS file
...
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-06-15 15:59:04 +08:00
a14c35274f
Fix bugs in dataservice ( #5457 )
...
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-06-15 15:59:04 +08:00
55eac3ee7f
Complete Flush procedure in DataNode ( #5454 )
...
* Complete Flush proceduer in DataNode
Resolves : #5220
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
* fix unittest
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-06-15 15:59:04 +08:00
59a3d83f97
Replace insertChannelsMap with channelsMgr ( #5453 )
...
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-06-15 15:59:04 +08:00
607a8ffa1f
Update WatchDmChannels signature ( #5447 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-06-15 15:59:04 +08:00
cfdd98b70f
Fix recovery branch after rebasing master ( #5444 )
...
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-06-15 15:59:04 +08:00
07c6a4a669
Add dataservice register and discovery ( #5435 )
...
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-06-15 15:59:04 +08:00
e3956ad13f
Change SaveBinLogPaths field2Path to repeated ( #8 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-06-15 15:59:04 +08:00
ef563fda8f
Updata flushe procedure
...
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-06-15 15:59:04 +08:00
c53afee616
DataNode scales flowgraph
...
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-06-15 15:59:04 +08:00
b9cf4c1095
Fix recovery ( #5417 )
...
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-06-15 15:59:04 +08:00
81ac20143b
Add persistency of datanode cluster ( #5387 )
...
We save channels registered in datanode in etcd and restore the cluster info after
restarting.
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-06-15 15:59:04 +08:00
3662b3f0e5
Refactor dataservice ( #5400 )
...
Pick up grpc calls to grpc_handler.go and refactor code format
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-06-15 15:59:01 +08:00
ef7339ceda
DataService stores flush msgstream pos at VChannel granularity ( #5397 )
...
SaveBinLogPaths saves stream positions at VChannel granularity
resolves #5396
Signed-off-by: Congqi Xia congqi.xia@zilliz.com
2021-06-15 15:54:57 +08:00
cb58a85a5f
bug fix and enable case ( #5765 )
...
Signed-off-by: zw <zw@milvus.io>
Co-authored-by: zw <zw@milvus.io>
2021-06-15 15:13:59 +08:00
a2787cad9a
Add test case for search with expressions. ( #5768 )
...
Signed-off-by: Binbin Lv <binbin.lv@zilliz.com>
2021-06-15 15:11:57 +08:00
bec9f2c182
Split segcore and plan proto for future feature ( #5767 )
...
* Split segcore and plan proto for future feature
Signed-off-by: fluorinedog <fluorinedog@gmail.com>
* lint
Signed-off-by: fluorinedog <fluorinedog@gmail.com>
2021-06-15 14:43:57 +08:00
cdbc6d2c94
Refactor query node and query service ( #5751 )
...
Signed-off-by: xige-16 <xi.ge@zilliz.com>
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
Co-authored-by: xige-16 <xi.ge@zilliz.com>
Co-authored-by: yudong.cai <yudong.cai@zilliz.com>
2021-06-15 12:41:40 +08:00
4165b761c9
fix not in bug ( #5741 )
...
Signed-off-by: fluorinedog <fluorinedog@gmail.com>
2021-06-11 17:08:25 +08:00
902307327b
[skip ci] Update partition tests for fixed issues. ( #5728 )
...
Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
2021-06-11 10:02:50 +08:00
006dae35c3
fix retrieve bug ( #5727 )
...
Signed-off-by: fluorinedog <fluorinedog@gmail.com>
2021-06-11 00:06:55 +08:00
faf5752096
Add test cases of connection interface ( #5732 )
...
* [skip ci] Add test cases of connection interface
Signed-off-by: wangting0128 <ting.wang@zilliz.com>
* [skip ci] Update test code
Signed-off-by: wangting0128 <ting.wang@zilliz.com>
* [skip ci] Update test codes
Signed-off-by: wangting0128 <ting.wang@zilliz.com>
2021-06-10 19:19:49 +08:00
9f4bb832b7
Add publish milvus images pipeline ( #5722 )
...
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
2021-06-10 17:33:49 +08:00
5b2f8d5552
[skip ci] Fix send email bug on jenkins pipeline failure ( #5721 )
...
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
2021-06-10 16:44:49 +08:00
f00b3bcbaa
[cronjob] Update periodically scheduled tasks to "50 22 * * *" ( #5715 )
...
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
2021-06-10 15:33:49 +08:00
e97936ba1c
Fix query hang bug ( #5709 )
...
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
2021-06-10 14:33:49 +08:00
congqixia
neza2017
XuanYang-cn
sunby
dragondriver
godchen
Cai Yudong
Xiangyu Wang
congqixia
del-zhenwu
binbin
FluorineDog
bigsheeper
yanliang567
紫晴
quicksilver
yukun