e9a8d1c404
Add vChannels in proxy for query results ( #5802 )
...
* Fix proxynode for new retrieve logic
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
* Remove querynodenum from proxynode and querynode
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
* Remove QueryNodeIDList from proxy
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
Co-authored-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-06-16 20:15:59 +08:00
18eb27aa7a
Tidy replica in datanode ( #5805 )
...
- Remove not using functions
- Remove `initNode` logic in ParamTable
- Delete collection.go and collection_test.go
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-06-16 19:03:57 +08:00
5ce06de3d1
Add context usage and fix defer issue ( #5796 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-06-16 18:35:59 +08:00
f28403b99b
clean proxy service ( #5812 )
...
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-06-16 18:19:57 +08:00
bbcacc1880
Remove action-hadolint ( #5806 )
...
* Update action-hadolint version to 1.19.0
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
* Update action-hadolint version to 1.18.0
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
* Update action-hadolint version to 1.16.1
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
* Remove reviewdog/action-hadolint action
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
2021-06-16 17:51:29 +08:00
9ae3da41cc
[skip ci]Update Mergify ( #5808 )
...
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-06-16 17:04:55 +08:00
71ed21f9bf
Pass smoke test ( #5797 )
...
* pass smoke test
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
2021-06-16 16:59:50 +08:00
d6f7fbf19f
release collection synchronously ( #5798 )
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-06-16 15:31:57 +08:00
bf39644457
Fix channels in query node and fix search ( #5795 )
...
* fix add channel error
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
* fix tSafe vChannel key
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-06-16 12:05:56 +08:00
1c6786f85c
Add blob info ( #5792 )
...
* Add blob info
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
* fix error
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
* fix error
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-06-16 12:03:57 +08:00
97049ce293
use dataservice's getRecoveryInfo to get load and watch info ( #5790 )
...
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-06-16 11:09:56 +08:00
587ccc0557
Optimize dml_channels ( #5783 )
...
* update timetickSync::UpdateTimeTick
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* update dml_channels.go
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* fix unittest
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* remove ProduceAll and BroadcastAll
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-06-15 21:11:58 +08:00
5c18138a6b
Fix retrieve_collection return nil vchannel ( #5787 )
...
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
2021-06-15 20:07:30 +08:00
58bf92a407
Add vChannel,pChannel in query node and get vChannel from master in query service ( #5784 )
...
* add vChannel,pChannel in query node and get vChannel from master in query service
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-06-15 20:06:10 +08:00
d997dd20af
Modify GetRecoveryInfo logic ( #5785 )
...
To satisfy QueryNode's request, GetRecoveryInfo interface should return
unflushed segments with start position. Because of using the same code
for getting seek position in QueryNode and DataNode before, we add a
flag to differentiate.
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-06-15 19:23:55 +08:00
75f2b3c2ba
Add Unittest for allocator ( #5781 )
...
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-06-15 19:11:55 +08:00
c2ecce61c0
Make pulsar client singleton ( #5782 )
...
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-06-15 19:03:56 +08:00
96cb24e566
fix session util ( #5780 )
...
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
2021-06-15 18:53:55 +08:00
324f5d1df7
Fix memory leak in indexservice ( #5779 )
...
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-06-15 18:25:55 +08:00
ebe5073749
Update proto
...
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-06-15 16:56:09 +08:00
9246c663fc
Add start position in segment ( #5753 )
...
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-06-15 16:39:01 +08:00
1d4ff7c8d7
Add waitgroup for starting server ( #5705 )
...
* Add waitgroup for starting server
Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>
* [skip ci] Just for skip ci
Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>
2021-06-15 16:39:01 +08:00
cf8953a347
Add log for indexservice ( #5754 )
...
Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>
2021-06-15 16:39:01 +08:00
fceba63a33
Add segment and channel msgs in retrieve results ( #5750 )
...
* Fix query hang bug (#5709 )
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
* Add segment and channel msgs in retrieve results
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
2021-06-15 16:38:59 +08:00
5c69c6902d
Refactor proxy channelTimeTick and fix search timeout ( #5760 )
...
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-06-15 16:36:36 +08:00
0f4bd50ce3
Support ETCD endpoints array ( #5755 )
2021-06-15 16:36:36 +08:00
5bcb94294c
Set start positions in datanode ( #5757 )
...
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-06-15 16:36:36 +08:00
7beab6eddc
Fix new miniokv retry logic ( #5749 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-06-15 16:36:36 +08:00
dea0a4e522
Idempotent of FlushSegment ( #5746 )
...
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-06-15 16:36:36 +08:00
81d3546b7c
Remove indexnode in indexservice when indexnode crashed ( #5719 )
...
* Remove indexnode in indexservice when indexnode crashed
Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>
2021-06-15 16:36:36 +08:00
561897c3d0
Load nodes when indexservices started ( #5740 )
...
Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>
2021-06-15 16:36:36 +08:00
17b19233dc
add BroadcastMany, send DdMsg to collection channels only ( #5734 )
...
* add BroadcastMany, send DdMsg to collection channels only
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* fix master unittest
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* fix time tick of masterservice
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
* fix time tick unittest of masterservice
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* skip dataservice ut
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* fix format issue
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* update timeticksync
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* update timeticksync
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* update SendDdXXXReq interface
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* update master unittest
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* roll back unecessary change
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* enable ut
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
Co-authored-by: yefu.chen <yefu.chen@zilliz.com>
2021-06-15 16:36:36 +08:00
ba9952e780
Fix unit test in dataservice ( #5745 )
...
After a unit test, the msg stream will be closed. But in pulsar go
client, the connection between client and server will be reconnected.
Then we rebuild the msgstream immediately and produce a message, this
message will be sent to the previous connection which cause timeout of
test. We append the channelName wil a random value.
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-06-15 16:36:36 +08:00
89c0acd8b8
Fix bug: describe_collection not return fail reason ( #5742 )
...
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-06-15 16:36:36 +08:00
f74ea3beee
[skip ci] Remove ProxyService ( #5738 )
...
* [skip ci] Remove ProxyService
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
* Change helm branch
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
* Fix bug: paramstable crashed
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-06-15 16:36:35 +08:00
8a0335be44
Fix dd node filter bug ( #5736 )
...
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-06-15 16:24:49 +08:00
33a43f48c8
[skip ci] Fix bug: Search timeout and refactor client init logic ( #5735 )
...
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-06-15 16:24:48 +08:00
ab2fd34a2f
fix double messages bug in datanode recovery ( #5733 )
...
* fix double messages bug in datanode recovery
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
* remove debug log
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-06-15 16:14:29 +08:00
ac9dde7352
fix seek on pulsar msgstream ( #5726 )
...
* fix seek on pulsar msgstream
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
* test seek on pulsar msgstream
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
* msg-seek test test
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
* add pure pulsar client seek test
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
2021-06-15 16:14:29 +08:00
fd07d544c3
[skip ci]add timerecord for debug ( #5729 )
...
* add timerecord for debug
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* update comments
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-06-15 16:14:29 +08:00
6ec2074da4
[skip ci] Fix bug: concurrent map read and map write ( #5713 )
...
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-06-15 16:14:29 +08:00
f2e551e9a0
Add asConsumer before seek ( #5706 )
...
* add asConsumer before seek
Signed-off-by: xige-16 <xi.ge@zilliz.com>
* get dmChannel from master
Signed-off-by: xige-16 <xi.ge@zilliz.com>
* use pChannel instead in queryservice
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-06-15 16:14:27 +08:00
0ddd13aeb3
[skip ci] Fix bug: tick timestamp always zero ( #5711 )
...
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-06-15 16:06:13 +08:00
7d9f299ff6
Change default unregister policy ( #5710 )
...
* Change default unassign policy
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
* Fix gofmt
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-06-15 16:06:13 +08:00
a3788a99c5
Fix unregister node info has no channel ( #5699 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-06-15 16:06:13 +08:00
f1bb02f724
Stop indexnode when milvus process stops ( #5694 )
...
Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>
2021-06-15 16:06:13 +08:00
a3679d5540
Add vchannel buffer for cluster ( #5691 )
...
* Add channel buffer for cluster
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
* Change default register policy
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-06-15 16:06:13 +08:00
46699c4c2b
fix datanode dropcollection error ( #5690 )
...
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-06-15 16:06:13 +08:00
351d87055a
fix ttmsgstream ( #5689 )
...
* fix msgstream
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
* fix asconsumer
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
2021-06-15 16:06:13 +08:00
c172af78bd
Add dataservice state check ( #5687 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-06-15 16:06:13 +08:00
yukun
XuanYang-cn
congqixia
Cai Yudong
quicksilver
Xiangyu Wang
neza2017
bigsheeper
godchen
xige-16
sunby
zhenshan.cao
cai.zhang