38f5b0826f
Add SaveBinlogPaths grpc in dataservice ( #5277 )
...
* Add SaveBinlogPaths grpc in dataservice
* fix merged compile error
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-05-20 11:34:45 +08:00
613cf21950
add cases for collection with data ( #5305 )
...
- add cases for collection with data
- cancel import *
See also: #5302 #5304
Signed-off-by: ThreadDao yufen.zong@zilliz.com
2021-05-20 03:03:45 +00:00
1c779f3efd
Refactor ddl binlog flush ( #5303 )
...
DDL won't auto-flush now
See also: #5289 , #5220
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-05-19 11:42:07 +00:00
a7bb701f73
Add Retrieve method grpc definition ( #5294 )
...
See also: #5253
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
2021-05-19 10:45:15 +00:00
e224d1e725
Add SessionManager ( #5288 )
...
Add SessionManager.
Resolves : #5174
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-05-19 10:36:05 +00:00
3db653f2d0
Update cpp thirdparty source URL ( #5295 )
...
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
Resolves : #5293
2021-05-19 09:39:29 +00:00
d5ab8ac345
Restore flush and stats stream pos ( #5284 )
...
Restore segment flush stream & statistic stream to last success pos
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-05-19 06:13:53 +00:00
4b712284f2
Add ddl flush design ( #5289 )
...
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-05-19 04:06:16 +00:00
9b37cab922
Add cases for creating collections ( #5286 )
...
- add cases for creating collections
- change unique str from var to function
- improve coding standards
See also: #5273 #5285
Signed-off-by: ThreadDao yufen.zong@zilliz.com
2021-05-19 03:34:52 +00:00
fc630bc1c8
Correct auto-flush behaviour ( #5282 )
...
Before this PR, DataNode considered auto-flush a valid flush
complete. It's wrong. So I open this PR to correct this behaviour
in DataNode.
Now binlog paths from auto-flush will be buffered in replica,
waiting until the manul flush to save into etcd all together.
See also: #5220 , #5268
A follow up job of #5271
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-05-18 11:45:00 +00:00
6dba0fd9cf
Add dataservice recover design docs ( #5212 )
...
This document includes changes about dataservice and new features
to support the recovery after it crashed.
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-05-18 11:20:26 +00:00
ccc0130bb4
Add exclusive parameter ( #5278 )
...
Add exclusive parameter.
Issue #5174
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-05-18 11:07:27 +00:00
d68ee3fbc2
query base on timeline ( #5279 )
...
query base on timeline, add timestamp on these queries:
- ShowCollections
- HasCollection
- DescribeCollection
Resolves : #5219
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
2021-05-18 09:12:17 +00:00
c3ac1375d8
Add buffer function for auto flush ( #5271 )
...
Auto-flush of a segment is not considered flush-completed.
So we need to buffer binlog paths generated by auto-flush.
See also: #5268 #5220
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-05-18 08:33:05 +00:00
49f6542b1d
query master service's with timestamp ( #5275 )
...
let meta support snapshot
so collection could query meta with timestamp
Resolves : #5219
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
2021-05-18 06:18:02 +00:00
c775744136
Add the design doc of retrieve entity ( #5274 )
...
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-05-18 03:44:04 +00:00
cca75583a6
Add session util ( #5269 )
...
Add session util.
Issue #5174
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-05-18 03:39:21 +00:00
ca03488c83
Update design about O1-1 ( #5268 )
...
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
The design procedure can be seen: #5220
2021-05-18 03:17:38 +00:00
e8ceb37f81
Add ShardsNum when create collection ( #5245 )
...
Updata PhysicalChannelNames and VirtualChannelNames based
on ShardsNum when create collection, and update collVChan2Chan
map.
Resolves : #5240
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-05-17 11:15:01 +00:00
5d0aab05b9
add collection cases level and change get_connection to create_collection ( #5249 )
...
- update get_connection to create_connection
- add case level for collection
- optimize collection cases
Signed-off-by: ThreadDao yufen.zong@zilliz.com
2021-05-17 09:37:17 +00:00
689f750c1a
Fix run pytest bug ( #5250 )
...
* Change volumes with pytest docker-compose
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
* Fix run pytest bug
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
2021-05-17 16:05:04 +08:00
16ad186874
Change volumes with pytest docker-compose ( #5247 )
...
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
2021-05-17 15:29:50 +08:00
1dabbfc835
Add collection cases ( #5239 )
...
Add a part of the test case for the Collection interface
See also: #5224 #5231
Signed-off-by: ThreadDao yufen.zong@zilliz.com
2021-05-17 02:14:35 +00:00
001795ee70
Optimize CreateIndex routine in master service ( #5238 )
...
Remove CreateIndexTask struct and indexTaskQueue
Resolves : #5228
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-05-15 10:08:08 +00:00
cd11c50932
[skip ci]Update mergify ( #5237 )
...
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-05-15 13:02:31 +08:00
b37861ad5f
Increase ephemeral storage size for jenkins slave ( #5235 )
...
Signed-off-by: jeffoverflow zhifeng.zhang@zilliz.com
2021-05-15 02:13:17 +00:00
a124568664
fix makefile ( #5236 )
...
just remove standalone and milvus from $GOPATH/bin,
not every thing, on command of make clean
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
2021-05-15 02:02:48 +00:00
d17ed98fff
Allow jenkins slave pod to remain active for a while ( #5232 )
...
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
2021-05-14 14:09:40 +00:00
08bb1b2ec3
Resend unsuccessful ddMsg when master start ( #5214 )
...
Resend unsuccessful ddMsg when master start
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-05-14 21:26:06 +08:00
9ffb42d448
Deny jenkins slave pod to remain active for a while ( #5225 )
...
* Deny jenkins slave pod to remain active for a while
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
2021-05-14 18:23:30 +08:00
0a21933d84
Specified Milvus Helm Chart path in E2E test ( #5199 )
...
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
2021-05-14 10:12:03 +00:00
168b917f5b
Add session util doc ( #5229 )
...
Add session util doc.
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-05-14 09:09:14 +00:00
9151e13480
Remove not using dependence and proto ( #5226 )
...
- FlushMsg proto message is no longer in use
- We don't depend on "github.com/prometheus/common/log"
Signed-off-by: yangxuan xuan.yang@zilliz.com
2021-05-14 08:50:55 +00:00
aa9ed7e98f
add datanode design doc ( #5206 )
...
* add datanode design doc
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-05-14 14:56:32 +08:00
b1e95ccf28
[skip-ci] master service recovery on power failure ( #5216 )
...
design doc about master service recover on power failure
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
2021-05-14 06:25:46 +00:00
4e6e502df1
Add data sorter in storage ( #5200 )
...
GetEntityByID needs the flushed segment to be sorted by RowID field,
then do binary search to get the target id and entities.
See also: #5177
Signed-off-by: fishpenguin kun.yu@zilliz.com
2021-05-14 02:59:49 +00:00
b74afd7a0e
Add service registration ( #5189 )
...
Add service registration.
Part of Issue #5174 .
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-05-14 02:05:18 +00:00
d35092cc09
Update check function ( #5205 )
...
Signed-off-by: wangting0128 <ting.wang@zilliz.com>
2021-05-14 01:27:41 +00:00
ec3d8c98a4
update test code ( #5197 )
...
update test code
2021-05-13 04:17:15 +00:00
8b7181c584
Update install_milvus.sh ( #5188 )
...
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
Delete milvus PVC does not affect E2E Test workflow
2021-05-13 01:47:26 +00:00
7b83b11e95
Save ddmsg info and type into ETCD ( #5173 )
...
Save ddmsg info and type into ETCD to support re-send ddmsg
when system restart or recover from failure (#5172 )
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-05-12 07:33:53 +00:00
d543e2c6cc
[skip ci]Update mergify ( #5180 )
...
Signed-off-by: scsven <wxyucs@gmail.com>
2021-05-12 10:36:00 +08:00
acc631d632
[skip ci] Change code owner for test folders ( #5178 )
...
Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
2021-05-12 09:42:26 +08:00
9e848b184e
Add publish-krte-images.yaml ( #5179 )
...
* Add publish-krte-images.yaml
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
* Update publish-krte-images.yaml
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
2021-05-11 19:28:29 +08:00
1c8717c2a3
add refactor test codes ( #5171 )
...
Signed-off-by: wangting0128 <ting.wang@zilliz.com>
2021-05-11 17:59:29 +08:00
2ae529a673
[skip ci]Update mergify rules ( #5170 )
...
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-05-11 10:50:17 +08:00
b9d3d965e4
[skip ci] Add new tests directories ( #5165 )
...
Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
2021-05-11 10:18:13 +08:00
64ab4d740b
Add row_count to partition statistics ( #5162 )
...
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-05-10 09:39:08 +00:00
a00422bac2
Remove unused interfaces in dataservice ( #5157 )
...
Remove unused interfaces in dataservice
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-05-10 09:03:24 +00:00
6f3a460b4e
Add pulsar seek example in pulsar_client_test.go ( #5154 )
...
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-05-10 02:32:10 +00:00
congqixia
ThreadDao
XuanYang-cn
yukun
godchen
quicksilver
sunby
neza2017
Xiangyu Wang
Cai Yudong
紫晴
yanliang567
dragondriver