milvus-io
/
milvus
mirror of https://github.com/milvus-io/milvus.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
7,243 Commits
130 Branches
115 Tags
556 MiB
Commit Graph

253 Commits (282d2624e0d8856065765b8086363ef3c469bb2a)

Author SHA1 Message Date
godchen 200801271d
Fix GetSession error (#5401) 
Fix GetSession error

Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-05-25 12:15:46 +00:00
godchen 17cabfbed3
Add datanode client retry (#5394) 
Add datanode client retry.

Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-05-25 07:47:08 +00:00
neza2017 c28c34e852
let master use session.ServerID as nodeID (#5385) 
let master use session.ServerID as nodeID

also see #5386 

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
 2021-05-25 07:06:05 +00:00
Xiangyu Wang 88b42304ef
Add dummy RPC (#5376) 
Adding a dummy RPC makes it easier to develop new RPCs in the future.
When developing a new function, the SDK is often implemented last. 
At this time, this is helpful for debugging.

Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-05-25 06:44:43 +00:00
Cai Yudong dab536a6cf
Rename some field names in master for better code readability (#5389) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-05-25 06:03:06 +00:00
Cai Yudong 4058350e30
update AddIndex not return error when fail (#5372) 
Resolves: #5332 

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-05-25 03:42:23 +00:00
Cai Yudong 39f8c9dd56
Save msg start position and end position when AddSegment and AddIndex (#5358) 
Resolves: #5332 

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-05-24 06:19:52 +00:00
neza2017 e326eaad1e
master client,reconnect if grpc failed (#5360) 
master client,reconnect if grpc failed

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
 2021-05-24 01:44:49 +00:00
godchen 0eee7e5252
Change session (#5329) 
Change session.

Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-05-21 11:28:52 +00:00
Cai Yudong 761350e44d
Refactor timetick msg handling between proxy node and master (#5321) 
Proxy node send channel timetick msg to master, master reduce timetick
msg for each channel and send the min timetick to msg stream.

Resolves: #5276 

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-05-21 08:08:12 +00:00
neza2017 c0daf8e41d
set timestamp on dd requests (#5298) 
- set dd request's time tick on master
- send time tick into dd channel

See also: #5291

Signed-off-by: yefu.chen yefu.chen@zilliz.com
 2021-05-20 06:14:14 +00:00
congqixia 38f5b0826f
Add SaveBinlogPaths grpc in dataservice (#5277) 
* Add SaveBinlogPaths grpc in dataservice

* fix merged compile error

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-05-20 11:34:45 +08:00
yukun a7bb701f73
Add Retrieve method grpc definition (#5294) 
See also: #5253 

Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-05-19 10:45:15 +00:00
godchen e224d1e725
Add SessionManager (#5288) 
Add SessionManager.
Resolves: #5174 

Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-05-19 10:36:05 +00:00
neza2017 49f6542b1d
query master service's with timestamp (#5275) 
let meta support snapshot
so collection could query meta with timestamp

Resolves: #5219

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
 2021-05-18 06:18:02 +00:00
Cai Yudong 001795ee70
Optimize CreateIndex routine in master service (#5238) 
Remove CreateIndexTask struct and indexTaskQueue

Resolves: #5228 

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-05-15 10:08:08 +00:00
Cai Yudong 08bb1b2ec3
Resend unsuccessful ddMsg when master start (#5214) 
Resend unsuccessful ddMsg when master start

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-05-14 21:26:06 +08:00
Cai Yudong 5f0006d0f9
Improve code coverage for distributed/datanode (#5133) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-05-08 06:07:04 +00:00
Cai Yudong d66a5acca3
improve code coverage for distributed/dataservice (#5123) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
improve code coverage for distributed/dataservice
 2021-05-07 07:43:52 +00:00
xige-16 1e7205fa03
fix querynode's port not set correctly when conflict with other node (#5066) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-05-07 12:03:53 +08:00
XuanYang-cn cebdfb34f7
Enable multiple datanode (#5068) 
Now it's able to start multiple datanode, however, they will process the same insert data.
Further changes are needed to truly enable multiple datanode:

While registering, dataservice need to return different insert channels for different datanode.

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2021-04-28 18:43:48 +08:00
godchen 060f7ca0d2
Add get index build progress proto (#5049) 
Add get index build progress proto.

Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-04-27 07:46:45 +00:00
neza2017 262c484cec
the return struct of "DescribeIndex" should contain "filed_name" (#5055) 
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
 2021-04-27 02:30:55 +00:00
neza2017 d7c0a72611
index name is unique in collection scope (#5046) 
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
 2021-04-26 15:53:47 +08:00
neza2017 ada58b5bde
add unit test for master's grpc server, the coverage of package "internal/distributed/masterservice" increase from 46.6% to 82.9% (#5019) 
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
 2021-04-24 09:23:35 +00:00
Xiangyu Wang 82ccd4cec0
Rename module (#4988) 
* Rename module

Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-04-22 14:45:57 +08:00
sunby b79a408491 Remove unused config item in dataservice and datanode 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-04-19 19:28:45 +08:00
cai.zhang 9ccaff8005 Add copyright 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2021-04-19 19:28:11 +08:00
XuanYang-cn 6deba1faa5 Add license in datanode package 
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
 2021-04-19 15:16:33 +08:00
cai.zhang 8a037a404e Add Copyright for indexnode and indexservice 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2021-04-19 13:50:12 +08:00
bigsheeper 71b8f309b2 Add licence to query node and query service 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-04-19 13:47:10 +08:00
neza2017 83647e8503 Add copyright in master service module 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2021-04-19 11:12:56 +08:00
dragondriver dc7db73251 Add copyright annotations to proxynode and proxyservice 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-04-19 10:09:43 +08:00
sunby 751c9d4e81 Refactor meta.go 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-04-09 09:55:04 +08:00
Xiangyu Wang 87a1e0b662 Reorganize msgstream 
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-04-02 13:48:25 +08:00
zhenshan.cao efbf4452c5 Add log for port changed or proto unmarshal error 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-03-30 20:19:30 +08:00
Xiangyu Wang 72917467ec Change singlenode logs output 
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-03-30 10:52:42 +08:00
neza2017 71614313b0 Fix master service 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2021-03-29 09:50:52 +08:00
neza2017 eca1e25470 Add msg id at master service 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2021-03-26 15:13:33 +08:00
bigsheeper 1634d75980 Release flowgraph properly 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-03-22 12:49:50 -05:00
dragondriver 49c6eeb052 Add useful debug log in proxy 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-03-22 19:28:43 +08:00
zhenshan.cao c2734fa55f Fix bug and enchance system 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-03-22 16:36:10 +08:00
neza2017 6eb45c92eb Fix flush segment 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2021-03-13 17:05:36 +08:00
groot 47d9471498 Change gracefultime to 1 second 
Signed-off-by: groot <yihua.mo@zilliz.com>
 2021-03-13 14:42:53 +08:00
cai.zhang 012f5f360e Add init log logic for index and proxy 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2021-03-12 19:47:37 +08:00
godchen f3649f0419 Refactor interface and proto 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-03-12 14:22:09 +08:00
ThreadDao ca916a5c47 Add ci nightly 
Signed-off-by: ThreadDao <yufen.zong@zilliz.com>
 2021-03-11 14:14:29 +08:00
xige-16 09ae985daa Fix wrong error code in master_service_test.go 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-03-10 22:06:22 +08:00
sunby 800e31ebe4 Fix grpc startup in query node 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-03-10 15:49:23 +08:00
del-zhenwu b1763c3690 Disable xdist in test cases 
Signed-off-by: del-zhenwu <zhenxiang.li@zilliz.com>
 2021-03-10 15:27:26 +08:00
neza2017 7b2d67242d Fix describe segment if index not exist 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2021-03-10 14:45:35 +08:00
cai.zhang 60b97f9b65 Replace log of indexnode and indexservice 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2021-03-10 09:56:09 +08:00
quicksilver 455cc59dbb Update publish-test-images.yaml 
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
 2021-03-09 13:55:35 +08:00
neza2017 55fc3f56aa Set grpc max recv mag as math.MaxInt32 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2021-03-08 15:49:42 +08:00
neza2017 014c4fe8ce Add config of minSegmentSizeToEnableIndex 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2021-03-08 15:46:51 +08:00
sunby aeb3038545 Remove GetCount in data service 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-03-08 10:49:44 +08:00
ThreadDao 7a3223dc81 tags collection stats 
Signed-off-by: ThreadDao <yufen.zong@zilliz.com>
 2021-03-08 10:09:48 +08:00
neza2017 3ef18e86f0 Add 0331 tag for test_partition 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2021-03-06 17:47:11 +08:00
neza2017 2835bcdc4e Update masterservice unittest 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2021-03-05 20:41:34 +08:00
quicksilver 74154a11a4 Fix deploy error during regression stage 
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
 2021-03-05 16:52:45 +08:00
BossZou 1efcf605e2 Enlarge Proxy node max size of reveivable grpc message 
Signed-off-by: BossZou <yinghao.zou@zilliz.com>
 2021-03-05 11:16:23 +08:00
sunby ddddd65d10 Delete internal/errors package 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-03-05 10:15:27 +08:00
bigsheeper 447a15207e Add zap log to query node 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-03-05 09:21:35 +08:00
dragondriver 801396e3ef Disable register link logic, use proxy node directly 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-03-04 22:27:12 +08:00
sunby dec0b40a66 Add log panic in log module 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-03-04 16:01:30 +08:00
godchen 3ba0dc2f44 Change doc 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-03-04 10:35:28 +08:00
bigsheeper d3db2b0d82 Add zap log in query service 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-02-27 10:11:52 +08:00
godchen 7b9fdd7f29 Add opentracing 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-02-26 17:44:24 +08:00
xige-16 50a5f97421 Delete componentStateResponse in queryService.proto 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-02-26 15:26:21 +08:00
neza2017 4ea27335fd Ignore index param from create_collection 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2021-02-26 14:40:53 +08:00
XuanYang-cn 10374f15d7 Update log in datanode 
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
 2021-02-26 10:13:36 +08:00
Cai Yudong 84110d2684 Add tests/benchmark and tests/python_test using new python SDK 
Signed-off-by: Cai Yudong <yudong.cai@zilliz.com>
 2021-02-25 17:35:36 +08:00
cai.zhang 1d899513dd Unify service name in config file 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2021-02-24 18:46:19 +08:00
bigsheeper 08a020798e Release collection and partitions 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-02-24 17:24:51 +08:00
neza2017 e396c3f4ad Add zap log 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2021-02-24 16:25:40 +08:00
XuanYang-cn fd562f9f9c Update doc: add proxy graph 
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
 2021-02-24 09:48:17 +08:00
zhenshan.cao 7c5e2d89f6 Read node port from config 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-02-23 18:08:17 +08:00
xige-16 d5379a82a8 Fix load binlog error when segment not flushed 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-02-23 14:13:33 +08:00
dragondriver b506754364 Add workflow of drop index in index service 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-02-23 11:57:18 +08:00
zhenshan.cao bbfcbbdd68 Fix ci: paramstable and startup logic refactor 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-02-23 11:40:30 +08:00
sunby ea1d9ea99a Replace log in dataservice 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-02-23 09:58:06 +08:00
BossZou 7ac1821cf1 Add RocksMQ throughout test 
Signed-off-by: BossZou <yinghao.zou@zilliz.com>
 2021-02-20 18:30:37 +08:00
yukun 346b0a9549 Add unittest for RmqTtMsgStream 
Signed-off-by: yukun <kun.yu@zilliz.com>
 2021-02-20 15:58:19 +08:00
neza2017 1d3fdebcf4 Add drop index for proxy 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2021-02-20 15:38:44 +08:00
cai.zhang 4c5a898e9a Add masterservice docker 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2021-02-20 11:15:19 +08:00
dragondriver 6ebb03791f Fix drop index interface 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-02-20 10:37:11 +08:00
zhenshan.cao b3891ca199 Correct docker file 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-02-20 10:15:37 +08:00
xige-16 450ea631bc Fix queryService assign queryChannel failure 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-02-18 16:26:02 +08:00
neza2017 85c1404393 Refactor master:index ids 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2021-02-11 08:41:59 +08:00
xige-16 70fe5233cf Watch queryChannel on query node 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-02-09 17:09:26 +08:00
neza2017 e2d8358cb4 Refactor master:index ids 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2021-02-09 13:11:55 +08:00
neza2017 4035083188 Support Index name 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2021-02-08 14:49:12 +08:00
groot dc3736281a Integrate message stream 
Signed-off-by: groot <yihua.mo@zilliz.com>
 2021-02-08 14:30:54 +08:00
sunby eba5b9ae27 Fix queryservice client init bug 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-02-07 13:53:40 +08:00
groot b034740c6c Make paramstable initialize once by using do_once 
Signed-off-by: groot <yihua.mo@zilliz.com>
 2021-02-06 13:39:15 +08:00
sunby 0111cba8ae Fix startup bugs in some modules 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-02-06 10:28:53 +08:00
XuanYang-cn dd72dfcbd1 Fix queryservice start 
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
 2021-02-05 17:57:41 +08:00
dragondriver e6aec3fd08 Hardcode the peer id of proxy node 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-02-05 14:09:55 +08:00
godchen 0f620ab149 Fix invalidate metacache error 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-02-05 11:49:13 +08:00
dragondriver e384222803 Fix the logic of node id allocator 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-02-05 11:25:52 +08:00