milvus-io
/
milvus
mirror of https://github.com/milvus-io/milvus.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
7,283 Commits
140 Branches
117 Tags
589 MiB
Commit Graph

247 Commits (24038be146dffc692ca162747af6473f76e71076)

Author SHA1 Message Date
XuanYang-cn cebdfb34f7
Enable multiple datanode (#5068) 
Now it's able to start multiple datanode, however, they will process the same insert data.
Further changes are needed to truly enable multiple datanode:

While registering, dataservice need to return different insert channels for different datanode.

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2021-04-28 18:43:48 +08:00
Xiangyu Wang 82ccd4cec0
Rename module (#4988) 
* Rename module

Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-04-22 14:45:57 +08:00
sunby fad94fb576
Remove unsed configs (#4990) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-04-22 03:04:59 +00:00
Xiangyu Wang c731e2df78 Add license header in util 
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-04-19 13:42:47 +08:00
XuanYang-cn e281f6a4a1 Remove useless configs in config files 
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
 2021-04-13 10:04:39 +08:00
godchen deba964590 Change name of kvBase 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-04-12 18:09:28 +08:00
Cai Yudong 687d48b75e Improve code readibility 
Signed-off-by: Cai Yudong <yudong.cai@zilliz.com>
 2021-04-12 14:11:06 +08:00
yukun 964e7e0a28 Add unittest for mqclient 
Signed-off-by: yukun <kun.yu@zilliz.com>
 2021-04-08 20:05:33 +08:00
dragondriver decc80a525 Add unittest to proxy service 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-04-06 14:12:57 +08:00
Xiangyu Wang 87a1e0b662 Reorganize msgstream 
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-04-02 13:48:25 +08:00
Xiangyu Wang 8b62790017 Reorganize mqclient 
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-04-02 10:01:11 +08:00
dragondriver 6796c7bfe7 Add unittest to function util 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-03-31 18:22:17 +08:00
yukun 18e0581d5c Destroy readOptions after use 
Signed-off-by: yukun <kun.yu@zilliz.com>
 2021-03-31 14:10:07 +08:00
yukun 604f96ed2b Replace consumers map by sync.map 
Signed-off-by: yukun <kun.yu@zilliz.com>
 2021-03-31 10:05:24 +08:00
sunby f8e879b423 Fix high cpu usage in singlenode 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-03-29 17:48:15 +08:00
sunby ff8ebd5f42 Fix bug of estimating record size 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-03-29 15:14:51 +08:00
yukun c4f9a6f22b Fix get position wrong 
Signed-off-by: yukun <kun.yu@zilliz.com>
 2021-03-27 17:39:12 +08:00
xige-16 a48d46cfaf Alter msgID’s type to bytes 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-03-27 09:46:54 +08:00
yukun 432b53149b Refactor msgstream 
Signed-off-by: yukun <kun.yu@zilliz.com>
 2021-03-26 20:10:11 +08:00
xige-16 7e60460aee close timeTickStream in flow graph when the collection is released 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-03-26 18:40:04 +08:00
neza2017 eca1e25470 Add msg id at master service 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2021-03-26 15:13:33 +08:00
dragondriver 41e1975611 Parallelize the processing of loading binlog and saving index files 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-03-26 11:19:02 +08:00
yukun 85bc2a54f0 Fix load twice hang bug 
Signed-off-by: yukun <kun.yu@zilliz.com>
 2021-03-25 18:49:41 +08:00
quicksilver 7ad9b36207 Update reviewdog/action-hadolint github action version to v1.16.1 
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
 2021-03-25 14:41:46 +08:00
bigsheeper 1634d75980 Release flowgraph properly 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-03-22 12:49:50 -05:00
紫晴 3be6dde459 Add smoke tag 
Signed-off-by: 紫晴 <ting.wang@zilliz.com>
 2021-03-23 14:35:30 +08:00
zhenshan.cao c2734fa55f Fix bug and enchance system 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-03-22 16:36:10 +08:00
Xiangyu Wang 9ba9bf5e8a Release memory after rocksdb.write() 
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-03-19 20:20:32 +08:00
yukun 0823382876 Refactor rmq_msgstream 
Signed-off-by: yukun <kun.yu@zilliz.com>
 2021-03-19 19:33:21 +08:00
yukun 207f6ae773 Fix rocksmq CreateConsumerGroup 
Signed-off-by: yukun <kun.yu@zilliz.com>
 2021-03-16 21:37:25 +08:00
XuanYang-cn 32660d547b Add endPositions in datanode 
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
 2021-03-16 17:55:42 +08:00
Xiangyu Wang fc6472eb08 Implement RocksMQ client 
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-03-13 11:46:50 +08:00
godchen f3649f0419 Refactor interface and proto 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-03-12 14:22:09 +08:00
xige-16 09ae985daa Fix wrong error code in master_service_test.go 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-03-10 22:06:22 +08:00
del-zhenwu b1763c3690 Disable xdist in test cases 
Signed-off-by: del-zhenwu <zhenxiang.li@zilliz.com>
 2021-03-10 15:27:26 +08:00
neza2017 7b2d67242d Fix describe segment if index not exist 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2021-03-10 14:45:35 +08:00
Xiangyu Wang c875fdf913 Add logs in rocksmq 
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-03-10 10:02:55 +08:00
yukun 0d499a9f99 Fix search waiting for seconds in singlenode 
Signed-off-by: yukun <kun.yu@zilliz.com>
 2021-03-09 17:42:13 +08:00
quicksilver 455cc59dbb Update publish-test-images.yaml 
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
 2021-03-09 13:55:35 +08:00
ThreadDao 7a3223dc81 tags collection stats 
Signed-off-by: ThreadDao <yufen.zong@zilliz.com>
 2021-03-08 10:09:48 +08:00
quicksilver 74154a11a4 Fix deploy error during regression stage 
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
 2021-03-05 16:52:45 +08:00
sunby ddddd65d10 Delete internal/errors package 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-03-05 10:15:27 +08:00
godchen 3ba0dc2f44 Change doc 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-03-04 10:35:28 +08:00
bigsheeper 456d0e8c23 Refactor time tick align strategy 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-02-27 18:33:29 +08:00
BossZou 7893ebdb40 Fix test insert case 
Signed-off-by: BossZou <yinghao.zou@zilliz.com>
 2021-02-26 17:59:13 +08:00
godchen 7b9fdd7f29 Add opentracing 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-02-26 17:44:24 +08:00
Xiangyu Wang 38266b8569 Lock channel at produce and consume 
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-02-25 17:44:29 +08:00
Cai Yudong 84110d2684 Add tests/benchmark and tests/python_test using new python SDK 
Signed-off-by: Cai Yudong <yudong.cai@zilliz.com>
 2021-02-25 17:35:36 +08:00
bigsheeper 7cb28428ce Prevent consuming after loading collection 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-02-25 15:08:50 +08:00
cai.zhang 1d899513dd Unify service name in config file 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2021-02-24 18:46:19 +08:00
zhenshan.cao 0679954922 Refactor tso and global id allocator 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-02-24 17:12:06 +08:00
zhenshan.cao 7c5e2d89f6 Read node port from config 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-02-23 18:08:17 +08:00
dragondriver b506754364 Add workflow of drop index in index service 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-02-23 11:57:18 +08:00
zhenshan.cao bbfcbbdd68 Fix ci: paramstable and startup logic refactor 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-02-23 11:40:30 +08:00
sunby ea1d9ea99a Replace log in dataservice 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-02-23 09:58:06 +08:00
yukun e786ad6aa8 Fix inconsistency of produce msg and consume msg with rmq 
Signed-off-by: yukun <kun.yu@zilliz.com>
 2021-02-22 16:34:15 +08:00
sunby b07a371f25 Create a goroutine to receive msg for every consumer 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-02-22 09:58:34 +08:00
BossZou 7ac1821cf1 Add RocksMQ throughout test 
Signed-off-by: BossZou <yinghao.zou@zilliz.com>
 2021-02-20 18:30:37 +08:00
yukun 346b0a9549 Add unittest for RmqTtMsgStream 
Signed-off-by: yukun <kun.yu@zilliz.com>
 2021-02-20 15:58:19 +08:00
groot af3736aff6 Fix rocksmq bug 
Signed-off-by: groot <yihua.mo@zilliz.com>
 2021-02-18 14:13:55 +08:00
groot 833f1d49ab Let singlenode use rocksmq 
Signed-off-by: groot <yihua.mo@zilliz.com>
 2021-02-09 15:57:10 +08:00
dragondriver da94fbed0a Parse to map when the key of index params is named params 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-02-07 21:32:37 +08:00
yukun 563a8d66ae Add RmqTtMsgStream 
Signed-off-by: yukun <kun.yu@zilliz.com>
 2021-02-07 15:10:13 +08:00
yukun 099220394e Add rmq_msgstream unittest and fix bugs 
Signed-off-by: yukun <kun.yu@zilliz.com>
 2021-02-06 11:39:12 +08:00
godchen c2ca2c276f Fix response check error 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-02-04 19:34:35 +08:00
bigsheeper c6c99ef32b Implement getSegmengInfo function 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-02-04 11:40:14 +08:00
Xiangyu Wang fd9e42d3ed Update msgstream/unmarshal interface 
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-02-03 17:30:10 +08:00
bigsheeper abffdbff62 Remove partition tag 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-02-03 16:44:38 +08:00
yukun 35f6f4b7c2 Add consume in rmq_msgstream 
Signed-off-by: yukun <kun.yu@zilliz.com>
 2021-02-03 15:18:54 +08:00
xige-16 4cd315408b Refactor query service’s meta interface 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-02-02 11:52:41 +08:00
Xiangyu Wang 99cef4b5c8 Add Distributed main entrypoint 
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-01-29 17:08:31 +08:00
zhenshan.cao ee0faddb07 Refactor proxy 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-01-29 09:27:26 +08:00
bigsheeper 4f914a2c30 Add docker file and main function of query node 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-01-27 09:50:52 +08:00
neza2017 d1b23cb8c5 Add debug info 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2021-01-26 19:24:09 +08:00
sunby f632ed0272 Fix data service init 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-01-26 18:45:12 +08:00
zhenshan.cao e83ac41bca Add component for index 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-01-26 18:01:32 +08:00
xige-16 7c210310dc Add load function for query service 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-01-26 15:13:20 +08:00
XuanYang-cn 6c303c67b2 Rearrange update segment statistics of start position procedure 
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
 2021-01-26 09:44:39 +08:00
cai.zhang f940cc455a Add dockerfile for index 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2021-01-26 09:38:40 +08:00
FluorineDog a8fd6e7686 Enable drop index/field_data and extract info visitor 
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
 2021-01-25 18:59:44 +08:00
yukun 5600d06583 Add global_rmq 
Signed-off-by: yukun <kun.yu@zilliz.com>
 2021-01-25 18:53:08 +08:00
XuanYang-cn b794921b7e Add enum Role in internal.proto 
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
 2021-01-23 18:56:08 +08:00
BossZou 01267776d2 Fix rocksmq consume invalid currendID 
Signed-off-by: BossZou <yinghao.zou@zilliz.com>
 2021-01-23 10:15:57 +08:00
xige-16 511aa419fa Refactor query service 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-01-22 14:28:06 +08:00
neza2017 9828a52999 Add create index 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2021-01-21 10:01:29 +08:00
FluorineDog 7f044fff82 Enable search for sealed 
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
 2021-01-21 15:29:52 +08:00
sunby be6775671b Add error returned by Init,Start,Stop functions 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-01-19 17:38:01 +08:00
xige-16 76bc365181 Add seek function for pulsarTtMsgStream 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-01-20 17:34:50 +08:00
yukun 527c0c49df Add rocksmq unittest with goroutines 
Signed-off-by: yukun <kun.yu@zilliz.com>
 2021-01-20 16:46:58 +08:00
BossZou 4588342fa1 Add IdAlocator service 
Signed-off-by: BossZou <yinghao.zou@zilliz.com>
 2021-01-20 15:02:23 +08:00
sunby 3602376a10 Rename GetServiceStates to GetComponentStates and reconstruct NodeStates and ServiceStates 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-01-20 11:02:29 +08:00
cai.zhang fb493cc235 Update interface name of GetIndexStates 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2021-01-19 18:32:57 +08:00
XuanYang-cn 6a13386393 Add DataNode package 
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
 2021-01-19 11:37:16 +08:00
neza2017 1ce32d8775 Add master service 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2021-01-19 09:45:42 +08:00
yukun b874a55c35 Add complex unittest for rocksmq 
Signed-off-by: yukun <kun.yu@zilliz.com>
 2021-01-19 10:22:27 +08:00
zhenshan.cao 76d92e73d1 Replace old proto with new proto 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-01-18 21:39:56 +08:00
yukun 3aca35b10c Add rocksmq unittest 
Signed-off-by: yukun <kun.yu@zilliz.com>
 2021-01-18 15:05:49 +08:00
bigsheeper 48821690b3 Rename Service interface to Component, and update segmentLoad strategy 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-01-18 10:38:41 +08:00
XuanYang-cn 37688559f1 Fix multi rowdata writenode crash bug 
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
 2021-01-16 15:06:19 +08:00
yukun 44c75bdd2a Add consume function in rocksmq 
Signed-off-by: yukun <kun.yu@zilliz.com>
 2021-01-15 20:12:26 +08:00