milvus-io
/
milvus
mirror of https://github.com/milvus-io/milvus.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
19,817 Commits
125 Branches
112 Tags
560 MiB
Commit Graph

248 Commits (be87ae6424242c10621fb9ada183d401a3d7c8be)

Author SHA1 Message Date
sunby 044b06421a
Use etcd to watch channel on datanode (#7115) 
issue: #6578
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-10-14 15:44:34 +08:00
JackLCL 41b4264ac2
[skip ci]Fix doc error in milvus_create_index_en.md (#9653) 
Signed-off-by: JackLCL <chenglong.li@zilliz.com>
 2021-10-11 19:06:38 +08:00
congqixia 619beb0d2c
Upgrade pulsar-client-go to BUGFIX commit (#8835) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-09-29 15:02:00 +08:00
XuanYang-cn ff8fef6ad0
Upgrade pulsar-client-go to 0.6.0 (#7909) 
MessageID interface of pulsar-client-go adds
4 more methods. So I add these methods into
`MessageID` interface.

Resolves: #7770

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2021-09-15 18:13:49 +08:00
Xiaofan 619c4393f3
Refine logs in proxy search path(#7357) (#7753) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2021-09-12 17:54:01 +08:00
dragondriver 960741ee16
Support more detailed metrics information (#7472) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-09-03 19:35:47 +08:00
Xiaofan 58b2089692
Import Embedded Etcd Kv Part 1(#7112) (#7174) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2021-08-24 09:45:51 +08:00
godchen db25550fd0
Add segment filter interface (#7231) 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-08-23 17:49:50 +08:00
dragondriver ad77a6e0d4
Fix #6859, increase the MaxCallRecvMsgSize and MaxCallSendMsgSize of grpc client (#6861) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-08-04 13:03:24 +08:00
godchen b9f5949680
Add vector chunk manager (#6613) 
* add file manager

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* add file manager

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* add vector file manager

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* rename

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* go fmt

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* fix ut error

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* fix action error

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* fix memory error

Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-07-20 15:06:09 +08:00
dragondriver 676a30e418
Use channelsMgr to manage search message stream (#5819) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-06-18 10:33:58 +08:00
godchen b7bf26b486
Update grpc trace (#5817) 
* Update grpc trace

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* go fmt

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* remove useless code in mod

Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-06-17 14:17:56 +08:00
dragondriver e76830ab09
Support column-based insert request in proxy (#5560) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-06-03 15:03:34 +08:00
zhenshan.cao 6766169878
Refactor repack logic for insertion (#5399) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-05-25 11:53:15 +00:00
zhenshan.cao 3e73775cc1
Add ComputeProduceChannelIndexes for msgstream (#5364) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-05-24 08:30:09 +00:00
godchen d189bf4d33
Change pulsar version (#5352) 
Change pulsar version.

Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-05-22 03:36:49 +00:00
zhenshan.cao 70241a8bf9
Cancel local timestamp caching logic (#5327) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-05-21 05:11:21 +00:00
godchen 895037889e
Upgrade grpc version from 1.26 to 1.27 (#5128) 
Upgrade grpc version from 1.26 to 1.27
Signed-off-by: godchen qingxiang.chen@zilliz.com
 2021-05-07 18:30:11 +08:00
FluorineDog 88f5642603
Add plan proto and support basic boolean expr parser (#5088) 
**What type of PR is this?**
- [x] Feature

**What this PR does / why we need it:**
This PR supports boolean expression as DSL.
1. The goal of this PR is to support predicates
    like `A > 3 && not B < 5 or C in [1, 2, 3]`. 
2. Defines `plan.proto`, as Intermediate Representation (IR) 
    used between go and cpp. 
3. Support expr parser, convert predicate expr to IR
    in proxynode, while doing static check there
4. Support IR to AST in cpp, enable the execution
 2021-04-29 08:48:06 +00:00
godchen 139b334732
Add kv etcd unittest and remove useless code (#4974) 
* Add kv etcd unittest and remove useless code

Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-04-21 18:45:35 +08:00
neza2017 56d367b1e9 Add unit test for master service 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2021-04-09 16:10:12 +08:00
sunby 1d7195e036 Add minio and etcd stats watcher 
Our system needs to watch the performance of etcd and minio. For now I just add the creation stats.

Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-04-08 14:14:43 +08:00
XuanYang-cn 35656ea9ce Update pulsar client version and fix cannot close tcp connection error 
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
 2021-03-27 13:58:52 +08:00
xige-16 7e60460aee close timeTickStream in flow graph when the collection is released 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-03-26 18:40:04 +08:00
zhenshan.cao c2734fa55f Fix bug and enchance system 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-03-22 16:36:10 +08:00
yukun 0823382876 Refactor rmq_msgstream 
Signed-off-by: yukun <kun.yu@zilliz.com>
 2021-03-19 19:33:21 +08:00
dragondriver af32f442bb Split big insert message into serveral smaller 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-03-09 16:18:58 +08:00
sunby aeb3038545 Remove GetCount in data service 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-03-08 10:49:44 +08:00
Xiangyu Wang e5d595564a Add log for singlenode 
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-03-05 18:16:50 +08:00
sunby ddddd65d10 Delete internal/errors package 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-03-05 10:15:27 +08:00
godchen 7b9fdd7f29 Add opentracing 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-02-26 17:44:24 +08:00
zhenshan.cao bbfcbbdd68 Fix ci: paramstable and startup logic refactor 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-02-23 11:40:30 +08:00
sunby ea1d9ea99a Replace log in dataservice 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-02-23 09:58:06 +08:00
sunby b07a371f25 Create a goroutine to receive msg for every consumer 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-02-22 09:58:34 +08:00
neza2017 1d3fdebcf4 Add drop index for proxy 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2021-02-20 15:38:44 +08:00
sunby 5769c000a1 Find failed python tests and add skip mark "r0.3-test" 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-02-19 15:37:04 +08:00
bigsheeper d8f41db823 Fix search crash error 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-02-19 13:56:01 +08:00
sunby 5b61182140 Add zap log 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-02-18 19:53:55 +08:00
Xiangyu Wang 99cef4b5c8 Add Distributed main entrypoint 
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-01-29 17:08:31 +08:00
dragondriver eba6bb288c Refactor the sructure of proto file 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-01-22 09:36:18 +08:00
xige-16 d22c2cd203 Add proto for query service loading collection 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-01-18 19:32:08 +08:00
GuoRentong f3d2afa4f4 Update doc:detailed interface definition 
Signed-off-by: GuoRentong <rentong.guo@zilliz.com>
 2021-01-11 18:35:54 +08:00
quicksilver a6690dbc99 Fix syntax error: Bad fd number 
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
 2021-01-11 15:25:19 +08:00
yukun 18cfd31d40 Fix Load and Remove with prefix 
Signed-off-by: yukun <kun.yu@zilliz.com>
 2021-01-11 15:17:06 +08:00
sunby 5dfe9448ae Modify test_index 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-01-08 16:00:16 +08:00
yukun d11f00cd9f Add rocksdb in kv 
Signed-off-by: yukun <kun.yu@zilliz.com>
 2021-01-08 19:07:57 +08:00
GuoRentong 82ad488555 Update doc: rocksmq 
Signed-off-by: GuoRentong <rentong.guo@zilliz.com>
 2021-01-08 18:37:24 +08:00
XuanYang-cn 65089ea362 Fix flush sync test bug and remove unecessary lock in meta_table 
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
 2021-01-08 17:20:59 +08:00
GuoRentong c4599e26de Update doc: node api 
Signed-off-by: GuoRentong <rentong.guo@zilliz.com>
 2021-01-08 16:59:19 +08:00
bigsheeper e13822043d Fix search test when partition not exists, add binary vector search validation 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-01-08 16:41:27 +08:00
godchen fa73f17f94 Add opentracing doc 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-01-08 15:32:38 +08:00
dragondriver ced56c84f8 Add support to build binary vector index 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-01-07 14:56:17 +08:00
godchen 8f1f118b26 Add query opentracing 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-01-07 14:24:35 +08:00
bigsheeper a1e2ff3c26 Add gcNode to write node 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-01-06 16:44:12 +08:00
neza2017 4015d7245d Merge operation 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2021-01-06 14:45:50 +08:00
XuanYang-cn d64aa37db3 Add KFlush msg type 
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
 2020-12-11 12:01:20 +08:00
neza2017 32445dd02b Add cpu profile 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2020-11-26 16:01:31 +08:00
cai.zhang 57da66ba72 Update read config logic in docker 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2020-11-26 15:18:36 +08:00
zhenshan.cao 045fc3d808 Fix segment fault 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2020-11-26 14:29:01 +08:00
XuanYang-cn 0a16a9a651 Add logic of getting milvus distributed config from master by grpc 
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
 2020-11-26 13:21:06 +08:00
zhenshan.cao f12366342f Fix bug and delete unused code 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2020-11-25 16:45:42 +08:00
zhenshan.cao 50d59d91ee Add segmentIDAssigner for proxy 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2020-11-19 21:02:31 +08:00
neza2017 429c94e18b Add grpc for proxy 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2020-11-19 17:09:22 +08:00
dragondriver c2382adf04 Initialize the task scheduler of Proxy 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2020-11-14 11:24:49 +08:00
cai.zhang 1cd2fd9954 Add the standard of golangci-lint 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2020-11-12 12:04:12 +08:00
bigsheeper c9c8fb9def Close pulsar channel 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2020-11-12 11:18:23 +08:00
cai.zhang 3b2c86c42d Fix bug for cmake_build dir was changed 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2020-11-12 10:45:46 +08:00
zhenshan.cao 170792d9bc Unify proto package and version 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2020-11-11 18:13:20 +08:00
bigsheeper b4bd9cf9d6 Add collection, segment, and partition unittest 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2020-11-11 11:22:19 +08:00
zhenshan.cao 9c490a209b Delete unused code 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2020-11-05 19:11:12 +08:00
neza2017 10217f9485 Add test for etcd kv 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2020-11-02 10:26:33 +08:00
zhenshan.cao 64295db471 Refact master and proxy and add etcdutil 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2020-10-15 21:31:50 +08:00
shengjh 3d7181617b Add ci and format code for proxy 
Signed-off-by: shengjh <1572099106@qq.com>
 2020-10-15 16:32:22 +08:00
xige-16 d7e6b99394 Add config reader 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2020-10-13 15:03:46 +08:00
neza2017 bb517eeca1 Add test for proxy-go 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2020-10-12 13:54:00 +08:00
bigsheeper d4e485d37a Add pulsar authentication 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2020-10-10 11:06:07 +08:00
neza2017 b0189c2450 Update time sync 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2020-10-09 15:37:24 +08:00
cai.zhang 046b74e8b8 Add the interface of S3 for storage 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2020-09-24 10:58:20 +08:00
zhenshan.cao 3acd55f675 Fix config path error 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2020-09-18 01:53:18 +08:00
FluorineDog a177f48577 Support Query Dispatch, enable offset for bitset 
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
 2020-09-17 18:37:54 +08:00
rain 7f798c5fce Synchronous creation of collection instead of goroutine 
Signed-off-by: rain <boyan.wang@zilliz.com>
 2020-09-17 10:00:03 +08:00
zhenshan.cao be58ccb65e Delete unused 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2020-09-12 16:57:37 +08:00
shengjh 4cf6e079bc Add etcd watcher 
Signed-off-by: shengjh <1572099106@qq.com>
 2020-09-11 18:55:43 +08:00
bigsheeper 1370da9498 Add message sending and key2segment, use hard code schema 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2020-09-09 15:24:07 +08:00
xige-16 81e70ee6c9 Hash message to partitioned topic 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2020-09-07 21:07:36 +08:00
zhenshan.cao a90b725ba8 Add timestamp for key2segment message 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2020-09-07 19:46:14 +08:00
rain 642554e7bd Refactor directory structure and update the default value of server 
Signed-off-by: rain <boyan.wang@zilliz.com>
 2020-09-07 19:19:50 +08:00
xige-16 d58f6ff817 Support write node send key2seg msg 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2020-09-06 19:39:35 +08:00
xige-16 c076bd4a73 Fix the error that write node cannot receive the message 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2020-09-06 15:56:10 +08:00
xige-16 6eff45989e Serialize produce message in proxy node 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2020-09-04 18:37:21 +08:00
xige-16 e13fc08d09 Refactor write node using message client 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2020-09-04 17:52:49 +08:00
cai.zhang ce95fd4f20 Add interface for generated_ids 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2020-09-04 11:25:20 +08:00
bigsheeper 947b223bae Add segment status and management 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2020-09-03 19:58:33 +08:00
become-nice 18cf816e56 Update the organization form of the stored prefix key 
Signed-off-by: become-nice <995581097@qq.com>
 2020-09-03 10:00:48 +08:00
bigsheeper bb9c906ef6 Use go mod instead of GO_PATH and add more cgo interfeces 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2020-09-01 16:23:39 +08:00
cai.zhang 26805ebd3a Add interface for minio and add grpc proto for master 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2020-08-29 17:40:29 +08:00
bigsheeper 6d777b7518 Init reader component 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2020-08-25 15:38:35 +08:00
become-nice b8f012d8f0 Add interface of writenode 
Signed-off-by: become-nice <995581097@qq.com>
 2020-08-24 11:58:51 +08:00