milvus-io
/
milvus
mirror of https://github.com/milvus-io/milvus.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
16,668 Commits
141 Branches
117 Tags
558 MiB
Commit Graph

305 Commits (ebc173cfb8ee7c12198c67a69388ce391463095b)

Author SHA1 Message Date
Cai Yudong 6323907566
Use CommonConfig in GlobalParams for all components (#15106) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-01-10 19:03:35 +08:00
Cai Yudong b6a48817b0
Add MinioConfig in GlobalParams for all components (#15099) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-01-10 17:29:34 +08:00
Cai Yudong ee60e95310
Use RocksmqConfig in GlobalParams (#15083) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-01-10 14:51:34 +08:00
Cai Yudong 7efb02a4c5
Use PulsarConfig in GlobalParams for all components (#15046) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-01-09 23:45:33 +08:00
Cai Yudong f8f61fb116
Use BaseParams etcd configuration for all components (#15012) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-01-07 16:15:26 +08:00
shaoyue 0b74c62aed
[skip e2e] Add comment for global param (#14987) 
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
 2022-01-07 13:27:38 +08:00
XuanYang-cn 06ae5b19a3
Remove not inuse configs (#14836) 
- segmentBinlgoSubPath
- collectionBinlogSubPath
- flushStreamPosSubPath
- statsStreamPosSubPath

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2022-01-06 14:49:20 +08:00
shaoyue 8b70ae0507
[skip e2e] Add comment for global param (#14926) 
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
 2022-01-06 13:51:21 +08:00
shaoyue 00d06f2562
[skip e2e] Add comment for base table (#14925) 
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
 2022-01-06 13:49:27 +08:00
guoxiangzhou 2836b1c670
[skip e2e] Modify code comments according to the naming conventions (#14896) 
Signed-off-by: guoxiangzhou <xiangzhou.guo@zilliz.com>
 2022-01-06 13:09:49 +08:00
Cai Yudong d01931d8d9
[skip e2e] Remove useless configuration rootcoord.timeout (#14852) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-01-05 19:05:25 +08:00
shaoyue bb7d0fb2ad
[skip e2e] Add comment for base table (#14804) 
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
 2022-01-05 13:56:05 +08:00
shaoyue b6b62c025e
[skip e2e] Add comment for base table (#14723) 
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
 2022-01-04 10:47:27 +08:00
guoxiangzhou 915f11b1cd
[skip e2e] Modify log text according to the naming conventions (#14716) 
Signed-off-by: guoxiangzhou <xiangzhou.guo@zilliz.com>
 2022-01-04 10:29:18 +08:00
guoxiangzhou d00d5bd651
[skip e2e] Modify log text according to the naming conventions (#14713) 
Signed-off-by: guoxiangzhou <xiangzhou.guo@zilliz.com>
 2022-01-04 10:27:22 +08:00
guoxiangzhou 0f477092b0
[skip e2e] Modify log text according to the naming conventions (#14715) 
Signed-off-by: guoxiangzhou <xiangzhou.guo@zilliz.com>
 2022-01-04 10:25:18 +08:00
guoxiangzhou d6ec5a0170
[skip e2e] Modify log text according to the naming conventions (#14712) 
Signed-off-by: guoxiangzhou <xiangzhou.guo@zilliz.com>
 2022-01-04 10:21:18 +08:00
zhenshan.cao 32a5126705
Add the limitation of the traceable interval to timeTravel (#14498) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-12-29 21:29:45 +08:00
Xiaofan 69087ff8dd
Support embedded etcd (#14333) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2021-12-29 14:35:21 +08:00
Cai Yudong d34bb26414
Set rolname when init GlobalParamTable (#14422) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-12-28 23:12:46 +08:00
Cai Yudong 30382d8bf9
Let rootcoord share proxy.TimeTickInterval (#14326) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-12-27 21:38:45 +08:00
zhenshan.cao 9a2c91d326
Fix bug: refresh config after session init (#14194) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-12-24 22:22:17 +08:00
Cai Yudong 891c202b73
Use GlobalParamTable for all components (#13885) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-12-23 18:39:11 +08:00
shaoyue 9672c4d7d0
[skip e2e] Add comment for ConvertRangeToIntRange (#13898) 
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
 2021-12-21 21:04:25 +08:00
Cai Yudong fe4830046c
Update base_param (#13863) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-12-21 15:53:51 +08:00
Cai Yudong 9d6ee4ab75
Remove useless var from base_param (#13603) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-12-17 19:09:23 +08:00
Cai Yudong cf50e9ac55
Use GrpcServerConfig and GrpcClientConfig for distributed components (#13469) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-12-17 10:23:15 +08:00
Cai Yudong 77b2112249
Rename param to base_param, rename basetable to base_table in package paramtable (#13407) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-12-15 12:05:55 +08:00
Cai Yudong 024c1ac412
Let standalone share same LogCfg with distributed components (#13332) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-12-14 18:31:25 +08:00
shaoyue c77ed0417f
[skip ci] Add comment for basetable (#12694) 
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
 2021-12-03 17:09:43 +08:00
Xiaofan d37fdf9379
Remove component.yaml (#11959) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2021-11-17 14:35:10 +08:00
Xiaofan 71e814f796
Remove common.yaml and component.yaml (#11661) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2021-11-12 21:25:08 +08:00
Xiaofan 3cd47cd89e
Remove common.yaml and component.yaml (#11662) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2021-11-12 18:53:11 +08:00
Xiaofan 5f08b67156
Move node.yaml to component.yaml (#11363) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2021-11-10 14:27:37 +08:00
BIKI DAS c41bc0dede
Combined function params (#10868) 
Signed-off-by: Biki-das <bikid475@gmail.com>
 2021-10-29 11:06:37 +08:00
BIKI DAS b1d6d52c36
Combined appends into a single call (#10865) 
Signed-off-by: Biki-das <bikid475@gmail.com>
 2021-10-29 10:58:38 +08:00
Xiaofan 93f205b49e
Rename param names (#10853) 
* Rename param names

Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>

* Rename configs

Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2021-10-28 23:16:40 +08:00
Xiaofan fce23538a8
Remote Common yaml (#10616) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2021-10-28 21:24:41 +08:00
BIKI DAS d55d3d63eb
Improved empty string tests (#10429) 
Signed-off-by: Biki-das <bikid475@gmail.com>
 2021-10-27 14:44:24 +08:00
zhenshan.cao 49e23ebc85
[skip ci]Fix golint error in basetable (#10698) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-10-26 22:35:05 +08:00
cai.zhang aff5cd9598
Supprot to read the configuration of minio from environment variables (#10534) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2021-10-26 15:10:03 +08:00
dragondriver 977e98d21b
[skip ci] Fix golint warning of paramtable (#9991) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-10-15 20:38:58 +08:00
zhenshan.cao bbafd05911
[skip ci]Fix golint error in paramtable (#9851) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-10-14 14:04:34 +08:00
Xiaofan 2c2b800ff4
Change Log Name with NodeId (#8721) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2021-10-01 08:52:50 +08:00
congqixia 6bbf0a8f8f
Read datanode insertbuf from env (#8942) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-09-30 17:45:07 +08:00
Xiaofan 93fc10f8a6
Remove channel&&common.yaml (#8716) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2021-09-27 22:32:02 +08:00
Xiaofan d38adda8c2
Rename log file based on node id (#8517) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2021-09-26 21:18:11 +08:00
Xiaofan 31ec399edb
Move channel configuration from channel.yaml to milvus.yaml (#8420) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2021-09-26 19:46:17 +08:00
congqixia 06d4faf771
Fix G109 from gosec, prevents overflow (#7896) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-09-14 18:35:48 +08:00
Cai Yudong 27dcf698d3
Support set segcore chunk_size via config file (#7635) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-09-11 14:40:01 +08:00
groot 89815a64d7
Update unittest name of paramtable (#7526) 
Signed-off-by: yhmo <yihua.mo@zilliz.com>
 2021-09-07 15:20:40 +08:00
dragondriver 960741ee16
Support more detailed metrics information (#7472) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-09-03 19:35:47 +08:00
Cai Yudong 7edd96989f
Improve codecov for paramtable and typeutil (#7451) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-09-02 22:18:10 +08:00
Xiaofan 58b2089692
Import Embedded Etcd Kv Part 1(#7112) (#7174) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2021-08-24 09:45:51 +08:00
dragondriver b3f10ae5cc
Fix 6990, enhance the log when parsing grpc recv/send parameters (#6993) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-08-12 16:16:09 +08:00
dragondriver ad77a6e0d4
Fix #6859, increase the MaxCallRecvMsgSize and MaxCallSendMsgSize of grpc client (#6861) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-08-04 13:03:24 +08:00
yukun 39614aa8eb
Add rocksmq_path in config and paramtable (#6099) 
* Add rocksmq_path in config and paramtable

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Add rocksdbPath in NewRmsFactory

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Change rdb default path to /vat/lib/milvus/rdb_data

Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-06-25 19:44:11 +08:00
Cai Yudong 7dbf33b788
Update component names in config files (#5989) 
* rename master/indexService in .yaml

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename proxyNode to proxy in config files

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* move master.yaml to root_coord.yaml, move proxy_node.yaml to proxy.yaml

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* move proto/query_service.proto to proto/query_coord.proto

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* move proxy_node_manager.go to proxy_manager.go

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add query_coord.pb.go

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update ci

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-06-22 19:08:03 +08:00
xige-16 2af1c27811
Rename queryservice to querycoordinator (#5931) 
* Rename queryservice to querycoordinator

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* fix param table

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* code format

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* fix unittest

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* service to coordinator

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* code format

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* set querycoord in ci

Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-06-22 16:44:09 +08:00
sunby 7f2d7c1da3
Rename dataservice to datacoordinator (#5912) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-06-21 18:22:13 +08:00
XuanYang-cn ab7f642740
Add alias in paramtable (#5878) 
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2021-06-19 12:38:06 +08:00
Cai Yudong 3b0951e3e6
Combine standalone binary into milvus (#5866) 
* rename service to coord for cmd

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update docs

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update variable name

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* optimize roles.go

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* support milvus run standalone

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* remove cmd/standalone

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix static-check

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename proxynode to proxy

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename service to coord for compoments

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add comments for codacy check

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* set helm chart branch

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-06-18 15:20:08 +08:00
yukun e9a8d1c404
Add vChannels in proxy for query results (#5802) 
* Fix proxynode for new retrieve logic

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Remove querynodenum from proxynode and querynode

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Remove QueryNodeIDList from proxy

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

Co-authored-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-06-16 20:15:59 +08:00
Cai Yudong f28403b99b
clean proxy service (#5812) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-06-16 18:19:57 +08:00
Cai Yudong 0f4bd50ce3 Support ETCD endpoints array (#5755) 2021-06-15 16:36:36 +08:00
XuanYang-cn cebdfb34f7
Enable multiple datanode (#5068) 
Now it's able to start multiple datanode, however, they will process the same insert data.
Further changes are needed to truly enable multiple datanode:

While registering, dataservice need to return different insert channels for different datanode.

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2021-04-28 18:43:48 +08:00
Xiangyu Wang 82ccd4cec0
Rename module (#4988) 
* Rename module

Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-04-22 14:45:57 +08:00
sunby fad94fb576
Remove unsed configs (#4990) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-04-22 03:04:59 +00:00
XuanYang-cn e281f6a4a1 Remove useless configs in config files 
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
 2021-04-13 10:04:39 +08:00
cai.zhang 1d899513dd Unify service name in config file 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2021-02-24 18:46:19 +08:00
zhenshan.cao 7c5e2d89f6 Read node port from config 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-02-23 18:08:17 +08:00
zhenshan.cao bbfcbbdd68 Fix ci: paramstable and startup logic refactor 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-02-23 11:40:30 +08:00
xige-16 7c210310dc Add load function for query service 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-01-26 15:13:20 +08:00
cai.zhang f940cc455a Add dockerfile for index 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2021-01-26 09:38:40 +08:00
xige-16 511aa419fa Refactor query service 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-01-22 14:28:06 +08:00
cai.zhang fb493cc235 Update interface name of GetIndexStates 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2021-01-19 18:32:57 +08:00
XuanYang-cn 6a13386393 Add DataNode package 
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
 2021-01-19 11:37:16 +08:00
sunby 0a383608f3 Remove mock client in master 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2020-12-31 11:20:03 +08:00
FluorineDog 89f38e459a Fix bool type 
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
 2021-01-04 17:52:42 +08:00
zhenshan.cao 6bd5702441 Add indexbuilder address to paramstable 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-01-04 14:53:20 +08:00
XuanYang-cn 32977e270c Add impl cgo of parquet 
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
 2020-12-08 14:41:04 +08:00
zhenshan.cao 7bbbc14637 Fix bug: address already used 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2020-12-07 15:22:20 +08:00
xige-16 37d7526d31 Fix search test failure 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2020-12-07 14:37:42 +08:00
neza2017 aeca8f85a5 Run cwrapper unittest in ci 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2020-12-05 17:39:58 +08:00
cai.zhang 806a97505c Add query node for CI progress 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2020-12-05 16:12:45 +08:00
neza2017 70710dee47 Add parquet payload 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2020-12-05 16:11:03 +08:00
FluorineDog 6412ebc0d4 Add support of metric type in schema, enable binary vector, fix segfault 
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
 2020-12-05 06:46:01 +08:00
zhenshan.cao 0b6fe0a8cf Refactor paramtable 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2020-12-04 14:18:17 +08:00
godchen 7ab5b5d80d Add MinIO kv implements 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2020-12-03 19:00:11 +08:00
dragondriver 2358fae703 Check the status of search results before reduce in Proxy 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2020-11-30 19:08:32 +08:00
bigsheeper 0cd3e8d86c Use exception instead of runtime error 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2020-11-30 17:58:23 +08:00
cai.zhang 546beb333d Compile master and proxy without cgo 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2020-11-30 16:40:13 +08:00
quicksilver 86573d0053 Create Jenkins pipeline 
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
 2020-11-30 10:07:04 +08:00
FluorineDog 77fa75b1ec Add binary insert and warper of binary search, rename vector 
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
 2020-11-30 05:18:44 +08:00
cai.zhang 85a544b79b Add dockerfile for each component 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2020-11-28 19:06:48 +08:00
cai.zhang 115d0732af Update docker-compose file for unittest 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2020-11-28 16:43:39 +08:00
dragondriver eb6c0e0a28 Fix the bug when reduce the multiple search result from query nodes 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2020-11-28 15:57:49 +08:00
godchen 92bf09c7e6 Add ddstream and scheduler implements 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2020-11-28 15:26:39 +08:00
dragondriver 235e723eb6 Read proxy id from param table instead of hardcode 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2020-11-27 11:17:12 +08:00
XuanYang-cn 3e605ae25f Rename reader to querynode 
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
 2020-11-27 10:56:53 +08:00
neza2017 32445dd02b Add cpu profile 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2020-11-26 16:01:31 +08:00
cai.zhang 57da66ba72 Update read config logic in docker 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2020-11-26 15:18:36 +08:00
cai.zhang 17c3a70d64 update ParameterTable 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2020-11-21 15:06:46 +08:00
neza2017 ce89f64bb3 Set master config 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2020-11-19 10:46:17 +08:00
cai.zhang 5bcabffdaf Replace conf struct with ParamsTable 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2020-11-16 21:10:43 +08:00