Commit Graph

746 Commits (438790744da2fb22e98276a8331f8a42c9b10389)

Author SHA1 Message Date
Jiquan Long e44afc7aa3
Fix ut: wait service ready after run Proxy server (#17208)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2022-05-25 18:56:00 +08:00
Ten Thousand Leaves 5e1e7a6896
Fix an issue that causes flush hanging (#17169)
issue: #16628
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-05-25 14:34:00 +08:00
bigsheeper 9eeec4a2d5
Add collection load cache and InvalidateCollMetaCache by collID (#16882)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-05-19 10:13:56 +08:00
Jiquan Long ca3ecec5b5
Fix ut data race (#16939)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2022-05-13 17:43:53 +08:00
exia 4ee78eb5ad
proxy support tls (#16706)
Signed-off-by: xiyichan <2863768433@qq.com>
2022-05-10 20:05:53 +08:00
Ten Thousand Leaves 1acd256481
Add DataQueryable and DataIndexed states for bulk load tasks (#16725)
issue: #16607

/kind enhancement

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-05-05 21:17:50 +08:00
Xiaofan 333b439a6c
Support config grpc log level (#16618)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-05-05 10:35:50 +08:00
cai.zhang 2be46a01c0
Reduce the number of retries and add error log (#16754)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2022-05-05 09:31:51 +08:00
shaoyue 2f49db9d3c
Implements remaining rest APIs (#16387)
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
2022-04-29 15:15:47 +08:00
xiyichan 056fd9aa17
Correct wrong words (#16490)
Signed-off-by: chenlifeng <chenlifeng0995@wezhuiyi.com>

Co-authored-by: chenlifeng <chenlifeng0995@wezhuiyi.com>
2022-04-27 19:51:46 +08:00
congqixia 0bf176a7e1
Add SyncReplicaSegments interface for QueryNode (#16672)
Add SyncReplicaSegments for LoadCollection/LoadPartition child task
This API notifies related ShardCluster to record segment Loaded
Preventing the Etcd events is delayed, which will cause some segment not search

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-04-27 10:41:46 +08:00
XuanYang-cn fbc7fe1cdc
Fix proxy unable to update cache (#16646)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-04-26 11:27:53 +08:00
groot bc5e9ec929
Add list import tasks api (#16605)
Signed-off-by: groot <yihua.mo@zilliz.com>
2022-04-25 17:37:46 +08:00
Xiaofan 89b4a34892
Fix Data race in NodeID (#16603)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-04-24 22:03:44 +08:00
congqixia 69252f812d
Implement memory replica in Proxy, QueryNode and QueryCoord (#16470)
Related to #16298 #16291 #16154
Co-authored-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: yangxuan <xuan.yang@zilliz.com>
Co-authored-by: yah01 <yang.cen@zilliz.com>
Co-authored-by: Letian Jiang <letian.jiang@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-04-20 16:15:41 +08:00
Ten Thousand Leaves 289e468a7a
Make bulk load fully work (#16512)
issue: #15604

/kind enhancement

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-04-20 14:03:40 +08:00
codeman f5e1ded854
fix #16504 (#16523)
Signed-off-by: kejiang <ke.jiang@zilliz.com>

Co-authored-by: kejiang <ke.jiang@zilliz.com>
2022-04-19 16:35:39 +08:00
Ten Thousand Leaves e76a8c5ec2
Update compile options and config for embedded Milvus (#16472)
/kind enhancement

issue: #15711
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-04-14 19:57:34 +08:00
codeman 4e12271d79
Support login with username and password (#15656) (#16341)
Signed-off-by: kejiang <ke.jiang@zilliz.com>

Co-authored-by: kejiang <ke.jiang@zilliz.com>
2022-04-11 19:49:34 +08:00
godchen bb7a0766fe
Add dependency factory (#16204)
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
2022-04-07 22:05:32 +08:00
XuanYang-cn 909dd73741
Avoid using specific port in ut (#16398)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-04-06 20:13:32 +08:00
Ten Thousand Leaves 1e6f6bd072
Implement set segment state call from RootCoord to DataCoord. (#16374)
/kind feature

issue: #15604
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-04-06 15:33:32 +08:00
XuanYang-cn 78200009a3
Impl GetReplicas in Proxy (#16390)
See also: #16298

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-04-06 14:57:31 +08:00
Letian Jiang 028c5cb882
Modify grpc interface for replica Search/Query in QueryNode (#16326)
Signed-off-by: Letian Jiang <letian.jiang@zilliz.com>
2022-04-02 14:15:31 +08:00
yah01 c005f07ccc
Modify proto to add GetReplicas() interface (#16328)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-04-01 16:15:29 +08:00
Ten Thousand Leaves 45be3deb3f
Implement RC, DC, DN calling path for import. (#16321)
/kind feature

issue: #15604
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-04-01 11:33:28 +08:00
XuanYang-cn 58addbf9f4
Add GetShardLeaders rpc in querycoord proto (#16299)
See also: #16298

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-03-31 16:39:29 +08:00
Jiquan Long ba37531456
Add support for loading multiple indexes (#16138)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2022-03-30 21:11:28 +08:00
congqixia 18a3e9f265
Add grpc interface for replica Search/Query in QueryNode (#16197)
Resolves #16195
Add Search and Query grpc interface in query proto and types/types.go

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-03-30 12:03:27 +08:00
groot 1c4b949a1d
modify import rpc interface (#16129)
Signed-off-by: groot <yihua.mo@zilliz.com>
2022-03-22 15:11:24 +08:00
groot 2078b24dca
Prepare for import (#16083)
Signed-off-by: groot <yihua.mo@zilliz.com>
2022-03-21 15:47:23 +08:00
shaoyue 97e5d77953
implements restful entities api (#15916)
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
2022-03-21 11:21:23 +08:00
Xiaofan 787d4f8bfd
Refine prometheus metrics and etcd log (#16084)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-03-18 15:51:22 +08:00
zhenshan.cao 0bd07daae3
Add grpc healthy check (#16050)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-03-15 15:05:21 +08:00
groot 89356ebcb9
Modify import rpc interfaces (#16006)
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2022-03-11 17:13:59 +08:00
Ten Thousand Leaves d5bae6710f
Make all logs configurable for standalone/embedded Milvus. (#15926)
Resolves: #15919, #15708
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-03-10 10:35:59 +08:00
groot 5fdef607d5
Add rpc interfaces for import (#15930)
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2022-03-09 18:33:59 +08:00
Xiaofan d013134b44
Enhance error message when coordinator not healthy (#15950)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-03-09 15:57:58 +08:00
Cai Yudong 3e862ecdfb
Merge msgChannelConfig and knowhereConfig into commonConfig (#15843)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-03-04 11:17:56 +08:00
jaime 7a3a721380
Reconstruct mqstream module (#15784)
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
2022-03-03 21:57:56 +08:00
shaoyue 83fdc1da7c
Add web server in proxy with sample handler (#15647)
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
2022-02-23 14:37:52 +08:00
Xiaofan d499ffc421
Fix confusing logs in rootcoord (#15431)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-02-10 13:51:47 +08:00
Xiaofan be42627ab2
Move Logutil to Util dir (#15447)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-02-09 18:55:46 +08:00
xige-16 55cdc5aa35
Get indexInfo and segment size by queryCoord (#14207)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-02-08 21:57:46 +08:00
Cai Yudong 1f337891e1
Rename base_param to service_param, rename global_param to component_param (#15463)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-02-08 20:57:47 +08:00
Cai Yudong f1ea9613ae
Add EtcdConfig and move PulsarConfig/RocksdbConfig/MinioConfig into BaseParamTable (#15434)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-02-07 10:09:45 +08:00
Cai Yudong 4bccc85534
Use MsgChannelConfig in GlobalParams for all components (#15163)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-02-02 00:35:43 +08:00
cai.zhang f564ea7fc7
RootCoord set healthy state after registered service (#15261)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2022-01-18 12:09:37 +08:00
Jiquan Long 675e6d352b
QueryNodes send search/query results by rpc (#15223)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2022-01-17 14:41:35 +08:00
zhenshan.cao 1601a61b78
Move logic of checking available port to service (#15222)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-01-14 21:17:34 +08:00
zhenshan.cao f14df1d2db
Initilize listener in service instead of ParamTable (#15220)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-01-14 18:29:34 +08:00
cai.zhang 745d5f5a8a
[skip e2e]Add comment for newGrpcClient (#15144)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2022-01-12 11:13:37 +08:00
Cai Yudong 6323907566
Use CommonConfig in GlobalParams for all components (#15106)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-01-10 19:03:35 +08:00
shaoyue cea5853cfb
[skip e2e] Add comment for datacoord client (#15088)
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
2022-01-10 18:31:51 +08:00
congqixia ca7988fec7
[skip e2e] Normalize DataNode in comment by convention (#15024)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-01-07 21:13:37 +08:00
Cai Yudong f8f61fb116
Use BaseParams etcd configuration for all components (#15012)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-01-07 16:15:26 +08:00
congqixia aeae62a1fb
[skip e2e] Add comment for ConnectionManager (#14906)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-01-06 18:01:26 +08:00
shaoyue 8818245d63
[skip e2e] Add comment for datacoord client params (#14924)
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
2022-01-06 13:03:30 +08:00
cai.zhang 47e19fdeab
[skip e2e]Reorder head files for indexcoord/client.go (#14729)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2022-01-04 10:55:18 +08:00
cai.zhang 162e053ae4
[skip e2e]Reorder head files for indexnode/service.go (#14727)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2022-01-04 10:53:18 +08:00
cai.zhang 75fe1a755d
[skip e2e]Reorder head files for indexnode/service_test.go (#14726)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2022-01-04 10:51:22 +08:00
cai.zhang 9baa6e84c3
[skip e2e]Reorder head files for indexcoord/service.go (#14663)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2021-12-31 16:13:19 +08:00
Xiaofan 69087ff8dd
Support embedded etcd (#14333)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2021-12-29 14:35:21 +08:00
shaoyue 017f04aea1
[skip e2e] Add comment for indexcoord.Params (#14438)
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
2021-12-28 19:36:51 +08:00
shaoyue fa3d42f8f8
[skip e2e] Add comment for datanode.Params (#14437)
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
2021-12-28 19:34:58 +08:00
shaoyue 17f8739068
[skip e2e] Add comment for datacoord.Params (#14436)
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
2021-12-28 19:33:19 +08:00
Jiquan Long 8a488f1f20
Refine startup workflow of proxy (#13459)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-12-25 16:10:17 +08:00
Cai Yudong 891c202b73
Use GlobalParamTable for all components (#13885)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-12-23 18:39:11 +08:00
jingkl 6c3f1a9162
[skip e2e]Add comments to proxy service (#14082)
Signed-off-by: jingkl <jingjing.jia@zilliz.com>
2021-12-23 18:17:15 +08:00
jingkl df546bc241
[skip e2e]Add comments to proxy service (#14081)
Signed-off-by: jingkl <jingjing.jia@zilliz.com>
2021-12-23 18:15:21 +08:00
jingkl df90402a7f
[skip e2e]Add comments to proxy service (#14080)
Signed-off-by: jingkl <jingjing.jia@zilliz.com>
2021-12-23 18:11:31 +08:00
shaoyue c65bb0d3c6
[skip e2e] Add comment for datacoord/client.Client (#14029)
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
2021-12-23 11:07:14 +08:00
jingkl e3e45ef069
[skip e2e]Add comments to proxy service (#13996)
Signed-off-by: jingkl <jingjing.jia@zilliz.com>
2021-12-22 19:37:19 +08:00
jingkl 7423da8cb0
[skip e2e]Add comments to proxy service (#13995)
Signed-off-by: jingkl <jingjing.jia@zilliz.com>
2021-12-22 19:35:27 +08:00
shaoyue 33963955dc
[skip e2e] Add comment for distributed/proxy.Server (#13991)
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
2021-12-22 19:29:50 +08:00
jingkl 1e658f26ea
[skip e2e]Add comments to proxy service (#13994)
Signed-off-by: jingkl <jingjing.jia@zilliz.com>
2021-12-22 19:13:14 +08:00
jingkl e4ef536bb5
[skip e2e]Add comments to proxy service (#13829)
Signed-off-by: jingkl <jingjing.jia@zilliz.com>
2021-12-21 15:25:08 +08:00
jingkl 0011f39530
[skip e2e]Add comments to proxy service (#13835)
Signed-off-by: jingkl <jingjing.jia@zilliz.com>
2021-12-21 15:12:55 +08:00
jingkl 398a11cda0
[skip e2e]Add comments to proxy service (#13831)
Signed-off-by: jingkl <jingjing.jia@zilliz.com>
2021-12-21 11:55:31 +08:00
jingkl 93e634c189
[skip e2e]Add comments to proxy service (#13756)
Signed-off-by: jingkl <jingjing.jia@zilliz.com>
2021-12-20 19:16:37 +08:00
jingkl 9e7f548ee6
[skip e2e]Add comments to proxy service (#13754)
Signed-off-by: jingkl <jingjing.jia@zilliz.com>
2021-12-20 18:58:50 +08:00
Cai Yudong a101d1d3c5
[skip e2e] Unify component import alias (#13765)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-12-20 16:43:05 +08:00
Cai Yudong 966119526a
Rename Params to ClientParams for all component clients (#13743)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-12-20 14:25:25 +08:00
jingkl 572afa502c
[skip e2e]Add comments to proxy service (#13565)
Signed-off-by: jingkl <jingjing.jia@zilliz.com>
2021-12-17 14:30:27 +08:00
jingkl 21189779ee
[skip e2e]Add comments to proxy service (#13564)
Signed-off-by: jingkl <jingjing.jia@zilliz.com>
2021-12-17 14:27:30 +08:00
Cai Yudong cf50e9ac55
Use GrpcServerConfig and GrpcClientConfig for distributed components (#13469)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-12-17 10:23:15 +08:00
Jiquan Long acbb680c91
[skip ci] Fix go fmt error (#13538)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-12-16 20:47:14 +08:00
jingkl fd33750aec
[skip ci] Add the comments to service (#13509)
Signed-off-by: jingkl <jingjing.jia@zilliz.com>
2021-12-16 16:39:52 +08:00
jingkl d4c998e6f5
[skip ci] Add the comments to service (#13508)
Signed-off-by: jingkl <jingjing.jia@zilliz.com>
2021-12-16 16:37:27 +08:00
jingkl 24b8b6e4c7
[skip ci] Add the comments to service (#13507)
Signed-off-by: jingkl <jingjing.jia@zilliz.com>
2021-12-16 16:33:44 +08:00
jingkl 2aa1d5d714
[skip ci] Add the comments to service (#13400)
Signed-off-by: jingkl <jingjing.jia@zilliz.com>
2021-12-15 17:01:49 +08:00
jingkl 706cc9d7c2
[skip ci] Add the comments to service (#13404)
Signed-off-by: jingkl <jingjing.jia@zilliz.com>
2021-12-15 16:59:56 +08:00
jingkl 3ea83d77cc
[skip ci] Add the comments to service (#13411)
Signed-off-by: jingkl <jingjing.jia@zilliz.com>
2021-12-15 12:00:20 +08:00
cai.zhang 9f23fc7f2a
Register the service when the component state is healthy (#13248)
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
2021-12-15 11:47:10 +08:00
congqixia 61f2db3102
Remove debug log from DataCoord client (#13313)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-12-15 11:09:24 +08:00
jingkl f311353182
[skip ci] Add the comments to service (#13316)
Signed-off-by: jingkl <jingjing.jia@zilliz.com>
2021-12-14 16:21:22 +08:00
jingkl 96a0352272
[skip ci] Add notes to service (#13315)
Signed-off-by: jingkl <jingjing.jia@zilliz.com>
2021-12-14 16:19:33 +08:00
jingkl 1950beba24
[skip ci] Add comments to service (#13314)
Signed-off-by: jingkl <jingjing.jia@zilliz.com>
2021-12-14 11:59:10 +08:00
Cai Yudong 973b98c721
Update parameter of API startGrpc (#13282)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-12-13 21:39:10 +08:00
jingkl d91d5d765b
[skip ci]Add notes with datanode service (#13230)
Signed-off-by: jingkl <jingjing.jia@zilliz.com>
2021-12-13 13:27:49 +08:00