milvus-io
/
milvus
mirror of https://github.com/milvus-io/milvus.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
19,918 Commits
133 Branches
115 Tags
559 MiB
Commit Graph

548 Commits (066c8ea17591e775f7f734defff8e3514abafc39)

Author SHA1 Message Date
MrPresent-Han 21b54709a2
enable delbuf auto flush function to avoid OOM when del msg accumulating (#20213) 
issue: #19713

Signed-off-by: MrPresent-Han <jamesharden11122@gmail.com>

Signed-off-by: MrPresent-Han <jamesharden11122@gmail.com>
Co-authored-by: MrPresent-Han <zilliz@zillizdeMacBook-Pro-2.local>
 2022-11-07 18:49:02 +08:00
xige-16 6377d9f375
Open c++ log (#20325) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>

Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-11-04 18:01:41 +08:00
zhagnlu 15b2d1cace
Change default thread core coefficient (#20120) (#20328) 
Signed-off-by: zhagnlu <lu.zhang@zilliz.com>

Signed-off-by: zhagnlu <lu.zhang@zilliz.com>
Co-authored-by: zhagnlu <lu.zhang@zilliz.com>
 2022-11-04 12:41:35 +08:00
zhagnlu cb2591d1fe
Parallel operations of disk-ann file manager (#20120) (#20122) 
Signed-off-by: zhagnlu <lu.zhang@zilliz.com>

Signed-off-by: zhagnlu <lu.zhang@zilliz.com>
Co-authored-by: zhagnlu <lu.zhang@zilliz.com>
 2022-11-03 14:39:40 +08:00
shaoyue 9fac1476f3
Add support for GCS(GoogleCloudStorage) with IAM (#20164) 
go mod tidy

Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>

Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
 2022-11-01 11:07:35 +08:00
bigsheeper b949fa0ea9
Increase default ttProtection max delay (#20125) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-10-27 17:37:32 +08:00
bigsheeper 99df2d9b54
Add binlogSize and ttDelay metrics monitor (#19872) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-10-19 15:13:26 +08:00
bigsheeper 760e9c5df2
Enable rate limit and update quota unit (#19816) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-10-17 16:23:25 +08:00
SimFG ae8ec28925
Delete the unrelated content in the yaml file (#19812) 
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2022-10-17 15:45:25 +08:00
SimFG a55f739608
Separate public proto files (#19782) 
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2022-10-16 20:49:27 +08:00
xige-16 a1db9038fb
Move disk index params to config file (#19714) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>

Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-10-14 17:51:24 +08:00
bigsheeper 22b32465e7
Make bulkLoad failed if quota exceeds and remove autoindex config (#19770) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-10-14 14:27:25 +08:00
bigsheeper f1e330a997
Add read result rate protection (#19728) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-10-13 14:57:24 +08:00
bigsheeper e6c76aba0e
Force deny writing if disk quota exceeded (#19680) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-10-12 10:03:22 +08:00
bigsheeper f98dbcf5be
Update quota params and disable limit for showCollection (#19641) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-10-09 14:30:58 +08:00
zhenshan.cao 0ac4bc32a5
Add logic of autoindex (#19570) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2022-10-08 15:38:58 +08:00
yah01 e6c7286bee
Remove task timeout (#19583) 
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-09-30 19:48:55 +08:00
SimFG ccf30d358c
Revert the `milvus.yaml` file (#19568) 
Signed-off-by: SimFG <bang.fu@zilliz.com>
 2022-09-29 21:53:31 +08:00
yanliang567 3e9376f951
Disable time travel, shorten growing segment lifetime (#19520) 
Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>

Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
 2022-09-29 19:42:55 +08:00
wayblink 54e2c079c4
MEP-30:Support basic coordinators active-standby mechanism #16901 (#19166) 
Signed-off-by: wayblink <anyang.wang@zilliz.com>

Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2022-09-29 18:35:02 +08:00
bigsheeper 5405dedd2e
Fix DDL rate limit unit, and improve log (#19504) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-09-28 20:58:54 +08:00
yah01 9dcee37e1c
Skip canceled tasks (#19513) 
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-09-28 19:58:54 +08:00
cai.zhang c58a8d8fd6
Support bind IndexNode mode (#19247) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>

Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2022-09-27 22:38:55 +08:00
Ten Thousand Leaves b30c9d4f60
Support Bulk Load in Milvus 2.2 (#18982) 
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-09-26 18:06:54 +08:00
bigsheeper 804c18df68
Update quota params (#19351) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-09-26 16:48:53 +08:00
zhagnlu 59f8e96522
Change compactor policy from loading all binlogs to loading serially (#13716) (#19280) 
Signed-off-by: zhagnlu <lu.zhang@zilliz.com>

Signed-off-by: zhagnlu <lu.zhang@zilliz.com>
Co-authored-by: zhagnlu <lu.zhang@zilliz.com>
 2022-09-25 20:12:52 +08:00
Bingyi Sun 29d2c0bd7b
Move task merge cap to config file (#19387) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@zilliz.com>
 2022-09-24 17:56:51 +08:00
SimFG 68a257458b
Implement the `hook` interceptor (#19294) 
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2022-09-23 09:50:52 +08:00
xige-16 1cd6e80c8a
Increase timeout of activate task (#19330) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>

Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-09-22 14:14:52 +08:00
xige-16 428840178c
Support diskann index for vector field (#19093) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>

Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-09-21 20:16:51 +08:00
Jiquan Long 5141e05c47
Add policy to seal long time idle segment (#19222) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2022-09-20 20:54:50 +08:00
bigsheeper 5143e2a75e
Disable quota and limits (#19242) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-09-20 09:34:50 +08:00
bigsheeper 5cdd6ca251
Add quotaCenter, rateLimiter and rateCollector (#18636) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-09-16 09:56:47 +08:00
Bingyi Sun 626854cf0c
Refactor QueryCoord (#18836) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: yah01 <yang.cen@zilliz.com>
Co-authored-by: Wei Liu <wei.liu@zilliz.com>
Co-authored-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: yah01 <yang.cen@zilliz.com>
Co-authored-by: Wei Liu <wei.liu@zilliz.com>
Co-authored-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-09-15 18:48:32 +08:00
bigsheeper 5eb6127fc5
Set max read concurrency ratio to improve concurrent read performance (#18911) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-08-30 19:50:57 +08:00
Xiaofan 8e22d03cf3
Support trigger compaction everyday to eliminiate deleted data in rocksmq (#18752) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>

Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-08-29 10:56:55 +08:00
cai.zhang c924f73105
Refactor for IndexCoord to support cloud (#18643) 
Co-authored-by: Zach41 <zongmei.zhang@zilliz.com>
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>

Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
Co-authored-by: Zach41 <zongmei.zhang@zilliz.com>
 2022-08-25 15:48:54 +08:00
wgcn d443b5420a
add user special kafka config (#18742) 
Signed-off-by: wgcn <1026688210@qq.com>

Signed-off-by: wgcn <1026688210@qq.com>
Co-authored-by: wanggang11335 <wanggang11335@autohome.com.cn>
 2022-08-25 11:02:53 +08:00
Xiaofan 8fe3895485
change retention time to 1 day (#18749) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>

Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-08-25 10:42:53 +08:00
SimFG ce434b496e
Implement the mysql metastore of the rbac (#18704) 
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2022-08-23 10:26:53 +08:00
codeman 6c3dbf0a07
Support meta storing on database (#17236) (#18541) 
Signed-off-by: kejiang <ke.jiang@zilliz.com>

Signed-off-by: kejiang <ke.jiang@zilliz.com>
Co-authored-by: kejiang <ke.jiang@zilliz.com>
 2022-08-11 12:12:38 +08:00
bigsheeper cef8b1e7cc
Enable jemalloc (#18349) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-07-20 22:22:31 +08:00
Jiquan Long 70246f82fd
Reduce chunk row size (#18320) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2022-07-19 13:50:28 +08:00
congqixia f7bd054adb
Set EnableGarbageCollection to true by default (#18305) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-07-18 16:34:28 +08:00
cai.zhang 0ad15a22c8
IndexCoord handle events correctly (#17878) 
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
 2022-07-07 14:44:21 +08:00
zhenshan.cao 1647bebaf4
Count growing segment number to estimate cpu usage (#17820) (#18027) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2022-07-04 15:10:20 +08:00
jaime 762e3b78a6
support configurable sasl mechanisms for kafka (#17542) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
 2022-06-29 10:32:17 +08:00
shaoyue 96b97f7f58
RESTful api use same port as metrics & healthcheck (#17732) 
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
 2022-06-24 15:34:14 +08:00
SimFG f4c6a6734e
Fix rpc retry policy not taking effect (#17673) 
Signed-off-by: SimFG <bang.fu@zilliz.com>
 2022-06-22 22:22:13 +08:00
Xiaofan d70f483806
Support rocksmq dynamic lru cache size (#17626) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-06-20 10:56:12 +08:00
bigsheeper 92d06b2e30
Purge memory by the memory state and try to purge after each search (#17565) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-06-17 17:46:10 +08:00
Xiaofan 1f6fbf91b2
Fix pulsar unsubsribe issue (#17562) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-06-16 17:28:11 +08:00
Ten Thousand Leaves 65ab847815
Add segment reference lock for bulk load (#17457) 
So we can support DataQueryable and DataIndexed states

issue: #16607
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-06-15 12:20:10 +08:00
Bingyi Sun b3f0bbadff
Adjust keepalive timeout to 20s as default (#17510) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
 2022-06-13 11:54:08 +08:00
yah01 a38790f586
Predict the loading memory usage more accurate (#17451) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-06-09 18:16:07 +08:00
shaoyue 76eaa3fc50
Support access s3 through AWS IAM role (#17292) 
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
 2022-06-02 19:42:03 +08:00
SimFG 0986c29d7f
proxy tls support one-way authentication (#17348) 
Signed-off-by: xiyichan <2863768433@qq.com>

Co-authored-by: SimFG <bang.fu@zilliz.com>
Signed-off-by: SimFG <bang.fu@zilliz.com>

Co-authored-by: xiyichan <2863768433@qq.com>
 2022-06-02 18:56:04 +08:00
jaime 5698bf4236
Add sasl configurations for kafka (#17323) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
 2022-06-02 12:12:03 +08:00
Jiquan Long 6bd786bf50
Reduce default seal propotion (#17287) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2022-05-31 12:02:04 +08:00
congqixia c88514bc49
Remove not used QueryChannel in Proxy and Query Cluster (#16856) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-05-30 19:50:04 +08:00
zhenshan.cao 2512e668f7
Update timestmap according to gracefultime for Bounded Consistency (#17171) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2022-05-24 12:05:59 +08:00
zhenshan.cao 08a4dff41d
Add logic for search merging and a simple task scheduler for read tasks (#17022) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>

Co-authored-by: cai.zhang <cai.zhang@zilliz.com>
Co-authored-by: bigsheeper <yihao.dai@zilliz.com>

Co-authored-by: cai.zhang <cai.zhang@zilliz.com>
Co-authored-by: bigsheeper <yihao.dai@zilliz.com>
 2022-05-23 16:41:58 +08:00
zhenshan.cao 246b22b5ea
Configure indexSliceSize to 16MB (#17080) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2022-05-23 15:01:58 +08:00
ownbylichaobao 2f8a7e7793
Support tls/ssl connection to etcd (#7703) (#17012) 
Signed-off-by: lichaobao <1527563274@qq.com>
 2022-05-20 12:29:19 +08:00
Jiquan Long 99b4042f8c
Disable segment small index (#16935) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2022-05-18 15:27:56 +08:00
exia 4ee78eb5ad
proxy support tls (#16706) 
Signed-off-by: xiyichan <2863768433@qq.com>
 2022-05-10 20:05:53 +08:00
Enwei Jiao 67ce206cf0
make SegmentMaxLifetime configurable (#16783) 
Signed-off-by: Enwei Jiao <jiaoew2011@gmail.com>
 2022-05-06 23:41:51 +08:00
Ten Thousand Leaves 1acd256481
Add DataQueryable and DataIndexed states for bulk load tasks (#16725) 
issue: #16607

/kind enhancement

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-05-05 21:17:50 +08:00
Xiaofan 333b439a6c
Support config grpc log level (#16618) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-05-05 10:35:50 +08:00
shaoyue fa97f86cda
Clean up not used id_cache related code (#16655) 
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
 2022-05-05 09:59:51 +08:00
Xiaofan 337ad53613
Support config small index nprobe and nlist (#16731) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-05-05 09:49:50 +08:00
Cai Yudong 6a62ff18bf
Support easylogging config for segcore and knowhere (#16751) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-05-03 08:39:49 +08:00
shaoyue 2f49db9d3c
Implements remaining rest APIs (#16387) 
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
 2022-04-29 15:15:47 +08:00
Xiaofan ed51ffd50f
Fix Compact Entity ttl param (#16665) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-04-28 11:51:47 +08:00
Ten Thousand Leaves 4855ba6fa3
Optimize Mac M1 support some tweaks to embedded Milvus (#16650) 
issue: #16500
issue: #16621

/kind enhancement

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-04-26 14:21:46 +08:00
Ten Thousand Leaves 14724013ba
[skip e2e]Update config for embedded Milvus (#16614) 
issue: #15711

/kind improvement

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-04-25 14:17:45 +08:00
codeman f5e1ded854
fix #16504 (#16523) 
Signed-off-by: kejiang <ke.jiang@zilliz.com>

Co-authored-by: kejiang <ke.jiang@zilliz.com>
 2022-04-19 16:35:39 +08:00
Ten Thousand Leaves e76a8c5ec2
Update compile options and config for embedded Milvus (#16472) 
/kind enhancement

issue: #15711
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-04-14 19:57:34 +08:00
jaime ed3c29eae7
Support Kafka (#16149) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
 2022-04-12 19:47:33 +08:00
Cai Yudong 71cd7ba67a
Add configuration common.indexSliceSize (#16438) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-04-08 20:29:33 +08:00
godchen bb7a0766fe
Add dependency factory (#16204) 
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
 2022-04-07 22:05:32 +08:00
Ten Thousand Leaves e2eed6fd94
Reduce ID cache duration from 60min to 5min. (#16409) 
/kind improvement

issue: #15593
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-04-07 12:21:31 +08:00
Ten Thousand Leaves 2bd0a9526a
Implement task expiring logic and make all deadlines configurable. (#16355) 
/kind feature

issue: #15604
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-04-03 11:37:29 +08:00
godchen 478f6ca11e
Add cache for vector chunk (#15912) 
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
 2022-03-26 22:05:26 +08:00
Bingyi Sun ca45e0bd00
Add new config file(not applied) (#15891) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
 2022-03-18 15:53:23 +08:00
Ten Thousand Leaves a3a29fed06
Add entry point for embedded Milvus. (#16008) 
/kind feature

issue: #15711
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-03-16 16:51:22 +08:00
Ten Thousand Leaves d5bae6710f
Make all logs configurable for standalone/embedded Milvus. (#15926) 
Resolves: #15919, #15708
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-03-10 10:35:59 +08:00
Cai Yudong 3e862ecdfb
Merge msgChannelConfig and knowhereConfig into commonConfig (#15843) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-03-04 11:17:56 +08:00
Nemo cdcb3627b4
Make Etcd log level configurable. (#15737) 
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-02-25 15:03:53 +08:00
shaoyue 83fdc1da7c
Add web server in proxy with sample handler (#15647) 
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
 2022-02-23 14:37:52 +08:00
JunSung Park f6873d3dc1
Implement automatic item expiration on compaction (#14418) 
* Implement automatic item expiration on compaction

Signed-off-by: JunSung Park <writtic@gmail.com>

* Change entityExpiration value comparison logic

Signed-off-by: JunSung Park <writtic@gmail.com>

* Add logging for the number of expired entity

Signed-off-by: JunSung Park <writtic@gmail.com>

* Set default value of CompactionEntityExpiration to off

Signed-off-by: JunSung Park <writtic@gmail.com>

* Fix errors for code checker

Signed-off-by: JunSung Park <writtic@gmail.com>

* Set default entityExpiration to max value

Signed-off-by: JunSung Park <writtic@gmail.com>
 2022-02-07 22:45:46 +08:00
Cai Yudong f1ea9613ae
Add EtcdConfig and move PulsarConfig/RocksdbConfig/MinioConfig into BaseParamTable (#15434) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-02-07 10:09:45 +08:00
Cai Yudong 4bccc85534
Use MsgChannelConfig in GlobalParams for all components (#15163) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-02-02 00:35:43 +08:00
xige-16 1cd2363bd8
Delete usless params SkipQueryChannelRecovery (#15289) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-01-21 21:43:40 +08:00
XuanYang-cn e69e7de9c2
[skip e2e]Remove not in use config proxy msgStream insert and search (#14877) 
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2022-01-06 15:13:20 +08:00
XuanYang-cn 06ae5b19a3
Remove not inuse configs (#14836) 
- segmentBinlgoSubPath
- collectionBinlogSubPath
- flushStreamPosSubPath
- statsStreamPosSubPath

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2022-01-06 14:49:20 +08:00
Cai Yudong d01931d8d9
[skip e2e] Remove useless configuration rootcoord.timeout (#14852) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-01-05 19:05:25 +08:00
XuanYang-cn c64ad70d43
[skip e2e]Correct comments for proxy config (#14855) 
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2022-01-05 15:31:20 +08:00
zhenshan.cao 32a5126705
Add the limitation of the traceable interval to timeTravel (#14498) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-12-29 21:29:45 +08:00
Cai Yudong 30382d8bf9
Let rootcoord share proxy.TimeTickInterval (#14326) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-12-27 21:38:45 +08:00
ThreadDao 1dd8d2f397
Enable auto compaction (#13086) 
Signed-off-by: ThreadDao <yufen.zong@zilliz.com>
 2021-12-10 14:41:14 +08:00
congqixia aea5a61e9a
Add switch for auto compaction (#12974) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-12-08 19:47:05 +08:00
Bennu cb4711425e
[skip ci] Add comment for milvus.yaml (#12759) 
Signed-off-by: Bennu <yunmei.li@zilliz.com>
 2021-12-08 15:02:09 +08:00
Bennu d85240779f
[skip ci] Add comments for milvus.yaml (#12948) 
Signed-off-by: Bennu <yunmei.li@zilliz.com>
 2021-12-08 13:29:39 +08:00
Bennu 1d0847fb8a
[skip ci] Add comments for milvus.yaml (#12947) 
Signed-off-by: Bennu <yunmei.li@zilliz.com>
 2021-12-08 13:27:48 +08:00
XuanYang-cn f1ac6e2092
Enable compaction in config.yaml (#12844) 
Helm chart and python tests enabled compaction, yet
the milvus.yaml disabled. This caused local smoke tests
failure

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2021-12-07 19:35:13 +08:00
yanliang567 e464708556
[skip ci]Refine comments for milvus yaml (#12866) 
Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
 2021-12-07 14:45:45 +08:00
Bennu 4cd314d41c
[Skip ci] Add comments for milvus.yaml (#12842) 
Signed-off-by: Bennu <yunmei.li@zilliz.com>
 2021-12-07 14:14:08 +08:00
Bennu 86d3784277
[Skip ci] Add comments for milvus.yaml (#12841) 
Signed-off-by: Bennu <yunmei.li@zilliz.com>
 2021-12-07 14:08:12 +08:00
Bennu bcef4a1711
[skip ci] Add comments for milvus.yaml (#12848) 
Signed-off-by: Bennu <yunmei.li@zilliz.com>
 2021-12-07 14:05:46 +08:00
Bennu 69902378cc
[skip ci] Add comment for milvus.yaml (#12760) 
Signed-off-by: Bennu <yunmei.li@zilliz.com>
 2021-12-06 18:05:32 +08:00
Bennu 519fb5d9a2
[skip ci] Add comment for milvus.yaml (#12758) 
Signed-off-by: Bennu <yunmei.li@zilliz.com>
 2021-12-06 18:02:50 +08:00
xige-16 7d723abdff
Start balanceLoop when Params.autoBalance equal to true (#12660) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-12-06 10:19:41 +08:00
Bennu d1aeed2802
[skip ci] Add comments for milvus.yaml (#12676) 
Signed-off-by: Bennu <yunmei.li@zilliz.com>
 2021-12-03 13:34:06 +08:00
Bennu d076f3629e
[skip ci] Add comments for milvus.yaml (#12675) 
Signed-off-by: Bennu <yunmei.li@zilliz.com>
 2021-12-03 13:32:12 +08:00
Bennu 864165689d
[skip ci] Add comments for milvus.yaml (#12674) 
Signed-off-by: Bennu <yunmei.li@zilliz.com>
 2021-12-03 13:30:16 +08:00
Bennu d40f4a6c94
[skip ci] Add comment for milvus.yaml (#12610) 
Signed-off-by: Bennu <yunmei.li@zilliz.com>
 2021-12-02 18:19:33 +08:00
Bennu 856e2f4c9a
[skip ci] Add comment for milvus.yaml (#12609) 
Signed-off-by: Bennu <yunmei.li@zilliz.com>
 2021-12-02 18:17:37 +08:00
Bennu db9387da0e
[skip ci] Add comment for milvus.yaml (#12608) 
Signed-off-by: Bennu <yunmei.li@zilliz.com>
 2021-12-02 18:15:44 +08:00
groot 22f04f90ff
Enlarge field number limitation (#12517) 
Signed-off-by: yhmo <yihua.mo@zilliz.com>
 2021-12-01 15:53:31 +08:00
Bennu 2ebe59e0b4
[skip ci] Add comment for milvus.yaml (#12510) 
Signed-off-by: Bennu <yunmei.li@zilliz.com>
 2021-12-01 13:17:31 +08:00
Bennu b87756b54f
[skip ci] Add comment for milvus.yaml (#12508) 
Signed-off-by: Bennu <yunmei.li@zilliz.com>
 2021-12-01 13:15:35 +08:00
Bennu da0088996b
[Skip ci] Add comments for milvus.yaml (#12429) 
Signed-off-by: Bennu <yunmei.li@zilliz.com>
 2021-12-01 13:09:49 +08:00
Bennu 191f6996f8
[Skip ci] Add comments for milvus.yaml (#12384) 
Signed-off-by: Bennu <yunmei.li@zilliz.com>
 2021-12-01 13:07:53 +08:00
Bennu cbf877a5e2
[skip ci] Add comment for milvus.yaml (#12509) 
Signed-off-by: Bennu <yunmei.li@zilliz.com>
 2021-12-01 13:05:37 +08:00
Bennu 8ea9c00c8a
[Skip ci] Add comments for milvus.yaml (#12430) 
Signed-off-by: Bennu <yunmei.li@zilliz.com>
 2021-11-30 19:26:08 +08:00
groot a4f11c9da5
Change proxy default rpc transfer limitation (#12351) 
Signed-off-by: yhmo <yihua.mo@zilliz.com>
 2021-11-30 15:44:19 +08:00
Bennu aceb08a067
[Skip ci] Add comments for milvus.yaml (#12382) 
Signed-off-by: Bennu <yunmei.li@zilliz.com>
 2021-11-30 14:19:10 +08:00
Bennu 373290d351
[Skip ci] Add comments for milvus.yaml (#12381) 
Signed-off-by: Bennu <yunmei.li@zilliz.com>
 2021-11-30 14:17:12 +08:00
Bennu f034f893c9
[Skip ci] Add comments for milvus.yaml (#12428) 
Signed-off-by: Bennu <yunmei.li@zilliz.com>
 2021-11-30 14:05:57 +08:00
congqixia b99164f43d
Add datacoord GC configuration (#12061) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-11-18 22:29:40 +08:00
Xiaofan d37fdf9379
Remove component.yaml (#11959) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2021-11-17 14:35:10 +08:00
Xiangyu Wang 12f50cb22c
[skip ci]Update OWNERS files (#11898) 
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-11-16 15:41:11 +08:00
shaoyue 5b50731ff4
Fix searchResultBufFlags & queryResultBufFlags memory leak (#11585) 
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
 2021-11-16 14:13:12 +08:00
Bingyi Sun b8c5239784
Add compaction retention config (#11793) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
 2021-11-16 11:47:10 +08:00
Xiaofan 3cd47cd89e
Remove common.yaml and component.yaml (#11662) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2021-11-12 18:53:11 +08:00
xige-16 d857577a7b
Postpone the execution of handoff until index creation is complete (#11648) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-11-12 18:49:10 +08:00
Bingyi Sun 002b854590
Move compaction and garbage switch to milvus.yaml (#11654) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
 2021-11-11 23:38:42 +08:00
Cai Yudong 77b6439c65
[skip ci] Update configuration name (#11656) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-11-11 17:24:41 +08:00
yukun f51ef71b51
Change default retentionSizeInMB to 8192(8GB) (#11436) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-11-11 17:18:43 +08:00
Xiaofan 5f08b67156
Move node.yaml to component.yaml (#11363) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2021-11-10 14:27:37 +08:00
yanliang567 a5305a482e
[skip ci]Update milvus yaml config (#11361) 
Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
 2021-11-08 10:21:04 +08:00
yukun 8ec040a35c
Add rocksmqPageSize to config (#11113) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-11-08 10:17:18 +08:00
Bennu dcf4cecea4
Add comments for milvus.yaml (#11268) 
Signed-off-by: Bennu <yunmei.li@zilliz.com>
 2021-11-06 14:32:56 +08:00
sunby c26cd79532
Add compaction (#9906) 
issue: #9904
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-11-05 22:25:00 +08:00
Xiaofan d81b15163f
Seek Query/Query Result channel to Latest (#10985) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2021-11-05 14:57:44 +08:00
Bennu 9ece3b65e4
Add comments for milvus.yaml (#11269) 
Signed-off-by: Bennu <yunmei.li@zilliz.com>
 2021-11-04 19:19:21 +08:00
Bennu bb16d14ba5
Add comments for milvus.yaml (#11267) 
Signed-off-by: Bennu <yunmei.li@zilliz.com>
 2021-11-04 19:17:30 +08:00
Cai Yudong a2ad88237d
Add deltaChannels in rootcoord core (#11097) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-11-03 21:04:14 +08:00
Xiaofan 385eed1d17
Remove knowhere config (#10614) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2021-11-01 10:26:03 +08:00
Xiaofan 93f205b49e
Rename param names (#10853) 
* Rename param names

Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>

* Rename configs

Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2021-10-28 23:16:40 +08:00
Bennu 2c9776feaa
[skip ci] Add comments for milvus.yaml (#10825) 
Signed-off-by: Bennu <yunmei.li@zilliz.com>
 2021-10-28 20:22:59 +08:00
Bennu 997fed0ddd
[skip ci] Add comments for milvus.yaml (#10827) 
Signed-off-by: Bennu <yunmei.li@zilliz.com>
 2021-10-28 19:51:03 +08:00
Xiaofan cf574b34a0
[skip ci]Add etcd config comment (#10617) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2021-10-25 22:54:21 +08:00
xige-16 9ebabdac79
Add handoffTask (#10330) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-10-24 22:39:09 +08:00
bigsheeper 132f57f5b7
Set gracefulTime to 0 (#10114) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-10-21 10:55:12 +08:00
Bennu b57d7a7508
[skip ci] Add comments for milvus.yaml (#10061) 
Signed-off-by: Bennu <yunmei.li@zilliz.com>
 2021-10-18 18:02:57 +08:00
Bennu b9a904a1d3
[skip ci] Add comments for Milvus.yaml (#9565) 
Signed-off-by: Bennu <yunmei.li@zilliz.com>
 2021-10-12 08:20:33 +08:00
XuanYang-cn 532470a607
[skip ci]Update license for all configs (#9644) 
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2021-10-11 21:36:55 +08:00
XuanYang-cn 43da3e3b1e
[skip ci]Add license for configs (#9643) 
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2021-10-11 21:35:03 +08:00
Bennu e2638a3232
[skip ci] Add comments for Milvus.yaml (#9611) 
Signed-off-by: Bennu <yunmei.li@zilliz.com>
 2021-10-11 21:31:16 +08:00
Bennu ab72753c4b
[skip ci] Add comments for Milvus.yaml (#9609) 
Signed-off-by: Bennu <yunmei.li@zilliz.com>
 2021-10-11 21:29:27 +08:00
Bennu ce4e848663
Add comments for Milvus.yaml (#9553) 
Signed-off-by: Bennu <yunmei.li@zilliz.com>
 2021-10-11 21:21:47 +08:00
Bennu 24de3fa76d
Add comments for Milvus.yaml (#9555) 
Signed-off-by: Bennu <yunmei.li@zilliz.com>
 2021-10-11 21:19:40 +08:00
XuanYang-cn 8d8d53beeb
[skip ci]Change license on config files (#9642) 
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2021-10-11 21:15:00 +08:00
Bennu 8f3b8ed1da
[skip ci] Add comments for Milvus.yaml (#9610) 
Signed-off-by: Bennu <yunmei.li@zilliz.com>
 2021-10-11 16:28:05 +08:00
shiyu22 7caccd36ce
[skip ci]Update the comment of yaml to uppercase (#9502) 
Signed-off-by: shiyu22 <shiyu.chen@zilliz.com>
 2021-10-09 10:55:14 +08:00
Xiangyu Wang 270cdbd4e2
[skip ci]Update reviewers in OWNERS files (#9316) 
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-10-06 07:06:08 +08:00
XuanYang-cn 491744b954
[skip ci]Update comments for configs in datanode (#8976) 
Signed-off-by: Yang Xuan <xuan.yang@zilliz.com>
 2021-10-01 10:51:15 +08:00
bigsheeper 9d6c95bf25
Add cacheSize to prevent OOM in query node (#8765) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-09-28 22:24:03 +08:00
Xiaofan 93fc10f8a6
Remove channel&&common.yaml (#8716) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2021-09-27 22:32:02 +08:00
XuanYang-cn ebd894d8b7
Refactor buffer size calculation (#8565) 
This PR changes the datanode insertBufSize config
from 32000 number of rows to 16MB

Resolves: #7741

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2021-09-26 20:55:59 +08:00
Xiaofan 31ec399edb
Move channel configuration from channel.yaml to milvus.yaml (#8420) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2021-09-26 19:46:17 +08:00
Bennu 607e16f60f
[skip ci] add comments in query_node.yaml (#8585) 
Signed-off-by: Bennu <yunmei.li@zilliz.com>
 2021-09-26 16:53:58 +08:00
Xiaofan dbcacec0a7
[skip ci]Add comments for common conf (#8531) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2021-09-25 11:17:56 +08:00
Xiaofan d7f98ca4bb
[skip ci]Add comments for proxy conf (#8530) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2021-09-25 11:16:02 +08:00
Bennu aec4c98363
[skip ci] add comments in root_coord.yaml (#8507) 
Signed-off-by: Bennu <yunmei.li@zilliz.com>
 2021-09-24 20:55:56 +08:00
sunby 816bac82ec
Add root path in minio and replace etcd.rootPath in datanode (#8272) 
issue: #7905
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-09-24 20:50:23 +08:00
shiyu22 8cc78b2a97
[skip ci] Add comments about minio in yaml (#8493) 
Signed-off-by: shiyu22 <shiyu.chen@zilliz.com>
 2021-09-24 18:40:13 +08:00
Bennu 270200b93c
[skip ci] add comments in root_coord.yaml (#8478) 
Signed-off-by: Bennu <yunmei.li@zilliz.com>
 2021-09-24 16:57:55 +08:00
Bennu 865c67efdb
[skip ci] add comments in proxy.yaml (#8481) 
Signed-off-by: Bennu <yunmei.li@zilliz.com>
 2021-09-24 16:03:55 +08:00
shiyu22 11de20c8e5
[skip ci] Add comments for etcd (#8366) 
Signed-off-by: shiyu22 <shiyu.chen@zilliz.com>
 2021-09-23 16:38:24 +08:00
Xiaofan f5173b595f
Add a channel prefix for all channels (#8166) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2021-09-23 10:53:53 +08:00
shiyu22 51aa4f0254
Add comments with localStorage in yaml (#7892) 
Signed-off-by: shiyu22 <shiyu.chen@zilliz.com>

Add comments with localStorage in yaml

Signed-off-by: shiyu22 <shiyu.chen@zilliz.com>
 2021-09-22 19:23:59 +08:00
dragondriver a3215ceaab
[skip ci] Explain maxShardNum in proxy.yaml (#8236) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-09-18 18:26:08 +08:00
dragondriver 3738655ed0
[skip ci] Explain maxTaskNum in proxy.yaml (#8235) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-09-18 18:24:08 +08:00
dragondriver cd367c83d9
[skip ci] Explain timeTickInterval in proxy.yaml (#8233) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-09-18 18:21:57 +08:00
XuanYang-cn 46c72c57c2
Add BufferData in insertbufferNode (#8138) 
Optimize bufferInsertMsg param to avoid a second transfer
of endposition

See also: #8058, #7741

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2021-09-18 14:25:50 +08:00
Cai Yudong 83bf9c5efb
Add knowhere.yaml to support knowhere configuration (#7967) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-09-18 10:37:51 +08:00
Cai Yudong d10c6abc9e
Apply for msgstream from pool when creating collection (#7738) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-09-18 09:13:50 +08:00
dragondriver 7f048ea8c8
Support to set max task number of task queue (#7974) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-09-15 19:09:50 +08:00
Bennu 511612d3bf
[skip ci] Add comments to the file query_node.yaml (#7916) 
Signed-off-by: Bennu <yunmei.li@zilliz.com>
 2021-09-15 16:05:48 +08:00
Cai Yudong 14ef405a6b
Rename chunk_size to chunk_rows for better code readability (#7953) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-09-15 15:15:52 +08:00
Cai Yudong a0fd2707cb
Support config SIMD type (#7942) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-09-15 12:57:48 +08:00
Cai Yudong e6c384b4d4
Move segcore chunk_size configuration to querynode.yaml (#7913) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-09-15 10:35:52 +08:00
Bennu e28db5600d
[skip ci] Add comments to the file data_node.yaml (#7780) 
Signed-off-by: Bennu <yunmei.li@zilliz.com>
 2021-09-14 16:11:48 +08:00
ryjiang 9a9a1492d0
[skip ci]add comment for segcore config (#7840) 
Signed-off-by: ruiyi.jiang <ruiyi.jiang@zilliz.com>
 2021-09-13 20:52:58 +08:00
Bennu 888c7d7bfb
[skip ci] Add comments to the file query_node.yaml (#7707) 
Signed-off-by: Bennu <yunmei.li@zilliz.com>
 2021-09-11 14:52:01 +08:00
Cai Yudong 27dcf698d3
Support set segcore chunk_size via config file (#7635) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-09-11 14:40:01 +08:00
Bennu 04b8160cac
[skip ci]Add comments to the file data_coord.yaml (#7611) 
Signed-off-by: Bennu <yunmei.li@zilliz.com>
 2021-09-10 23:18:01 +08:00
Cai Yudong 4e23ed9a18
Revert apply physical channels from pool (d5f7e358 and 3b960969) (#7687) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-09-10 13:00:00 +08:00
shiyu22 fc962d856a
Add comments to log in yaml (#7614) 
Signed-off-by: shiyu22 <shiyu.chen@zilliz.com>
 2021-09-09 18:48:02 +08:00
zhenshan.cao b0524adbd8
Support specified shard number when create collection (#7482) 
Add shards_num to CollectionInfo

Modify dmlChannelNum config

Compatible with old meta

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-09-08 15:00:00 +08:00
Cai Yudong 3b9609692b
Apply for msgstream from pool when creating collection (#7377) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-08-31 18:35:58 +08:00
Cai Yudong f4b53df727
Revert "Apply for msgstream from pool when creating collection (#7323)" (#7376) 
This reverts commit e6de86a433.
 2021-08-31 11:07:58 +08:00
Cai Yudong e6de86a433
Apply for msgstream from pool when creating collection (#7323) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-08-31 09:47:58 +08:00
zhenshan.cao 7c60e6aa00
Avoid port conflicts between embedded etcd and external etcd (#7320) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-08-27 10:13:57 +08:00
Xiangyu Wang c62f23e201
[skip ci]Remove neza2017 from reviewers (#7324) 
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-08-27 00:27:36 +08:00
Xiaofan 58b2089692
Import Embedded Etcd Kv Part 1(#7112) (#7174) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2021-08-24 09:45:51 +08:00
godchen 8cc24e09c9
Make users get vectors without using local storage (#7134) 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-08-18 16:30:11 +08:00
yukun 847586eb95
Add rocksmq retention (#6617) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-08-16 18:46:10 +08:00
Xiangyu Wang 5e78fd6a97
[skip ci]Update approvers (#7063) 
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-08-12 19:36:09 +08:00
dragondriver ad77a6e0d4
Fix #6859, increase the MaxCallRecvMsgSize and MaxCallSendMsgSize of grpc client (#6861) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-08-04 13:03:24 +08:00
godchen 346e9cba4a
Add minio FGet method (#6386) 
* Add minio FGet method

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* go fmt

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* change el append

Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-07-09 11:44:40 +08:00
yukun 39614aa8eb
Add rocksmq_path in config and paramtable (#6099) 
* Add rocksmq_path in config and paramtable

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Add rocksdbPath in NewRmsFactory

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Change rdb default path to /vat/lib/milvus/rdb_data

Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-06-25 19:44:11 +08:00
Cai Yudong a09a3a1905
rename service to coord (#6020) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-06-23 16:14:08 +08:00
Cai Yudong 7dbf33b788
Update component names in config files (#5989) 
* rename master/indexService in .yaml

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename proxyNode to proxy in config files

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* move master.yaml to root_coord.yaml, move proxy_node.yaml to proxy.yaml

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* move proto/query_service.proto to proto/query_coord.proto

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* move proxy_node_manager.go to proxy_manager.go

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add query_coord.pb.go

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update ci

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-06-22 19:08:03 +08:00
xige-16 2af1c27811
Rename queryservice to querycoordinator (#5931) 
* Rename queryservice to querycoordinator

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* fix param table

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* code format

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* fix unittest

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* service to coordinator

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* code format

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* set querycoord in ci

Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-06-22 16:44:09 +08:00
sunby d2c8462dac
Rename params in data_coord.yaml (#5968) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-06-22 16:18:21 +08:00
sunby 7f2d7c1da3
Rename dataservice to datacoordinator (#5912) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-06-21 18:22:13 +08:00
sunby da1f9f9241
Rename dataservice to datacoordinator (#5904) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-06-21 11:40:15 +08:00
XuanYang-cn ab7f642740
Add alias in paramtable (#5878) 
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2021-06-19 12:38:06 +08:00
yukun e9a8d1c404
Add vChannels in proxy for query results (#5802) 
* Fix proxynode for new retrieve logic

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Remove querynodenum from proxynode and querynode

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Remove QueryNodeIDList from proxy

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

Co-authored-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-06-16 20:15:59 +08:00
XuanYang-cn 75f2b3c2ba
Add Unittest for allocator (#5781) 
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2021-06-15 19:11:55 +08:00
Cai Yudong 0f4bd50ce3 Support ETCD endpoints array (#5755) 2021-06-15 16:36:36 +08:00
congqixia ef7339ceda DataService stores flush msgstream pos at VChannel granularity (#5397) 
SaveBinLogPaths saves stream positions at VChannel granularity

resolves #5396

Signed-off-by: Congqi Xia congqi.xia@zilliz.com
 2021-06-15 15:54:57 +08:00
Xiangyu Wang ca0993c274 Update OWNERS 
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-05-30 18:58:47 +08:00
congqixia 4aa6a8fe18
Dataservice use lastest grpc design and saves ddl/dml pos pair (#5348) 
Dataservice use lastest grpc design and saves ddl/dml pos pair

* Update SaveBinlogPaths grpc definition
* Saves segment ddl & dml flush pos pair(start, end)

See also #5220 

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-05-21 11:38:58 +00:00
neza2017 c0daf8e41d
set timestamp on dd requests (#5298) 
- set dd request's time tick on master
- send time tick into dd channel

See also: #5291

Signed-off-by: yefu.chen yefu.chen@zilliz.com
 2021-05-20 06:14:14 +00:00
congqixia 38f5b0826f
Add SaveBinlogPaths grpc in dataservice (#5277) 
* Add SaveBinlogPaths grpc in dataservice

* fix merged compile error

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-05-20 11:34:45 +08:00
congqixia d5ab8ac345
Restore flush and stats stream pos (#5284) 
Restore segment flush stream & statistic stream to last success pos

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-05-19 06:13:53 +00:00
XuanYang-cn cebdfb34f7
Enable multiple datanode (#5068) 
Now it's able to start multiple datanode, however, they will process the same insert data.
Further changes are needed to truly enable multiple datanode:

While registering, dataservice need to return different insert channels for different datanode.

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2021-04-28 18:43:48 +08:00
bigsheeper c4b379be8b
Set search graceful time from config (#5014) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-04-24 07:53:37 +00:00
bigsheeper dadb02db75
Remove unused params in query node (#4987) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-04-23 10:07:45 +08:00
XuanYang-cn 5645718311
edit datanode config files (#4992) 
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2021-04-22 11:19:48 +08:00
sunby fad94fb576
Remove unsed configs (#4990) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-04-22 03:04:59 +00:00
sunby b79a408491 Remove unused config item in dataservice and datanode 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-04-19 19:28:45 +08:00
XuanYang-cn e281f6a4a1 Remove useless configs in config files 
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
 2021-04-13 10:04:39 +08:00
zhenshan.cao c68bcfea16 Increase insert buff size 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-03-19 11:51:16 +08:00
zhenshan.cao ab200e5598 Change querynode num to one 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-03-16 21:30:44 +08:00
dragondriver af32f442bb Split big insert message into serveral smaller 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-03-09 16:18:58 +08:00
Cai Yudong c795cdbe2a Change minSegmentSizeToEnableIndex default to 1024 
Signed-off-by: Cai Yudong <yudong.cai@zilliz.com>
 2021-03-07 13:37:25 -06:00
sunby 76480fcefe Change default log output to stdout 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-03-08 16:24:55 +08:00
neza2017 014c4fe8ce Add config of minSegmentSizeToEnableIndex 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2021-03-08 15:46:51 +08:00
dragondriver 801396e3ef Disable register link logic, use proxy node directly 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-03-04 22:27:12 +08:00
bigsheeper d3db2b0d82 Add zap log in query service 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-02-27 10:11:52 +08:00
cai.zhang 1d899513dd Unify service name in config file 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2021-02-24 18:46:19 +08:00
zhenshan.cao 7c5e2d89f6 Read node port from config 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-02-23 18:08:17 +08:00
sunby 5769c000a1 Find failed python tests and add skip mark "r0.3-test" 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-02-19 15:37:04 +08:00
bigsheeper d8f41db823 Fix search crash error 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-02-19 13:56:01 +08:00