Commit Graph

334 Commits (2ff469771c47078b00b9f37a55dfef5cf7484b6d)

Author SHA1 Message Date
Enwei Jiao 958e94f6f0
Use Conan as c++ package manager (#19920)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>

Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2022-11-23 10:39:11 +08:00
Jenny Li 44aed995ae
[skip e2e] Add time caculate for cpp ut && go ut (#20552)
Signed-off-by: Jenny Li <jing.li@zilliz.com>

Signed-off-by: Jenny Li <jing.li@zilliz.com>
2022-11-14 14:43:06 +08:00
wei liu e7df499397
fix make test-querycoord (#20116)
Signed-off-by: wei.liu <wei.liu@zilliz.com>

Signed-off-by: wei.liu <wei.liu@zilliz.com>
2022-10-27 17:23:32 +08:00
SimFG b213297d06
Fix the `stop` error when no `milvus` process (#20101)
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
2022-10-26 21:13:32 +08:00
SimFG 1b7968557c
Improve the `stop` command (#19981)
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
2022-10-21 18:45:30 +08:00
SimFG e10d21ef90
Improve the `download_milvus_proto` bash (#19894)
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
2022-10-19 20:45:32 +08:00
jaime b79687687d
Replace segment index file path with index file id within segment index meta (#19866)
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
2022-10-19 16:55:27 +08:00
SimFG 2b512a3371
Improve the making milvus process (#19833)
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
2022-10-17 19:19:25 +08:00
SimFG a55f739608
Separate public proto files (#19782)
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
2022-10-16 20:49:27 +08:00
Jenny Li 9aa307d851
Clean unnecessary deps to make milvus (#19711)
Signed-off-by: Jenny Li <jing.li@zilliz.com>

Signed-off-by: Jenny Li <jing.li@zilliz.com>
2022-10-12 14:33:23 +08:00
Jiquan Long a8a074162f
Add meta migration tool (#19709)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2022-10-12 11:37:23 +08:00
jaime 31db01b6ae
Support TTL for collection level (#19383)
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
2022-10-10 20:31:22 +08:00
SimFG a85e44d169
Support the `RELEASE` param to update the milvus/api version dynamically (#19656)
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
2022-10-09 14:28:59 +08:00
zhenshan.cao 0ac4bc32a5
Add logic of autoindex (#19570)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-10-08 15:38:58 +08:00
xige-16 6466291953
Add make option to compile for disk index (#19530)
Signed-off-by: xige-16 <xi.ge@zilliz.com>

Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-09-29 11:08:54 +08:00
Jiquan Long 22477d4601
UnWatch channels if failed to create collection (#19390)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2022-09-23 16:56:50 +08:00
xige-16 428840178c
Support diskann index for vector field (#19093)
Signed-off-by: xige-16 <xi.ge@zilliz.com>

Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-09-21 20:16:51 +08:00
SimFG d7f38a803d
Separate some proto files (#19218)
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
2022-09-16 16:56:49 +08:00
Bingyi Sun 626854cf0c
Refactor QueryCoord (#18836)
Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: yah01 <yang.cen@zilliz.com>
Co-authored-by: Wei Liu <wei.liu@zilliz.com>
Co-authored-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: yah01 <yang.cen@zilliz.com>
Co-authored-by: Wei Liu <wei.liu@zilliz.com>
Co-authored-by: Congqi Xia <congqi.xia@zilliz.com>
2022-09-15 18:48:32 +08:00
xige-16 4de1bfe5bc
Add cpp data codec (#18538)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
Co-authored-by: zhagnlu lu.zhang@zilliz.com

Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-09-09 22:12:34 +08:00
zhenshan.cao b512072b64
Add missed unittest of querycoord (#19120)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-09-08 20:38:35 +08:00
zhenshan.cao 43d367b7d8
Add submodule unit test command entry (#19108)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-09-08 17:36:35 +08:00
Jiquan Long a5e2d6b6fb
Refactor RootCoord (#18930)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
Co-authored-by: xaxys <tpnnghd@163.com>

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
Co-authored-by: xaxys <tpnnghd@163.com>
2022-09-05 13:29:11 +08:00
congqixia 43c14055f6
Fix test compact logic and data race (#18932)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-08-31 12:10:57 +08:00
Match_yc 2ef8b15dde
add libaio as a necessary dependency (#18923)
Signed-off-by: matchyc <dawnlight.yc@protonmail.com>

Signed-off-by: matchyc <dawnlight.yc@protonmail.com>
2022-08-31 10:24:58 +08:00
SimFG 69c0b2fb49
Improve the storage method of the grant info in the metastore (#18817)
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
2022-08-26 19:22:56 +08:00
cai.zhang c924f73105
Refactor for IndexCoord to support cloud (#18643)
Co-authored-by: Zach41 <zongmei.zhang@zilliz.com>
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>

Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
Co-authored-by: Zach41 <zongmei.zhang@zilliz.com>
2022-08-25 15:48:54 +08:00
XuanYang-cn 769606cce2
Add zstd lib in docker images (#18823)
See also: #18304, #18335

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-08-25 14:24:53 +08:00
Ji Bin f3c755602b
Adaptation for latest MinGW64 (#18699)
Signed-off-by: Ji Bin <matrixji@live.com>

Signed-off-by: Ji Bin <matrixji@live.com>
2022-08-25 10:56:53 +08:00
SimFG ce434b496e
Implement the mysql metastore of the rbac (#18704)
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
2022-08-23 10:26:53 +08:00
codeman 0a84326581
Modify name length and delete unique constraint for index (#18685) (#18687)
Signed-off-by: kejiang <ke.jiang@zilliz.com>

Signed-off-by: kejiang <ke.jiang@zilliz.com>
Co-authored-by: kejiang <ke.jiang@zilliz.com>
2022-08-17 14:10:49 +08:00
jaime 74a87f8000
Disable ut about kafka temporarily (#18611)
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
2022-08-12 13:30:38 +08:00
bigsheeper 004ba37436
Set LD_PRELOAD env for jemalloc (#18612)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-08-11 19:50:37 +08:00
codeman 6c3dbf0a07
Support meta storing on database (#17236) (#18541)
Signed-off-by: kejiang <ke.jiang@zilliz.com>

Signed-off-by: kejiang <ke.jiang@zilliz.com>
Co-authored-by: kejiang <ke.jiang@zilliz.com>
2022-08-11 12:12:38 +08:00
jaime 4a32c1bb8e
Enable kafka ut with a mocked kafka server (#18366)
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
2022-08-03 19:06:35 +08:00
Su Yang 554994d8d7
Remove invalid build parameters for openblas (https://github.com/milvus-io/milvus/issues/18360) (#18464)
Signed-off-by: soulteary <soulteary@gmail.com>
2022-08-02 11:32:33 +08:00
Enwei Jiao aab060aeff
upgrade Go 1.18 in go.mod (#18165)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2022-08-01 14:28:33 +08:00
Enwei Jiao 283f5731d2
config from etcd (#18421)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2022-08-01 10:04:33 +08:00
Bingyi Sun 1fd3ded815
Add querycoord v2 dir and proto (#18032)
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
2022-07-04 16:56:20 +08:00
Enwei Jiao ea901393ef
fix windows ldflags (#17807)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2022-06-28 19:58:25 +08:00
Enwei Jiao 16c3aedc15
refine complie configuration (#17502)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2022-06-24 21:12:15 +08:00
bigsheeper cdcdfa1ea5
Disable jemalloc and use malloc_trim instead (#17538)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-06-14 11:12:09 +08:00
bigsheeper c77bc81057
Set jemalloc LD_PRELOAD env to linux image (#17494)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-06-10 21:58:08 +08:00
bigsheeper 2d9a52206d
Use jemalloc in QueryNode, DataNode and IndexNode (#17470)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-06-10 11:24:08 +08:00
Ji Bin 572cd0277f
[skip e2e] Lock msys package's versions (#17414)
Signed-off-by: Ji Bin <matrixji@live.com>
2022-06-07 19:56:05 +08:00
zhenshan.cao 5f44e45480
Add metrics on QueryNode and Proxy (#17328)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-06-02 16:06:03 +08:00
zhenshan.cao 5ad89ff4e8
Add offline build logic for arrow dependency (#17266)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-05-30 14:54:02 +08:00
SimFG 731870211a
Support role-based access control (only pb) (#16987)
Signed-off-by: SimFG <bang.fu@zilliz.com>
2022-05-28 00:04:01 +08:00
Ten Thousand Leaves 5e1e7a6896
Fix an issue that causes flush hanging (#17169)
issue: #16628
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-05-25 14:34:00 +08:00
Jiquan Long 8a958e1ea3
Fix missing dependencies in install_deps.sh (#17186)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2022-05-24 17:19:59 +08:00
Jiquan Long 98ceb162aa
Support antlr as plan parser (#16696)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>

Co-authored-by: xaxys <tpnnghd@163.com>

Co-authored-by: xaxys <tpnnghd@163.com>
2022-05-06 17:43:51 +08:00
Ji Bin abdbc65988
[skip e2e] Force go 1.17 for Code Checker MinGW (#16797)
- Also lock rocksdb to 6.26

Signed-off-by: Ji Bin <matrixji@live.com>
2022-05-06 16:27:50 +08:00
jaime 68b1b82faf
Remove DataKV interface (#16692)
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
2022-04-28 21:03:47 +08:00
Ten Thousand Leaves 4855ba6fa3
Optimize Mac M1 support some tweaks to embedded Milvus (#16650)
issue: #16500
issue: #16621

/kind enhancement

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-04-26 14:21:46 +08:00
groot bc5e9ec929
Add list import tasks api (#16605)
Signed-off-by: groot <yihua.mo@zilliz.com>
2022-04-25 17:37:46 +08:00
Ten Thousand Leaves f93862fa2b
Fix build and tests for Mac M1, also enabled more tests for OSX systems (#16552)
issue: #16500

/kind enhancement

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-04-20 23:03:41 +08:00
Ten Thousand Leaves 65e01bce90
[skip e2e] Add dependency install script for embedded Milvus (#16529)
issue: #15711

/kind enhancement

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-04-20 17:25:41 +08:00
Ten Thousand Leaves e76a8c5ec2
Update compile options and config for embedded Milvus (#16472)
/kind enhancement

issue: #15711
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-04-14 19:57:34 +08:00
jaime ed3c29eae7
Support Kafka (#16149)
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
2022-04-12 19:47:33 +08:00
groot be8d9a8b6b
Support import numpy file (#16348)
Signed-off-by: groot <yihua.mo@zilliz.com>
2022-04-03 11:27:29 +08:00
Ji Bin 3cd28420f1
Support compile under windows (#15786)
This patch makes compile milvus under windows(MSYS), including:
- some cpp adaptation for compile under msys/gcc-10.3
- install toolchain scripts for setup from MinGW/MSYS `scripts/install_deps_msys.sh`
- adaptation for POSIX API use in golang
  * using gofrs/flock instead of syscall.Flock
  * using x/exp/mmap instead of syscall.Mmap
- introducing github actions for build milvus.exe under windows/MSYS
- rocksdb's patch for MSYS
- adaptation for compile knowhere under windows
- a windows package script for pack zip file, `scripts/package_windows.sh`

issue #7706

Signed-off-by: Ji Bin <matrixji@live.com>
2022-03-17 17:17:22 +08:00
edward.zeng d4ad785b76
Bump go version in documentation (#15979)
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
2022-03-16 18:51:21 +08:00
Ten Thousand Leaves a3a29fed06
Add entry point for embedded Milvus. (#16008)
/kind feature

issue: #15711
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-03-16 16:51:22 +08:00
groot 5fdef607d5
Add rpc interfaces for import (#15930)
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2022-03-09 18:33:59 +08:00
jaime 7a3a721380
Reconstruct mqstream module (#15784)
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
2022-03-03 21:57:56 +08:00
Nemo a77e0562b2
Make C++ log level configurable and add log configuration while building embedded milvus. (#15743)
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-03-01 10:15:55 +08:00
Jenny Li b19602903a
[skip e2e]Add ccache into install mac deps (#15675)
Signed-off-by: Jenny Li <jing.li@zilliz.com>
2022-02-22 09:33:51 +08:00
jaime 307a8ce535
Support compile and run on Mac (#15491)
Co-authored-by: jaime <yun.zhang@zilliz.com>
Co-authored-by: Cai Yudong <yudong.cai@zilliz.com>
Co-authored-by: Jenny Li <jing.li@zilliz.com>
Co-authored-by: Nemo <yuchen.gao@zilliz.com>
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Co-authored-by: Cai Yudong <yudong.cai@zilliz.com>
Co-authored-by: Jenny Li <jing.li@zilliz.com>
Co-authored-by: Nemo <yuchen.gao@zilliz.com>
2022-02-09 14:27:46 +08:00
Jenny Li 97eff51b00
[skip e2e] Update comment to uppercase (#14710)
Signed-off-by: Jenny Li <jing.li@zilliz.com>
2022-01-04 19:13:45 +08:00
xige-16 0ddf890a32
Print test info when run ut (#14470)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-12-29 13:49:20 +08:00
godchen 675e4dcaa2
Remove dablooms (#13881)
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
2021-12-21 18:24:58 +08:00
Jenny Li 27a0fab3b8
[skip ci] Add comment for run_cpp_codecov.sh (#13477)
Signed-off-by: Jenny Li <jing.li@zilliz.com>
2021-12-16 09:15:27 +08:00
groot c0e719119c
[skip ci] Fix script bug (#13401)
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2021-12-15 10:33:35 +08:00
dragondriver e7552cc570
Optimize the compile workflow of storage (#13103)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-12-14 10:33:21 +08:00
XuanYang-cn 383f14999d
Make Arrow compilation easier (#13005)
replace all the bundled libraries with arrow_bundled_dependencies

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-12-08 20:55:05 +08:00
Jenny Li cf07b46bca
[skip ci] Add comment for go-run-unittest.sh (#11948)
Signed-off-by: Jenny Li <jing.li@zilliz.com>
2021-11-17 10:37:16 +08:00
Xiangyu Wang 12f50cb22c
[skip ci]Update OWNERS files (#11898)
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-11-16 15:41:11 +08:00
edward.zeng 2527a74104
[skip ci] Add comment for script installing dependencies (#11834)
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
2021-11-15 19:59:36 +08:00
Xiaofan 33df88a3a2
[skip ci]Fix typo in stop.sh (#10637)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2021-10-26 14:14:39 +08:00
shiyu22 c28085a381
Update the README (#9942)
Signed-off-by: shiyu22 <shiyu.chen@zilliz.com>
2021-10-15 15:22:34 +08:00
shiyu22 251ffea494
[skip ci] Update the comments (#9633)
Signed-off-by: shiyu22 <shiyu.chen@zilliz.com>
2021-10-11 19:19:02 +08:00
zwd1208 9c01d08d14
[skip ci]Update minio uppercase (#9663)
Signed-off-by: Weida Zhu <weida.zhu@zilliz.com>
2021-10-11 18:58:29 +08:00
XuanYang-cn 4a32f1b0fc
[skip ci]Add LF AI&Data foundation license (#9507)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-10-09 18:01:27 +08:00
edward.zeng 06483babef
[skip ci] Refine the script comment (#9548)
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
2021-10-09 17:55:42 +08:00
zwd1208 711031640f
[skip ci]Update scripts README.md (#9474)
Signed-off-by: Weida Zhu <weida.zhu@zilliz.com>
2021-10-08 18:05:23 +08:00
edward.zeng 5079878baf
[skip ci] Fix script indentation (#9453)
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
2021-10-08 13:20:59 +08:00
groot e4e7c05b3b
[skip ci] Modify script description (#9402)
Signed-off-by: groot <yihua.mo@zilliz.com>
2021-10-07 22:14:57 +08:00
edward.zeng d278fffca1
[skip ci] Refine scripts readme (#9408)
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
2021-10-07 18:34:42 +08:00
groot 4be169635d
[skip ci] Modify script description (#9401)
Signed-off-by: groot <yihua.mo@zilliz.com>
2021-10-07 18:01:28 +08:00
groot 9bc2c9f419
[skip ci] Modify script description (#9400)
Signed-off-by: groot <yihua.mo@zilliz.com>
2021-10-07 17:42:49 +08:00
groot e03b6be678
[skip ci] Modify script description (#9338)
Signed-off-by: groot <yihua.mo@zilliz.com>
2021-10-06 19:28:47 +08:00
groot e4dc44b41f
[skip ci] Modify script description (#9337)
Signed-off-by: groot <yihua.mo@zilliz.com>
2021-10-06 19:26:54 +08:00
groot e2a3d0a71c
[skip ci] Modify script/readme (#9336)
Signed-off-by: groot <yihua.mo@zilliz.com>
2021-10-06 19:25:06 +08:00
groot 418c195ae9
[skip ci] Modify description of script (#9256)
Signed-off-by: groot <yihua.mo@zilliz.com>
2021-10-05 12:32:26 +08:00
groot 9982e107de
[skip ci] Modify description of script (#9255)
Signed-off-by: groot <yihua.mo@zilliz.com>
2021-10-05 12:28:13 +08:00
groot f4aaf4ee04
[skip ci] Modify description of script (#9254)
Signed-off-by: groot <yihua.mo@zilliz.com>
2021-10-05 12:26:02 +08:00
groot 7b5d9a3c7a
[skip ci] Modify description of script (#9186)
Signed-off-by: groot <yihua.mo@zilliz.com>
2021-10-04 09:06:06 +08:00
groot 033befa2bc
[skip ci] Modify description of script (#9185)
Signed-off-by: groot <yihua.mo@zilliz.com>
2021-10-04 09:03:55 +08:00
groot 94af9ec47d
[skip ci] Add description to coverage script (#9138)
Signed-off-by: Yihua <michaelmo@YihuadeMacBook-Pro.local>

Co-authored-by: Yihua <michaelmo@YihuadeMacBook-Pro.local>
2021-10-03 17:48:10 +08:00
groot 234fd9b512
[skip ci] Fix script comment (#9062)
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2021-10-02 11:21:56 +08:00