milvus-io
/
milvus
mirror of https://github.com/milvus-io/milvus.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
16,465 Commits
127 Branches
113 Tags
553 MiB
Commit Graph

296 Commits (baa035885d7a9fbd9d29612b0164ec4a6130dca9)

Author SHA1 Message Date
cai.zhang 0ad15a22c8
IndexCoord handle events correctly (#17878) 
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
 2022-07-07 14:44:21 +08:00
Xiaofan a9b1d71a8f
change the etcd compareAndSwap interface (#18068) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-07-06 13:54:21 +08:00
cai.zhang 899a9bf86d
IndexNode still writes finished state to ETCD when task was deleted (#17761) 
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
 2022-06-28 12:08:17 +08:00
Enwei Jiao 16c3aedc15
refine complie configuration (#17502) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2022-06-24 21:12:15 +08:00
cai.zhang 3685ff9e0c
Mark indexing task to retry when minio occurs error (#17153) 
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
 2022-05-23 15:47:58 +08:00
cai.zhang e05cfe194e
Fix data race for indexcoord test (#17057) 
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
 2022-05-19 17:25:56 +08:00
cai.zhang b37b87eb97
The GetIndexFilePaths interface judges whether the index is complete (#17052) 
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
 2022-05-17 20:41:56 +08:00
cai.zhang b0c9f25352
Sort out the log of IndexCoord (#16810) 
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
 2022-05-09 12:05:52 +08:00
Cai Yudong 6a62ff18bf
Support easylogging config for segcore and knowhere (#16751) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-05-03 08:39:49 +08:00
xige-16 515d0369de
Support string type in segcore (#16546) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
Co-authored-by: dragondriver <jiquan.long@zilliz.com>

Co-authored-by: dragondriver <jiquan.long@zilliz.com>
 2022-04-29 13:35:49 +08:00
zhenshan.cao 0a953948af
Simplify monitoring metrics (#16687) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2022-04-27 23:03:47 +08:00
Xiaofan b657b10eb5
Remove useless log (#16624) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-04-27 10:01:47 +08:00
Xiaofan 89b4a34892
Fix Data race in NodeID (#16603) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-04-24 22:03:44 +08:00
Ji Bin a5ac637ba5
Adaptation windows for Knowhere 1.1.3 (#16444) 
fixes #16443

Signed-off-by: Ji Bin <matrixji@live.com>
 2022-04-13 14:39:34 +08:00
Cai Yudong 71cd7ba67a
Add configuration common.indexSliceSize (#16438) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-04-08 20:29:33 +08:00
godchen bb7a0766fe
Add dependency factory (#16204) 
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
 2022-04-07 22:05:32 +08:00
cai.zhang e73139deeb
Compressed IndexNode log (#16365) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2022-04-06 11:33:31 +08:00
cai.zhang 04076393c5
Fix log error (#16363) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2022-04-06 10:13:32 +08:00
Jiquan Long f8d9bc919d
Unify interface of vector index & scalar index. (#15959) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2022-03-21 14:23:24 +08:00
godchen 78557ca6d7
Replace minio kv with minio chunk manager (#15936) 
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
 2022-03-17 18:03:23 +08:00
Ji Bin 3cd28420f1
Support compile under windows (#15786) 
This patch makes compile milvus under windows(MSYS), including:
- some cpp adaptation for compile under msys/gcc-10.3
- install toolchain scripts for setup from MinGW/MSYS `scripts/install_deps_msys.sh`
- adaptation for POSIX API use in golang
  * using gofrs/flock instead of syscall.Flock
  * using x/exp/mmap instead of syscall.Mmap
- introducing github actions for build milvus.exe under windows/MSYS
- rocksdb's patch for MSYS
- adaptation for compile knowhere under windows
- a windows package script for pack zip file, `scripts/package_windows.sh`

issue #7706

Signed-off-by: Ji Bin <matrixji@live.com>
 2022-03-17 17:17:22 +08:00
zhenshan.cao f3eeecf146
Reduce the number of metrics (#16030) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2022-03-15 21:51:21 +08:00
cai.zhang 2512dff261
Fix misuse of timerecord (#15868) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2022-03-04 15:45:56 +08:00
Cai Yudong 3e862ecdfb
Merge msgChannelConfig and knowhereConfig into commonConfig (#15843) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-03-04 11:17:56 +08:00
cai.zhang 09a27d1620
Add prometheus metrics for indexnode (#15640) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2022-03-02 20:49:56 +08:00
zhenshan.cao f1aec49a74
Fixbug: building index return nil by mistake when error not occur (#15809) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2022-03-02 09:41:56 +08:00
zhenshan.cao 142848fcc3
Abandon using protobuf to pass binaryset parameter (#15626) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2022-02-18 18:39:50 +08:00
jaime 307a8ce535
Support compile and run on Mac (#15491) 
Co-authored-by: jaime <yun.zhang@zilliz.com>
Co-authored-by: Cai Yudong <yudong.cai@zilliz.com>
Co-authored-by: Jenny Li <jing.li@zilliz.com>
Co-authored-by: Nemo <yuchen.gao@zilliz.com>
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Co-authored-by: Cai Yudong <yudong.cai@zilliz.com>
Co-authored-by: Jenny Li <jing.li@zilliz.com>
Co-authored-by: Nemo <yuchen.gao@zilliz.com>
 2022-02-09 14:27:46 +08:00
Cai Yudong 1f337891e1
Rename base_param to service_param, rename global_param to component_param (#15463) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-02-08 20:57:47 +08:00
Cai Yudong f1ea9613ae
Add EtcdConfig and move PulsarConfig/RocksdbConfig/MinioConfig into BaseParamTable (#15434) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-02-07 10:09:45 +08:00
zhenshan.cao 3f48a40ab0
Load index meta in every retry (#15376) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2022-01-25 13:28:05 +08:00
zhenshan.cao 6bc2efe429
Fixbug: IndexNode should panic when save meta failed to MetaKV (#15347) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2022-01-24 17:18:46 +08:00
zhenshan.cao 871e093dc8
Call release memory in case error occured (#15237) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2022-01-17 14:43:38 +08:00
Jiquan Long 2819f6cc6f
Release memory to OS after index is built (#15196) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2022-01-13 20:27:34 +08:00
cai.zhang 89c9073ab3
[skip e2e]Add comment for msgIndexNodeIsUnhealthy (#15142) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2022-01-12 11:11:36 +08:00
cai.zhang 4daf0fb6b5
[skip e2e]Add comment for errIndexNodeIsUnhealthy (#15141) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2022-01-12 11:09:36 +08:00
Cai Yudong 951249c6c1
Use KnowhereConfig for indexnode and querynode (#15112) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-01-11 09:35:34 +08:00
zhenshan.cao 1aafb56694
Clarify the index building process (#15044) 
Reduce peak memory usage

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2022-01-10 20:51:34 +08:00
Cai Yudong b6a48817b0
Add MinioConfig in GlobalParams for all components (#15099) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-01-10 17:29:34 +08:00
congqixia fc256e7924
Index parallel respect CPU quota (#14991) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-01-07 17:31:22 +08:00
Cai Yudong f8f61fb116
Use BaseParams etcd configuration for all components (#15012) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-01-07 16:15:26 +08:00
Jiquan Long 32eed4f025
Check leakage of index CGO object (#14630) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2022-01-06 23:28:15 +08:00
cai.zhang 2327d53915
[skip e2e]Add comment for setID (#14841) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2022-01-05 13:07:25 +08:00
cai.zhang 47b89ba84a
[skip e2e]Add comment for checkIndexMeta (#14842) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2022-01-05 13:05:25 +08:00
shaoyue fc415c0fb5
[skip e2e] Add comment for indexnode.Params (#14721) 
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
 2022-01-04 10:45:31 +08:00
cai.zhang d4c325adbb
[skip e2e]Reorder head files for indexnode/task.go (#14661) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2021-12-31 16:11:24 +08:00
cai.zhang f1c3f465a5
[skip e2e]Reorder head files for indexnode/metrics_info.go (#14660) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2021-12-31 16:09:18 +08:00
cai.zhang f96cd84efb
[skip e2e]Reorder head files for indexnode/indexnode_test.go (#14609) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2021-12-30 17:02:50 +08:00
cai.zhang 911fcf83a2
[skip e2e]Reorder head files for indexnode/indexnode_mock_test.go (#14608) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2021-12-30 17:01:07 +08:00
cai.zhang 4297ff9bd0
[skip e2e]Reorder head files for indexnode/error_test.go (#14607) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2021-12-30 16:36:24 +08:00
Xiaofan 69087ff8dd
Support embedded etcd (#14333) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2021-12-29 14:35:21 +08:00
Xiaofan 1408926cd9
Remove logs without enough info (#14440) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2021-12-29 10:06:47 +08:00
edward.zeng 9977bea982
[skip e2e] Refine indexnode task scheduler comment (#14450) 
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
 2021-12-28 19:50:25 +08:00
xige-16 a8cb995216
Improve log for build index in indexNode (#14051) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-12-23 22:00:16 +08:00
Cai Yudong 891c202b73
Use GlobalParamTable for all components (#13885) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-12-23 18:39:11 +08:00
JackLCL 2c0d9397fb
[skip e2e]Fix error log output format in task.go (#14028) 
Signed-off-by: JackLCL <chenglong.li@zilliz.com>
 2021-12-23 14:28:25 +08:00
cai.zhang 1dab9ccc7f
[skip e2e]Add commont for initMinIOAccessKeyID in indexnode (#13975) 
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
 2021-12-22 17:11:35 +08:00
cai.zhang 595692c350
[skip e2e]Add commont for initMinIOAddress in indexnode (#13973) 
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
 2021-12-22 17:03:43 +08:00
cai.zhang 9f23fc7f2a
Register the service when the component state is healthy (#13248) 
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
 2021-12-15 11:47:10 +08:00
JackLCL 7cb35e4172
[skip ci]Fix error log output format in task.go (#13102) 
Signed-off-by: JackLCL <chenglong.li@zilliz.com>
 2021-12-09 19:45:28 +08:00
cai.zhang 342200ce13
Estimate the memory size of the index before building the index (#12973) 
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
 2021-12-09 14:19:40 +08:00
yanliang567 694e3f72a1
[skip ci]Refine etcd name in comments (#12798) 
Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
 2021-12-06 18:59:55 +08:00
congqixia 38956215aa
Update indexnode state to unhealthy before stopping (#12744) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-12-06 10:39:36 +08:00
godchen 5e3e3b3b41
Remove grpc returned error (#12537) 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-12-01 22:17:46 +08:00
jaime 90a1d016a0
Refine return value (#11730) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Co-authored-by: yun.zhang <yun.zhang@zilliz.com>
 2021-11-30 19:32:17 +08:00
dragondriver b6d0150dc5
Fix health check when disconnect to etcd (#12292) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-11-26 11:39:16 +08:00
dragondriver 6c4c0ef6b5
Add more deploy metrics (#12248) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-11-24 15:43:15 +08:00
godchen 9d5bcd3e3a
Close event and binlog reader (#12173) 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-11-22 17:27:14 +08:00
congqixia 4121e31df1
Send SIGINT to runner goroutine after etcd disconnects (#12163) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-11-22 16:23:17 +08:00
dragondriver ee0f753f7a
Fix datarace between GetComponentStates and Register (#11935) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-11-19 13:57:12 +08:00
Baiqiang Zhao ba9977951a
Fix log in indexnode (#11988) 
Signed-off-by: Baiqiang Zhao <zhaobaiqiang@apache.org>
 2021-11-17 16:41:13 +08:00
congqixia 5edbb82610
Add session revoke (#11908) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-11-16 22:31:14 +08:00
Xiangyu Wang 12f50cb22c
[skip ci]Update OWNERS files (#11898) 
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-11-16 15:41:11 +08:00
Cai Yudong 95fff75dec
Use API HandleCStatus to handle CStatus returned by CGo in indexnode (#11788) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-11-15 15:27:10 +08:00
Xiaofan 5f08b67156
Move node.yaml to component.yaml (#11363) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2021-11-10 14:27:37 +08:00
XuanYang-cn 35e8779bd9
Rename IndexRootPath to IndexStorageRootPath (#11236) 
IndexRootPath is index file's blob storage prefix.
But the name is confusing with the MetaRootPath, which is prefix of etcd.

This PR changes the IndexRootPath to IndexStorageRootPath to
elimilate the confusion.

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2021-11-05 11:45:00 +08:00
XuanYang-cn a086042b45
[skip ci]Update in task (#11032) 
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2021-11-01 22:51:41 +08:00
XuanYang-cn 41e720aad8
[skip ci]Update license for in pt (#11031) 
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2021-11-01 22:49:57 +08:00
XuanYang-cn e6a87ba94f
[skip ci]Update license for in in (#11030) 
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2021-11-01 22:48:06 +08:00
XuanYang-cn 8566d64f4a
[skip ci]Update license for in errors (#11028) 
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2021-11-01 22:46:00 +08:00
Xiaofan 385eed1d17
Remove knowhere config (#10614) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2021-11-01 10:26:03 +08:00
Xiaofan 2df2d488e1
Fix Unit test fatal fail (#10933) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2021-10-30 10:24:38 +08:00
godchen bdbe9067f2
Check indexnode mock error (#10204) 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-10-29 11:48:39 +08:00
Xiaofan 93f205b49e
Rename param names (#10853) 
* Rename param names

Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>

* Rename configs

Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2021-10-28 23:16:40 +08:00
godchen 92dfacc2d7
Check index node test error (#10205) 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-10-28 17:24:29 +08:00
Xiaofan 6fddcb3925
Fix Session checker for proxy (#10737) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2021-10-27 21:58:33 +08:00
cai.zhang aff5cd9598
Supprot to read the configuration of minio from environment variables (#10534) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2021-10-26 15:10:03 +08:00
zhenshan.cao 83684628b2
Fix bug:Index building task canceled by indexcoord by mistake (#10314) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-10-20 21:02:35 +08:00
groot d33e810aa1
[skip ci] Fix a typo (#10155) 
Signed-off-by: yhmo <yihua.mo@zilliz.com>
 2021-10-19 16:24:35 +08:00
congqixia ec6e8e4127
Hide liveChn into sessionutil and fix liveness init order (#9860) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-10-14 16:40:35 +08:00
dragondriver 72d3d470ba
Use the server id as identifier in system topology graph (#9825) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-10-14 11:12:38 +08:00
Xiaofan 91d8f85df7
Refine log in task.go (#8992) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2021-10-13 23:24:32 +08:00
NotRyan 539fac4f83
Log language changes in Indexnode (#9692) 
Signed-off-by: NotRyan <ryan.chan@zilliz.com>

Co-authored-by: NotRyan <ryan.chan@zilliz.com>
 2021-10-13 10:54:33 +08:00
cai.zhang 6938e7b8e6
[skip ci]Fix golint in indexnode/param_table.go (#9629) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2021-10-11 19:17:02 +08:00
cai.zhang 7801357dd3
Retry index building task in case of internal error such as minio failure (#9552) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2021-10-11 16:38:33 +08:00
cai.zhang 6b58c5930e
[skip ci]Fix golint for BaseTask in indexnode.go (#9495) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2021-10-09 18:27:00 +08:00
cai.zhang e368b51d3e
[skip ci]Fix golint for task executing in indexnode.go (#9494) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2021-10-09 18:24:58 +08:00
godchen faac37e726
Remove index registration after ut (#9558) 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-10-09 15:47:00 +08:00
cai.zhang 3b7b10c18e
[skip ci]Fix golint for IndexBuildTask in indexnode.go (#9493) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2021-10-09 00:05:49 +08:00
Xiaofan 0f65b845d2
Refine Index Create Log (#9233) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2021-10-07 22:20:50 +08:00