milvus-io
/
milvus
mirror of https://github.com/milvus-io/milvus.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
16,271 Commits
146 Branches
120 Tags
573 MiB
Commit Graph

212 Commits (3c9840d6d3daff609a57cf71f44642c657f60e60)

Author SHA1 Message Date
Xiaofan b6cca25d1f
Refine rocksdb (#12743) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2021-12-17 23:44:42 +08:00
jaime 0a99e1aeba
[skip ci] Fix go format error (#13212) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Co-authored-by: yun.zhang <yun.zhang@zilliz.com>
 2021-12-11 22:23:13 +08:00
yukun 197bcbba08
[skip ci]Format rocksdbkv log (#13191) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-12-10 21:16:04 +08:00
yukun f6ec741114
[skip ci]Format rocksdbkv log (#13190) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-12-10 21:14:14 +08:00
jaime ea7d8be913
[skip ci] Refine error message (#11728) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Co-authored-by: yun.zhang <yun.zhang@zilliz.com>
 2021-12-10 16:50:52 +08:00
yukun c71b4d32f6
[skip ci]Format rocksdbkv log (#13094) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-12-10 10:08:45 +08:00
yukun 506ca42c5a
[skip ci]Format rocksdbkv log (#13093) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-12-10 10:06:31 +08:00
jaime 7691876fff
[skip ci] Fix golint error in mem_kv (#13020) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Co-authored-by: yun.zhang <yun.zhang@zilliz.com>
 2021-12-09 10:49:25 +08:00
jaime efaca29ee4
[skip ci] fix a grammar error of the comment in etcd_kv (#13019) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Co-authored-by: yun.zhang <yun.zhang@zilliz.com>
 2021-12-09 09:23:08 +08:00
shaoyue 34fadc9ab6
[skip ci] Add comment for EmbedEtcdKV (#12964) 
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
 2021-12-08 15:11:33 +08:00
edward.zeng d3c23ca5e9
[skip ci] Refine etcd kv comment (#12872) 
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
 2021-12-07 15:55:03 +08:00
edward.zeng d1c7a1574d
[skip ci] Refine rocksdb kv comment (#12867) 
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
 2021-12-07 14:47:37 +08:00
jaime e4746e8c6e
[skip ci] Add a function comment for LoadWithPrefix2 (#12828) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Co-authored-by: yun.zhang <yun.zhang@zilliz.com>
 2021-12-07 09:42:15 +08:00
edward.zeng 1d2311d4da
[skip ci] Refine minio kv comment (#12724) 
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
 2021-12-03 19:21:33 +08:00
shaoyue bbdd1dbef9
[skip ci] Add comment for etcd_kv.go (#12449) 
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
 2021-11-30 16:58:14 +08:00
shaoyue f00ee2a3b1
[skip ci] Add comment for mem_kv.go (#12448) 
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
 2021-11-30 16:08:59 +08:00
edward.zeng c9d31f0b73
[skip ci] Refine etcd kv comment (#12374) 
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
 2021-11-29 19:25:49 +08:00
edward.zeng 772857d9c1
[skip ci] Refine minio kv comment (#12369) 
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
 2021-11-29 17:55:41 +08:00
bigsheeper 18463ebde4
Reduce memory cost when load from minio (#12055) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-11-23 14:49:15 +08:00
edward.zeng 3cfc740a05
Refine minio kv FGetObject function (#12078) 
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
 2021-11-21 22:53:13 +08:00
edward.zeng 8f40450d93
[skip ci] Refine rocksdb kv comment (#12093) 
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
 2021-11-18 19:27:13 +08:00
edward.zeng a92a6f34b4
[skip ci] Fix typo in etcd kv comment (#12091) 
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
 2021-11-18 19:25:19 +08:00
edward.zeng 77264ae493
[skip ci] Refine etcd kv comment (#12090) 
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
 2021-11-18 19:23:25 +08:00
yukun 3b51232d59
[skip ci]Format rocksdbkv (#12034) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-11-17 22:03:24 +08:00
yukun 18b9f7810c
[skip ci]Format rocksdbkv (#12033) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-11-17 22:01:31 +08:00
congqixia b7de3bda3b
Fix mem_kv.go slice 0 length declaration (#11996) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-11-17 20:57:17 +08:00
edward.zeng 2b566eb622
[skip ci] Refine comments in mem kv (#12006) 
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
 2021-11-17 19:27:27 +08:00
edward.zeng bcb7cce026
[skip ci] Refine comment in minio kv (#12004) 
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
 2021-11-17 19:25:20 +08:00
edward.zeng 776a01b5b9
[skip ci] Add comment to embed etcd kv (#12003) 
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
 2021-11-17 19:23:25 +08:00
shaoyue 6e5334ab11
[skip ci] Add comments for exported funcs in mem_kv.go (#11979) 
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
 2021-11-17 16:11:16 +08:00
shaoyue 585d9be439
[skip ci] Add comments for exported funcs in etcd_kv.go (#11975) 
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
 2021-11-17 16:09:23 +08:00
shaoyue 6542b092b2
[skip ci] Add comments of exported items in etcd_kv.go (#11884) 
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
 2021-11-16 17:41:11 +08:00
Xiangyu Wang 12f50cb22c
[skip ci]Update OWNERS files (#11898) 
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-11-16 15:41:11 +08:00
shaoyue 53c29088b0
[skip ci] Add comment for exported funcs in mem_key.go (#11877) 
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
 2021-11-16 13:23:10 +08:00
yukun 5e19629b3b
[skip ci]Format rocksdbkv (#11854) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-11-16 09:23:45 +08:00
yukun 36966504b1
[skip ci]Format rocksdbkv (#11853) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-11-16 09:21:48 +08:00
yukun 8593d75fd1
[skip ci]Format rocksdbkv (#11852) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-11-16 09:19:55 +08:00
XuanYang-cn 72c0711919
Refine minio kv test (#11255) 
This PR refines the minio_kv_test, makes sure the minIO test
is in the scope of configed rootPath

See also: #11237

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2021-11-05 23:05:10 +08:00
Xiaofan 2b0f1a14ac
Remove secretkey and accesskey log print for MinIO (#11130) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2021-11-03 13:04:09 +08:00
dragondriver 232f75a001
[skip ci] Fix golint warning of kv (#11087) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-11-02 21:40:12 +08:00
dragondriver c208363b0d
[skip ci] Fix golint warning of MinIOKV (#10913) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-10-29 17:51:28 +08:00
XuanYang-cn 41f1378a01
[skip ci]Update license for kv kv (#10896) 
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2021-10-29 13:15:04 +08:00
XuanYang-cn 848fa77d13
[skip ci]Update license for kv rocksdb (#10895) 
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2021-10-29 13:13:15 +08:00
XuanYang-cn 005d177f68
[skip ci]Update license for kv etcdkv (#10894) 
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2021-10-29 13:11:24 +08:00
XuanYang-cn 978229517d
[skip ci]Update license for kv mem (#10893) 
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2021-10-29 13:09:33 +08:00
XuanYang-cn 08cd25c14f
[skip ci]Update license for kv minio (#10892) 
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2021-10-29 13:07:43 +08:00
XuanYang-cn 8885c510ef
[skip ci]Update license for kv etcd (#10891) 
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2021-10-29 12:12:44 +08:00
congqixia 95b8953bac
Add kv Value interface (#10469) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-10-29 00:16:38 +08:00
zhenshan.cao 5320698e29
[skip ci]Fix golint error in mem_kv (#10862) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-10-28 23:54:55 +08:00
zhenshan.cao 25551ffa2f
[skip ci]Fix golint error in memory_kv (#10861) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-10-28 23:42:39 +08:00
Xiaofan 93f205b49e
Rename param names (#10853) 
* Rename param names

Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>

* Rename configs

Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2021-10-28 23:16:40 +08:00
BIKI DAS 1f2f76789e
Combined function params (#10786) 
Signed-off-by: Biki-das <bikid475@gmail.com>
 2021-10-28 15:58:29 +08:00
zhenshan.cao d598c78bbf
[skip ci]Fix golint error in minio_kv (#10702) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-10-26 22:42:25 +08:00
zhenshan.cao dcac9c71b1
[skip ci]Fix golint error in minio_kv (#10701) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-10-26 22:40:34 +08:00
zhenshan.cao 4ca4c81ce3
[skip ci]Fix golint error in minio_kv (#10700) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-10-26 22:38:44 +08:00
zhenshan.cao 58d202f200
[skip ci]Fix golint error in minio_kv (#10699) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-10-26 22:36:53 +08:00
cai.zhang 5b42a3223c
Increase compatibility for EstimateMemorySize interface (#10603) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2021-10-26 15:34:21 +08:00
congqixia ed2b8d67c0
Reduce info saved in SnapshotMeta (#10288) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-10-21 14:04:36 +08:00
godchen 8daeb0d519
Change bloom filter use pk (#10193) 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-10-19 20:18:47 +08:00
zhenshan.cao e97b62cf67
Change RocksDBLRUCache size to 0 (#9820) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-10-13 20:28:34 +08:00
Cai Yudong c7566345eb
Simplify meta_snapshot interface (#9778) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-10-13 15:54:33 +08:00
dragondriver 41bd866ff6
Support LoadPartial interface for DataKV (#9554) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-10-09 19:33:03 +08:00
yukun a6e97e0d44
Change LRUCache size of rocksdb to 512MB (#9509) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-10-09 11:08:56 +08:00
yukun d9115fe78d
Set PinL0FilterAndIndexBlocksInCache to true for rocksdb (#8855) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-10-08 10:19:06 +08:00
zhenshan.cao 405fe29b0b
[skip ci] fix golint error in metakv_factory.go (#9152) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-10-03 20:52:20 +08:00
yukun 2e77c1d1cd
Format rocksdb_kv code (#8982) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-09-30 22:40:14 +08:00
yukun e85cbea18d
Set CacheIndexAndFilterBlocks to true for rocksdb (#8851) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-09-29 18:45:59 +08:00
godchen 167069c810
[skip ci]Add kv comment (#8880) 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-09-29 18:39:59 +08:00
yukun c438001a62
Fix rocksdb_kv value free (#8798) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-09-28 22:34:04 +08:00
yukun 7c2729916b
Fix rocksdb_kv error process (#8744) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-09-28 20:36:04 +08:00
yukun 93fa86daf2
Fix rocksdb_kv memory leak (#8743) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-09-28 19:36:04 +08:00
yukun aea7cc1630
Add rocksdb_kv ut (#8769) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-09-28 17:48:08 +08:00
ZhiShen 0e767f0e1b
Change cmake version (#8666) 
Change cmake version from 3.16 to 3.18

1. Change the CMakeList's minimum require version.

2. Change the describtion document of cmake version.

Resolves: milvus-io#8407

Signed-off-by: Shen Zhi <m13120163046@163.com>
 2021-09-28 14:00:03 +08:00
Xiangyu Wang ea53d52dba
Fix golint warnings in rockskv (#8723) 
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-09-28 08:28:02 +08:00
cai.zhang 171f362b86
Use recursion to delete objects in minio (#8671) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2021-09-27 16:20:06 +08:00
groot e4e762f2e8
Reduce go unittest time cost (#8503) 
Signed-off-by: yhmo <yihua.mo@zilliz.com>
 2021-09-26 12:50:24 +08:00
Xiaofan 2f92f78e7b
Remove meaningless Etcd kv logs (#8019) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2021-09-22 20:27:54 +08:00
yukun 06c9f03c59
Refactor rocksmq close logic and improve codecov (#8278) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-09-22 17:21:00 +08:00
godchen c8c10bb3d6
Remove performanceTest (#8164) 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-09-18 10:13:52 +08:00
godchen 74bb712527
[skip ci]Add minio comment (#8165) 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-09-17 23:59:47 +08:00
yukun 5a303e7672
Fix rocksmq load with prefix (#7678) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-09-10 10:22:01 +08:00
sunby 62f4cb5e1b
Remove stats watcher (#7501) 
issue: #7500
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-09-07 16:11:07 +08:00
dragondriver 960741ee16
Support more detailed metrics information (#7472) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-09-03 19:35:47 +08:00
zhenshan.cao 7c60e6aa00
Avoid port conflicts between embedded etcd and external etcd (#7320) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-08-27 10:13:57 +08:00
Xiangyu Wang c62f23e201
[skip ci]Remove neza2017 from reviewers (#7324) 
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-08-27 00:27:36 +08:00
sunby 4d56d53c0b
Add a tool to move binlog path to SegmentInfo (#7243) 
issue: #7214
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-08-24 15:51:51 +08:00
Xiaofan 58b2089692
Import Embedded Etcd Kv Part 1(#7112) (#7174) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2021-08-24 09:45:51 +08:00
XuanYang-cn 9e71daa759
Fix rocksdb not installed problem (#7200) 2021-08-20 17:46:48 +08:00
Cai Yudong 305fa9c17e
Fix timestamp go back issue for timeticksync (#7123) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-08-18 14:36:10 +08:00
yukun 847586eb95
Add rocksmq retention (#6617) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-08-16 18:46:10 +08:00
XuanYang-cn 6c5be83ea1
Refactor NewEtcdKV API (#7075) 
This PR mainly did the following 3 things:

- Refactor NewEtcdKV API
- Add unittests for `kv/etcd/`, raise the coverage
up to 94%
- Correct some APIs from version to revision

The old NewEtcdKV() API took `clientv3.Client` as
one of the input, making everyone who using this API
had to create a new `clientv3.Client`. So there're a lot
of repeated codes everywhere.

Meanwhile, someplace didn't even check whether the
`clientv3.New()` returned error or not, and the old API
also didn't check whether the clientv3.Client was Nil.
This caused issue#6955.

Ectd's version and revision have nothing alike, they
represent different concepts, but our APIs kind of
mixed them all up. So I correct those APIs' names.

Resolves: #6955

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2021-08-13 11:04:09 +08:00
Xiangyu Wang 5e78fd6a97
[skip ci]Update approvers (#7063) 
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-08-12 19:36:09 +08:00
congqixia 4d5eeb2890
Fix minio object leaking goroutine (#6944) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-08-02 15:25:24 +08:00
neza2017 5adb69161f
Set CollectionInfo.CreateTime to the timestamp when save the metadata into etcd, (#6763) 
not the timestamp when RootCoord receive the CreateCollection Request

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
 2021-07-23 14:36:12 +08:00
cai.zhang a3c662c7bf
Add watchWithVersion interface for etcdKV (#6663) 
* Add watchWithVersion interface for etcdKV

Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>

* Add unittest

Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>

* Add LoadWithVersion interface

Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>
 2021-07-21 18:20:11 +08:00
godchen b9f5949680
Add vector chunk manager (#6613) 
* add file manager

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* add file manager

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* add vector file manager

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* rename

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* go fmt

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* fix ut error

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* fix action error

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* fix memory error

Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-07-20 15:06:09 +08:00
cai.zhang f3a5b46288
Fix bug for repeatedly create bucket in minio (#6477) 
* Fix bug for repeatedly create bucket in minio

Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>

* Simplify code

Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>

* Add log

Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>
 2021-07-14 09:55:46 +08:00
godchen 346e9cba4a
Add minio FGet method (#6386) 
* Add minio FGet method

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* go fmt

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* change el append

Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-07-09 11:44:40 +08:00
quicksilver 7d920e9c9c
Fix compile error on CentOS (#6359) 
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
 2021-07-08 15:43:45 +08:00
godchen 99be4c09a5
change retry (#5996) 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-06-23 09:24:10 +08:00