milvus-io
/
milvus
mirror of https://github.com/milvus-io/milvus.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
19,497 Commits
125 Branches
112 Tags
560 MiB
Commit Graph

247 Commits (1cd15d9322f8b3f0437b01467a42253321171cf4)

Author SHA1 Message Date
gcmutator 6edd06083f
chore: remove repetitive words (#31153) 
Signed-off-by: gcmutator <329964069@qq.com>
 2024-03-20 10:17:07 +08:00
yiwangdr 85246c1f5d
enhance: disallow tikv/rawkv usages (#30027) 
Data write through rawkv API may pollute tikv data. It should be
disallowed.
We will add this check to all repos that involves metadata access.
In the longer term, we should have a metadata service that implements
access control.

relate: #30029

Signed-off-by: yiwangdr <yiwangdr@gmail.com>
 2024-02-06 10:41:41 +08:00
PowderLi 08ca0a2ca5
feat: support etcd authentication (#30226) 
issue: #28895
add 3 configuration for ETCD config

Signed-off-by: PowderLi <min.li@zilliz.com>
 2024-01-24 11:35:00 +08:00
wei liu 9fc5f1176c
fix: Drop segment meta info with prefix (#29856) 
If segment has more than 128 log fils, drop segment will exceed etcd txn
ops limit, which will failed the drop segment request
This PR drop segment meta info with prefix, to avoid drop segment meta
failed

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-01-11 12:22:51 +08:00
congqixia a2fe9dad49
enhance: Make etcd kv request timeout configurable (#28661) 
See also #28660
This pr add request timeout config item for etcd kv request timeout
 Sync the default timeout value to same value for etcdKV & tikv config

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-11-23 19:34:23 +08:00
yah01 c96d07682e
enhance: enhance the accuracy of memory usage (#28554) 
before this, Milvus use container/system's memory info to get the memory
usage, which could be inaccurate.

we allocates the memory by private anon mmap,
then `rss - shared` would be the accurate memory usage

resolve #28553

---------

Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-11-23 15:12:23 +08:00
Bingyi Sun d7145e2c06
enhance: Update golangci_lint version (#28535) 
Update golangci lint and fix some warnings

Signed-off-by: sunby <sunbingyi1992@gmail.com>
 2023-11-21 10:04:21 +08:00
Xiaofan 38aa53f523
Fix datacoord start timeout (#28026) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2023-10-31 10:04:12 +08:00
yah01 15746b5b5f
Refine KV errors (#27588) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-10-13 17:15:34 +08:00
congqixia 5d558623fe
Add revive sub-lints and fix existing problems (#27495) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-10-07 20:53:38 +08:00
congqixia cd5f03f80c
Add var-name sub linter in revive (#27424) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-10-07 10:09:31 +08:00
congqixia a3dd2756cf
Add predicates for TxnKV operations (#27365) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-09-27 10:21:26 +08:00
jaime 7f7c71ea7d
Decoupling client and server API in types interface (#27186) 
Co-authored-by:: aoiasd <zhicheng.yue@zilliz.com>

Signed-off-by: jaime <yun.zhang@zilliz.com>
 2023-09-26 09:57:25 +08:00
SimFG 26f06dd732
Format the code (#27275) 
Signed-off-by: SimFG <bang.fu@zilliz.com>
 2023-09-21 09:45:27 +08:00
yiwangdr f85af0732c
remove MultiRemoveWithPrefix (#26924) 
Signed-off-by: yiwangdr <yiwangdr@gmail.com>
 2023-09-12 15:07:17 +08:00
yiwangdr 337edc321b
tikv integration (#26246) 
Signed-off-by: yiwangdr <yiwangdr@gmail.com>
 2023-09-07 07:25:14 +08:00
Enwei Jiao fb0705df1b
Decouple basetable and componentparam (#26725) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-09-05 10:31:48 +08:00
congqixia 1a8cf5c415
Organize all mockery generation commands in Makefile (#26826) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-09-04 21:19:48 +08:00
smellthemoon eae02de8bd
Change the key to make read faster in rocksmq (#26404) 
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2023-08-21 12:58:20 +08:00
Bingyi Sun 6539396bbe
Add HasPrefix in kv (#25462) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@zilliz.com>
 2023-07-11 10:28:28 +08:00
Bingyi Sun f75201bbfd
Add Has interface in kv (#25439) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@zilliz.com>
 2023-07-10 19:40:27 +08:00
xige-16 33c2012675
Add more metrics (#25081) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2023-06-26 17:52:44 +08:00
yiwangdr c7b851f870
add interface for non-watch metakv (#25092) 
Signed-off-by: yiwangdr <yiwangdr@gmail.com>
 2023-06-26 09:20:44 +08:00
yiwangdr 4387f36897
make etcdKV private (#24778) 
Signed-off-by: yiwangdr <yiwangdr@gmail.com>
 2023-06-13 10:52:38 +08:00
congqixia b77bee6708
Remove unused iterator initialization (#24743) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-06-08 19:10:35 +08:00
yah01 ebd0279d3f
Check error by Error() and NoError() for better report message (#24736) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-06-08 15:36:36 +08:00
yiwangdr 37c02c9927
clean up meta interface (#24729) 
Signed-off-by: yiwangdr <yiwangdr@gmail.com>
 2023-06-08 14:18:35 +08:00
SimFG d246defa4d
Return error response when an unimplemented request is received (#24545) 
Signed-off-by: SimFG <bang.fu@zilliz.com>
 2023-05-31 20:24:31 +08:00
hulk 1aeff84293
Use MGet to query multi keys in RocksDB KV (#24362) 
RocksDB can performance optimizations such as async-io by io_uring.
This would benefit the performance a lot when keys are lying in
different levels of the LSM tree.

Signed-off-by: git-hulk <hulk.website@gmail.com>
 2023-05-26 11:07:27 +08:00
jaime c9d0c157ec
Move some modules from internal to public package (#22572) 
Signed-off-by: jaime <yun.zhang@zilliz.com>
 2023-04-06 19:14:32 +08:00
Enwei Jiao 697dedac7e
Use cockroachdb/errors to replace other error pkg (#22390) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-02-26 11:31:49 +08:00
wayblink 88293b8b79
Add log to analysis when etcd txn operation failed (#21558) 
Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2023-02-02 15:57:51 +08:00
congqixia 5986106037
Make paramtable init only once (#21782) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-01-19 14:53:44 +08:00
Jiquan Long 9fd9bed2b9
Remove collection meta after GC finished (#21595) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2023-01-12 09:55:42 +08:00
jaime 58b79eb74c
Add based on timetravel GC for snapshot KV (#21417) 
Signed-off-by: jaime <yun.zhang@zilliz.com>
 2023-01-04 21:37:35 +08:00
Enwei Jiao 2ecdb4ba4a
Etcd config source support TLS (#20874) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>

Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2022-11-30 18:23:15 +08:00
Enwei Jiao c05b9ad539
Add event dispatcher for config (#20393) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>

Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2022-11-17 18:59:09 +08:00
XuanYang-cn 22fb1d3b93
Remove dup mock_kv.go and mock_txn_kv.go (#20267) 
Replace them with mockery generated mocks in kv/mocks

See also: #19289

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2022-11-03 14:41:35 +08:00
Eng Zer Jun 438790744d
test: use `T.Setenv` to set env vars in tests (#19649) 
This commit replaces `os.Setenv` with `t.Setenv` in tests. The
environment variable is automatically restored to its original value
when the test and all its subtests complete.

Reference: https://pkg.go.dev/testing#T.Setenv
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
 2022-10-11 18:43:23 +08:00
Ten Thousand Leaves b30c9d4f60
Support Bulk Load in Milvus 2.2 (#18982) 
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-09-26 18:06:54 +08:00
XuanYang-cn 253fb514bd
Add Revert in DataCoord meta catalog (#19313) 
See also: #19072

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2022-09-23 10:50:51 +08:00
Bingyi Sun 626854cf0c
Refactor QueryCoord (#18836) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: yah01 <yang.cen@zilliz.com>
Co-authored-by: Wei Liu <wei.liu@zilliz.com>
Co-authored-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: yah01 <yang.cen@zilliz.com>
Co-authored-by: Wei Liu <wei.liu@zilliz.com>
Co-authored-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-09-15 18:48:32 +08:00
Jiquan Long a5e2d6b6fb
Refactor RootCoord (#18930) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
Co-authored-by: xaxys <tpnnghd@163.com>

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
Co-authored-by: xaxys <tpnnghd@163.com>
 2022-09-05 13:29:11 +08:00
jaime 816967c000
Add metrics for etcd (#18962) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
 2022-09-02 17:46:59 +08:00
jaime 237d912625
Add log for the large value during etcd saving (#18229) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
 2022-09-01 09:47:02 +08:00
SimFG 1366f130a2
Add a specific `KeyNotExistError` err for the metastore `load` interface (#18582) 
Signed-off-by: SimFG <bang.fu@zilliz.com>
 2022-08-10 17:20:41 +08:00
Jiquan Long e7b3bacbec
Refine catalog of partition & alias. (#18546) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2022-08-10 10:22:38 +08:00
Ten Thousand Leaves 097f144db1
Improve slow Etcd logs by specifying keys (#18547) 
/kind improvement

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-08-09 21:14:37 +08:00
Enwei Jiao 283f5731d2
config from etcd (#18421) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2022-08-01 10:04:33 +08:00
Xiaofan a9b1d71a8f
change the etcd compareAndSwap interface (#18068) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-07-06 13:54:21 +08:00