milvus-io
/
milvus
mirror of https://github.com/milvus-io/milvus.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
18,542 Commits
146 Branches
120 Tags
575 MiB
Commit Graph

426 Commits (038eebba4db629edd2498abf1b51009a4b91bf22)

Author SHA1 Message Date
yah01 d69440524b
fix: bypass growing index if no index meta (#28791) 
we shouldn't panic if no index meta, just skip building it
fix #28022

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-11-30 14:10:27 +08:00
congqixia 1dc086496f
fix: schema->size() check logic with system field (#28802) 
Now segcore load system field info as well, the growing segment
assertion shall not pass with "+ 2" value
This will cause all growing segments load failure
Fix #28801
Related to #28478
See also #28524

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-11-29 22:40:28 +08:00
zhagnlu 0d9d098186
enhance: Add precheck when chunk manager init (#28330) 
#28329

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2023-11-23 19:56:32 +08:00
Bingyi Sun e8c5e75786
enhance: ignore proto generated files (#28537) 
ignore proto generated files
issue: https://github.com/milvus-io/milvus/issues/28566

---------

Signed-off-by: sunby <sunbingyi1992@gmail.com>
 2023-11-22 10:20:22 +08:00
PowderLi a1c505dbd5
add internal storage metrics (#28278) 
/kind improvement
issue: #28277

Signed-off-by: PowderLi <min.li@zilliz.com>
 2023-11-19 17:22:25 +08:00
yah01 f7d2ab6677
enhance: reduce 1x copy for variable length field while retrieving (#28345) 
- Reduce 1x copy for varchar/string/JSON/array types while retrieving
- Reduce 1x copy for int8/int16 while retrieving

Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-11-15 18:08:20 +08:00
MrPresent-Han 836f300536
support skip-index based on chunk-metrics to accelerate expr filter(#27925) (#28297) 
related: #27925

Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
 2023-11-15 11:20:19 +08:00
Xu Tong 8ec85f5f4c
Add template for VectorMemIndex (#28324) 
Signed-off-by: Writer-X <1256866856@qq.com>
 2023-11-11 13:20:22 +08:00
yah01 267c67dfee
enhance: reduce 1x copy while retrieving data from growing segment (#28323) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-11-10 15:44:22 +08:00
cai.zhang 01c5484817
No hit when the index exceeds the array length (#28301) 
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2023-11-10 10:04:19 +08:00
PowderLi 9f9726f8b7
print azure sdk log (#28240) 
Signed-off-by: PowderLi <min.li@zilliz.com>
 2023-11-08 17:50:18 +08:00
cai.zhang 2b5f632fa4
Fix bug for constructing ArrayView with fixed-length type (#28185) 
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2023-11-07 23:38:21 +08:00
PowderLi 7bb0fa9c70
reduce useless ObjectExists (#28156) 
replace ListBlobs() with GetProperties()
unified style std::string& / char*
config azure requestTimeoutMs

Signed-off-by: PowderLi <min.li@zilliz.com>
 2023-11-07 16:32:20 +08:00
yah01 863e26969a
Reduce 1x memory copy for retrieving data (#28106) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-11-06 17:48:19 +08:00
yah01 dc89730a50
Support collection-level mmap control (#26901) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-11-02 23:52:16 +08:00
Enwei Jiao f8dd589755
Refactor collection's cgo call (#28055) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-11-02 13:02:13 +08:00
Enwei Jiao 8ae9c947ae
Use OpenDAL to access object store (#25642) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-11-01 09:00:14 +08:00
cqy123456 4fbe3c9142
replace loaded binlog with binlog index for search performance (#27673) 
Signed-off-by: cqy123456 <qianya.cheng@zilliz.com>
 2023-11-01 02:20:15 +08:00
yihao.dai ab6b0103a3
Get vector concurrently (#27838) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-10-30 15:44:12 +08:00
Enwei Jiao 4a33391b8f
rename createindex (#27903) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-10-27 10:12:14 +08:00
Alexander Guzhva fbaec4c1a2
Fix non-removed 2 GB temporary file in test_utils.cpp (#27651) 
Signed-off-by: Alexandr Guzhva <alexanderguzhva@gmail.com>
 2023-10-18 02:24:09 +08:00
PowderLi 09d8b76048
[restful] new context with grpc metadata (#27668) 
Signed-off-by: PowderLi <min.li@zilliz.com>
 2023-10-17 20:00:14 +08:00
Xiaofan d83869aaeb
Refine minio chunks manager (#27510) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2023-10-13 14:15:35 +08:00
Gao 7a65b6fb85
Limit faiss ivf index build thread num and fix ut (#27567) 
Signed-off-by: chasingegg <chao.gao@zilliz.com>
 2023-10-11 10:33:33 +08:00
yihao.dai 106c17f304
Make read ahead policy in ChunkCache configurable (#27291) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-09-28 15:47:27 +08:00
Enwei Jiao b80a3e19d3
Add code for PanicInfo (#27364) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-09-27 12:01:28 +08:00
foxspy 5db4a0489e
dynamic index version control (#27335) 
Co-authored-by: longjiquan <jiquan.long@zilliz.com>
 2023-09-25 21:39:27 +08:00
foxspy 370b6fde58
milvus support multi index engine (#27178) 
Co-authored-by: longjiquan <jiquan.long@zilliz.com>
 2023-09-22 09:59:26 +08:00
yah01 93e2eb78c9
Delete only if primary keys exist (#25292) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-09-20 19:03:25 +08:00
cai.zhang a362bb1457
Support array datatype (#26369) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2023-09-19 14:23:23 +08:00
PowderLi 4feb3fa7c6
support azure (#26398) 
Signed-off-by: PowderLi <min.li@zilliz.com>
 2023-09-19 10:01:23 +08:00
yihao.dai bb6711f28c
Add ChunkCache: support get vector from storage (#26142) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-09-15 10:21:20 +08:00
Enwei Jiao 0afdfdb9af
Remove other Exceptions, keeps SegcoreError only (#27017) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-09-14 14:05:20 +08:00
yah01 3d05ddf505
Reduce cpp test time (#27043) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-09-13 15:41:18 +08:00
cqy123456 0ff4ddc76c
remove VectorMemNMIndex (#27000) 
Signed-off-by: cqy123456 <qianya.cheng@zilliz.com>
 2023-09-12 17:13:18 +08:00
Enwei Jiao c3f15c6b95
Refactor duplicate error class into one place (#26985) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-09-11 20:43:17 +08:00
Xu Tong 9166011c4a
Add float16 vector (#25852) 
Signed-off-by: Writer-X <1256866856@qq.com>
 2023-09-08 10:03:16 +08:00
Cai Yudong 8dc16b599b
Add binary metric types SUBSTRUCTURE/SUPERSTRUCTURE back (#26766) 
Signed-off-by: Yudong Cai <yudong.cai@zilliz.com>
 2023-08-31 20:07:00 +08:00
yah01 ba882b49b6
Optimize query/search on growing segment while output vector field (#26542) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-08-24 09:46:24 +08:00
zhagnlu 0cb3f88e73
Fix unit test case of thread pool (#26501) 
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2023-08-21 21:50:22 +08:00
zhagnlu 79ca04b995
fix unstable test case for thread pool (#26465) 
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2023-08-18 22:42:20 +08:00
cai.zhang c073aa0dc3
Fix bug for json_contains_all has multiple array elements (#26446) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2023-08-18 22:36:19 +08:00
cqy123456 fd37860e57
init knowhere build/search thread pool; (#26449) 
Signed-off-by: cqy123456 <qianya.cheng@zilliz.com>
 2023-08-18 14:00:19 +08:00
zhagnlu bdc8c507ea
Refactor thread pool to dynamic threads mode (#26407) 
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2023-08-18 11:58:19 +08:00
xige-16 50a77ef1f7
Fix serialize string index failed when greater than 2G (#26392) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2023-08-17 21:16:19 +08:00
zhagnlu 709352f96c
Repeat multi times for performace compare test because of cpu cache (#26394) 
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2023-08-17 15:18:17 +08:00
Jiquan Long 95bcf3e900
Fix error message of proto parser not clear (#26365) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2023-08-16 20:20:17 +08:00
Patrick Weizhi Xu 09da953a19
Use Knowhere AIO Context Init Defalut Value and Panic when Fail (#26286) 
Signed-off-by: Patrick Weizhi Xu <weizhi.xu@zilliz.com>
 2023-08-13 20:53:30 +08:00
xige-16 5b8d716cbc
Add ut for growing segment load binlog (#26268) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2023-08-13 20:41:31 +08:00
cai.zhang a0198ce8ae
Support json contains feature (#25384) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2023-08-11 17:09:30 +08:00