milvus-io
/
milvus
mirror of https://github.com/milvus-io/milvus.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
19,141 Commits
111 Branches
170 Tags
585 MiB
Commit Graph

48 Commits (8a219e0102a50c4197d1eb3fbcf6db1bf0cf222f)

Author SHA1 Message Date
Cai Yudong 8a219e0102
feat: Support knowhere trace using OpenTelemetry (#30750) 
Issue: #21508

Signed-off-by: Yudong Cai <yudong.cai@zilliz.com>
 2024-02-28 12:29:00 +08:00
MrPresent-Han 77eb6defb1
feat: support groupby on growing and non-indexed sealed egment(#30307) (#30644) 
related: #30308

Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
 2024-02-21 14:02:53 +08:00
zhagnlu 976b6fc0e4
enhance: change opendal as compile configurable (#30384) 
#30373

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2024-02-20 19:16:52 +08:00
MrPresent-Han 92d1d744ae
fix: groupby results lack good results(#29883) (#30428) 
related: #29883

Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
 2024-02-06 17:08:34 +08:00
Patrick Weizhi Xu 0907d76253
enhance: pass partition key scalar info if enabled when build vector index (#29931) 
issue: #29892 

Pass optional scalar IVF offsets to Cardinal

Signed-off-by: Patrick Weizhi Xu <weizhi.xu@zilliz.com>
 2024-01-24 00:04:55 +08:00
Bingyi Sun 8030b90891
fix: correct file name when loading index (#29985) 
issue: #29973

Signed-off-by: sunby <sunbingyi1992@gmail.com>
 2024-01-16 10:24:52 +08:00
yah01 f2e36db488
enhance: optimize the loading index performance (#29894) 
this utilizes concurrent loading

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2024-01-12 17:44:51 +08:00
Xu Tong e429965f32
Add float16 approve for multi-type part (#28427) 
issue：https://github.com/milvus-io/milvus/issues/22837

Add bfloat16 vector, add the index part of float16 vector.

Signed-off-by: Writer-X <1256866856@qq.com>
 2024-01-11 15:48:51 +08:00
congqixia d6429933a7
enhance: make Load process traceable in querynode & segcore (#29858) 
See also #29803

This PR:
- Add trace span for `LoadIndex` & `LoadFieldData` in segment loader
- Add `TraceCtx` parameter for `Index.Load` in segcore
- Add span for ReadFiles & Engine Load for Memory/Disk Vector index

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-01-10 21:58:51 +08:00
MrPresent-Han 9e2e7157e9
feat: support search_group_by for milvus(#25324) (#28983) 
related: #25324

Search GroupBy function, used to aggregate result entities based on a
specific scalar column.
several points to mention:

1. Temporarliy, the whole groupby is implemented separated from
iterative expr framework **for the first period**
2. In the long term, the groupBy operation will be incorporated into the
iterative expr framework:https://github.com/milvus-io/milvus/pull/28166
3. This pr includes some unrelated mocked interface regarding alterIndex
due to some unworth-to-mention reasons. All these un-associated content
will be removed before the final pr is merged. This version of pr is
only for review
4. All other related details were commented in the files comparison

Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
 2024-01-05 15:50:47 +08:00
yah01 aef483806d
enhance: improve the segcore logs (#29372) 
- remove the streaming logging
- refine existing logs

fix #29366

---------

Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-12-23 21:52:43 +08:00
yah01 8f89e9cf75
enhance: remove all unnecessary string formatting (#29323) 
done by two regex expressions:
- `PanicInfo\((.+),[. \n]+fmt::format\(([.\s\S]+?)\)\)`
- `AssertInfo\((.+),[. \n]+fmt::format\(([.\s\S]+?)\)\)`

related: #28811

---------

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-12-20 10:04:43 +08:00
zhagnlu a602171d06
enhance: Refactor runtime and expr framework (#28166) 
#28165

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2023-12-18 12:04:42 +08:00
Bingyi Sun 36f69ea031
feat: integrate storagev2 in building index of segcore (#28768) 
issue: https://github.com/milvus-io/milvus/issues/28655

---------

Signed-off-by: sunby <sunbingyi1992@gmail.com>
 2023-12-05 16:48:54 +08:00
Xu Tong 8ec85f5f4c
Add template for VectorMemIndex (#28324) 
Signed-off-by: Writer-X <1256866856@qq.com>
 2023-11-11 13:20:22 +08:00
Enwei Jiao b80a3e19d3
Add code for PanicInfo (#27364) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-09-27 12:01:28 +08:00
foxspy 5db4a0489e
dynamic index version control (#27335) 
Co-authored-by: longjiquan <jiquan.long@zilliz.com>
 2023-09-25 21:39:27 +08:00
foxspy 370b6fde58
milvus support multi index engine (#27178) 
Co-authored-by: longjiquan <jiquan.long@zilliz.com>
 2023-09-22 09:59:26 +08:00
Enwei Jiao 0afdfdb9af
Remove other Exceptions, keeps SegcoreError only (#27017) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-09-14 14:05:20 +08:00
Enwei Jiao c3f15c6b95
Refactor duplicate error class into one place (#26985) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-09-11 20:43:17 +08:00
yah01 9004601817
Enable vector index mmap (#26750) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-08-31 17:31:11 +08:00
MrPresent-Han 7d5a4b2994
add more event for segcore search(#26277) (#26688) 
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
 2023-08-30 14:15:01 +08:00
yah01 300fef446b
Enable mmap for vector index (#25877) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-08-10 13:59:15 +08:00
yah01 12a7747a95
Fix miss index file while slice meta contains raw data only (#26190) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-08-09 10:55:15 +08:00
yah01 07f08daf1a
Fix failed to load index due to lost binary (#26135) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-08-07 14:53:07 +08:00
MrPresent-Han 5634ba777d
add new threadpool with various priority to avoid deadlock(#25781) (#26028) 
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
 2023-08-03 09:31:07 +08:00
yah01 9618bd9b42
Set channel capacity before consuming it (#25895) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-07-26 17:35:01 +08:00
yah01 227d2c8b3a
Reduce loading index memory usage (#25698) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-07-19 14:02:57 +08:00
yah01 cb721781f3
Improve error message throwed from knowhere (#25473) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-07-11 17:26:29 +08:00
yihao.dai 37fe3393d1
Remove const to enable move semantics and improve schema util functions (#25193) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-06-29 11:04:22 +08:00
xige-16 04082b3de2
Migrate the ability to upload and download binlog to cpp (#22984) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2023-06-25 14:38:44 +08:00
Cai Yudong 1b3c4b26f1
Optimize range search result sort in segcore (#24837) 
Signed-off-by: Yudong Cai <yudong.cai@zilliz.com>
 2023-06-13 19:22:38 +08:00
Cai Yudong 58192f55f6
Update knowhere version for new GetVectorByIds interface (#24034) 
Signed-off-by: Yudong Cai <yudong.cai@zilliz.com>
 2023-05-11 18:05:20 +08:00
foxspy 6f4ed517de
add growing segment index (#23615) 
Signed-off-by: xianliang <xianliang.li@zilliz.com>
 2023-04-26 10:14:41 +08:00
yihao.dai 092d743917
Add support for getting vectors by ids (#23450) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-04-23 09:00:32 +08:00
congqixia 03a4f60b6e
Fix knowhere cpp header files name (#23330) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-04-10 20:00:30 +08:00
Cai Yudong 0e9a4478e3
Remove useless index mode (#22934) 
Signed-off-by: Yudong Cai <yudong.cai@zilliz.com>
 2023-03-23 21:39:59 +08:00
yah01 3202eb0d9f
Optimize rounding distances, avoid promoting to double (#22846) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-03-20 14:09:56 +08:00
yah01 bdd6bc7695
Re-format cpp code (#22513) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-03-02 15:55:49 +08:00
smellthemoon cf999b68dd
Fix range search can not get status from knowhere (#22384) 
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2023-02-24 17:59:45 +08:00
smellthemoon 7a4dfcc72b
Add status error output (#22325) 
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2023-02-22 15:51:44 +08:00
smellthemoon 9e0ec15436
Support range search (#21652) 
Signed-off-by: smellthemoon <xinguo.li@zilliz.com>
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: jaime <yun.zhang@zilliz.com>
 2023-02-21 09:48:32 +08:00
presburger 9950cacd10
support knowhere 2.0 (#21857) 
Signed-off-by: Yusheng.Ma <Yusheng.Ma@zilliz.com>
 2023-02-10 14:24:32 +08:00
xige-16 4a66965df4
Delete RAW_DATA copy when load IVF_FLAT index data (#20274) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>

Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-11-05 17:33:05 +08:00
xige-16 158787811e
Move assemble/disassemble func to core (#19420) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>

Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-10-16 21:17:25 +08:00
congqixia a20d83bccd
Assemble RAW_DATA files before storing into index.raw_data (#19771) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-10-14 12:03:24 +08:00
xige-16 d4bc00423c
Fix start milvus failed on macos (#19394) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>

Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-09-23 16:54:50 +08:00
xige-16 428840178c
Support diskann index for vector field (#19093) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>

Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-09-21 20:16:51 +08:00