ea7307747a
fix: fix pk in [..] skip next batch when using multi-chunk segment ( #43619 )
...
pr: #43618
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2025-07-31 16:59:37 +08:00
71c0f64a16
fix: [2.5] fix incorrect bitset for the division comparison when the right is < 0 ( #43180 )
...
issue: #42900
pr: #43179
Upd: also handles Inf and NaN values, and the division by zero case for
fp32 and fp64
Signed-off-by: Alexandr Guzhva <alexanderguzhva@gmail.com>
2025-07-16 15:36:52 +08:00
21d6866a56
fix: fix text match index / json key stats index leak when segment released ( #43308 )
...
pr: https://github.com/milvus-io/milvus/pull/42655
issue: https://github.com/milvus-io/milvus/issues/42626
Signed-off-by: SpadeA <tangchenjie1210@gmail.com>
2025-07-15 15:22:18 +08:00
4fe8011a70
enhance: refine variable-length-type memory usage( #38736 ) ( #43093 )
...
related: #38736
pr: https://github.com/milvus-io/milvus/pull/39578
Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
2025-07-04 18:26:45 +08:00
fe05970eba
fix is_not_in for trie index ( #42886 )
...
pr: #42716
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2025-06-25 17:30:43 +08:00
017eb9ffe2
fix: fix some bugs discovered by chaos tests --- cherry pick ( #42909 )
...
master: https://github.com/milvus-io/milvus/pull/42906
issue: https://github.com/milvus-io/milvus/issues/42870
---------
Signed-off-by: SpadeA <tangchenjie1210@gmail.com>
2025-06-24 17:10:42 +08:00
94005f8e01
enhance: tantivy collector set bitset directly cp-39748 ( #42694 )
...
issue: https://github.com/milvus-io/milvus/issues/39755
pr: #39748
It also cherry-pick #40608
---------
Signed-off-by: SpadeA <tangchenjie1210@gmail.com>
2025-06-16 16:58:40 +08:00
97fcc1586c
fix: [2.5] Fix the bug of valid data write corruption ( #42555 )
...
issue: https://github.com/milvus-io/milvus/issues/42554
pr: https://github.com/milvus-io/milvus/pull/42556
---------
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2025-06-06 19:22:43 +08:00
532f10f343
enhance: Support cast function for json index ( #42504 )
...
issue: #41948
pr: #41949
---------
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2025-06-05 20:26:33 +08:00
66e28f605f
fix: fix is null bug for marisa index ( #42421 )
...
pr: #42420
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2025-06-03 14:52:32 +08:00
0574fc7b7b
enhance: support TTL expiration with queries returning no results ( #41960 )
...
support TTL expiration with queries returning no results
issue:https://github.com/milvus-io/milvus/issues/41959
pr:https://github.com/milvus-io/milvus/pull/41720
---------
Signed-off-by: Xianhui.Lin <xianhui.lin@zilliz.com>
2025-05-26 15:52:28 +08:00
813bcb14a7
fix: [2.5]prevent crash when contains_all/any is used with empty array ( #41756 )
...
issue: https://github.com/milvus-io/milvus/issues/41348
related and optimized by https://github.com/milvus-io/milvus/pull/41347
master pr: #41739
---------
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
Co-authored-by: Sangho Park <hoyaspark@gmail.com>
2025-05-14 14:30:22 +08:00
5b8ea84d38
fix: add params to ignore config type exception ( #41777 )
...
pr: #41776
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2025-05-13 11:28:57 +08:00
418c35630b
fix: GetValueFromConfig return nullopt instead of exception ( #41711 )
...
pr: #41709
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2025-05-09 11:20:54 +08:00
22582a45b0
feat: Add json index support for json contains expr ( #41478 ) ( #41658 )
...
pr: #41478
issue: #35528
---------
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2025-05-07 10:40:53 +08:00
69a80b9ce3
enhance: resize high priority wqthreadpool dynamically( #40838 ) ( #41549 )
...
related: #40838
Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
2025-04-28 00:44:39 +08:00
4ac57f1217
enhance: support binary range expression for json path index ( #41317 )
...
pr: #41025
issue: #35528
---------
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2025-04-24 20:04:39 +08:00
8b3353cdab
feat: [2.5] Support run analyzer and more tokenizer. ( #41444 )
...
relate: https://github.com/milvus-io/milvus/issues/39659
https://github.com/milvus-io/milvus/issues/39705
pr: https://github.com/milvus-io/milvus/pull/40416
https://github.com/milvus-io/milvus/pull/40458
https://github.com/milvus-io/milvus/pull/39723
https://github.com/milvus-io/milvus/pull/40813
https://github.com/milvus-io/milvus/pull/39854
https://github.com/milvus-io/milvus/pull/39660
---------
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2025-04-24 10:48:39 +08:00
afc0ca77e4
enhance: Optimize index format for improved load performance ( #41041 )
...
related: #40838
pr: https://github.com/milvus-io/milvus/pull/40839
Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
2025-04-15 07:07:38 +08:00
2498eb0691
enhance: add optimize for like expr ( #41222 )
...
pr: #41066 cherry-pick from master
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2025-04-14 21:06:32 +08:00
cb0f984155
enhance: Revert "separate for index completed ( #40873 )" ( #41152 )
...
This reverts commit 23e579e324#40873
issue: #39519
Signed-off-by: Li Liu <li.liu@zilliz.com>
2025-04-08 17:36:30 +08:00
9eb74d7418
fix: Fix the bug that null data can not filtered by null expr ( #41135 )
...
issue: #41063
pr: #41124
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2025-04-08 00:26:26 +08:00
23e579e324
separate for index completed ( #40873 )
...
related: https://github.com/milvus-io/milvus/issues/40781
Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
2025-04-05 10:20:24 +08:00
240f766511
fix: Align brute force search with json index for exists expr ( #41056 )
...
issue: #35528
pr: #41004
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2025-04-02 18:28:23 +08:00
a906466d8f
fix: [2.5]Add bypass thread pool mode to avoid growing indexes blocking insert/load ( #41013 )
...
issue: #40825
related: #41012
pr: #41012
Signed-off-by: xianliang.li <xianliang.li@zilliz.com>
2025-04-02 16:58:25 +08:00
0d3bd3131c
fix:fix format error for json ( #41031 )
...
#41026
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2025-04-02 14:32:32 +08:00
c7ca352de5
enhance: Initialize field id to avoid negative number ( #38789 ) ( #41017 )
...
pr: #38789
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2025-04-01 10:08:21 +08:00
9108cee958
feat: add json null/exists expression ( #41002 )
...
issue: #35528
pr: https://github.com/milvus-io/milvus/pull/41004
---------
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2025-03-31 21:52:23 +08:00
d420f100c0
enhance: refine array view to optimize memory usage( #38736 ) ( #40206 )
...
related: #38736
pr: https://github.com/milvus-io/milvus/pull/38808
Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
2025-03-28 11:08:18 +08:00
deed5b5df4
enhance:change multi or expr to in expr ( #40751 )
...
pr: #40757
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2025-03-27 11:08:20 +08:00
7c716d61dc
fix:[2.5]wrong p.InterimIndexRefineQuantType default value and reduce ut run time ( #40830 )
...
issue: https://github.com/milvus-io/milvus/issues/27678
Signed-off-by: cqy123456 <qianya.cheng@zilliz.com>
2025-03-21 19:10:13 +08:00
2fbfd86146
fix:[2.5]fix growing index ut ( #40724 )
...
issue: https://github.com/milvus-io/milvus/issues/27678
Signed-off-by: cqy123456 <qianya.cheng@zilliz.com>
2025-03-18 19:04:15 +08:00
705b3c90a5
fix: Failed to rolling upgrade from v2.5.6 to new 2.5 version when enable JsonKeyStats ( #40661 )
...
fix: Failed to rolling upgrade from v2.5.6 to new 2.5 version when
enable JsonKeyStats.The reason is that the file path of the jsonkeyindex
has changed.
issue: https://github.com/milvus-io/milvus/issues/40649
，https://github.com/milvus-io/milvus/issues/40669
https://github.com/milvus-io/milvus/issues/40707
master-pr: https://github.com/milvus-io/milvus/pull/38039
---------
Signed-off-by: Xianhui.Lin <xianhui.lin@zilliz.com>
2025-03-18 17:32:16 +08:00
b0078ceae3
fix: Fix json test ( #40710 )
...
issue: #35528
pr: https://github.com/milvus-io/milvus/pull/40671
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2025-03-18 12:52:13 +08:00
b8abd91962
enhance[2.5]: add more config for interimIndex to support more refine mode ( #40429 )
...
issue: https://github.com/milvus-io/milvus/issues/27678
Signed-off-by: cqy123456 <qianya.cheng@zilliz.com>
2025-03-17 22:04:12 +08:00
9145885f40
enhance: Record simdjson error ( #40671 )
...
issue: #35528
pr: #40647
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2025-03-15 12:48:08 +08:00
ae8a9cebb8
fix: fix json index not-equal filter ( #40648 )
...
issue: #35528
pr: https://github.com/milvus-io/milvus/pull/40647
---------
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2025-03-14 21:36:10 +08:00
6b9e141ada
enhance: reorder sub expr for conjunct expr ( #40186 )
...
pr:#39872
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2025-03-14 15:16:08 +08:00
f5e9dea2aa
fix: [2.5]fix the garbage cleanup logic of jsonkey stats && improve json key stats filer ( #40039 )
...
fix: fix the garbage collection cleanup logic of jsonkey stats &&
improve json key stats filer
issue: https://github.com/milvus-io/milvus/issues/36995
https://github.com/milvus-io/milvus/issues/40034
https://github.com/milvus-io/milvus/issues/40041
https://github.com/milvus-io/milvus/issues/40106
https://github.com/milvus-io/milvus/issues/40138
pr: https://github.com/milvus-io/milvus/pull/38039
---------
Signed-off-by: Xianhui.Lin <xianhui.lin@zilliz.com>
2025-03-13 20:18:10 +08:00
683b26ffb7
feat: cherry pick json path index ( #40313 )
...
issue: #35528
pr: #36750
this pr includes json path index pr and some related prs:
1. update tantivy version #39253
2. json path index #36750
3. fall back to brute force #40076
4. term filter #40140
5. bug fix #40336
---------
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2025-03-10 22:14:05 +08:00
345ca01634
fix: [2.5]fix concurrent issues in null offset #40363 ( #40365 )
...
issue: https://github.com/milvus-io/milvus/issues/40308
cherry-pick: https://github.com/milvus-io/milvus/pull/40363
---------
Signed-off-by: SpadeA <tangchenjie1210@gmail.com>
2025-03-05 16:54:09 +08:00
d66d3f0d47
enhance:decrease dump snapshot limit from 10w to 1w ( #40102 )
...
pr: #40101
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2025-02-22 11:29:53 +08:00
f0964f769d
enhance: [2.5]Add json key inverted index in stats for optimization ( #39876 )
...
Add json key inverted index in stats for optimization
issue: https://github.com/milvus-io/milvus/issues/36995
pr: https://github.com/milvus-io/milvus/pull/38039
---------
Signed-off-by: Xianhui.Lin <xianhui.lin@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2025-02-16 20:12:15 +08:00
d121ac3a7a
enhance: [2.5]intermin index support different index type and more data type(fp16/bf16) ( #39180 )
...
issue: https://github.com/milvus-io/milvus/issues/27678
related: https://github.com/milvus-io/milvus/pull/39753
some raw data status will change:
Intermin index has raw data:
<html xmlns:v="urn:schemas-microsoft-com:vml"
xmlns:o="urn:schemas-microsoft-com:office:office"
xmlns:x="urn:schemas-microsoft-com:office:excel"
xmlns="http://www.w3.org/TR/REC-html40 ">
<head>
<meta name=ProgId content=Excel.Sheet>
<meta name=Generator content="Microsoft Excel 15">
<link id=Main-File rel=Main-File
href="file:////Users/cqy/Library/Group%20Containers/UBF8T346G9.Office/TemporaryItems/msohtmlclip/clip.htm">
<link rel=File-List
href="file:////Users/cqy/Library/Group%20Containers/UBF8T346G9.Office/TemporaryItems/msohtmlclip/clip_filelist.xml">
<!--table
{mso-displayed-decimal-separator:"\.";
mso-displayed-thousand-separator:"\,";}
@page
{margin:.75in .7in .75in .7in;
mso-header-margin:.3in;
mso-footer-margin:.3in;}
.font5
{color:windowtext;
font-size:9.0pt;
font-weight:400;
font-style:normal;
text-decoration:none;
font-family:等线;
mso-generic-font-family:auto;
mso-font-charset:134;}
tr
{mso-height-source:auto;
mso-ruby-visibility:none;}
col
{mso-width-source:auto;
mso-ruby-visibility:none;}
br
{mso-data-placement:same-cell;}
td
{padding-top:1px;
padding-right:1px;
padding-left:1px;
mso-ignore:padding;
color:black;
font-size:12.0pt;
font-weight:400;
font-style:normal;
text-decoration:none;
font-family:等线;
mso-generic-font-family:auto;
mso-font-charset:134;
mso-number-format:General;
text-align:general;
vertical-align:middle;
border:none;
mso-background-source:auto;
mso-pattern:auto;
mso-protection:locked visible;
white-space:nowrap;
mso-rotate:0;}
ruby
{ruby-align:left;}
rt
{color:windowtext;
font-size:9.0pt;
font-weight:400;
font-style:normal;
text-decoration:none;
font-family:等线;
mso-generic-font-family:auto;
mso-font-charset:134;
mso-char-type:none;
display:none;}
-->
</head>
<body link="#0563C1" vlink="#954F72">
sparse vector | growing segment | sealed segment
-- | -- | --
BM25 | no | no
IP | yes | no
| |
dense vector | growing segment | sealed segment
ivf flat cc | yes | yes
scann_dvr | no | no
</body>
</html>
Signed-off-by: cqy123456 <qianya.cheng@zilliz.com>
2025-02-13 09:56:47 +08:00
0f4d3ef8b6
enhance[2.5]: move GetVector_EmptySparseVector ut to growing test ( #39286 )
...
issue: https://github.com/milvus-io/milvus/issues/35853
pr: https://github.com/milvus-io/milvus/pull/39234
Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com>
2025-02-12 16:00:49 +08:00
6cdabd9002
fix:fix GetValueFromConfig for bool type ( #39527 )
...
pr: #39526
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2025-01-23 00:03:07 +08:00
1602390734
enhance: support null expr( #38772 ) ( #38931 )
...
pr: #38772
issue: #31728
---------
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2025-01-16 10:55:07 +08:00
1d9788e4bc
fix: [2.5] enable to build index with single segment ( #39272 )
...
issue: #39232
cherry-pick https://github.com/milvus-io/milvus/pull/39233
---------
Signed-off-by: SpadeA-Tang <tangchenjie1210@gmail.com>
2025-01-16 09:59:02 +08:00
b4ddf746c1
enhance: [2.5][bitset] extend op_find() to be able to search both 0 and 1 ( #39237 )
...
Cherry-pick from master
pr: #39176 #39229
issue: #39124
`bitset::find_first()` and `bitset::find_next()` now accept one more
parameter, which allows to search for `0` bit instead of `1` bit
---------
Signed-off-by: Alexandr Guzhva <alexanderguzhva@gmail.com>
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
Co-authored-by: Alexander Guzhva <alexanderguzhva@gmail.com>
2025-01-14 17:59:00 +08:00
adfc3f945e
enhance: record memory size (uncompressed) item for index ( #38844 )
...
issue: #38715
pr: #38770
- Current milvus use a serialized index size(compressed) for estimate
resource for loading.
- Add a new field MemSize (before compressing) for index to estimate
resource.
---------
Signed-off-by: chyezh <chyezh@outlook.com>
2025-01-14 10:33:06 +08:00
zhagnlu
Alexander Guzhva
Spade A
Chun Han
Bingyi Sun
Xianhui Lin
cai.zhang
aoiasd
liliu-z
foxspy
cqy123456
Buqian Zheng
smellthemoon
congqixia
Zhen Ye