04175d8549
enhance: [2.5]compress jsonstatslog path only save filename ( #39979 )
...
enhance: compress jsonstatslog path only save filename
issue: https://github.com/milvus-io/milvus/issues/36995
pr: https://github.com/milvus-io/milvus/pull/38039
Signed-off-by: Xianhui.Lin <xianhui.lin@zilliz.com>
2025-02-19 10:20:52 +08:00
63576738b6
fix: [2.5]exists expr on nested dictionaries fail in JSONkeyIndex ( #39972 )
...
fix: exists expr on nested dictionaries fail in JSONkeyIndex
issue: https://github.com/milvus-io/milvus/issues/39963
pr: https://github.com/milvus-io/milvus/pull/38039
Signed-off-by: Xianhui.Lin <xianhui.lin@zilliz.com>
2025-02-19 10:18:51 +08:00
7a92121c92
fix: [2.5] Fix failed to seek to earliest position ( #39966 )
...
If it is the earliest message ID, skip the seek to prevent failure.
issue: https://github.com/milvus-io/milvus/issues/39964
pr: https://github.com/milvus-io/milvus/pull/39965
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2025-02-18 20:36:53 +08:00
d60ba1828b
test: [cherry-pick] Enable more test cases for auto index ( #39939 ) ( #39959 )
...
related pr: #39939
Signed-off-by: binbin lv <binbin.lv@zilliz.com>
2025-02-18 17:42:52 +08:00
486e54c2c4
test: update test cases and sdk version[2.5] ( #39442 )
...
pr: #39443
Signed-off-by: nico <cheng.yuan@zilliz.com>
2025-02-18 17:20:52 +08:00
d92ffb66a1
fix: [skip e2e] [2.5]data race in load test ( #39846 )
...
Related to https://github.com/milvus-io/milvus/pull/39701
pr: #39845
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2025-02-18 15:04:50 +08:00
bf54f47c34
enhance: [2.5] use rated logger for high frequency log in dist handler ( #39452 ) ( #39928 )
...
pr: #39452
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2025-02-18 14:32:52 +08:00
7fb69960d1
test: [GoSDK][2.5] Sync gosdk test commits from master branch ( #39838 )
...
Cherry-pick from master
pr: #39484 #39570 #39537 #39824 #39579
---------
Signed-off-by: ThreadDao <yufen.zong@zilliz.com>
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
Signed-off-by: zhuwenxing <wxzhuyeah@gmail.com>
Co-authored-by: zhuwenxing <wenxing.zhu@zilliz.com>
2025-02-18 11:11:08 +08:00
7ea0af572a
enhance: [2.5] Optimize the result format of GetQueryNodeDistribution ( #39664 ) ( #39926 )
...
pr: #39664
Use string array for SealedSegmentIDs to prevent precision loss in JSON
parsers. Large integers (int64) may be incorrectly rounded when parsed
as double.
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2025-02-18 09:59:07 +08:00
2eaa1a4869
fix: [2.5] Use `text_log` prefix for TextMatchIndex null offset file ( #39936 )
...
Cherry-pick from master
pr: #39935
Related to #39933
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2025-02-18 00:43:07 +08:00
de8abe7970
fix:reset default auto index for varchar ( #39908 )
...
pr: #39819
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2025-02-17 20:55:26 +08:00
52434ccc78
enhance: [2.5] Limit the speed of the generating stats task ( #39645 )
...
master pr: #39644
---------
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2025-02-17 16:06:17 +08:00
8067113133
enhance: [cp25]Enable to observe write amplification ( #39661 ) ( #39743 )
...
pr: #39661
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2025-02-17 16:00:17 +08:00
56c1a8d462
fix: ignore growing option is lost at hibridsearch ( #39900 )
...
issue: #39892
pr: #39799
Signed-off-by: chyezh <chyezh@outlook.com>
2025-02-17 15:30:15 +08:00
3a951f2160
test: Cherry pick sparse invert index algo tests ( #39816 )
...
related issue: https://github.com/milvus-io/milvus/issues/39332
related pr: #39691
---------
Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
2025-02-17 15:06:15 +08:00
ee25af4c9b
enhance: Add configs for compaction schedule ( #39010 ) ( #39511 )
...
pr: #39010
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2025-02-17 11:44:15 +08:00
8c86e4e6ac
fix: [2.5]AlterCollection unable to modify ConsistencyLevel ( #39902 )
...
fix: AlterCollection unable to modify ConsistencyLevel
issue: https://github.com/milvus-io/milvus/issues/39707
pr: https://github.com/milvus-io/milvus/pull/39708
Signed-off-by: Xianhui.Lin <xianhui.lin@zilliz.com>
2025-02-17 10:00:15 +08:00
9918e1008d
fix: Fix import failed due to 0 row num ( #39887 ) ( #39904 )
...
issue: https://github.com/milvus-io/milvus/issues/39885
pr: https://github.com/milvus-io/milvus/pull/39886
---------
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
Co-authored-by: yihao.dai <yihao.dai@zilliz.com>
2025-02-17 01:36:15 +08:00
f0964f769d
enhance: [2.5]Add json key inverted index in stats for optimization ( #39876 )
...
Add json key inverted index in stats for optimization
issue: https://github.com/milvus-io/milvus/issues/36995
pr: https://github.com/milvus-io/milvus/pull/38039
---------
Signed-off-by: Xianhui.Lin <xianhui.lin@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2025-02-16 20:12:15 +08:00
e372b1733c
enhance: [2.5]update raft to cuvs ( #39221 )
...
pr: #39873
issue: #39883
Signed-off-by: yusheng.ma <yusheng.ma@zilliz.com>
2025-02-14 16:52:14 +08:00
401794f405
fix: [2.5] Fix module for long type ( #39802 )
...
issue: #39640
master pr: #39722
---------
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2025-02-14 15:18:21 +08:00
5da9262f58
fix: [2.5] Add and use lifetime context for compaction trigger ( #39857 ) ( #39880 )
...
Cherry-pick from master
pr: #39857
Related to #39856
This PR add lifetime bound context for compaction trigger and use it
instead of context.Background in case of rootcoord down and some grpc
call retry forever
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2025-02-14 14:18:14 +08:00
418f971d2d
fix: [2.5] ReEnqueue L0 compaction task when preCheck failed ( #39871 )
...
issue: #39868
master pr: #39870
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2025-02-14 13:38:15 +08:00
ba0edb8383
enhance: [2.5]return topks when search in restful v2 ( #39839 )
...
pr: #39812
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2025-02-14 11:30:14 +08:00
9407a3c9b1
fix: [2.5] Check collection released before target checks ( #39843 )
...
Cherry-pick from master
pr: #39841
Related to #39840
The target could be updated async in previous code. This PR make remove
collection from target observer block until all tasks related in
dispatchers are removed preventing the metrics being updated after
collection released.
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2025-02-13 20:00:15 +08:00
f85bf59921
fix: rootcoord graceful stop failed and limit resource of ci ( #39793 )
...
issue: #38399
pr: #39781
---------
Signed-off-by: chyezh <chyezh@outlook.com>
2025-02-13 19:34:13 +08:00
aa74e8ba3d
enhance: [2.5][GoSDK] Add WithEnableMatch syntactic sugar ( #39837 ) ( #39853 )
...
Cherry-pick from master
pr: #39837
Related to #35922
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2025-02-13 19:28:15 +08:00
16ea4fcb90
Revert "fix:reset default auto index type for scalar ( #37086 )" ( #39820 )
...
This reverts commit eeb67a3845#39819
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2025-02-13 17:36:14 +08:00
a50249e0bb
enhance: [2.5] dump pprof info if component stop progress timeout ( #39760 )
...
issue: #39735
pr: #39726
---------
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2025-02-13 15:30:47 +08:00
df28d2200d
fix: [2.5] Remove load field & schema column size check #39834 ( #39835 )
...
Cherry-pick from master
pr: #39833
Related to #39788
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2025-02-13 12:42:53 +08:00
cb1bf6d122
fix:[2.5] remove the mmap.enable param in the type param when creating index ( #39806 )
...
Because when GetIndexParams is used, index params and type params are
concatenated, so when loading index, the mmap.enable parameter in type
params is also referenced.
- issue: #39801
- pr: #39803
Signed-off-by: SimFG <bang.fu@zilliz.com>
2025-02-13 10:08:53 +08:00
d121ac3a7a
enhance: [2.5]intermin index support different index type and more data type(fp16/bf16) ( #39180 )
...
issue: https://github.com/milvus-io/milvus/issues/27678
related: https://github.com/milvus-io/milvus/pull/39753
some raw data status will change:
Intermin index has raw data:
<html xmlns:v="urn:schemas-microsoft-com:vml"
xmlns:o="urn:schemas-microsoft-com:office:office"
xmlns:x="urn:schemas-microsoft-com:office:excel"
xmlns="http://www.w3.org/TR/REC-html40 ">
<head>
<meta name=ProgId content=Excel.Sheet>
<meta name=Generator content="Microsoft Excel 15">
<link id=Main-File rel=Main-File
href="file:////Users/cqy/Library/Group%20Containers/UBF8T346G9.Office/TemporaryItems/msohtmlclip/clip.htm">
<link rel=File-List
href="file:////Users/cqy/Library/Group%20Containers/UBF8T346G9.Office/TemporaryItems/msohtmlclip/clip_filelist.xml">
<!--table
{mso-displayed-decimal-separator:"\.";
mso-displayed-thousand-separator:"\,";}
@page
{margin:.75in .7in .75in .7in;
mso-header-margin:.3in;
mso-footer-margin:.3in;}
.font5
{color:windowtext;
font-size:9.0pt;
font-weight:400;
font-style:normal;
text-decoration:none;
font-family:等线;
mso-generic-font-family:auto;
mso-font-charset:134;}
tr
{mso-height-source:auto;
mso-ruby-visibility:none;}
col
{mso-width-source:auto;
mso-ruby-visibility:none;}
br
{mso-data-placement:same-cell;}
td
{padding-top:1px;
padding-right:1px;
padding-left:1px;
mso-ignore:padding;
color:black;
font-size:12.0pt;
font-weight:400;
font-style:normal;
text-decoration:none;
font-family:等线;
mso-generic-font-family:auto;
mso-font-charset:134;
mso-number-format:General;
text-align:general;
vertical-align:middle;
border:none;
mso-background-source:auto;
mso-pattern:auto;
mso-protection:locked visible;
white-space:nowrap;
mso-rotate:0;}
ruby
{ruby-align:left;}
rt
{color:windowtext;
font-size:9.0pt;
font-weight:400;
font-style:normal;
text-decoration:none;
font-family:等线;
mso-generic-font-family:auto;
mso-font-charset:134;
mso-char-type:none;
display:none;}
-->
</head>
<body link="#0563C1" vlink="#954F72">
sparse vector | growing segment | sealed segment
-- | -- | --
BM25 | no | no
IP | yes | no
| |
dense vector | growing segment | sealed segment
ivf flat cc | yes | yes
scann_dvr | no | no
</body>
</html>
Signed-off-by: cqy123456 <qianya.cheng@zilliz.com>
2025-02-13 09:56:47 +08:00
7ca5e8309b
enhance: [GoSDK][2.5] Sync gosdk commits from master branch ( #39823 )
...
Cherry-pick from master
pr: #38791 #38812 #38940 #39515 #39580 #39662 #39665 #39751
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2025-02-12 18:46:47 +08:00
0f4d3ef8b6
enhance[2.5]: move GetVector_EmptySparseVector ut to growing test ( #39286 )
...
issue: https://github.com/milvus-io/milvus/issues/35853
pr: https://github.com/milvus-io/milvus/pull/39234
Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com>
2025-02-12 16:00:49 +08:00
dd516a5716
fix: [2.5]not pass the indexname when drop properties ( #39679 )
...
pr: #39678
issue: #38967
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2025-02-12 15:52:48 +08:00
1b14f5a7ab
fix: [2.5] Segments return both growing&sealed result ( #39789 )
...
Cherry-pick from master
pr: #39787
Previous PR #38311
Logic bug found in #38348
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2025-02-12 14:44:48 +08:00
82dc57ace0
fix: [skip e2e][2.5] pr conflict cause ut failed ( #39810 )
...
Related to https://github.com/milvus-io/milvus/pull/39701 &
https://github.com/milvus-io/milvus/issues/39681
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2025-02-12 11:44:51 +08:00
667c84740c
enhance: [2.5] Add metrics for parse expression ( #39716 )
...
master pr: #39654
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2025-02-12 10:30:46 +08:00
c2986d4ab9
fix: [2.5] Fix concurrent map ( #39776 )
...
issue: https://github.com/milvus-io/milvus/issues/39778
pr: https://github.com/milvus-io/milvus/pull/39775
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2025-02-11 22:12:46 +08:00
4322a0d49a
fix: [2.5] Resolve conflict on qc task test ( #39797 )
...
Cherry-pick from master
pr: #39796
Related to #39701 & #39681
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2025-02-11 18:52:45 +08:00
11cba57dc7
fix: [2.5] load collection stucks if compaction/gc happens ( #39761 )
...
issue: #39680
pr: #39701
if compaction/gc happens, load collection may stuck due to
SegmentNotFound, we should trigger UpdateNextTarget to get a new data
view to execute loading operation.
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2025-02-11 15:48:50 +08:00
969e34d540
fix: [2.5]uneven distribution caused by executing task delta cache leak ( #39759 )
...
issue: #39681
pr: #39702
this PR maintain workload effect in action instead of computing workload
effect from target, which may cause leak if target changes.
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2025-02-11 14:32:46 +08:00
c0cc8a59f7
fix: [2.5] Return early when skip load pk index ( #39763 )
...
Cherry pick from master
pr: #39762
Previous PR #39437 only print log and add index while load operation is
still executed. This PR return early when segment decides not to load PK
index.
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2025-02-11 11:04:50 +08:00
f542c866b3
fix: [2.5] root check the role list if `rootShouldBindRole` is true ( #39714 )
...
- issue: #39712
- pr: #39713
Signed-off-by: SimFG <bang.fu@zilliz.com>
2025-02-11 10:10:51 +08:00
87b39c2022
fix: [skip e2e] update the collection detail page logic ( #39739 )
...
issue: #36621
pr: #39740
Signed-off-by: mimo.oyn <ning.ouyang@zilliz.com>
2025-02-10 15:26:45 +08:00
b936061ec6
enhance: keep consistent of memory and meta of broadcaster ( #39721 )
...
issue: #38399
pr: #39720
Signed-off-by: chyezh <chyezh@outlook.com>
2025-02-10 10:50:44 +08:00
4a8e6fc59c
fix: [2.5] Fix flowgraph leak ( #39686 )
...
Consider lag targets when determining whether to remove the dispatcher
manager.
issue: https://github.com/milvus-io/milvus/issues/39642
pr: https://github.com/milvus-io/milvus/pull/39656
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2025-02-08 15:40:45 +08:00
8f2e7e93b3
fix: [skip e2e] update web ui tasks and query segments list ( #39676 )
...
issue: #36621
pr: #39693
Signed-off-by: mimo.oyn <ning.ouyang@zilliz.com>
2025-02-08 09:50:48 +08:00
b02d6423a3
enhance: [2.5]update knowhere version ( #39673 )
...
/kind branch-feature
release note :
https://github.com/zilliztech/knowhere/releases/tag/v2.5.3
Signed-off-by: xianliang.li <xianliang.li@zilliz.com>
2025-02-07 12:46:45 +08:00
858dc10ef9
enhance: broadcast with event-based notification ( #39550 )
...
issue: #38399
pr: #39522
- broadcast message can carry multi resource key now.
- implement event-based notification for broadcast messages
- broadcast message use broadcast id as a unique identifier in message
- broadcasted message on vchannels keep the broadcasted vchannel now.
- broadcasted message and broadcast message have a common broadcast
header now.
---------
Signed-off-by: chyezh <chyezh@outlook.com>
2025-02-07 11:50:50 +08:00
Xianhui Lin
yihao.dai
binbin
nico
wei liu
ThreadDao
congqixia
zhagnlu
cai.zhang
XuanYang-cn
Zhen Ye
yanliang567
zhenshan.cao
presburger
smellthemoon
SimFG
cqy123456
Buqian Zheng
MimoN
foxspy