c1c1f2df4e
test: [2.5] add test cases for template param ( #39475 )
...
issue: #33419
pr: #38867
- add case for Get
- add case for TemplateParam
Signed-off-by: ThreadDao <yufen.zong@zilliz.com>
2025-01-22 09:55:24 +08:00
a90bd9e417
test:[cp2.5] add search iterator v2 case for milvus client ( #39447 )
...
issue: #37548
pr: #39400
Signed-off-by: qixuan <673771573@qq.com>
2025-01-22 09:53:06 +08:00
8475d8b193
fix: cherry pick warmup async ( #39402 ) ( #39474 )
...
related pr: https://github.com/milvus-io/milvus/pull/38690
issue: https://github.com/milvus-io/milvus/issues/38692
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2025-01-21 15:25:05 +08:00
20c6de156e
feat: update milvusproto version to 2.5.5 ( #39478 )
...
feat: update milvusproto version to 2.5
issue: https://github.com/milvus-io/milvus/issues/39093
Signed-off-by: Xianhui.Lin <xianhui.lin@zilliz.com>
2025-01-21 15:11:05 +08:00
f912cbf86d
enhance:[Cherry-pick] skip load bm25 sparse row data ( #39207 )
...
pr: https://github.com/milvus-io/milvus/pull/39078
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2025-01-21 11:05:04 +08:00
f7799f0add
fix:[2.5] check utf-8 format for varchar with analyzer open ( #39439 )
...
relate: https://github.com/milvus-io/milvus/issues/39285
pr: https://github.com/milvus-io/milvus/pull/39299
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2025-01-21 10:57:03 +08:00
83666f1bff
enhance: [2.5] Enable hook for importv2 ( #39430 )
...
/kind improvement
pr: https://github.com/milvus-io/milvus/pull/39429
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2025-01-21 10:29:04 +08:00
3b3959c625
enhance: [2.5]Add schema update time verification for insert and upsert to use cache ( #39382 )
...
enhance: Add schema update time verification for insert and upsert to
use cache
issue: https://github.com/milvus-io/milvus/issues/39093
Related to pr: https://github.com/milvus-io/milvus/pull/39096
---------
Signed-off-by: Xianhui.Lin <xianhui.lin@zilliz.com>
2025-01-21 10:19:11 +08:00
817b616eb4
fix: [2.5]Restore the compacting state for stats task during recovery ( #39460 )
...
issue: #39333
master pr: #39459
---------
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2025-01-21 01:03:05 +08:00
15d60c60c2
enhance: [2.5] remain token after authorize ( #39455 )
...
relate: https://github.com/milvus-io/milvus/issues/39285
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2025-01-20 20:27:04 +08:00
f62eca09a0
enhance: set materializedView.enabled default to true ( #39450 )
...
pr: #39449
Signed-off-by: chasingegg <chao.gao@zilliz.com>
2025-01-20 19:55:04 +08:00
e4c6ef9d57
enhance: [2.5] serializes the resp body and attach some details ( #39401 )
...
issue: #37611
pr: #39140
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2025-01-20 18:17:10 +08:00
ff53d4f742
enhance: [2.5]add some apis in Restful ( #39425 )
...
pr: #39105
issue: #38709
---------
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2025-01-20 18:15:22 +08:00
8e2e828a78
enhance: [2.5] Skip loading pk index for sorted segment in loader ( #39438 )
...
Cherry pick from master
pr: #39437
Related to #39339
Previous PR #39389 only skips append index into segment
Also related to #39428
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2025-01-20 18:07:04 +08:00
964000f645
fix: deleted the sealed segment data accidentally ( #39422 )
...
issue:https://github.com/milvus-io/milvus/issues/39333
pr: https://github.com/milvus-io/milvus/pull/39421
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2025-01-20 17:49:03 +08:00
6342237ca8
enhance: add workaround for concurrent tantivy reload and directory removal ( #39441 )
...
related pr: https://github.com/milvus-io/milvus/pull/39440
issue: https://github.com/milvus-io/milvus/issues/39428
---------
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2025-01-20 17:33:05 +08:00
3893f24323
fix: apply custom producer config for kafkaHealthCheck ( #39364 )
...
issue: https://github.com/milvus-io/milvus/issues/39287
pr: https://github.com/milvus-io/milvus/pull/39283
Signed-off-by: DLT1412 <tuduc93@gmail.com>
2025-01-20 17:27:04 +08:00
6f9f840080
fix: Fix panic caused by removing directory ( #38622 ) ( #39434 )
...
related pr: https://github.com/milvus-io/milvus/pull/38622
issue: https://github.com/milvus-io/milvus/issues/39428
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2025-01-20 16:13:03 +08:00
8c35064d01
enhance: [2.5] clean up logs of empty op key ( #39394 )
...
cherry-pick from master: https://github.com/milvus-io/milvus/pull/39335
related: https://github.com/milvus-io/milvus/issues/39334
Signed-off-by: shaoting-huang <shaoting.huang@zilliz.com>
2025-01-20 12:15:10 +08:00
4464966462
enhance: [2.5] Remove frequent observe log ( #39414 )
...
/kind improvement
pr: https://github.com/milvus-io/milvus/pull/39413
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2025-01-20 11:01:10 +08:00
6d5f174142
enhance: [2.5] add recalls response for restful api ( #39409 )
...
issue: #37899
pr: #39408
Signed-off-by: chasingegg <chao.gao@zilliz.com>
2025-01-20 10:59:10 +08:00
efe76905c5
fix: correct remote centroids path in clustering compaction ( #39399 )
...
issue: https://github.com/milvus-io/milvus/issues/39353
pr: #39398
Signed-off-by: chasingegg <chao.gao@zilliz.com>
2025-01-20 10:57:21 +08:00
c9b0859b16
fix: [cp25]Record active collections for l0Policy ( #39217 ) ( #39383 )
...
By recording the active collection lists, The l0 compaction trigger of
view change and idle won't influence each other.
Also this pr replaces the L0View cache with real L0 segments' change.
Save some memory and make L0 compaction triggers more accurate.
See also: #39187
pr: #39217
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2025-01-20 10:47:03 +08:00
57bcb1bfc0
fix: sparse: add inverted_index_algo and dim_max_score_ratio config ( #39362 )
...
issue: #39332
pr: #39358
Signed-off-by: Shawn Wang <shawn.wang@zilliz.com>
2025-01-18 12:17:03 +08:00
f704e3edef
test: [skip e2e][cherry-pick]fix pytest_addoption type ( #39397 )
...
pr: https://github.com/milvus-io/milvus/pull/38860
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2025-01-17 16:59:02 +08:00
b2f4df49d6
test: [cherry-pick]add scalar index for compatibility testing ( #39380 )
...
pr: https://github.com/milvus-io/milvus/pull/39291
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2025-01-17 16:11:02 +08:00
0c00f94965
fix: [2.5] Skip load extra indexes for sorted segment pk field ( #39390 )
...
Cherry-pick from master
pr: #39389
Related to #39339
Extra indexes can be ignored for most cases since sorted pk column
already provided indexing features
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2025-01-17 14:53:03 +08:00
bb32bda423
enhance: [2.5] Sync multipleChunkEnabled default value & milvus yaml ( #39373 )
...
Cherry-pick from master
pr: #39372
The default value and yaml have different values which may cause
confusion when upgrading from older version.
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2025-01-17 12:07:02 +08:00
89a183c7c2
enhance: [2.5] enable task delta cache ( #39349 )
...
When there are many segment tasks in the querycoord scheduler, the
traversal in GetSegmentTaskDelta checks becomes time-consuming. This PR
adds caching for segment deltas.
issue: https://github.com/milvus-io/milvus/issues/37630
pr: https://github.com/milvus-io/milvus/pull/39307
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
Co-authored-by: Wei Liu <wei.liu@zilliz.com>
2025-01-17 12:01:03 +08:00
5330fc981c
fix: panic when streaming release if using msgstream ( #39375 )
...
issue: #39367
pr: #39374
Signed-off-by: chyezh <chyezh@outlook.com>
2025-01-17 11:55:03 +08:00
56cb1683eb
fix: Fix performance issue and use after free bug ( #39343 )
...
cherry pick some fixes in https://github.com/milvus-io/milvus/pull/39249
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2025-01-17 11:51:03 +08:00
47e410feb1
fix: [2.5] fix assignment operator in AssertInfo to comparison operator ( #39351 )
...
cp https://github.com/milvus-io/milvus/pull/39347
issue: #39346
Signed-off-by: SpadeA-Tang <tangchenjie1210@gmail.com>
2025-01-17 11:27:02 +08:00
0391b8c7d3
fix: update rg failed when alter database ( #39356 )
...
issue: #36933
pr: #39355
Signed-off-by: chyezh <chyezh@outlook.com>
2025-01-17 10:37:03 +08:00
b69994272f
enhance: [2.5] Limit the maximum number of segments restored and fail the job if saving the binlog fails ( #39359 )
...
1. Limit the maximum number of restored segments to 1024.
2. Fail the import job if saving binlog fails.
3. Fail the import job if saving the import task fails to prevent
repeatedly generating dirty importing segments.
issue: https://github.com/milvus-io/milvus/issues/39331
pr: https://github.com/milvus-io/milvus/pull/39344
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2025-01-17 10:27:04 +08:00
e7520599df
fix: [2.5] remove param num_build_thread ( #39341 )
...
issue: #39090
pr: #39340
Signed-off-by: xianliang.li <xianliang.li@zilliz.com>
2025-01-16 18:05:01 +08:00
6773fb10a8
enhance: [2.5] Read metadata concurrently to accelerate recovery ( #38900 )
...
Read metadata such as segments, binlogs, and partitions concurrently at
the collection level.
issue: https://github.com/milvus-io/milvus/issues/37630
pr: https://github.com/milvus-io/milvus/pull/38403
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2025-01-16 17:53:01 +08:00
9d2a0e775c
fix: [2.5] Fix slow dist handle and slow observe ( #38905 )
...
1. Provide partition&channel level indexing in the collection target.
2. Make SegmentAction not wait for distribution.
3. Remove scheduler and target manager mutex
4. Optimize logging to reduce CPU overhead.
issue: https://github.com/milvus-io/milvus/issues/37630
pr: https://github.com/milvus-io/milvus/pull/38566
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2025-01-16 17:07:02 +08:00
e68d161e60
enhance: Bump milvus & proto version to v2.5.4 ( #39330 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2025-01-16 15:39:03 +08:00
448eb3b63a
fix: [2.5]bulkimport binlogs skip diskquota check ( #39319 )
...
bulkimport binlogs skip diskquota check
issue:https://github.com/milvus-io/milvus/issues/38845
pr: https://github.com/milvus-io/milvus/pull/39315
Signed-off-by: Xianhui.Lin <xianhui.lin@zilliz.com>
2025-01-16 15:33:05 +08:00
29dad64341
fix: [2.5] Fix consume blocked due to too many consumers ( #38915 )
...
This PR limits the maximum number of consumers per pchannel to 10 for
each QueryNode and DataNode.
issue: https://github.com/milvus-io/milvus/issues/37630
pr: https://github.com/milvus-io/milvus/pull/38455
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2025-01-16 15:19:03 +08:00
4cb47b3f67
feat: [2.5] root privileges can be customized ( #39324 )
...
- issue: #39184
- pr: #39191
Signed-off-by: SimFG <bang.fu@zilliz.com>
2025-01-16 14:27:11 +08:00
c945efa853
enhance: [2.5] Remove mutex from SegmentManger in querynode ( #39051 ) ( #39282 )
...
Remove mutex from SegmentManger in querynode to prevent mutex
contention.
issue: https://github.com/milvus-io/milvus/issues/37630
pr: https://github.com/milvus-io/milvus/pull/39051
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2025-01-16 13:27:02 +08:00
c741b8be2b
fix: [2.5] Remove frequently updating metric to avoid mutex contention ( #38778 )
...
issue: https://github.com/milvus-io/milvus/issues/37630
Reduce the frequency of `updateIndexTasksMetrics` to avoid holding the
mutex for long periods.
pr: https://github.com/milvus-io/milvus/pull/38775
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2025-01-16 11:51:02 +08:00
1602390734
enhance: support null expr( #38772 ) ( #38931 )
...
pr: #38772
issue: #31728
---------
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2025-01-16 10:55:07 +08:00
76ed552b00
enhance: Add logs for check health failed ( #39208 ) ( #39302 )
...
pr: #39208
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2025-01-16 10:31:04 +08:00
21df11b37c
enhance: [2.5] Add try-catch and return CStatus for NewCollection ( #39279 ) ( #39303 )
...
Cherry pick from master
pr: #39279
Related to #28795
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2025-01-16 10:13:06 +08:00
1d9788e4bc
fix: [2.5] enable to build index with single segment ( #39272 )
...
issue: #39232
cherry-pick https://github.com/milvus-io/milvus/pull/39233
---------
Signed-off-by: SpadeA-Tang <tangchenjie1210@gmail.com>
2025-01-16 09:59:02 +08:00
b658467107
enhance: [2.5] Unify LoadStateLock RLock & PinIf ( #39206 ) ( #39255 )
...
Cherry-pick from master
pr: #39206 #39308
Related to #39205
This PR merge `RLock` & `PinIfNotReleased` into `PinIf` function
preventing segment being released before any Read operation finished.
---------
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2025-01-16 00:33:03 +08:00
9b916f2cf8
enhance: [2.5] Skip creating partition rate limiters when not enable ( #38911 )
...
issue: https://github.com/milvus-io/milvus/issues/37630
pr: https://github.com/milvus-io/milvus/pull/38816
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2025-01-15 22:19:02 +08:00
dd44a58381
enhance: [2.5] pass partition key scalar info if enable for vector mem index ( #39245 )
...
issue: #34332
pr: #39123
---------
Signed-off-by: chasingegg <chao.gao@zilliz.com>
2025-01-15 21:45:01 +08:00
ThreadDao
qixuan
Bingyi Sun
Xianhui Lin
aoiasd
yihao.dai
cai.zhang
Gao
smellthemoon
congqixia
zhenshan.cao
Duc Le Tu
sthuang
XuanYang-cn
sparknack
zhuwenxing
Zhen Ye
Spade A
foxspy
SimFG
wei liu