Commit Graph

21762 Commits (a945345110acce92da67ee5ed2f61ebbc93cf198)

Author SHA1 Message Date
SimFG a945345110
fix: [2.5] use the different msg type for the OperatePrivilegeV2 api (#40193)
- issue: #40178
- pr: #40192

Signed-off-by: SimFG <bang.fu@zilliz.com>
2025-03-03 10:20:01 +08:00
yihao.dai 893caee467
fix: [2.5] Fix task delta cache data race (#40262)
issue: https://github.com/milvus-io/milvus/issues/40258

pr: https://github.com/milvus-io/milvus/pull/40259

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2025-03-02 16:52:10 +08:00
congqixia 3f76e1c6d1
enhance: [GoSDK][2.5] Use pkg v2 module path (#40134) (#40269)
Cherry-pick from master
pr: #40134
Related to #39095
Previous PR #39990 update pkg module path using "/v2" package name, this
PR update milvusclient go sdk dependency for this update

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2025-03-01 11:00:01 +08:00
wei liu 82c000a4b2
fix: task delta cache leak due to duplicate task id (#40184)
issue: #40052
pr: #40183

task delta cache rely on the taskID is unique, so it incDeltaCache at
AddTask, and decDeltaCache at RemoveTask, but the taskID allocator is
not atomic, which cause two task with same taskID, in such case, it will
call incDeltaCache twice, but call decDeltaCacheOnce, which cause delta
cache leak.

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2025-02-28 10:22:08 +08:00
zhuwenxing 00eda6fe31
test: [cherry-pick]add concurrent index creation verification (#40175)
pr: https://github.com/milvus-io/milvus/pull/40174

/kind improvement

Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2025-02-27 20:11:59 +08:00
zhuwenxing bc318732ad
test: fix checker function name, release mistake and add nullable (#40092)
pr: https://github.com/milvus-io/milvus/pull/40135

/kind improvement

Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2025-02-27 20:10:09 +08:00
wei liu 14f05650e3
enhance: clean shard location cache after collection released (#40228)
issue: #40077
pr: #40088

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2025-02-27 19:42:05 +08:00
congqixia 2f4f5945cb
enhance: [2.5] Unify milvus version def (#40229)
Cherry-pick from master
pr: #40227 
Related to #40216

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2025-02-27 14:17:58 +08:00
zhikunyao 0ff731487f
enhance: bump etcd version to 3.5.18 (#40230)
Signed-off-by: Zhikun Yao <zhikun.yao@zilliz.com>
2025-02-27 11:52:00 +08:00
sre-ci-robot bc77bc18c9
[automated] Bump milvus version to v2.5.5 (#40224)
Bump milvus version to v2.5.5
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2025-02-27 10:56:06 +08:00
nico 1d048cb25a
enhance: [skip e2e] change nightly running time[2.5] (#40218)
pr: #40219

Signed-off-by: nico <cheng.yuan@zilliz.com>
2025-02-26 19:18:08 +08:00
yanliang567 84f7e2d895
test: Update search iterator result check for dup ids (#40190)
Related issue: #40161 
1. fix the search iterator result assertion for dup ids
2. use normalized vectors for iterator tests

---------

Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
2025-02-26 16:49:59 +08:00
zhuwenxing 32c00dbc1b
test: fix duplicate import file names in tests (#40182)
pr: https://github.com/milvus-io/milvus/pull/40181

/kind improvement

Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2025-02-26 14:02:05 +08:00
zhenshan.cao 27fb8d9512
fix: create multiple identical indexes by accident (#40180)
issue: https://github.com/milvus-io/milvus/issues/40163
pr: https://github.com/milvus-io/milvus/pull/40179

---------

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2025-02-25 22:19:56 +08:00
yihao.dai eee98fd044
fix: [2.5] Fix collections with duplicate names can be created (#40147)
This PR introduces two restrictions:

1. Before dropping a collection, all aliases associated with that
collection must be dropped.
2. When creating a collection, if the collection name duplicates any
alias, the collection creation will fail.

issue: https://github.com/milvus-io/milvus/issues/40142

pr: https://github.com/milvus-io/milvus/pull/40143

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2025-02-25 22:07:56 +08:00
zhagnlu d60511abc3
Revert "fix:fix create duplicated index when concurrency request (#40… (#40176)
…162)"

This reverts commit 34f907fd5d.
pr: #40164

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2025-02-25 17:55:11 +08:00
zhagnlu c45771af6a
enhance: add monitor metric for retrieve raw data (#40155)
pr: #40141

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2025-02-25 14:41:56 +08:00
zhagnlu 34f907fd5d
fix:fix create duplicated index when concurrency request (#40162)
pr: #40164

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2025-02-25 13:49:07 +08:00
cai.zhang 4a75fdac03
enhance: Remove task meta when task is no need to do (#40146)
issue: #39911 

master pr: #39084

---------

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2025-02-25 10:35:57 +08:00
nico 0c6518f344
test: add iterator check and update sdk version and cases[2.5] (#40132)
pr: #39798 #39960

Signed-off-by: nico <cheng.yuan@zilliz.com>
2025-02-24 19:13:56 +08:00
Bingyi Sun 29579a8ec9
fix: Fix search failure of null expression (#40128)
issue: https://github.com/milvus-io/milvus/issues/40095
pr: #40129

Signed-off-by: sunby <sunbingyi1992@gmail.com>
2025-02-24 18:35:55 +08:00
cai.zhang 69f6849dc4
fix: [2.5] Fix bug where prefix matching fails when wildcards are in prefix (#40021)
issue: #40019 

master pr: #40020

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2025-02-24 18:03:55 +08:00
yihao.dai b6b03ff74c
enhance: [2.5] Accelerate listing objects during binlog import (#40048)
issue: https://github.com/milvus-io/milvus/issues/40030

pr: https://github.com/milvus-io/milvus/pull/40047

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2025-02-24 15:59:56 +08:00
cai.zhang fdad35e668
fix: [2.5] Set task version for stats task (#40130)
issue: #40034 

master pr: #40035

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2025-02-24 15:43:55 +08:00
Xianhui Lin a4eb2ce224
fix: [2.5]Revert qc statschecker for json key stats (#40125)
Revert qc statschecker for json key stats
issue:https://github.com/milvus-io/milvus/issues/36995
pr:https://github.com/milvus-io/milvus/pull/39876

Signed-off-by: Xianhui.Lin <xianhui.lin@zilliz.com>
2025-02-24 13:31:55 +08:00
Chun Han cb82b040a7
enhance: support create collection with description(#40022) (#40028)
related: #40022
pr: https://github.com/milvus-io/milvus/pull/40023

Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
2025-02-24 09:55:57 +08:00
cai.zhang 6c81463263
fix: [2.5] Fix task scheduler dead lock (#40121)
issue: #39101 

master pr: #39084

Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2025-02-23 18:29:55 +08:00
cai.zhang 9cbab98382
enhance: [2.5] Export request timeout interval in config (#40118) 2025-02-23 13:47:12 +08:00
congqixia 709594f158
enhance: [2.5] Use v2 package name for pkg module (#40117)
Cherry-pick from master
pr: #39990
Related to #39095

https://go.dev/doc/modules/version-numbers

Update pkg version according to golang dep version convention

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2025-02-23 00:46:01 +08:00
congqixia ecdbc12678
enhance: [2.5] Remove hardcoded partition num in restful handler (#40113)
Cherry-pick from master
pr: #40112
The partition num shall be determined by core logic if user did not
specifiy the partition num in request.

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2025-02-22 16:58:17 +08:00
Xianhui Lin c1de61ff7c
fix: [2.5]Replace the position of EnabledJSONKeyStats (#40108)
Replace the position of EnabledJSONKeyStats
issue: https://github.com/milvus-io/milvus/issues/36995
pr: https://github.com/milvus-io/milvus/pull/38039

---------

Signed-off-by: Xianhui.Lin <xianhui.lin@zilliz.com>
2025-02-22 14:35:54 +08:00
Chun Han 2677f0b834
fix: cancel sub contexts casade when http request timeout(#40030) (#40060)
related: #40030
pr: https://github.com/milvus-io/milvus/pull/40059

Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
2025-02-22 11:39:53 +08:00
congqixia cb0dc6293c
enhance: [2.5] Sync proxy.maxTaskNum default value to 1024 (#40073)
Cherry-pick from master
pr: #40072
Sync milvus.yaml value to default value for maxTaskNum

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2025-02-22 11:31:53 +08:00
zhagnlu d66d3f0d47
enhance:decrease dump snapshot limit from 10w to 1w (#40102)
pr: #40101

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2025-02-22 11:29:53 +08:00
cai.zhang be30f5301d
enhance: Refine index task scheduler policy (#40104)
master pr: #39084 

issue: #39101

---------

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2025-02-22 11:25:54 +08:00
congqixia 898606ae4c
enhance: [2.5] Avoid stringtoslicebytes copy for BatchPKExists (#40097)
Cherry-pick from master
pr: #40096 
Using unsafe.Slice to convert string to []byte by directly using
underlying data could avoid lots of copy and cpu time

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2025-02-21 21:41:21 +08:00
cai.zhang 0caad5fe37
enhance: [2.5] Add metrics for proxy queue (#40071)
master pr: #40070

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2025-02-21 21:39:54 +08:00
yihao.dai b8a758b6c4
enhance: [2.5] Add get vector latency metric and refine request limit error message (#40085)
issue: https://github.com/milvus-io/milvus/issues/40078

pr: https://github.com/milvus-io/milvus/pull/40083

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2025-02-21 20:19:55 +08:00
congqixia 113d17646b
fix: [2.5] Add sub task pool for multi-stage tasks (#40081)
Cherry-pick from master
pr: #40079
Related to #40078

Add a subTaskPool to execute sub task in case of logic deadlock
described in issue.

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2025-02-21 19:17:55 +08:00
Chun Han 39b1f4aaea
enhance: support return configable properties when describing index(#39951) (#40043)
related: #39951
pr: https://github.com/milvus-io/milvus/pull/40042

Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
2025-02-21 18:51:55 +08:00
wei liu 82fb0bf9c1
fix: [2.5] task delta cache leak on reduce task (#40056)
issue: #40052
pr: #40055

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2025-02-21 16:49:54 +08:00
wei liu e42c944e04
fix: [2.5] querycoord panic in cornor case (#40058)
issue: #40050 
pr: #40057

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2025-02-21 11:19:58 +08:00
congqixia b7c631f0b7
enhance: [2.5] Remove frequent json stats distribution log (#40027)
Info log is too frequent only printing the json stats & index info

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2025-02-20 13:33:57 +08:00
zhenshan.cao f142b5bff8
fix: Fix wrong trigger of json key index (#40026)
pr: https://github.com/milvus-io/milvus/pull/38039

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2025-02-20 03:59:11 +08:00
SimFG 37874ed6d7
fix: [2.5] enhance isBalanced function to correctly count quote pairs (#40002)
- issue: #39999
- pr: #40001

Signed-off-by: SimFG <bang.fu@zilliz.com>
2025-02-19 20:19:00 +08:00
zhagnlu ce6d8ba836
enhance: optimize expr performace for some points (#39695) (#39938)
1. skip get expr arguments which deserialize proto for every batch
execute.
2. replace unordered_set with sort array that has better performace for
small set.
pr: #39695

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2025-02-19 19:52:53 +08:00
congqixia cae8f75e47
enhance: Bump milvus version to 2.5.5 (#40017)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2025-02-19 18:48:54 +08:00
XuanYang-cn bd5c56a77b
fix: [cp25]Negative -1 executing compaction tasks (#39955)
See also: #39675
pr: #39954

---------

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2025-02-19 18:42:53 +08:00
congqixia 96922c0a87
fix: [2.5] Use start pos ts instead for sealSegmentByLifetime policy (#39994)
Cherry-pick from master
pr: #39982
Related to #39981

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2025-02-19 17:48:55 +08:00
XuanYang-cn cc6b2ff150
fix: [cp25]A segment may never transfer from sealed to flushing (#39996)
See also: #39717
pr: #39993

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2025-02-19 16:50:59 +08:00