6ab8e84f1e
fix: [2.5] improve error handling and unit tests for InitMetaCache function ( #40324 )
...
- issue: #40320
- pr: #40322
Signed-off-by: SimFG <bang.fu@zilliz.com>
2025-03-05 11:08:13 +08:00
cc7d4ce399
fix: [2.5] Add duplication param check for create index ( #40254 ) ( #40330 )
...
Cherry-pick from master
pr: #40254
Related to #40156
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2025-03-05 10:10:01 +08:00
a945345110
fix: [2.5] use the different msg type for the OperatePrivilegeV2 api ( #40193 )
...
- issue: #40178
- pr: #40192
Signed-off-by: SimFG <bang.fu@zilliz.com>
2025-03-03 10:20:01 +08:00
709594f158
enhance: [2.5] Use v2 package name for pkg module ( #40117 )
...
Cherry-pick from master
pr: #39990
Related to #39095
https://go.dev/doc/modules/version-numbers
Update pkg version according to golang dep version convention
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2025-02-23 00:46:01 +08:00
2677f0b834
fix: cancel sub contexts casade when http request timeout( #40030 ) ( #40060 )
...
related: #40030
pr: https://github.com/milvus-io/milvus/pull/40059
Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
2025-02-22 11:39:53 +08:00
0caad5fe37
enhance: [2.5] Add metrics for proxy queue ( #40071 )
...
master pr: #40070
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2025-02-21 21:39:54 +08:00
b8a758b6c4
enhance: [2.5] Add get vector latency metric and refine request limit error message ( #40085 )
...
issue: https://github.com/milvus-io/milvus/issues/40078
pr: https://github.com/milvus-io/milvus/pull/40083
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2025-02-21 20:19:55 +08:00
113d17646b
fix: [2.5] Add sub task pool for multi-stage tasks ( #40081 )
...
Cherry-pick from master
pr: #40079
Related to #40078
Add a subTaskPool to execute sub task in case of logic deadlock
described in issue.
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2025-02-21 19:17:55 +08:00
3c2d8c1419
enhance: [2.5] Add management api to check querycoord balance status ( #37784 ) ( #39909 )
...
issue: #37783
pr: #37784
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2025-02-19 10:56:49 +08:00
7ea0af572a
enhance: [2.5] Optimize the result format of GetQueryNodeDistribution ( #39664 ) ( #39926 )
...
pr: #39664
Use string array for SealedSegmentIDs to prevent precision loss in JSON
parsers. Large integers (int64) may be incorrectly rounded when parsed
as double.
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2025-02-18 09:59:07 +08:00
de8abe7970
fix:reset default auto index for varchar ( #39908 )
...
pr: #39819
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2025-02-17 20:55:26 +08:00
56c1a8d462
fix: ignore growing option is lost at hibridsearch ( #39900 )
...
issue: #39892
pr: #39799
Signed-off-by: chyezh <chyezh@outlook.com>
2025-02-17 15:30:15 +08:00
f0964f769d
enhance: [2.5]Add json key inverted index in stats for optimization ( #39876 )
...
Add json key inverted index in stats for optimization
issue: https://github.com/milvus-io/milvus/issues/36995
pr: https://github.com/milvus-io/milvus/pull/38039
---------
Signed-off-by: Xianhui.Lin <xianhui.lin@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2025-02-16 20:12:15 +08:00
16ea4fcb90
Revert "fix:reset default auto index type for scalar ( #37086 )" ( #39820 )
...
This reverts commit eeb67a3845#39819
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2025-02-13 17:36:14 +08:00
667c84740c
enhance: [2.5] Add metrics for parse expression ( #39716 )
...
master pr: #39654
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2025-02-12 10:30:46 +08:00
858dc10ef9
enhance: broadcast with event-based notification ( #39550 )
...
issue: #38399
pr: #39522
- broadcast message can carry multi resource key now.
- implement event-based notification for broadcast messages
- broadcast message use broadcast id as a unique identifier in message
- broadcasted message on vchannels keep the broadcasted vchannel now.
- broadcasted message and broadcast message have a common broadcast
header now.
---------
Signed-off-by: chyezh <chyezh@outlook.com>
2025-02-07 11:50:50 +08:00
ddc5b299ad
enhance: expose more metrics data ( #39466 )
...
issue: #36621 #39417
pr: #39456
1. Adjust the server-side cache size.
2. Add source information for configurations.
3. Add node ID for compaction and indexing tasks.
4. Resolve localhost access issues to fix health check failures for
etcd.
Signed-off-by: jaime <yun.zhang@zilliz.com>
2025-02-07 11:48:45 +08:00
8a0f0f4099
enhance: [2.5] Add dsl log field for HybridSearch ( #39584 ) ( #39598 )
...
Cherry-pick from master
pr: #39584
The dsl fields are separated in sub request structs and cannot be easily
printed before. This PR adds a log helper to print the dsl expressions
of HybridSearchRequest.
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2025-02-07 09:58:43 +08:00
c8327934a6
fix: [2.5]not enable rate limiter for restful v1 ( #39555 )
...
issue: #39556
pr: #39553
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2025-01-24 14:05:13 +08:00
6f7b2b4e75
enhance: [2.5] Refine error msg for schema & index checking ( #39533 ) ( #39565 )
...
Cherry-pick from master
pr: #39533
The error message was malformated or missing some meta info, say field
name. This PR recitfies some message format and add field name in error
message when type param check fails.
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2025-01-24 13:43:06 +08:00
30411d6d3a
fix: [2.5] deny to set the mmap param for the alter index api ( #39520 )
...
- issue: #39517
- pr: #39518
Signed-off-by: SimFG <bang.fu@zilliz.com>
2025-01-22 23:55:06 +08:00
f7799f0add
fix:[2.5] check utf-8 format for varchar with analyzer open ( #39439 )
...
relate: https://github.com/milvus-io/milvus/issues/39285
pr: https://github.com/milvus-io/milvus/pull/39299
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2025-01-21 10:57:03 +08:00
3b3959c625
enhance: [2.5]Add schema update time verification for insert and upsert to use cache ( #39382 )
...
enhance: Add schema update time verification for insert and upsert to
use cache
issue: https://github.com/milvus-io/milvus/issues/39093
Related to pr: https://github.com/milvus-io/milvus/pull/39096
---------
Signed-off-by: Xianhui.Lin <xianhui.lin@zilliz.com>
2025-01-21 10:19:11 +08:00
15d60c60c2
enhance: [2.5] remain token after authorize ( #39455 )
...
relate: https://github.com/milvus-io/milvus/issues/39285
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2025-01-20 20:27:04 +08:00
6773fb10a8
enhance: [2.5] Read metadata concurrently to accelerate recovery ( #38900 )
...
Read metadata such as segments, binlogs, and partitions concurrently at
the collection level.
issue: https://github.com/milvus-io/milvus/issues/37630
pr: https://github.com/milvus-io/milvus/pull/38403
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2025-01-16 17:53:01 +08:00
448eb3b63a
fix: [2.5]bulkimport binlogs skip diskquota check ( #39319 )
...
bulkimport binlogs skip diskquota check
issue:https://github.com/milvus-io/milvus/issues/38845
pr: https://github.com/milvus-io/milvus/pull/39315
Signed-off-by: Xianhui.Lin <xianhui.lin@zilliz.com>
2025-01-16 15:33:05 +08:00
4cb47b3f67
feat: [2.5] root privileges can be customized ( #39324 )
...
- issue: #39184
- pr: #39191
Signed-off-by: SimFG <bang.fu@zilliz.com>
2025-01-16 14:27:11 +08:00
dd44a58381
enhance: [2.5] pass partition key scalar info if enable for vector mem index ( #39245 )
...
issue: #34332
pr: #39123
---------
Signed-off-by: chasingegg <chao.gao@zilliz.com>
2025-01-15 21:45:01 +08:00
4c91e05a5d
enhance: fix inconsistenty of alias and db for query iterator( #39045 ) ( #39248 )
...
related: #39045
pr: https://github.com/milvus-io/milvus/pull/39216
Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
2025-01-15 10:36:59 +08:00
4a47f760b3
fix: [2.5] rbac custom group privilege level check ( #39164 ) ( #39224 )
...
cherry-pick from master: https://github.com/milvus-io/milvus/pull/39164
related: https://github.com/milvus-io/milvus/issues/39086
Signed-off-by: shaoting-huang <shaoting.huang@zilliz.com>
2025-01-14 16:50:59 +08:00
99a8274326
enhance: Add primary field name in SearchResult and QueryResults ( #39222 )
...
pr: https://github.com/milvus-io/milvus/pull/39220
issue: https://github.com/milvus-io/milvus/issues/39219
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2025-01-14 16:10:58 +08:00
3059fae40d
fix: [2.5] fix restore rbac empty meta crash ( #39143 )
...
cp from master: https://github.com/milvus-io/milvus/pull/39141
related: https://github.com/milvus-io/milvus/issues/38985
Signed-off-by: shaoting-huang <shaoting.huang@zilliz.com>
2025-01-13 12:04:58 +08:00
95809ca767
enhance: make new go package to manage proto ( #39128 )
...
issue: #39095
pr: #39114
---------
Signed-off-by: chyezh <chyezh@outlook.com>
2025-01-10 10:53:01 +08:00
0693634f62
enhance: add db name in replica description ( #38673 )
...
issue: #36621
pr: #38672
Signed-off-by: jaime <yun.zhang@zilliz.com>
2025-01-09 19:43:04 +08:00
4b62dfd11f
enhance: [2.5] Use mvcc timestamp as guarantee ts if set ( #38980 ) ( #39019 )
...
Cherry pick from master
pr: #38980
When MvccTimestamp is set, it could be used as guarantee timestamp
directly instead of new ts allocated by scheduler reducing the waiting
time when delegator has tsafe lag
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2025-01-07 19:18:57 +08:00
1d55ad6a1d
enhance: [2.5] add param for tuning max VARCHAR length and restore limit to 65535 ( #38883 )
...
issue: #38882
pr: https://github.com/milvus-io/milvus/pull/38884
Signed-off-by: Patrick Weizhi Xu <weizhi.xu@zilliz.com>
(cherry picked from commit a64a7377ea8258efe36c74bd1cf9db556726ba2d)
2025-01-02 13:56:54 +08:00
a2c4cd59ce
fix: drop partition can not be successful if load failed[2.5] ( #38874 )
...
fix https://github.com/milvus-io/milvus/issues/38649
pr: #38793
when partition load failed, the partition drop will also fail due to the
wrong error message
Signed-off-by: xiaofanluan <xiaofan.luan@zilliz.com>
2025-01-02 09:56:53 +08:00
ef400227ad
enhance: [2.5][cp] speed up search iterator stage 1 ( #38678 )
...
pr: https://github.com/milvus-io/milvus/pull/37947
issue: https://github.com/milvus-io/milvus/issues/37548
Signed-off-by: Patrick Weizhi Xu <weizhi.xu@zilliz.com>
(cherry picked from commit 9016c4adcd765c0766b01e7e5d465c915e176a6f)
2024-12-27 18:48:52 +08:00
a4901ef7ec
fix: [2.5] fix privilege group list and list collections ( #38738 )
...
cherry-pick from: https://github.com/milvus-io/milvus/pull/38684
related: https://github.com/milvus-io/milvus/issues/37031
Signed-off-by: shaoting-huang <shaoting.huang@zilliz.com>
2024-12-25 18:06:50 +08:00
c51b458156
fix: [2.5]fix altercollectionfield interceptor dbname ( #38663 )
...
fix altercollectionfield interceptor dbname
issue: https://github.com/milvus-io/milvus/issues/37436
master pr: https://github.com/milvus-io/milvus/pull/38662
Signed-off-by: Xianhui.Lin <xianhui.lin@zilliz.com>
2024-12-24 10:54:50 +08:00
7fe2cd0518
fix: [2.5] Support hybrid search with expression template ( #38625 )
...
issue: https://github.com/milvus-io/milvus/issues/38616
master pr: https://github.com/milvus-io/milvus/pull/38623
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-12-20 18:28:20 +08:00
3cf9b6bae5
enhance: [2.5]field stringtype maxlength raise to 1M ( #38614 )
...
enhance: field stringtype maxlength raise to 1M
issue: https://github.com/milvus-io/milvus/issues/37436
master pr: https://github.com/milvus-io/milvus/pull/38592
Signed-off-by: Xianhui.Lin <xianhui.lin@zilliz.com>
2024-12-20 17:08:45 +08:00
ca7ec23198
enhance: Use partitionID when delete by partitionKey ( #38231 )
...
When delete by partition_key, Milvus will generates L0 segments
globally. During L0 Compaction, those L0 segments will touch all
partitions collection wise. Due to the false-positive rate of segment
bloomfilters, L0 compactions will append false deltalogs to completed
irrelevant partitions, which causes *partition deletion amplification.
This PR uses partition_key to set targeted partitionID when producing
deleteMsgs into MsgStreams. This'll narrow down L0 segments scope to
partition level, and remove the false-positive influence
collection-wise.
However, due to DeleteMsg structure, we can only label one partition to
one deleteMsg, so this enhancement fails if user wants to delete over 2
partition_keys in one deletion.
See also: #34665
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-12-20 11:18:46 +08:00
894c203f27
enhance: allow hyphen in partition name ( #38494 )
...
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2024-12-18 15:04:45 +08:00
4d5572d7ed
enhance: [skip e2e] refine display format ( #38537 )
...
issue: #36621
Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-12-17 22:30:44 +08:00
93fba1d5ab
fix: access log remove new log not the old and skip rotate empty log ( #38309 )
...
relate: https://github.com/milvus-io/milvus/issues/38293
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-12-17 19:16:44 +08:00
2afe2eaf3e
feat: support to replicate collection when the services contains the system tt msg ( #37559 )
...
- issue: #37105
---------
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-12-17 09:08:46 +08:00
fa8ac09550
fix: the issue of replicate message exception when the ttMsgEnable config is changed dynamically ( #38178 )
...
- issue: #38177
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-12-14 23:24:51 +08:00
27229f7907
enhance: refine exists log print with ctx ( #38080 )
...
issue: #35917
Refines exists log print with ctx
Signed-off-by: tinswzy <zhenyuan.wei@zilliz.com>
2024-12-14 22:36:44 +08:00
b416ff8c6c
enhance: altercollectionfield support mmap_enable convert to mmap.enable ( #38448 )
...
altercollectionfield support mmap_enable convert to mmap.enable
issue: https://github.com/milvus-io/milvus/issues/37436
---------
Signed-off-by: Xianhui.Lin <xianhui.lin@zilliz.com>
2024-12-13 20:02:43 +08:00
SimFG
congqixia
Chun Han
cai.zhang
yihao.dai
wei liu
zhagnlu
Zhen Ye
Xianhui Lin
jaime
smellthemoon
aoiasd
Gao
sthuang
zhenshan.cao
Patrick Weizhi Xu
Xiaofan
XuanYang-cn
Bingyi Sun
tinswzy