Commit Graph

426 Commits (2.5)

Author SHA1 Message Date
congqixia e13b55fc0a
fix: [2.5] Remove trimming space logic when validating collection name (#43064) (#43138)
Cherry-pick from master
pr: #43064
Related to #43031

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2025-07-07 09:54:46 +08:00
aoiasd d9706150d2
enhance: [2.5] run analyzer should get database name from grpc context (#42399)
pr:https://github.com/milvus-io/milvus/pull/42398

---------

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2025-05-30 15:48:30 +08:00
aoiasd 198ff1f150
enhance: [2.5] support run analyzer by loaded collection field (#42119)
relate: https://github.com/milvus-io/milvus/issues/42094
pr: https://github.com/milvus-io/milvus/pull/42113

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2025-05-29 10:26:30 +08:00
groot a48001f999
feat: support to drop properties of field (#41954)
issue: https://github.com/milvus-io/milvus/issues/41990
pr: https://github.com/milvus-io/milvus/pull/41996

Signed-off-by: yhmo <yihua.mo@zilliz.com>
2025-05-27 14:32:34 +08:00
SimFG 94e2871348
enhance: [2.5] implement authorization checks for DescribeCollection and DescribeDatabase tasks (#41799)
- issue: #41694
- pr: #41798

---------

Signed-off-by: SimFG <bang.fu@zilliz.com>
2025-05-15 17:48:23 +08:00
SimFG 4c656e8a7a
enhance: support more ddl apis to replicate according to cdc (#41679)
- issue: #41677
- pr: #41678

Signed-off-by: SimFG <bang.fu@zilliz.com>
2025-05-08 20:02:58 +08:00
Xianhui Lin 17465ffe9e
feat: [2.5]add str update ts in describecollection (#41600)
issue:https://github.com/milvus-io/milvus/issues/39093
pr:https://github.com/milvus-io/milvus/pull/39096

Signed-off-by: Xianhui.Lin <xianhui.lin@zilliz.com>
2025-04-29 17:50:53 +08:00
SimFG 18eb627533
fix: [2.5] Update logging context and upgrade dependencies (#41319)
- issue: #41291
- pr: #41318

---------

Signed-off-by: SimFG <bang.fu@zilliz.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2025-04-24 23:50:40 +08:00
aoiasd 544493e3e2
feat:[2.5] support multi analyzer for bm25 function (#41456)
relate: https://github.com/milvus-io/milvus/issues/41213
pr: https://github.com/milvus-io/milvus/pull/41351

---------

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2025-04-23 20:52:39 +08:00
Xianhui Lin 28c9dedb04
feat: [2.5]Add support for modifying max capacity of array fields (#41406)
feat: Add support for modifying max capacity of array fields

This commit adds support for modifying the max capacity of array fields
in the alterCollectionFieldTask function. It checks if the field is an
array type and then validates and updates the max capacity value. This
change improves the flexibility of array fields in the collection.

Issue: https://github.com/milvus-io/milvus/issues/41363
pr:https://github.com/milvus-io/milvus/pull/41404

---------

Signed-off-by: Xianhui.Lin <xianhui.lin@zilliz.com>
2025-04-21 18:00:37 +08:00
Buqian Zheng c9a354d436
enhance: [2.5] weighted reranker to allow skip score normalization (#40905)
issue: https://github.com/milvus-io/milvus/issues/40836
pr: https://github.com/milvus-io/milvus/pull/40903

Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com>
2025-04-02 17:30:22 +08:00
congqixia 709594f158
enhance: [2.5] Use v2 package name for pkg module (#40117)
Cherry-pick from master
pr: #39990
Related to #39095

https://go.dev/doc/modules/version-numbers

Update pkg version according to golang dep version convention

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2025-02-23 00:46:01 +08:00
congqixia 113d17646b
fix: [2.5] Add sub task pool for multi-stage tasks (#40081)
Cherry-pick from master
pr: #40079
Related to #40078

Add a subTaskPool to execute sub task in case of logic deadlock
described in issue.

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2025-02-21 19:17:55 +08:00
Zhen Ye 56c1a8d462
fix: ignore growing option is lost at hibridsearch (#39900)
issue: #39892
pr: #39799

Signed-off-by: chyezh <chyezh@outlook.com>
2025-02-17 15:30:15 +08:00
Xianhui Lin 3b3959c625
enhance: [2.5]Add schema update time verification for insert and upsert to use cache (#39382)
enhance: Add schema update time verification for insert and upsert to
use cache
issue: https://github.com/milvus-io/milvus/issues/39093
Related to pr: https://github.com/milvus-io/milvus/pull/39096

---------

Signed-off-by: Xianhui.Lin <xianhui.lin@zilliz.com>
2025-01-21 10:19:11 +08:00
Gao dd44a58381
enhance: [2.5] pass partition key scalar info if enable for vector mem index (#39245)
issue: #34332 
pr: #39123

---------

Signed-off-by: chasingegg <chao.gao@zilliz.com>
2025-01-15 21:45:01 +08:00
Chun Han 4c91e05a5d
enhance: fix inconsistenty of alias and db for query iterator(#39045) (#39248)
related: #39045
pr: https://github.com/milvus-io/milvus/pull/39216

Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
2025-01-15 10:36:59 +08:00
Zhen Ye 95809ca767
enhance: make new go package to manage proto (#39128)
issue: #39095
pr: #39114

---------

Signed-off-by: chyezh <chyezh@outlook.com>
2025-01-10 10:53:01 +08:00
Patrick Weizhi Xu 1d55ad6a1d
enhance: [2.5] add param for tuning max VARCHAR length and restore limit to 65535 (#38883)
issue: #38882
pr: https://github.com/milvus-io/milvus/pull/38884

Signed-off-by: Patrick Weizhi Xu <weizhi.xu@zilliz.com>
(cherry picked from commit a64a7377ea8258efe36c74bd1cf9db556726ba2d)
2025-01-02 13:56:54 +08:00
Xiaofan a2c4cd59ce
fix: drop partition can not be successful if load failed[2.5] (#38874)
fix https://github.com/milvus-io/milvus/issues/38649
pr: #38793
when partition load failed, the partition drop will also fail due to the
wrong error message

Signed-off-by: xiaofanluan <xiaofan.luan@zilliz.com>
2025-01-02 09:56:53 +08:00
Patrick Weizhi Xu ef400227ad
enhance: [2.5][cp] speed up search iterator stage 1 (#38678)
pr: https://github.com/milvus-io/milvus/pull/37947
issue: https://github.com/milvus-io/milvus/issues/37548

Signed-off-by: Patrick Weizhi Xu <weizhi.xu@zilliz.com>
(cherry picked from commit 9016c4adcd765c0766b01e7e5d465c915e176a6f)
2024-12-27 18:48:52 +08:00
Xianhui Lin 3cf9b6bae5
enhance: [2.5]field stringtype maxlength raise to 1M (#38614)
enhance: field stringtype maxlength raise to 1M
issue: https://github.com/milvus-io/milvus/issues/37436
master pr: https://github.com/milvus-io/milvus/pull/38592

Signed-off-by: Xianhui.Lin <xianhui.lin@zilliz.com>
2024-12-20 17:08:45 +08:00
SimFG 2afe2eaf3e
feat: support to replicate collection when the services contains the system tt msg (#37559)
- issue: #37105

---------

Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-12-17 09:08:46 +08:00
tinswzy 27229f7907
enhance: refine exists log print with ctx (#38080)
issue: #35917 
Refines exists log print with ctx

Signed-off-by: tinswzy <zhenyuan.wei@zilliz.com>
2024-12-14 22:36:44 +08:00
Xianhui Lin b416ff8c6c
enhance: altercollectionfield support mmap_enable convert to mmap.enable (#38448)
altercollectionfield support mmap_enable convert to mmap.enable 
issue: https://github.com/milvus-io/milvus/issues/37436

---------

Signed-off-by: Xianhui.Lin <xianhui.lin@zilliz.com>
2024-12-13 20:02:43 +08:00
Xianhui Lin f2d79b3d3a
enhance: altercollection validation logic (#38419)
altercollection validation logic
issue: https://github.com/milvus-io/milvus/issues/37436

---------

Signed-off-by: Xianhui.Lin <xianhui.lin@zilliz.com>
2024-12-12 21:42:42 +08:00
Xianhui Lin db05d4f976
enhance: alterindex & altercollection supports altering properties (#37437)
enhance : 

1. alterindex delete properties
We have introduced a new parameter deleteKeys to the alterindex
functionality, which allows for the deletion of properties within an
index. This enhancement provides users with the flexibility to manage
index properties more effectively by removing specific keys as needed.
2. altercollection delete properties
We have introduced a new parameter deleteKeys to the altercollection
functionality, which allows for the deletion of properties within an
collection. This enhancement provides users with the flexibility to
manage collection properties more effectively by removing specific keys
as needed.

3.support altercollectionfield
We currently support modifying the fieldparams of a field in a
collection using altercollectionfield, which only allows changes to the
max-length attribute.
Key Points:
- New Parameter - deleteKeys: This new parameter enables the deletion of
specified properties from an index. By passing a list of keys to
deleteKeys, users can remove the corresponding properties from the
index.

- Mutual Exclusivity: The deleteKeys parameter cannot be used in
conjunction with the extraParams parameter. Users must choose one
parameter to pass based on their requirement. If deleteKeys is provided,
it indicates an intent to delete properties; if extraParams is provided,
it signifies the addition or update of properties.

issue: https://github.com/milvus-io/milvus/issues/37436

---------

Signed-off-by: Xianhui.Lin <xianhui.lin@zilliz.com>
2024-12-11 10:20:42 +08:00
congqixia fec31fedce
enhance: Move `SyncCreatedPartition` step to proxy (#38296)
Related to #38275

This PR move sync created partition step to proxy to avoid potential
logic deadlock when create partition happens with target segment change.

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-12-10 12:30:43 +08:00
Xiaofan 33bfb25c73
enhance: refine meta cache log and logic (#37318)
related to #36989
add more logs in proxy meta cache and make it clearer

Signed-off-by: xiaofanluan <xiaofan.luan@zilliz.com>
2024-11-19 10:50:33 +08:00
Chun Han 2d29dcd30c
enhance:refine group_strict_size parameter(#37482) (#37483)
related: #37482

Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
2024-11-12 09:56:28 +08:00
aoiasd 12951f0abb
enhance: rename tokenizer to analyzer and check analyzer params (#37478)
relate: https://github.com/milvus-io/milvus/issues/35853

---------

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-11-10 16:12:26 +08:00
wayblink d119a2541a
fix: fix hasCollection response has no status (#37254)
issue: #37257

Signed-off-by: wayblink <anyang.wang@zilliz.com>
2024-11-01 15:40:21 +08:00
cai.zhang 2ef6cbbf59
feat: The expression supports filling elements through templates (#37033)
issue: #36672

The expression supports filling elements through templates, which helps
to reduce the overhead of parsing the elements.

---------

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-10-31 14:20:22 +08:00
congqixia d8c1bd24f2
enhance: Utilize proxy metacache for `HasCollection` (#37185)
Related to #37183

Utilize proxy metacache for `HasCollection` request, if collection
exists in metacache, it could be deducted that collection must exist in
system.

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-10-28 18:54:23 +08:00
congqixia d07338b4b9
enhance: Ignore index check for non-loaded vector field (#36170)
Related to #35996

For `Field Partial Load` feature, Milvus shall ignore index check for
non-loaded vector field.

Also, this PR unifies the logic of index check for load collection and
load partitions tasks.

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-09-14 11:01:08 +08:00
smellthemoon 2d05b7f219
fix: to add check in partition key field when set nullable==true (#36218)
#36213

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-09-14 10:53:08 +08:00
aoiasd da227ff9a1
feat: Support create collection with functions (#35973)
relate: https://github.com/milvus-io/milvus/issues/35853
Support create collection with functions. Prepare for support bm25
function.

---------

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-09-12 10:43:06 +08:00
Jiquan Long 89bf226f0b
feat: support keyword text match (#35923)
fix: #35922

---------

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2024-09-10 15:11:08 +08:00
Chun Han e480b103bd
feat: supporing hybrid search group_by (#35982)
related: #35096

Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
2024-09-08 17:09:04 +08:00
Chun Han 4641fd9195
enhance: make search groupby stop when reaching topk groups (#35814)
related: #33544

Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
2024-09-02 18:25:03 +08:00
congqixia 985d84d3ee
enhance: Add skip load validation for create collection task (#35737)
Related to #35415

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-08-29 10:05:08 +08:00
Zhen Ye 99dff06391
enhance: using streaming service in insert/upsert/flush/delete/querynode (#35406)
issue: #33285

- using streaming service in insert/upsert/flush/delete/querynode
- fixup flusher bugs and refactor the flush operation
- enable streaming service for dml and ddl
- pass the e2e when enabling streaming service
- pass the integration tst when enabling streaming service

---------

Signed-off-by: chyezh <chyezh@outlook.com>
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-08-29 10:03:08 +08:00
Chun Han bfd9d86fe9
feat: support groupby size on go-layer(#33544) (#33845)
related: #33544

Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
2024-08-27 14:21:00 +08:00
SimFG 731d45abbe
enhance: provide more general configuration to control mmap behavior (#35359)
- issue: #35273

Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-08-21 00:22:54 +08:00
congqixia 2fbc628994
feat: Support field partial load collection (#35416)
Related to #35415

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-08-20 16:49:02 +08:00
congqixia de8a266d8a
enhance: Enable linux code checker (#35084)
See also #34483

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-07-30 15:53:51 +08:00
wei liu c45f38aa61
enhance: Update protobuf-go to protobuf-go v2 (#34394)
issue: #34252

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-07-29 11:31:51 +08:00
Jiquan Long a2ac84bd64
feat: record the duration waiting in the proxy queue (#34744)
fix: https://github.com/milvus-io/milvus/issues/34743

---------

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2024-07-23 14:23:52 +08:00
SimFG 88c38a3756
fix: `execute` method should return an error when result is a failure (#34813)
- issue: #34812

Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-07-20 23:13:40 +08:00
Patrick Weizhi Xu 104d0966b7
feat: support partition key isolation (#34336)
issue: #34332

---------

Signed-off-by: Patrick Weizhi Xu <weizhi.xu@zilliz.com>
2024-07-11 19:01:35 +08:00