Commit Graph

749 Commits (2.5)

Author SHA1 Message Date
cai.zhang 867cab9ac1
fix: [2.5] Call AlterCollection when only rename collection (#43421)
issue: #43407 

master pr: #43420

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2025-07-18 15:47:07 +08:00
cai.zhang 513ff7d331
enhance: [2.5] Skip modify field meta when rename collection or rename dbName (#42876)
issue: #42873 

master pr: #42875

---------

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2025-06-23 18:24:43 +08:00
groot a48001f999
feat: support to drop properties of field (#41954)
issue: https://github.com/milvus-io/milvus/issues/41990
pr: https://github.com/milvus-io/milvus/pull/41996

Signed-off-by: yhmo <yihua.mo@zilliz.com>
2025-05-27 14:32:34 +08:00
SimFG 94e2871348
enhance: [2.5] implement authorization checks for DescribeCollection and DescribeDatabase tasks (#41799)
- issue: #41694
- pr: #41798

---------

Signed-off-by: SimFG <bang.fu@zilliz.com>
2025-05-15 17:48:23 +08:00
groot bf8e6e8ac0
enhance: Alter collection description (#41547)
issue: https://github.com/milvus-io/milvus/issues/41557
master pr: https://github.com/milvus-io/milvus/pull/41558

Signed-off-by: yhmo <yihua.mo@zilliz.com>
2025-05-12 11:22:56 +08:00
Xianhui Lin dbcdb30db2
fix: [2.5]set quota center metrics configuration before watch (#41708)
fix: [2.5]set quota center metrics configuration before watch
pr:https://github.com/milvus-io/milvus/pull/41706
issue:https://github.com/milvus-io/milvus/issues/35177

Signed-off-by: Xianhui.Lin <xianhui.lin@zilliz.com>
2025-05-09 14:30:55 +08:00
Xianhui Lin 17465ffe9e
feat: [2.5]add str update ts in describecollection (#41600)
issue:https://github.com/milvus-io/milvus/issues/39093
pr:https://github.com/milvus-io/milvus/pull/39096

Signed-off-by: Xianhui.Lin <xianhui.lin@zilliz.com>
2025-04-29 17:50:53 +08:00
SimFG 6e18ededab
fix: [2.5] mockery too unavailable after upgrade golang version (#41522)
- issue: ##41291
- pr: #41481

Signed-off-by: SimFG <bang.fu@zilliz.com>
2025-04-25 14:40:40 +08:00
SimFG 18eb627533
fix: [2.5] Update logging context and upgrade dependencies (#41319)
- issue: #41291
- pr: #41318

---------

Signed-off-by: SimFG <bang.fu@zilliz.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2025-04-24 23:50:40 +08:00
Xiaowei Shi ca5199c5de
fix: check PreCreatedTopic first in shard number validation (#41274) (#41421)
cherry-pick from master: https://github.com/milvus-io/milvus/pull/41274
related: https://github.com/milvus-io/milvus/issues/41271

Signed-off-by: Xiaowei Shi <shallwe.shih@gmail.com>
2025-04-21 21:52:38 +08:00
sthuang f047e9cd80
enhance: optimize error log message for list policy (#41251) (#41368)
cherry-pick from master: https://github.com/milvus-io/milvus/pull/41251
related: #41250

Signed-off-by: shaoting-huang <shaoting.huang@zilliz.com>
2025-04-18 10:58:37 +08:00
sthuang b7858d24a0
fix: [2.5] show collections support custom privilege groups granted objects (#41204)
cherry-pick from master: https://github.com/milvus-io/milvus/pull/41203
related: https://github.com/milvus-io/milvus/issues/41200

Signed-off-by: shaoting-huang <shaoting.huang@zilliz.com>
2025-04-10 10:54:26 +08:00
SimFG 1263505808
feat: [2.5] support to deny dll according to database property (#40784)
- issue: #40762
- pr: #40764

Signed-off-by: SimFG <bang.fu@zilliz.com>
2025-03-23 11:18:28 +08:00
Zhen Ye 167d058682
enhance: remove redundant resource key watch operation, just keep consistency of wal (#40503)
issue: #38399
pr: #40235

related PR: #39522

- Just implement exclusive broadcaster between broadcast message with
same resource key to keep same order in different wal.
- After simplify the broadcast model, original watch-based broadcast is
too complicated and redundant, remove it.
- Add metrics for broadcast.

---------

Signed-off-by: chyezh <chyezh@outlook.com>
2025-03-18 15:46:25 +08:00
SimFG a3c60a03b5
feat: [2.5] add DDLDB rate type and related quota configurations (#40653)
- issue: #40650
- pr: #40651

Signed-off-by: SimFG <bang.fu@zilliz.com>
2025-03-14 15:36:17 +08:00
congqixia 004668b14c
fix: [2.5] Add duplicated type/index params check creating collection (#40465)
Cherry-pick from master
pr: #40462
Related to #40461

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2025-03-11 10:14:11 +08:00
yihao.dai eee98fd044
fix: [2.5] Fix collections with duplicate names can be created (#40147)
This PR introduces two restrictions:

1. Before dropping a collection, all aliases associated with that
collection must be dropped.
2. When creating a collection, if the collection name duplicates any
alias, the collection creation will fail.

issue: https://github.com/milvus-io/milvus/issues/40142

pr: https://github.com/milvus-io/milvus/pull/40143

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2025-02-25 22:07:56 +08:00
congqixia 709594f158
enhance: [2.5] Use v2 package name for pkg module (#40117)
Cherry-pick from master
pr: #39990
Related to #39095

https://go.dev/doc/modules/version-numbers

Update pkg version according to golang dep version convention

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2025-02-23 00:46:01 +08:00
Xianhui Lin 8c86e4e6ac
fix: [2.5]AlterCollection unable to modify ConsistencyLevel (#39902)
fix: AlterCollection unable to modify ConsistencyLevel
issue: https://github.com/milvus-io/milvus/issues/39707
pr: https://github.com/milvus-io/milvus/pull/39708

Signed-off-by: Xianhui.Lin <xianhui.lin@zilliz.com>
2025-02-17 10:00:15 +08:00
Zhen Ye f85bf59921
fix: rootcoord graceful stop failed and limit resource of ci (#39793)
issue: #38399
pr: #39781

---------

Signed-off-by: chyezh <chyezh@outlook.com>
2025-02-13 19:34:13 +08:00
SimFG f542c866b3
fix: [2.5] root check the role list if `rootShouldBindRole` is true (#39714)
- issue: #39712
- pr: #39713

Signed-off-by: SimFG <bang.fu@zilliz.com>
2025-02-11 10:10:51 +08:00
Zhen Ye 858dc10ef9
enhance: broadcast with event-based notification (#39550)
issue: #38399
pr: #39522

- broadcast message can carry multi resource key now.
- implement event-based notification for broadcast messages
- broadcast message use broadcast id as a unique identifier in message
- broadcasted message on vchannels keep the broadcasted vchannel now.
- broadcasted message and broadcast message have a common broadcast
header now.

---------

Signed-off-by: chyezh <chyezh@outlook.com>
2025-02-07 11:50:50 +08:00
sthuang bc91b2a4ad
fix: [2.5] metastore privilege name check with privilege name all (#39492)
cherry-pick from master: https://github.com/milvus-io/milvus/pull/39476
related: https://github.com/milvus-io/milvus/issues/39365

Signed-off-by: shaoting-huang <shaoting.huang@zilliz.com>
2025-01-26 12:17:19 +08:00
Xianhui Lin 3b3959c625
enhance: [2.5]Add schema update time verification for insert and upsert to use cache (#39382)
enhance: Add schema update time verification for insert and upsert to
use cache
issue: https://github.com/milvus-io/milvus/issues/39093
Related to pr: https://github.com/milvus-io/milvus/pull/39096

---------

Signed-off-by: Xianhui.Lin <xianhui.lin@zilliz.com>
2025-01-21 10:19:11 +08:00
sthuang 8c35064d01
enhance: [2.5] clean up logs of empty op key (#39394)
cherry-pick from master: https://github.com/milvus-io/milvus/pull/39335
related: https://github.com/milvus-io/milvus/issues/39334

Signed-off-by: shaoting-huang <shaoting.huang@zilliz.com>
2025-01-20 12:15:10 +08:00
Zhen Ye 0391b8c7d3
fix: update rg failed when alter database (#39356)
issue: #36933
pr: #39355

Signed-off-by: chyezh <chyezh@outlook.com>
2025-01-17 10:37:03 +08:00
yihao.dai 6773fb10a8
enhance: [2.5] Read metadata concurrently to accelerate recovery (#38900)
Read metadata such as segments, binlogs, and partitions concurrently at
the collection level.

issue: https://github.com/milvus-io/milvus/issues/37630

pr: https://github.com/milvus-io/milvus/pull/38403

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2025-01-16 17:53:01 +08:00
SimFG 4cb47b3f67
feat: [2.5] root privileges can be customized (#39324)
- issue: #39184
- pr: #39191

Signed-off-by: SimFG <bang.fu@zilliz.com>
2025-01-16 14:27:11 +08:00
yihao.dai 9b916f2cf8
enhance: [2.5] Skip creating partition rate limiters when not enable (#38911)
issue: https://github.com/milvus-io/milvus/issues/37630

pr: https://github.com/milvus-io/milvus/pull/38816

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2025-01-15 22:19:02 +08:00
yihao.dai 8bf0f0b921
fix: [2.5] Fix rootcoord meta mutex contention (#38902)
RootCoord meta uses copy-on-write, allowing the removal of unnecessary
copies.

issue: https://github.com/milvus-io/milvus/issues/37630

pr: https://github.com/milvus-io/milvus/pull/38799

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2025-01-15 20:11:08 +08:00
sthuang 4a47f760b3
fix: [2.5] rbac custom group privilege level check (#39164) (#39224)
cherry-pick from master: https://github.com/milvus-io/milvus/pull/39164
related: https://github.com/milvus-io/milvus/issues/39086

Signed-off-by: shaoting-huang <shaoting.huang@zilliz.com>
2025-01-14 16:50:59 +08:00
Zhen Ye 54036bcafd
enhance: add broadcast operation for msgstream (#39119)
issue: #38399
pr: #39040

- make broadcast service available for msgstream by reusing the
architecture streaming service

---------

Signed-off-by: chyezh <chyezh@outlook.com>
2025-01-14 15:11:00 +08:00
SimFG 7c8db6f61f
fix: [2.5] use the object heap to keep the min ddl ts order (#39195)
- issue: #39002
- pr: #39118

Signed-off-by: SimFG <bang.fu@zilliz.com>
2025-01-14 14:23:05 +08:00
SimFG 68734fe4f7
enhance: [2.5] extract the task step execution process (#39196)
- issue: #38322
- pr: #38323

Signed-off-by: SimFG <bang.fu@zilliz.com>
2025-01-13 21:58:58 +08:00
tinswzy 39fa00fc1a
enhance: add intent ctx to rootcoord init (#38948)
issue: #38439 
cherry pick "enhance: add intent ctx to rootcoord init"

Signed-off-by: tinswzy <zhenyuan.wei@zilliz.com>
2025-01-11 23:54:57 +08:00
Zhen Ye 95809ca767
enhance: make new go package to manage proto (#39128)
issue: #39095
pr: #39114

---------

Signed-off-by: chyezh <chyezh@outlook.com>
2025-01-10 10:53:01 +08:00
Zhen Ye 6f1febe881
enhance: move streaming coord from datacoord to rootcoord (#39009)
issue: #38399
pr: #39007

We want to support broadcast operation for both streaming and msgstream.
But msgstream can be only sent message from rootcoord and proxy.
So this pr move the streamingcoord to rootcoord to make easier
implementation.

---------

Signed-off-by: chyezh <chyezh@outlook.com>
2025-01-07 17:56:56 +08:00
sthuang 147c6b099b
fix: [2.5] expand privilege group when list policy in rootcoord (#38760)
cherry-pick from: https://github.com/milvus-io/milvus/pull/38758
related: https://github.com/milvus-io/milvus/issues/38757

Signed-off-by: shaoting-huang <shaoting.huang@zilliz.com>
2024-12-26 15:34:49 +08:00
sthuang a4901ef7ec
fix: [2.5] fix privilege group list and list collections (#38738)
cherry-pick from: https://github.com/milvus-io/milvus/pull/38684
related: https://github.com/milvus-io/milvus/issues/37031

Signed-off-by: shaoting-huang <shaoting.huang@zilliz.com>
2024-12-25 18:06:50 +08:00
jaime 11bedf5e76
fix: Revert "Expose metrics of stanby coordinators (#27698)" (#38621)
issue: #38608
pr: #38620

Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-12-20 18:04:47 +08:00
jaime 78438ef41e
fix: revert optimize CPU usage for CheckHealth requests (#35589) (#38555)
issue: #35563

Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-12-19 00:38:45 +08:00
sthuang de8858931a
fix: rbac revoke check if there is same privilege in other privilege groups granted (#38558)
related: https://github.com/milvus-io/milvus/issues/38557

Signed-off-by: shaoting-huang <shaoting.huang@zilliz.com>
2024-12-18 18:46:45 +08:00
yihao.dai 1aa31e2a9b
fix: Fix inaccurate general count (#38524)
Checking general count should only count healthy collections and
partitions.

issue: https://github.com/milvus-io/milvus/issues/37630

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-12-17 23:24:45 +08:00
yihao.dai 01fa2f8da5
fix: Fix compile error (#38510)
/kind improvement

related: #37559

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-12-17 12:16:19 +08:00
jaime 28fdbc4e30
enhance: optimize CPU usage for CheckHealth requests (#35589)
issue: #35563
1. Use an internal health checker to monitor the cluster's health state,
storing the latest state on the coordinator node. The CheckHealth
request retrieves the cluster's health from this latest state on the
proxy sides, which enhances cluster stability.
2. Each health check will assess all collections and channels, with
detailed failure messages temporarily saved in the latest state.
3. Use CheckHealth request instead of the heavy GetMetrics request on
the querynode and datanode

Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-12-17 11:02:45 +08:00
SimFG 2afe2eaf3e
feat: support to replicate collection when the services contains the system tt msg (#37559)
- issue: #37105

---------

Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-12-17 09:08:46 +08:00
SimFG fa8ac09550
fix: the issue of replicate message exception when the ttMsgEnable config is changed dynamically (#38178)
- issue: #38177

Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-12-14 23:24:51 +08:00
tinswzy 27229f7907
enhance: refine exists log print with ctx (#38080)
issue: #35917 
Refines exists log print with ctx

Signed-off-by: tinswzy <zhenyuan.wei@zilliz.com>
2024-12-14 22:36:44 +08:00
Xianhui Lin f2d79b3d3a
enhance: altercollection validation logic (#38419)
altercollection validation logic
issue: https://github.com/milvus-io/milvus/issues/37436

---------

Signed-off-by: Xianhui.Lin <xianhui.lin@zilliz.com>
2024-12-12 21:42:42 +08:00
Xianhui Lin d0a8110a7a
enhance: alterdatabase support delete property (#38380)
alterdatabase support delete property
issue: https://github.com/milvus-io/milvus/issues/38379

---------

Signed-off-by: Xianhui.Lin <xianhui.lin@zilliz.com>
2024-12-12 17:04:43 +08:00