6bc799061e
fix: fix privilege group list and list collections ( #38684 )
...
related: #37031
* built-in privilege group privileges in listPrivilegeGroups() should be
the same as in milvus.yaml
* collections granted by collection level built-in privilege group
should be list in showCollections()
Signed-off-by: shaoting-huang <shaoting.huang@zilliz.com>
2024-12-25 18:00:51 +08:00
ea339c13c6
test: add cases for async milvus client ( #38699 )
...
issue: #38697
Signed-off-by: ThreadDao <yufen.zong@zilliz.com>
2024-12-25 16:54:50 +08:00
9c3f59dbbe
fix: Prevent balancer from overloading the same QueryNode ( #38719 )
...
issue: #38718
The balancer calculates the workload of executing tasks as an ongoing
score for target nodes. However, a logic issue arises when
GetSegmentTaskDelta or GetChannelTaskDelta is called with
collectionID=-1, which incorrectly returns zero.
Due to the incorrect global score, the executing task's workload is not
properly reflected for each collection. Consequently, each collection
submits its own balance task, leading to the balancer assigning
excessive tasks to the same QueryNode.
---------
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-12-25 16:36:49 +08:00
acc8fb7af6
enhance: eliminate compile warnings (part2) ( #38535 )
...
See #38435
---------
Signed-off-by: Ted Xu <ted.xu@zilliz.com>
2024-12-25 15:30:50 +08:00
5395ec19ad
enhance: add mem size for index file and fallback to multiply with serialized size ( #38716 )
...
issue: #38715
Signed-off-by: chyezh <chyezh@outlook.com>
2024-12-25 14:46:49 +08:00
c731357538
enhance: Add missing delete metrics ( #38634 )
...
Add 2 counter metrics:
- Total delete entries from deltalog:
milvus_datanode_compaction_delete_count
- Total missing deletes: milvus_datanode_compaction_missing_delete_count
See also: #34665
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-12-25 10:24:50 +08:00
285289d5b0
enhance: implement kafka for wal ( #38598 )
...
issue: #38399
---------
Signed-off-by: chyezh <chyezh@outlook.com>
2024-12-25 10:22:50 +08:00
05d9f8562f
fix: fix complie error generated by conflict pr ( #38729 )
...
issue: #38399
Signed-off-by: chyezh <chyezh@outlook.com>
2024-12-25 10:14:49 +08:00
118678bd91
enhance: make runtime config into a global environment table ( #38671 )
...
issue: #38399
---------
Signed-off-by: chyezh <chyezh@outlook.com>
2024-12-24 21:00:49 +08:00
69a9fd6ead
enhance: enable rmq for streaming ( #38669 )
...
issue: #38399
---------
Signed-off-by: chyezh <chyezh@outlook.com>
2024-12-24 20:24:48 +08:00
2b53b0905e
fix: Fix 0 read count during import ( #38694 )
...
issue: https://github.com/milvus-io/milvus/issues/38693
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-12-24 20:06:49 +08:00
5304707393
test: Add test cases for rbac v2 ( #38556 )
...
Signed-off-by: binbin lv <binbin.lv@zilliz.com>
2024-12-24 18:06:48 +08:00
c458dec206
fix: [skip e2e]update webui collection and query pages ( #38700 )
...
issue: #36621
* enhance collection detail page list columns
* enhance query page segments state detail logic
Signed-off-by: mimo.oyn <ning.ouyang@zilliz.com>
2024-12-24 17:36:48 +08:00
636e107e44
fix: Remove sync task after finished ( #38681 )
...
Related to #38680
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-12-24 13:34:48 +08:00
877cd80d93
Update all contributors
...
Signed-off-by: sre-ci-robot <sre-ci-robot@zilliz.com>
2024-12-24 00:01:37 +00:00
6884319d03
test: [CP] add alter tests ( #38659 )
...
related issue: https://github.com/milvus-io/milvus/issues/38471
---------
Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
2024-12-23 19:42:49 +08:00
d718bbb4de
fix: fix altercollectionfield interceptor dbname ( #38662 )
...
fix altercollectionfield interceptor dbname
issue: https://github.com/milvus-io/milvus/issues/37436
Signed-off-by: Xianhui.Lin <xianhui.lin@zilliz.com>
2024-12-23 18:24:48 +08:00
82d85e8c57
test: modify part of milvus client test cases with load func ( #38654 )
...
/kind improvment
/cc @yanliang567 @binbinlv
Signed-off-by: laurazhao0611 <laura.zhao@zilliz.com>
2024-12-23 17:16:48 +08:00
bb5f38e574
enhance: Return collection not loaded rather than not found on querynode ( #38593 )
...
issue: #38586
---------
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-12-23 14:36:48 +08:00
2356425530
fix: Support hybrid search with expression template ( #38623 )
...
issue: #38616
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-12-23 14:32:51 +08:00
205231b9c7
fix: Decode unicode for json key in expression ( #38651 )
...
issue: #38626
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-12-23 13:40:49 +08:00
9e1bb3bd8f
[automated] Bump milvus version to v2.5.0 ( #38640 )
...
Bump milvus version to v2.5.0
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-12-23 11:48:54 +08:00
5afd0c0a2b
fix: Revert "Expose metrics of stanby coordinators ( #27698 )" ( #38620 )
...
issue: #38608
Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-12-23 11:46:57 +08:00
984a605d47
enhance: use lazy initializing client for streaming node ( #38400 )
...
issue: #38399
---------
Signed-off-by: chyezh <chyezh@outlook.com>
2024-12-23 11:36:47 +08:00
f499ca47a0
fix: [skip e2e]update webui ( #36621 ) ( #38633 )
...
issue: #36621
* add collection detail page
* enhance query page segments & channels state display
Signed-off-by: mimo.oyn <ning.ouyang@zilliz.com>
2024-12-23 11:00:49 +08:00
407035c994
[automated] Update Knowhere Commit ( #38641 )
...
Update Knowhere Commit
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-12-22 00:44:46 +08:00
cce25ecdbc
[automated] Update Knowhere Commit ( #38635 )
...
Update Knowhere Commit
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-12-21 00:50:46 +08:00
90de37e60a
enhance: field stringtype maxlength raise to 1M ( #38592 )
...
enhance: field stringtype maxlength raise to 1M
issue: https://github.com/milvus-io/milvus/issues/37436
Signed-off-by: Xianhui.Lin <xianhui.lin@zilliz.com>
2024-12-20 14:32:45 +08:00
c39db11509
enhance: [GoSDK] Sync API names and add missing APIs ( #38603 )
...
Related to #31293
- Rename `UsingDatabase` to `UseDatabase`
- Uncomment default value methods
- Add missing RBAC APIs
- Add some resource group APIs
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-12-20 11:52:46 +08:00
ca7ec23198
enhance: Use partitionID when delete by partitionKey ( #38231 )
...
When delete by partition_key, Milvus will generates L0 segments
globally. During L0 Compaction, those L0 segments will touch all
partitions collection wise. Due to the false-positive rate of segment
bloomfilters, L0 compactions will append false deltalogs to completed
irrelevant partitions, which causes *partition deletion amplification.
This PR uses partition_key to set targeted partitionID when producing
deleteMsgs into MsgStreams. This'll narrow down L0 segments scope to
partition level, and remove the false-positive influence
collection-wise.
However, due to DeleteMsg structure, we can only label one partition to
one deleteMsg, so this enhancement fails if user wants to delete over 2
partition_keys in one deletion.
See also: #34665
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-12-20 11:18:46 +08:00
a728646534
enhance: [GoSDK] Add content-type header for bulk import req ( #38581 )
...
Related to #31293
Previous PR: #38493
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-12-20 10:28:47 +08:00
c0b855dc75
fix: ChannelManager concurret Release and Watch bug ( #38590 )
...
See also: #38589
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-12-19 22:50:47 +08:00
8fcb33c21d
fix:fix delete record assert failed ( #38580 )
...
#38472
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-12-19 18:22:47 +08:00
3d360c0624
fix: SyncSegments rpc always failed ( #38578 )
...
miss the patch due to code branching
previous pr: #38032
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
Co-authored-by: Wei Liu <wei.liu@zilliz.com>
2024-12-19 15:40:45 +08:00
b537a72309
fix: interted index out of range ( #38577 )
...
issue: #38546 , #38486
Signed-off-by: chyezh <chyezh@outlook.com>
2024-12-19 15:20:47 +08:00
306e5e6898
enhance: clean compaction task in compactionHandler ( #38170 )
...
issue: #35711
---------
Signed-off-by: wayblink <anyang.wang@zilliz.com>
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
Co-authored-by: wayblink <anyang.wang@zilliz.com>
2024-12-19 12:36:47 +08:00
01cfb1fd97
enhance: [GoSDK] support expression template ( #38568 )
...
Related to #36672
This PR add
- Expression template for search, query & hybrid search
- fix hybrid search rerank param
- add reranker interface(migrate from go sdk old repo)
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-12-19 11:20:47 +08:00
78438ef41e
fix: revert optimize CPU usage for CheckHealth requests ( #35589 ) ( #38555 )
...
issue: #35563
Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-12-19 00:38:45 +08:00
134d06f7e6
fix: need pass parameter to listPrivilegeGroups ( #38562 )
...
issue: https://github.com/milvus-io/milvus/issues/38563
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-12-18 20:28:51 +08:00
de8858931a
fix: rbac revoke check if there is same privilege in other privilege groups granted ( #38558 )
...
related: https://github.com/milvus-io/milvus/issues/38557
Signed-off-by: shaoting-huang <shaoting.huang@zilliz.com>
2024-12-18 18:46:45 +08:00
1639779b5d
fix: [skip e2e]metrics josn tag unmarshall failed ( #38554 )
...
issue: #38551
Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-12-18 18:02:46 +08:00
a348122758
fix: Support get segments from current segments view ( #38512 )
...
issue: #38511
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-12-18 18:00:54 +08:00
06d410b70f
enhance: update knowhere version ( #38544 )
...
related: #37730
Signed-off-by: xianliang.li <xianliang.li@zilliz.com>
2024-12-18 16:52:45 +08:00
87056be748
fix: fix snapshot or size when query ( #38549 )
...
#38472
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-12-18 16:42:45 +08:00
894c203f27
enhance: allow hyphen in partition name ( #38494 )
...
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2024-12-18 15:04:45 +08:00
5394f47866
enhance:[skip e2e] Add new maintainers ( #38547 )
...
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2024-12-18 12:54:20 +08:00
6ee94d00b9
fix:fix calculate arrow nest type and add ut ( #38527 )
...
#37767
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-12-18 11:54:44 +08:00
fb0e689617
test: update sdk version and add cases ( #38478 )
...
Signed-off-by: nico <cheng.yuan@zilliz.com>
2024-12-18 10:10:57 +08:00
ffd3c5d2f5
[automated] Update Knowhere Commit ( #38542 )
...
Update Knowhere Commit
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-12-18 01:24:47 +08:00
7a05b5bbea
fix: Fix stats task wrong RootPath when upload binlog ( #38539 )
...
issue: #38336
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-12-18 00:18:45 +08:00
sthuang
ThreadDao
wei liu
Ted Xu
Zhen Ye
XuanYang-cn
yihao.dai
binbin
MimoN
congqixia
sre-ci-robot
yanliang567
Xianhui Lin
laurazhao0611
cai.zhang
sre-ci-robot
jaime
zhagnlu
smellthemoon
foxspy
Bingyi Sun
zhenshan.cao
nico