b2b7cca910
fix: [2.5] set iterative filter hint to false when no expr specified ( #39034 )
...
issue: https://github.com/milvus-io/milvus/issues/39013
pr: #39033
Signed-off-by: chasingegg <chao.gao@zilliz.com>
2025-01-08 12:58:56 +08:00
4b62dfd11f
enhance: [2.5] Use mvcc timestamp as guarantee ts if set ( #38980 ) ( #39019 )
...
Cherry pick from master
pr: #38980
When MvccTimestamp is set, it could be used as guarantee timestamp
directly instead of new ts allocated by scheduler reducing the waiting
time when delegator has tsafe lag
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2025-01-07 19:18:57 +08:00
6f1febe881
enhance: move streaming coord from datacoord to rootcoord ( #39009 )
...
issue: #38399
pr: #39007
We want to support broadcast operation for both streaming and msgstream.
But msgstream can be only sent message from rootcoord and proxy.
So this pr move the streamingcoord to rootcoord to make easier
implementation.
---------
Signed-off-by: chyezh <chyezh@outlook.com>
2025-01-07 17:56:56 +08:00
0b62c1e692
enhance: [2.5] Optimize retrieve performance utilizing bitset SIMD methods ( #39041 )
...
Cherry pick from master
pr: #39004 #39037
Related to #39003
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2025-01-07 17:45:03 +08:00
b457c2f415
enhance: [2.5]Add missing delete metrics ( #38634 ) ( #38747 )
...
Add 2 counter metrics:
- Total delete entries from deltalog:
milvus_datanode_compaction_delete_count
- Total missing deletes: milvus_datanode_compaction_missing_delete_count
See also: #34665
pr: #38634
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2025-01-07 11:20:56 +08:00
e6dd3e5a57
fix: [2.5]Remove valid expressions from invalid expressions ( #38999 )
...
issue: #39014
master pr: #38957
master pr: #39012
---------
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2025-01-06 18:02:55 +08:00
b9da6a58e2
enhance: [2.5]Return collection not loaded rather than not found on querynode ( #38593 ) ( #38918 )
...
issue: #38586
master pr: #38593
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2025-01-05 18:55:00 +08:00
018fbb48b1
fix: search req transfer correctly ( #38954 )
...
issue: #37899
pr: #38955
Signed-off-by: chasingegg <chao.gao@zilliz.com>
2025-01-03 10:02:54 +08:00
4533fc8f9c
fix: [2.5] expression with capital AND and OR are not recognized ( #38928 )
...
fix https://github.com/milvus-io/milvus/issues/38864
AND, OR and NOT is not recognized in milvus parser
master pr: #38927
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2025-01-02 15:04:58 +08:00
1d55ad6a1d
enhance: [2.5] add param for tuning max VARCHAR length and restore limit to 65535 ( #38883 )
...
issue: #38882
pr: https://github.com/milvus-io/milvus/pull/38884
Signed-off-by: Patrick Weizhi Xu <weizhi.xu@zilliz.com>
(cherry picked from commit a64a7377ea8258efe36c74bd1cf9db556726ba2d)
2025-01-02 13:56:54 +08:00
a2c4cd59ce
fix: drop partition can not be successful if load failed[2.5] ( #38874 )
...
fix https://github.com/milvus-io/milvus/issues/38649
pr: #38793
when partition load failed, the partition drop will also fail due to the
wrong error message
Signed-off-by: xiaofanluan <xiaofan.luan@zilliz.com>
2025-01-02 09:56:53 +08:00
6fa096eb39
fix:[Cherry-pick] bm25 import segment loss stats ( #38881 )
...
relate: https://github.com/milvus-io/milvus/issues/38854
pr: https://github.com/milvus-io/milvus/pull/38855
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-12-31 19:24:54 +08:00
e389ae215c
fix: [2.5][skip e2e]update webui logic ( #38754 )
...
issue: #36621
pr: #38755
* update collection detail page logic
* fix data page segments list
* remove useless js file
2024-12-31 18:25:00 +08:00
71dea30d44
fix: [2.5] Release lock when return function ( #38863 )
...
issue: #38851
master pr: #38856
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-12-30 22:44:39 +08:00
015e50e4de
fix: [2.5] Only generate the index_null_offset file when the field support null value ( #38834 )
...
issue: #38832
master pr: #38833
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2024-12-30 17:02:52 +08:00
04f07f00b7
fix: the retrieve plan on heap is used after free when reduce ( #38841 )
...
issue: #38731
pr: #38840
Signed-off-by: chyezh <chyezh@outlook.com>
2024-12-30 15:52:51 +08:00
ef400227ad
enhance: [2.5][cp] speed up search iterator stage 1 ( #38678 )
...
pr: https://github.com/milvus-io/milvus/pull/37947
issue: https://github.com/milvus-io/milvus/issues/37548
Signed-off-by: Patrick Weizhi Xu <weizhi.xu@zilliz.com>
(cherry picked from commit 9016c4adcd765c0766b01e7e5d465c915e176a6f)
2024-12-27 18:48:52 +08:00
3d723676ac
enhance: [2.5]update knowhere version ( #38785 )
...
/kind branch-feature
release note: https://github.com/zilliztech/knowhere/releases/tag/v2.5.1
Signed-off-by: xianliang.li <xianliang.li@zilliz.com>
2024-12-27 10:24:50 +08:00
147c6b099b
fix: [2.5] expand privilege group when list policy in rootcoord ( #38760 )
...
cherry-pick from: https://github.com/milvus-io/milvus/pull/38758
related: https://github.com/milvus-io/milvus/issues/38757
Signed-off-by: shaoting-huang <shaoting.huang@zilliz.com>
2024-12-26 15:34:49 +08:00
930f9e9ef8
fix:[Cherr-pick] sealed segment get empty index params when brute force search for bm25 ( #38752 )
...
relate: https://github.com/milvus-io/milvus/issues/38236
pr: https://github.com/milvus-io/milvus/pull/38707
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-12-26 10:26:51 +08:00
d47372eb9c
fix: report error when hints not supported ( #38730 )
...
issue: #38705
pr: #38717
---------
Signed-off-by: chasingegg <chao.gao@zilliz.com>
2024-12-25 19:02:56 +08:00
a4901ef7ec
fix: [2.5] fix privilege group list and list collections ( #38738 )
...
cherry-pick from: https://github.com/milvus-io/milvus/pull/38684
related: https://github.com/milvus-io/milvus/issues/37031
Signed-off-by: shaoting-huang <shaoting.huang@zilliz.com>
2024-12-25 18:06:50 +08:00
f441ccdbe9
fix: [2.5] Prevent balancer from overloading the same QueryNode ( #38724 )
...
issue: #38718
pr: #38719
The balancer calculates the workload of executing tasks as an ongoing
score for target nodes. However, a logic issue arises when
GetSegmentTaskDelta or GetChannelTaskDelta is called with
collectionID=-1, which incorrectly returns zero.
Due to the incorrect global score, the executing task's workload is not
properly reflected for each collection. Consequently, each collection
submits its own balance task, leading to the balancer assigning
excessive tasks to the same QueryNode.
---------
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-12-25 16:16:49 +08:00
cb0618b2d4
fix: [2.5] Querycoord will trigger unexpected balance task after restart ( #38725 )
...
issue: https://github.com/milvus-io/milvus/issues/38606
pr: https://github.com/milvus-io/milvus/pull/38630
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-12-25 16:14:49 +08:00
b16d04d7cc
fix: Fix update loading collection's load config doesn't work ( #38737 )
...
issue: #38594
pr: #38595
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-12-25 15:02:50 +08:00
52de43dbeb
enhance: add multiply factor when loading index ( #38722 )
...
issue: #38715
pr: #38716
Signed-off-by: chyezh <chyezh@outlook.com>
2024-12-25 10:50:58 +08:00
bca21bde30
fix: [2.5] Fix 0 read count during import ( #38695 )
...
issue: https://github.com/milvus-io/milvus/issues/38693
pr: https://github.com/milvus-io/milvus/pull/38694
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-12-24 22:08:49 +08:00
91130909a5
fix: [2.5] Remove sync task after finished ( #38681 ) ( #38687 )
...
Cherry-pick from master
pr: #38681
Related to #38680
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-12-24 19:02:49 +08:00
18cc83b9f6
fix: [skip e2e]update webui collection and query pages ( #38701 )
...
pr: #38700
issue: #36621
* enhance collection detail page list columns
* enhance query page segments state detail logic
Signed-off-by: mimo.oyn <ning.ouyang@zilliz.com>
2024-12-24 17:34:49 +08:00
c51b458156
fix: [2.5]fix altercollectionfield interceptor dbname ( #38663 )
...
fix altercollectionfield interceptor dbname
issue: https://github.com/milvus-io/milvus/issues/37436
master pr: https://github.com/milvus-io/milvus/pull/38662
Signed-off-by: Xianhui.Lin <xianhui.lin@zilliz.com>
2024-12-24 10:54:50 +08:00
bb3d993da5
fix: [2.5] Decode unicode for json key in expression ( #38653 )
...
issue: #38626
master pr: #38651
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-12-23 13:58:49 +08:00
7fe2cd0518
fix: [2.5] Support hybrid search with expression template ( #38625 )
...
issue: https://github.com/milvus-io/milvus/issues/38616
master pr: https://github.com/milvus-io/milvus/pull/38623
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-12-20 18:28:20 +08:00
8cbe8da9a7
fix: [skip e2e]update webui ( #36621 ) ( #38632 )
...
issue: #36621
* add collection detail page
* enhance query page segments & channels state display
Signed-off-by: mimo.oyn <ning.ouyang@zilliz.com>
2024-12-20 18:27:04 +08:00
11bedf5e76
fix: Revert "Expose metrics of stanby coordinators ( #27698 )" ( #38621 )
...
issue: #38608
pr: #38620
Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-12-20 18:04:47 +08:00
3cf9b6bae5
enhance: [2.5]field stringtype maxlength raise to 1M ( #38614 )
...
enhance: field stringtype maxlength raise to 1M
issue: https://github.com/milvus-io/milvus/issues/37436
master pr: https://github.com/milvus-io/milvus/pull/38592
Signed-off-by: Xianhui.Lin <xianhui.lin@zilliz.com>
2024-12-20 17:08:45 +08:00
ca7ec23198
enhance: Use partitionID when delete by partitionKey ( #38231 )
...
When delete by partition_key, Milvus will generates L0 segments
globally. During L0 Compaction, those L0 segments will touch all
partitions collection wise. Due to the false-positive rate of segment
bloomfilters, L0 compactions will append false deltalogs to completed
irrelevant partitions, which causes *partition deletion amplification.
This PR uses partition_key to set targeted partitionID when producing
deleteMsgs into MsgStreams. This'll narrow down L0 segments scope to
partition level, and remove the false-positive influence
collection-wise.
However, due to DeleteMsg structure, we can only label one partition to
one deleteMsg, so this enhancement fails if user wants to delete over 2
partition_keys in one deletion.
See also: #34665
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-12-20 11:18:46 +08:00
c0b855dc75
fix: ChannelManager concurret Release and Watch bug ( #38590 )
...
See also: #38589
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-12-19 22:50:47 +08:00
8fcb33c21d
fix:fix delete record assert failed ( #38580 )
...
#38472
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-12-19 18:22:47 +08:00
3d360c0624
fix: SyncSegments rpc always failed ( #38578 )
...
miss the patch due to code branching
previous pr: #38032
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
Co-authored-by: Wei Liu <wei.liu@zilliz.com>
2024-12-19 15:40:45 +08:00
b537a72309
fix: interted index out of range ( #38577 )
...
issue: #38546 , #38486
Signed-off-by: chyezh <chyezh@outlook.com>
2024-12-19 15:20:47 +08:00
306e5e6898
enhance: clean compaction task in compactionHandler ( #38170 )
...
issue: #35711
---------
Signed-off-by: wayblink <anyang.wang@zilliz.com>
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
Co-authored-by: wayblink <anyang.wang@zilliz.com>
2024-12-19 12:36:47 +08:00
78438ef41e
fix: revert optimize CPU usage for CheckHealth requests ( #35589 ) ( #38555 )
...
issue: #35563
Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-12-19 00:38:45 +08:00
134d06f7e6
fix: need pass parameter to listPrivilegeGroups ( #38562 )
...
issue: https://github.com/milvus-io/milvus/issues/38563
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-12-18 20:28:51 +08:00
de8858931a
fix: rbac revoke check if there is same privilege in other privilege groups granted ( #38558 )
...
related: https://github.com/milvus-io/milvus/issues/38557
Signed-off-by: shaoting-huang <shaoting.huang@zilliz.com>
2024-12-18 18:46:45 +08:00
a348122758
fix: Support get segments from current segments view ( #38512 )
...
issue: #38511
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-12-18 18:00:54 +08:00
06d410b70f
enhance: update knowhere version ( #38544 )
...
related: #37730
Signed-off-by: xianliang.li <xianliang.li@zilliz.com>
2024-12-18 16:52:45 +08:00
87056be748
fix: fix snapshot or size when query ( #38549 )
...
#38472
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-12-18 16:42:45 +08:00
894c203f27
enhance: allow hyphen in partition name ( #38494 )
...
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2024-12-18 15:04:45 +08:00
6ee94d00b9
fix:fix calculate arrow nest type and add ut ( #38527 )
...
#37767
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-12-18 11:54:44 +08:00
ffd3c5d2f5
[automated] Update Knowhere Commit ( #38542 )
...
Update Knowhere Commit
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-12-18 01:24:47 +08:00
Gao
congqixia
Zhen Ye
XuanYang-cn
cai.zhang
Patrick Weizhi Xu
Xiaofan
aoiasd
MimoN
foxspy
sthuang
wei liu
yihao.dai
Xianhui Lin
jaime
zhagnlu
smellthemoon
Bingyi Sun
sre-ci-robot