2eaa1a4869
fix: [2.5] Use `text_log` prefix for TextMatchIndex null offset file ( #39936 )
...
Cherry-pick from master
pr: #39935
Related to #39933
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2025-02-18 00:43:07 +08:00
de8abe7970
fix:reset default auto index for varchar ( #39908 )
...
pr: #39819
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2025-02-17 20:55:26 +08:00
52434ccc78
enhance: [2.5] Limit the speed of the generating stats task ( #39645 )
...
master pr: #39644
---------
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2025-02-17 16:06:17 +08:00
8067113133
enhance: [cp25]Enable to observe write amplification ( #39661 ) ( #39743 )
...
pr: #39661
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2025-02-17 16:00:17 +08:00
56c1a8d462
fix: ignore growing option is lost at hibridsearch ( #39900 )
...
issue: #39892
pr: #39799
Signed-off-by: chyezh <chyezh@outlook.com>
2025-02-17 15:30:15 +08:00
3a951f2160
test: Cherry pick sparse invert index algo tests ( #39816 )
...
related issue: https://github.com/milvus-io/milvus/issues/39332
related pr: #39691
---------
Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
2025-02-17 15:06:15 +08:00
ee25af4c9b
enhance: Add configs for compaction schedule ( #39010 ) ( #39511 )
...
pr: #39010
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2025-02-17 11:44:15 +08:00
8c86e4e6ac
fix: [2.5]AlterCollection unable to modify ConsistencyLevel ( #39902 )
...
fix: AlterCollection unable to modify ConsistencyLevel
issue: https://github.com/milvus-io/milvus/issues/39707
pr: https://github.com/milvus-io/milvus/pull/39708
Signed-off-by: Xianhui.Lin <xianhui.lin@zilliz.com>
2025-02-17 10:00:15 +08:00
9918e1008d
fix: Fix import failed due to 0 row num ( #39887 ) ( #39904 )
...
issue: https://github.com/milvus-io/milvus/issues/39885
pr: https://github.com/milvus-io/milvus/pull/39886
---------
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
Co-authored-by: yihao.dai <yihao.dai@zilliz.com>
2025-02-17 01:36:15 +08:00
f0964f769d
enhance: [2.5]Add json key inverted index in stats for optimization ( #39876 )
...
Add json key inverted index in stats for optimization
issue: https://github.com/milvus-io/milvus/issues/36995
pr: https://github.com/milvus-io/milvus/pull/38039
---------
Signed-off-by: Xianhui.Lin <xianhui.lin@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2025-02-16 20:12:15 +08:00
e372b1733c
enhance: [2.5]update raft to cuvs ( #39221 )
...
pr: #39873
issue: #39883
Signed-off-by: yusheng.ma <yusheng.ma@zilliz.com>
2025-02-14 16:52:14 +08:00
401794f405
fix: [2.5] Fix module for long type ( #39802 )
...
issue: #39640
master pr: #39722
---------
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2025-02-14 15:18:21 +08:00
5da9262f58
fix: [2.5] Add and use lifetime context for compaction trigger ( #39857 ) ( #39880 )
...
Cherry-pick from master
pr: #39857
Related to #39856
This PR add lifetime bound context for compaction trigger and use it
instead of context.Background in case of rootcoord down and some grpc
call retry forever
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2025-02-14 14:18:14 +08:00
418f971d2d
fix: [2.5] ReEnqueue L0 compaction task when preCheck failed ( #39871 )
...
issue: #39868
master pr: #39870
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2025-02-14 13:38:15 +08:00
ba0edb8383
enhance: [2.5]return topks when search in restful v2 ( #39839 )
...
pr: #39812
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2025-02-14 11:30:14 +08:00
9407a3c9b1
fix: [2.5] Check collection released before target checks ( #39843 )
...
Cherry-pick from master
pr: #39841
Related to #39840
The target could be updated async in previous code. This PR make remove
collection from target observer block until all tasks related in
dispatchers are removed preventing the metrics being updated after
collection released.
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2025-02-13 20:00:15 +08:00
f85bf59921
fix: rootcoord graceful stop failed and limit resource of ci ( #39793 )
...
issue: #38399
pr: #39781
---------
Signed-off-by: chyezh <chyezh@outlook.com>
2025-02-13 19:34:13 +08:00
aa74e8ba3d
enhance: [2.5][GoSDK] Add WithEnableMatch syntactic sugar ( #39837 ) ( #39853 )
...
Cherry-pick from master
pr: #39837
Related to #35922
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2025-02-13 19:28:15 +08:00
16ea4fcb90
Revert "fix:reset default auto index type for scalar ( #37086 )" ( #39820 )
...
This reverts commit eeb67a3845#39819
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2025-02-13 17:36:14 +08:00
a50249e0bb
enhance: [2.5] dump pprof info if component stop progress timeout ( #39760 )
...
issue: #39735
pr: #39726
---------
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2025-02-13 15:30:47 +08:00
df28d2200d
fix: [2.5] Remove load field & schema column size check #39834 ( #39835 )
...
Cherry-pick from master
pr: #39833
Related to #39788
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2025-02-13 12:42:53 +08:00
cb1bf6d122
fix:[2.5] remove the mmap.enable param in the type param when creating index ( #39806 )
...
Because when GetIndexParams is used, index params and type params are
concatenated, so when loading index, the mmap.enable parameter in type
params is also referenced.
- issue: #39801
- pr: #39803
Signed-off-by: SimFG <bang.fu@zilliz.com>
2025-02-13 10:08:53 +08:00
d121ac3a7a
enhance: [2.5]intermin index support different index type and more data type(fp16/bf16) ( #39180 )
...
issue: https://github.com/milvus-io/milvus/issues/27678
related: https://github.com/milvus-io/milvus/pull/39753
some raw data status will change:
Intermin index has raw data:
<html xmlns:v="urn:schemas-microsoft-com:vml"
xmlns:o="urn:schemas-microsoft-com:office:office"
xmlns:x="urn:schemas-microsoft-com:office:excel"
xmlns="http://www.w3.org/TR/REC-html40 ">
<head>
<meta name=ProgId content=Excel.Sheet>
<meta name=Generator content="Microsoft Excel 15">
<link id=Main-File rel=Main-File
href="file:////Users/cqy/Library/Group%20Containers/UBF8T346G9.Office/TemporaryItems/msohtmlclip/clip.htm">
<link rel=File-List
href="file:////Users/cqy/Library/Group%20Containers/UBF8T346G9.Office/TemporaryItems/msohtmlclip/clip_filelist.xml">
<!--table
{mso-displayed-decimal-separator:"\.";
mso-displayed-thousand-separator:"\,";}
@page
{margin:.75in .7in .75in .7in;
mso-header-margin:.3in;
mso-footer-margin:.3in;}
.font5
{color:windowtext;
font-size:9.0pt;
font-weight:400;
font-style:normal;
text-decoration:none;
font-family:等线;
mso-generic-font-family:auto;
mso-font-charset:134;}
tr
{mso-height-source:auto;
mso-ruby-visibility:none;}
col
{mso-width-source:auto;
mso-ruby-visibility:none;}
br
{mso-data-placement:same-cell;}
td
{padding-top:1px;
padding-right:1px;
padding-left:1px;
mso-ignore:padding;
color:black;
font-size:12.0pt;
font-weight:400;
font-style:normal;
text-decoration:none;
font-family:等线;
mso-generic-font-family:auto;
mso-font-charset:134;
mso-number-format:General;
text-align:general;
vertical-align:middle;
border:none;
mso-background-source:auto;
mso-pattern:auto;
mso-protection:locked visible;
white-space:nowrap;
mso-rotate:0;}
ruby
{ruby-align:left;}
rt
{color:windowtext;
font-size:9.0pt;
font-weight:400;
font-style:normal;
text-decoration:none;
font-family:等线;
mso-generic-font-family:auto;
mso-font-charset:134;
mso-char-type:none;
display:none;}
-->
</head>
<body link="#0563C1" vlink="#954F72">
sparse vector | growing segment | sealed segment
-- | -- | --
BM25 | no | no
IP | yes | no
| |
dense vector | growing segment | sealed segment
ivf flat cc | yes | yes
scann_dvr | no | no
</body>
</html>
Signed-off-by: cqy123456 <qianya.cheng@zilliz.com>
2025-02-13 09:56:47 +08:00
7ca5e8309b
enhance: [GoSDK][2.5] Sync gosdk commits from master branch ( #39823 )
...
Cherry-pick from master
pr: #38791 #38812 #38940 #39515 #39580 #39662 #39665 #39751
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2025-02-12 18:46:47 +08:00
0f4d3ef8b6
enhance[2.5]: move GetVector_EmptySparseVector ut to growing test ( #39286 )
...
issue: https://github.com/milvus-io/milvus/issues/35853
pr: https://github.com/milvus-io/milvus/pull/39234
Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com>
2025-02-12 16:00:49 +08:00
dd516a5716
fix: [2.5]not pass the indexname when drop properties ( #39679 )
...
pr: #39678
issue: #38967
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2025-02-12 15:52:48 +08:00
1b14f5a7ab
fix: [2.5] Segments return both growing&sealed result ( #39789 )
...
Cherry-pick from master
pr: #39787
Previous PR #38311
Logic bug found in #38348
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2025-02-12 14:44:48 +08:00
82dc57ace0
fix: [skip e2e][2.5] pr conflict cause ut failed ( #39810 )
...
Related to https://github.com/milvus-io/milvus/pull/39701 &
https://github.com/milvus-io/milvus/issues/39681
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2025-02-12 11:44:51 +08:00
667c84740c
enhance: [2.5] Add metrics for parse expression ( #39716 )
...
master pr: #39654
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2025-02-12 10:30:46 +08:00
c2986d4ab9
fix: [2.5] Fix concurrent map ( #39776 )
...
issue: https://github.com/milvus-io/milvus/issues/39778
pr: https://github.com/milvus-io/milvus/pull/39775
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2025-02-11 22:12:46 +08:00
4322a0d49a
fix: [2.5] Resolve conflict on qc task test ( #39797 )
...
Cherry-pick from master
pr: #39796
Related to #39701 & #39681
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2025-02-11 18:52:45 +08:00
11cba57dc7
fix: [2.5] load collection stucks if compaction/gc happens ( #39761 )
...
issue: #39680
pr: #39701
if compaction/gc happens, load collection may stuck due to
SegmentNotFound, we should trigger UpdateNextTarget to get a new data
view to execute loading operation.
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2025-02-11 15:48:50 +08:00
969e34d540
fix: [2.5]uneven distribution caused by executing task delta cache leak ( #39759 )
...
issue: #39681
pr: #39702
this PR maintain workload effect in action instead of computing workload
effect from target, which may cause leak if target changes.
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2025-02-11 14:32:46 +08:00
c0cc8a59f7
fix: [2.5] Return early when skip load pk index ( #39763 )
...
Cherry pick from master
pr: #39762
Previous PR #39437 only print log and add index while load operation is
still executed. This PR return early when segment decides not to load PK
index.
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2025-02-11 11:04:50 +08:00
f542c866b3
fix: [2.5] root check the role list if `rootShouldBindRole` is true ( #39714 )
...
- issue: #39712
- pr: #39713
Signed-off-by: SimFG <bang.fu@zilliz.com>
2025-02-11 10:10:51 +08:00
87b39c2022
fix: [skip e2e] update the collection detail page logic ( #39739 )
...
issue: #36621
pr: #39740
Signed-off-by: mimo.oyn <ning.ouyang@zilliz.com>
2025-02-10 15:26:45 +08:00
b936061ec6
enhance: keep consistent of memory and meta of broadcaster ( #39721 )
...
issue: #38399
pr: #39720
Signed-off-by: chyezh <chyezh@outlook.com>
2025-02-10 10:50:44 +08:00
4a8e6fc59c
fix: [2.5] Fix flowgraph leak ( #39686 )
...
Consider lag targets when determining whether to remove the dispatcher
manager.
issue: https://github.com/milvus-io/milvus/issues/39642
pr: https://github.com/milvus-io/milvus/pull/39656
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2025-02-08 15:40:45 +08:00
8f2e7e93b3
fix: [skip e2e] update web ui tasks and query segments list ( #39676 )
...
issue: #36621
pr: #39693
Signed-off-by: mimo.oyn <ning.ouyang@zilliz.com>
2025-02-08 09:50:48 +08:00
b02d6423a3
enhance: [2.5]update knowhere version ( #39673 )
...
/kind branch-feature
release note :
https://github.com/zilliztech/knowhere/releases/tag/v2.5.3
Signed-off-by: xianliang.li <xianliang.li@zilliz.com>
2025-02-07 12:46:45 +08:00
858dc10ef9
enhance: broadcast with event-based notification ( #39550 )
...
issue: #38399
pr: #39522
- broadcast message can carry multi resource key now.
- implement event-based notification for broadcast messages
- broadcast message use broadcast id as a unique identifier in message
- broadcasted message on vchannels keep the broadcasted vchannel now.
- broadcasted message and broadcast message have a common broadcast
header now.
---------
Signed-off-by: chyezh <chyezh@outlook.com>
2025-02-07 11:50:50 +08:00
ddc5b299ad
enhance: expose more metrics data ( #39466 )
...
issue: #36621 #39417
pr: #39456
1. Adjust the server-side cache size.
2. Add source information for configurations.
3. Add node ID for compaction and indexing tasks.
4. Resolve localhost access issues to fix health check failures for
etcd.
Signed-off-by: jaime <yun.zhang@zilliz.com>
2025-02-07 11:48:45 +08:00
c32ec451b0
fix: [2.5] Use param item formatter to avoid SetConfig to overlay ( #39597 ) ( #39636 )
...
Cherry-pick from master
pr: #39597
Related to #39596
When updating the build param configuration, the `Formatter` could be
used to do so and completed avoid touching the `overlay` config items
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2025-02-07 10:02:44 +08:00
8a0f0f4099
enhance: [2.5] Add dsl log field for HybridSearch ( #39584 ) ( #39598 )
...
Cherry-pick from master
pr: #39584
The dsl fields are separated in sub request structs and cannot be easily
printed before. This PR adds a log helper to print the dsl expressions
of HybridSearchRequest.
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2025-02-07 09:58:43 +08:00
aee9368c6e
enhance: [2.5] update knowhere version and IsAdditionalScalarSupported interface ( #39574 )
...
pr: #39573
---------
Signed-off-by: chasingegg <chao.gao@zilliz.com>
2025-02-05 19:53:11 +08:00
22a69b5399
enhance: [2.5]Only check L0 compaction with same channel when stating ( #39543 )
...
issue: #39333
master pr: #39459
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2025-02-05 17:17:11 +08:00
7fb91d15d3
enhance: [skip e2e] [2.5]upgrade upload-artifact version ( #39638 )
...
https://github.com/milvus-io/milvus/pull/39391
---------
Signed-off-by: Yellow Shine <sammy.huang@zilliz.com>
2025-02-05 14:19:15 +08:00
bc91b2a4ad
fix: [2.5] metastore privilege name check with privilege name all ( #39492 )
...
cherry-pick from master: https://github.com/milvus-io/milvus/pull/39476
related: https://github.com/milvus-io/milvus/issues/39365
Signed-off-by: shaoting-huang <shaoting.huang@zilliz.com>
2025-01-26 12:17:19 +08:00
8934672687
enhance: [2.5] Skip update index metrics if index dropped ( #39458 ) ( #39572 )
...
Cherry-pick from master
pr: #39458
Related to #39457
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2025-01-24 18:21:06 +08:00
a48749cc11
enhance: [2.5] Use mockery pkg config for datacoord&datanode ( #39567 ) ( #39577 )
...
Cherry-pick from master
pr: #39567
Related to #38339
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2025-01-24 17:21:13 +08:00
congqixia
zhagnlu
cai.zhang
XuanYang-cn
Zhen Ye
yanliang567
Xianhui Lin
zhenshan.cao
presburger
smellthemoon
wei liu
SimFG
cqy123456
Buqian Zheng
yihao.dai
MimoN
foxspy
jaime
Gao
yellow-shine
sthuang