b88d140219
enhance: use soft node affinity instead of hard nodeSelector ( #32675 )
...
pr: #32677
Signed-off-by: Liang Huang <sammy.huang@zilliz.com>
2024-04-29 14:25:36 +08:00
868723340c
enhance: [2.4] the list collection and database api ( #32602 )
...
issue: https://github.com/milvus-io/milvus/issues/32550
pr: #32576
/kind improvement
---------
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-04-29 14:21:26 +08:00
5da6db9252
enhance: [2.4] add the skip auto id and partition key check config ( #32597 )
...
/kind improvement
issue: https://github.com/milvus-io/milvus/issues/32591
pr: #32592
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-04-29 10:19:31 +08:00
05c14d4c21
enhance: update knowhere version ( #32559 )
...
master pr: https://github.com/milvus-io/milvus/pull/32468
Signed-off-by: chasingegg <chao.gao@zilliz.com>
2024-04-28 19:31:25 +08:00
51e08556a9
fix: Correct the negative queryable num entities metric ( #32361 ) ( #32581 )
...
issue: https://github.com/milvus-io/milvus/issues/32281
pr: https://github.com/milvus-io/milvus/pull/32361
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-04-28 15:13:26 +08:00
6178355e2f
fix: Remove watching config event in rate limiter ( #32313 ) ( #32314 )
...
Remove watching config event in rate limiter to prevent object leaks.
Instead, reset limiter rates periodically.
issue: https://github.com/milvus-io/milvus/issues/32312
pr: https://github.com/milvus-io/milvus/pull/32313
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-04-28 11:35:28 +08:00
193f88bec3
enhance: [2.4] Make applyDelete work in paralell in segment level ( #32291 ) ( #32604 )
...
Cherry-pick from master
pr: #32291
`applyDelete` used to be serial for delete entries on each segments.
This PR make it work in parallel with errgroup to improve performance
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-04-26 18:09:27 +08:00
4f465343d3
enhance: [2.4] update the go-api version for the list api ( #32606 )
...
issue: https://github.com/milvus-io/milvus/issues/32550
pr: #32605
/kind improvement
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-04-26 17:49:33 +08:00
cd830f8cbc
fix: [cp]Zero flushReq metric for all sealed segs ( #32531 )
...
See also: #32399
pr: #32404
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-04-26 15:17:28 +08:00
b62fba47b2
enhance: bind e2e to dedicated nodes ( #32579 )
...
pr:#32600
---------
Signed-off-by: Liang Huang <sammy.huang@zilliz.com>
2024-04-26 14:13:30 +08:00
8e7c9ee00e
fix: correct AssignSegmentID return and add retry for loadCollectionF… ( #32349 )
...
pr: #32335
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-04-25 14:31:25 +08:00
08e633187e
test: update pymilvus version and modify test cases ( #32573 )
...
issue: #32376
pr: #32572
Signed-off-by: binbin lv <binbin.lv@zilliz.com>
2024-04-25 08:57:31 +08:00
e1b4ef7451
enhance: [2.4] Make LeaderView Manager filter use map index ( #32505 ) ( #32528 )
...
Cherry-pick from master
pr: #32505
See also #32165
Change `LeaderViewFilter` to interface to provided map key to avoid
iterating all key-values in LeaderViewManager
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-04-24 18:01:25 +08:00
1aba9eb616
fix: [2.4] Remove metrics after flowgraph closed ( #32515 ) ( #32539 )
...
Cherry-pick from master
pr: #32515
See also #32403
`fg_buffer_size` was decreased after metrics removed in flowgraph
ddnode, which make metrics value negative.
This PR move remove metrics logic into `dataSyncService.Close`
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-04-24 14:21:24 +08:00
bafa67c1b6
test: [cherry pick] enable groupby tests cases as master ( #32519 )
...
cherry pick from master to 2.4
related pr: https://github.com/milvus-io/milvus/pull/31903
Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
2024-04-24 10:27:23 +08:00
7cace01d22
fix: [restful v2] parameters of create collection ( #32486 )
...
issue: #31176
master pr: #32485
Signed-off-by: PowderLi <min.li@zilliz.com>
2024-04-24 09:43:28 +08:00
71c30c9144
enhance: [2.4] Make querycoordv2 collection observer task driven ( #32441 ) ( #32493 )
...
Cherry-pick from master
pr: #32441
See also #32440
- Add loadTask in collection observer
- For load collection/partitions, load task shall timeout as a whole
- Change related constructor to load jobs
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-04-23 20:09:23 +08:00
e7b3651ef0
test: [cherry-pick]fix delete expr ( #32378 )
...
pr: https://github.com/milvus-io/milvus/pull/32377
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2024-04-23 14:11:23 +08:00
969e9cc63e
fix: add bitset sve support macro protect ( #32464 )
...
pr: #32463
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-04-22 12:29:24 +08:00
8b0d445aac
enhance: [2.4] Manual release pool after save targets ( #32358 ) ( #32457 )
...
Cherry-pick from master
pr: #32358
See also #31632
Release conc.Pool after usage to clean worker and stop background purge
and ticktock.
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-04-22 10:11:22 +08:00
87719caab4
enhance: [2.4]speed up minor functions calls in datacoord ( #32477 )
...
Related to https://github.com/milvus-io/milvus/issues/32165
Duplicate of #32389 but for 2.4 branch.
Signed-off-by: yiwangdr <yiwangdr@gmail.com>
2024-04-20 11:45:21 +08:00
238f9a4ab0
enhance: Update tls user doc ( #32354 )
...
pr: #32353
Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-04-18 21:05:20 +08:00
eba9c74c5a
enhance: revert [skip e2e] use pull_request_target event instead (#32… ( #32429 )
...
pr: #32426
This reverts commit 6a4062d08c778ac7ba1675d03ae365e236b32078.
Signed-off-by: Liang Huang <sammy.huang@zilliz.com>
2024-04-18 20:08:04 +08:00
21474c1ab0
fix: [restful]use default search parameter `nq: 0` ( #32356 )
...
issue: #32225
master pr: #32355
1. v1 can only accept one vector, but v2 accept list of vectors
2. cannot get dbName from AliasReq #31978
---------
Signed-off-by: PowderLi <min.li@zilliz.com>
2024-04-18 14:35:19 +08:00
8f7ac8f7b8
enhance: [2.4] Use map instead of slice to maintain channel info ( #32273 ) ( #32316 )
...
Cherry-pick from master
pr: #32273
See also #32165
`ChannelManager.Match` is a frequent operation for datacoord. When the
collection number is large, iteration over all channels will cost lots
of CPU time and time consuming.
This PR change the data structure storing datanode-channel info to map
avoiding this iteration when checking channel existence.
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-04-17 19:07:20 +08:00
dff96c323b
test: [cherry-pick]use float32 to do import ( #32327 )
...
pr: https://github.com/milvus-io/milvus/pull/32276
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2024-04-17 10:57:20 +08:00
82cf91fd72
enhance: hotfix for pipeline ( #32338 )
...
Signed-off-by: Liang Huang <sammy.huang@zilliz.com>
2024-04-16 22:58:48 +08:00
4c636e8a7a
[automated] Bump milvus version to v2.4.0 ( #32337 )
...
Bump milvus version to v2.4.0
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-04-16 22:55:20 +08:00
cee6071300
fix: remove sve flags ( #32275 )
...
pr: #32270
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-04-16 20:27:32 +08:00
b564f065c7
enhance: hot fix publish pipeline ( #32331 )
...
pr: #32330
Signed-off-by: Liang Huang <sammy.huang@zilliz.com>
2024-04-16 19:48:25 +08:00
0e5118b1ba
fix: remove useless log of quota center ( #32307 ) ( #32310 )
...
/kind improvement
pr: #32307
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2024-04-16 18:39:19 +08:00
52cc1ed9d8
enhance: [2.4] make the auth error message more suitable ( #32298 )
...
issue: #32252
pr: #32253
/kind improvement
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-04-16 17:13:19 +08:00
d565d64941
enhance: [2.4] make datanode exit and case `TestProxy` faster ( #32293 )
...
/kind improvement
issue: #32219
pr: #32218
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-04-16 16:37:20 +08:00
ffb6edd433
enhance: Bump milvus-proto to v2.4.0 ( #32315 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-04-16 16:23:19 +08:00
bda2f8d1ac
[automated] Update Pytest image changes ( #32278 )
...
Update Pytest image changes
See changes:
f6377a5018sre-ci-robot@users.noreply.github.com
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-04-16 09:47:20 +08:00
9b25ce882b
test: add test cases for code change ( #32290 )
...
issue: #31368
pr: #32289
Signed-off-by: binbin lv <binbin.lv@zilliz.com>
2024-04-16 09:45:19 +08:00
018a784989
enhance: speed up GetByCollection/AndNode (2.4 patch) ( #32234 )
...
Related to https://github.com/milvus-io/milvus/issues/32165
Avoid iterating through all replicas/collections if possible. Iteration
is expensive when there are large number of replicas/collections.
Signed-off-by: yiwangdr <yiwangdr@gmail.com>
2024-04-15 19:53:21 +08:00
f6377a5018
test: enable bf16 and f16 test cases ( #32082 ) ( #32259 )
...
issue: #31625
pr: #32082
Signed-off-by: binbin lv <binbin.lv@zilliz.com>
2024-04-15 19:39:31 +08:00
9e3099e20e
enhance: [2.4] Maintain collection-patitions mapping in qc meta ( #32227 ) ( #32249 )
...
Cherry-pick from master
pr: #32227
Related to #32165
Add collection to partitionIDs mapping to avoid interation on all
partitions loaded when trying to get all partitions with collection id
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-04-15 18:15:19 +08:00
2726d639f9
fix: fix collectionInfo leak in datacoord ( #32243 )
...
issue: #32029
pr: #32175
lack of logic to clean collection info in datacoord's meta, This PR
clean collection info after drop channel, to avoid collection info leak
in datacoord
---------
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-04-15 17:21:19 +08:00
0df0972eb4
enhance: support disable search optimization ( #32142 )
...
pr: https://github.com/milvus-io/milvus/pull/32141
---------
Signed-off-by: chasingegg <chao.gao@zilliz.com>
2024-04-15 17:17:20 +08:00
3cacfeba0b
fix: panic when get empty ids( #32220 ) ( #32258 )
...
issue: #32223
pr: #32220
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-04-15 16:53:23 +08:00
c2ab387b1e
fix: [cherry-pick] more comprehensive check on sparse index and value. ( #32251 )
...
issue: #29419
pr: #32250
Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com>
2024-04-15 16:51:24 +08:00
1e4f5ee978
[automated] Update gpu Builder image changes ( #32050 )
...
Update gpu Builder image changes
See changes:
d8c50f024csre-ci-robot@users.noreply.github.com
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-04-15 14:39:19 +08:00
8894149a63
test: support enable or disable multiple vector fields in all data type tests ( #31201 ) ( #32246 )
...
issue: #29799
pr: #31201
Signed-off-by: binbin lv <binbin.lv@zilliz.com>
2024-04-15 14:25:20 +08:00
0a3a483d02
enhance: [cherry-pick]Use channel manager interface in server_test (… ( #32211 )
...
…#31621)
Tidy the following test codes
- Remove channel in newTestServer
- Remove newTestServerWithMeta
- Remove newTestServer2
- Remove testDataCoordBase
- Use the same func for handleTTmsg and handleRPCTTmsg
See also: #31620
pr: #31621
---------
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-04-15 11:57:20 +08:00
e50599ba10
fix: Skip to load l0 segment on old version query node ( #32131 )
...
issue: #32107
pr: #32124
during rolling upgrade progress, skip to load l0 segment on old version
query node
---------
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-04-15 11:23:23 +08:00
b68af208bc
fix: Use pk from binlog during import ( #32118 ) ( #32194 )
...
During binlog import, even if the primary key's autoID is set to true,
the primary key from the binlog should be used instead of being
reassigned.
issue: https://github.com/milvus-io/milvus/discussions/31943 ,
https://github.com/milvus-io/milvus/issues/28521
pr: https://github.com/milvus-io/milvus/pull/32118
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-04-15 10:31:19 +08:00
68b7469d83
enhance: [skip e2e] use pull_request_target event instead ( #32241 )
...
issue: #32235
pr: #32236
Signed-off-by: Liang Huang <sammy.huang@zilliz.com>
2024-04-14 23:35:22 +08:00
5f7edc7304
fix: lack good results for insufficient ef( #29883 ) ( #32151 )
...
related: #29883
pr: https://github.com/milvus-io/milvus/pull/32080
see also: #29883
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
2024-04-13 22:13:23 +08:00
sammy.huang
SimFG
Gao
yihao.dai
congqixia
XuanYang-cn
zhagnlu
binbin
yanliang567
PowderLi
zhuwenxing
yiwangdr
jaime
sre-ci-robot
Jiquan Long
wei liu
smellthemoon
Buqian Zheng
Chun Han