3c31499bbc
enhance: [2.4]Assert insert data length not overflow int ( #33248 ) ( #33271 )
...
Cherry-pick from master
pr: #33248
When InsertData is too large for cpp proto unmarshalling, the error
message is confusing since the length is overflowed
This PR adds assertion for insert data length.
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-05-22 19:07:40 +08:00
64d0a71ba0
fix: arm-based gpu image ( #33276 )
...
Signed-off-by: Liang Huang <sammy.huang@zilliz.com>
2024-05-22 17:08:46 +08:00
e5ca1f7c45
fix: Return error when startup Delete/AddNode fail ( #33193 ) ( #33258 )
...
See also: #33151 , #33149
pr: #33193
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-05-22 14:49:40 +08:00
4b8680894f
fix: Clean offline node from resource group after qc restart ( #33233 )
...
issue: #33200 #33207
pr: #33232
pr#33104 causes the offline node will be kept in resource group after qc
recover, and offline node will be assign to new replica as rwNode, then
request send to those node will fail by NodeNotFound.
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-05-22 14:07:39 +08:00
1f23c39700
fix:[Cherry-Pick] can't generate traceID when use noop exporter ( #33191 ) ( #33208 )
...
relate: https://github.com/milvus-io/milvus/issues/33190
pr: https://github.com/milvus-io/milvus/pull/33191
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-05-22 11:13:39 +08:00
2a38f5b598
enhance: [2.4] add config to control whether to init public role permissions ( #33174 )
...
issue: #33164
pr: #33165
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-05-21 22:39:46 +08:00
00b05fcc02
fix: Remove L0 compactor in completedCompactor ( #33169 ) ( #33216 )
...
See also: #33168
pr: #33169
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-05-21 19:07:39 +08:00
7013d2f906
feat: support arm-based image build and pull request ( #33225 )
...
Signed-off-by: Liang Huang <sammy.huang@zilliz.com>
2024-05-21 16:54:43 +08:00
e5db2f7e77
fix: [2.4] Check schema without vector field in proxy ( #33211 ) ( #33218 )
...
Cherry-pick from master
pr: #33211
Related to #33199
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-05-21 16:37:44 +08:00
b2f7d7ba4e
fix: [2.4]channel manager's goroutine run order ( #33121 )
...
See also: #33117
pr: #33118
---------
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-05-21 14:31:39 +08:00
a27a2e8021
fix: try best to get enough query results ( #33178 ) ( #33179 )
...
issue: https://github.com/milvus-io/milvus/issues/33137
pr: #33178
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2024-05-21 14:07:45 +08:00
5370c39a23
fix: [2.4] Mark channel checkpoint dropped prevent cp lag metrics leakage ( #32454 ) ( #33201 )
...
Cherry-pick from master
pr: #33198
See also #31506 #31508
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-05-21 11:57:51 +08:00
2f260cd33b
enhance: reduce memory when read field( #33195 ) ( #33196 )
...
pr:https://github.com/milvus-io/milvus/pull/33195
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-05-20 21:17:39 +08:00
cf7765915c
enhance: [2.4]Refine channel mgr v2 implementation ( #33156 ) ( #33172 )
...
Cherry-pick from master
pr: #33156
Related to #25309
- Remove ctx from struct
- Add ctx parameters for internal check logic methods
- Add Waitgroup to make sure worker goroutine quit before close returns
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-05-20 18:25:38 +08:00
6b02c085f6
fix: [2.4] reset the quota value when init the limiter ( #33152 )
...
issue: #33107
pr: #33111
/kind improvement
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-05-20 15:17:38 +08:00
9ae4945df2
fix: query node may stuck at stopping progress ( #33104 ) ( #33154 )
...
issue: #33103
pr: #33104
when try to do stopping balance for stopping query node, balancer will
try to get node list from replica.GetNodes, then check whether node is
stopping, if so, stopping balance will be triggered for this replica.
after the replica refactor, replica.GetNodes only return rwNodes, and
the stopping node maintains in roNodes, so balancer couldn't find
replica which contains stopping node, and stopping balance for replica
won't be triggered, then query node will stuck forever due to
segment/channel doesn't move out.
---------
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-05-20 15:01:43 +08:00
780f3137f3
[automated] Update Pytest image changes ( #33091 )
...
Update Pytest image changes
See changes:
9536e79363sre-ci-robot@users.noreply.github.com
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-05-17 15:09:36 +08:00
9536e79363
test: update pymilvus version ( #33077 )
...
issue: #17614
pr: #33079
Signed-off-by: binbin lv <binbin.lv@zilliz.com>
2024-05-16 14:51:34 +08:00
b2d83d3354
enhance: Bump milvus version to v2.4.2 ( #33048 )
...
Bumping version to v2.4.2. Also bump milvus-proto version to v2.4.3.
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-05-15 13:57:35 +08:00
259bc97a2b
fix: Fix segments lost in flush response ( #33061 )
...
#33055
Signed-off-by: wayblink <anyang.wang@zilliz.com>
2024-05-15 13:49:34 +08:00
6ea7633bd5
enhance: Add memory size for binlog ( #33025 )
...
issue: #33005
1. add `MemorySize` field for insert binlog.
2. `LogSize` means the file size in the storage object.
3. `MemorySize` means the size of the data in the memory.
---------
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2024-05-15 12:59:34 +08:00
d4837307b3
fix: Make submit idempotent ( #33053 )
...
issue: #33054
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-05-14 22:09:34 +08:00
6fc00e42d7
fix: Fix binlog import ( #33027 )
...
binlog import should only import into one partition.
issue: https://github.com/milvus-io/milvus/issues/32807
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-05-14 21:45:32 +08:00
3777eb2f68
enhance: update knowhere new version ( #33043 )
...
for milvus new release, knowhere update to latest release version
Signed-off-by: chasingegg <chao.gao@zilliz.com>
2024-05-14 20:45:32 +08:00
f48a7ff8ff
enhance: use Delete instead of DeletePartialMatch to remove metrics ( #33029 )
...
Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-05-14 18:49:33 +08:00
89a7c34c7a
fix: exceed etcd limit ( #33041 )
...
#32974
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-05-14 18:45:32 +08:00
dea7faf394
fix: Remove `RLock` re-entry in `GetNumRowsOfPartition` ( #33045 )
...
Resolves #33044
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-05-14 17:51:32 +08:00
3d85e6e028
fix: etcd txn exceeds limit due to too many fields ( #33040 )
...
fix : #33038
---------
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2024-05-14 17:05:33 +08:00
ba3b2a91a0
test: Remove useless common types and refine error assert in negative cases ( #33023 )
...
Related issue: #32653
1. Remove some meaningless common types
2. Refine error assertion in negative cases
3. Remove some dup tests
---------
Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
2024-05-14 15:03:33 +08:00
1d48d0aeb2
enhance: use different value to get related data size according to segment type ( #33017 )
...
issue: #30436
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-05-14 14:59:33 +08:00
4fc7915c70
enhance: unify data generation test APIs ( #32955 )
...
Issue: #22837
Signed-off-by: Cai Yudong <yudong.cai@zilliz.com>
2024-05-14 14:33:33 +08:00
96489b814d
fix: remove busy log ( #33042 )
...
issue: #32963
Signed-off-by: chyezh <chyezh@outlook.com>
2024-05-14 14:20:32 +08:00
861977ab60
fix: Start `LeaderCacheObserver` before `SyncAll` ( #33035 )
...
Related to #33033
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-05-14 13:25:32 +08:00
40e5c0be70
fix: Enable to balance released standby channel ( #32986 )
...
See also: #32879
---------
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-05-14 11:51:32 +08:00
dc058eaf61
fix: should init access logger before init server ( #32976 )
...
relate: https://github.com/milvus-io/milvus/issues/32968
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-05-14 10:47:32 +08:00
3871009cf5
test: add timeout for GetMetrics grpc interface ( #33024 )
...
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2024-05-14 10:29:32 +08:00
4ae7cabb04
fix: Remove channel when create flowgraph timeout ( #33014 )
...
See also #33013
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-05-14 10:07:33 +08:00
cba2c7a3be
enhance: clean channel node info in meta store ( #32988 )
...
issue: #32910
see also: #32911
when channel exclusive mode is enabled, replica will record channel node
info in meta store, and if the balance policy changes, which means
channel exclusive mode is disabled, we should clean up the channel node
info in meta store, and stop to balance node between channels.
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-05-14 10:05:40 +08:00
a984e46a29
enhance: Remove rootcoord from datanode broker ( #32818 )
...
issue: https://github.com/milvus-io/milvus/issues/32827
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-05-14 10:03:32 +08:00
09cca1fb51
[automated] Update Knowhere Commit ( #33030 )
...
Update Knowhere Commit
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-05-14 01:47:32 +08:00
9e3f3d99f7
test: add timeout for import testcases ( #33006 )
...
add timeout for import testcases
---------
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2024-05-13 19:35:32 +08:00
efdbd8e7c1
enhance: Enable to upload by batch ( #32788 )
...
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-05-13 19:07:32 +08:00
29b621f759
fix: Make compactor able to clear empty segments ( #32821 )
...
See also: #32553
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-05-13 18:21:32 +08:00
ba625835bc
enhance: Add metrics for segment index files size ( #32979 )
...
issue:#32980
Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-05-13 17:59:32 +08:00
293f14a8b9
fix: remove redundant replica recover ( #32985 )
...
issue: #22288
- replica recover should be only triggered by replica recover
Signed-off-by: chyezh <chyezh@outlook.com>
2024-05-13 15:25:32 +08:00
f6777267e3
enhance: add score compute consistency config for knowhere ( #32997 )
...
issue: https://github.com/milvus-io/milvus/issues/32583
related: #32584
Signed-off-by: xianliang.li <xianliang.li@zilliz.com>
2024-05-13 14:21:31 +08:00
4031abd2fa
enhance: change default partition num to 16 when using partition key ( #32950 )
...
/kind improvement
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-05-13 14:19:31 +08:00
12ec3d61d9
fix: Fill deltalog entry num & time range in L0 compactions ( #33004 )
...
Resolves #33003
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-05-13 14:13:31 +08:00
4724779b3b
enhance: remove fallback keys for config generator ( #32946 )
...
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2024-05-13 13:33:31 +08:00
230c88b1de
enhance: improve reduce performance on very large topk ( #32871 )
...
fix #32870
reduce cpu usage for reduce on large k
Signed-off-by: xiaofanluan <xiaofan.luan@zilliz.com>
2024-05-13 11:55:31 +08:00
congqixia
sammy.huang
XuanYang-cn
wei liu
aoiasd
SimFG
Jiquan Long
smellthemoon
sre-ci-robot
binbin
wayblink
cai.zhang
yihao.dai
Gao
jaime
yanliang567
Cai Yudong
chyezh
zhuwenxing
foxspy
Bingyi Sun
Xiaofan