1d0138d16e
fix: limit minimal pq code size for diskann index ( #29785 )
...
issue: #29784
limit the minimal pq code size to avoid floating-exception
Signed-off-by: xianliang <xianliang.li@zilliz.com>
2024-01-09 15:48:55 +08:00
3787da8e7d
enhance: [hotfix-2.3.3] Add concurrency for datacoord segment GC ( #29562 )
...
issue: https://github.com/milvus-io/milvus/issues/29553
pr: #29561
/kind improvement
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-01-03 14:22:49 +08:00
5194240b08
enhance: [Cherry-pick] make sure stream closed ( #29458 )
...
relate: https://github.com/milvus-io/milvus/issues/28367
pr:https://github.com/milvus-io/milvus/pull/29456
---------
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2023-12-27 11:34:46 +08:00
2d517039b5
fix: [Cherry-Pick] msgstream memory leak caused by config event ( #29418 )
...
relate: https://github.com/milvus-io/milvus/issues/28620
https://github.com/milvus-io/milvus/issues/28367
pr: https://github.com/milvus-io/milvus/pull/29266
https://github.com/milvus-io/milvus/pull/29377
https://github.com/milvus-io/milvus/pull/29403
---------
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2023-12-25 11:18:44 +08:00
38f50f6e4a
enhance: [hotfix-2.3.3] Support to get the param value in the runtime ( #29396 )
...
pr: https://github.com/milvus-io/milvus/pull/29297
/kind improvement
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-12-22 19:12:43 +08:00
5b6a894e7c
fix: [hotfix-2.3.3]Clean the compaction plan info to avoid the object leak ( #29369 )
...
issue: https://github.com/milvus-io/milvus/issues/29296
pr: https://github.com/milvus-io/milvus/pull/29365
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-12-21 15:10:42 +08:00
1ddb12ba92
enhance: enhance the accuracy of memory usage ( #28554 ) ( #28559 ) ( #29381 )
...
before this, Milvus use container/system's memory info to get the memory
usage, which could be inaccurate.
we allocates the memory by private anon mmap,
then rss - shared would be the accurate memory usage
resolve https://github.com/milvus-io/milvus/issues/28553
pr: #28554
---------
Signed-off-by: yah01 <yah2er0ne@outlook.com>
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-12-21 15:00:43 +08:00
747934eb2b
enhance: make consistency level used in delete configurable ( #29280 ) ( #29285 )
...
pr: #29280
issue: #29279
---------
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2023-12-20 14:14:05 +08:00
b9757f8433
enhance: [hotfix] Add channel checkpoint ts lag metric ( #28767 ) ( #29324 )
...
Cherry-pick from master
pr: #28767
See also #28765
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-12-20 11:04:45 +08:00
869f89cf33
fix: [hotfix]parse logID from logPath if copyDeltalog find logID not provided ( #29278 )
...
Cherry-pick from master
pr: #29273
See also: #29272
This PR add `getDeltaLogID` to safely return logID when Binlog struct
has zero value logID. It parses logID from logPath if the format is
valid. Otherwise, this function shall return error.
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-12-19 14:02:40 +08:00
d34ada83cb
enhance: Remove rpc during querycoord start ( #28396 ) ( #29264 )
...
issue: #28332
pr: #28396
during querycoord's recover, it try to call `DescribeCollection` and
`ShowPartitions` to root coord, to checker whether collection or
partition has been released in rootcoord. but if rootcoord isn't not
ready yet, the rpc will fail, the querycoord panic.
to fix this, we remove rpc call during querycoord's start
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-12-18 14:13:35 +08:00
8e19049b5c
enhance: remove merger for load segments ( #29062 ) ( #29078 )
...
remove merger as now QueryNode could load segments concurrently
fix https://github.com/milvus-io/milvus/issues/29063
pr: #29062
---------
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-12-08 22:31:58 +08:00
e29a8e5c2f
enhance: [hotfix] Add client connect wrapper to keep connection alive ( #29079 )
...
Cherry-pick from master
pr: #29058
See also #29057
Add wrapper to maintain client&connection, when reset operation is
needed, close method shall wait until all on-going request return
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-12-08 18:48:33 +08:00
125bd39ade
enhance: improve load speed ( #28518 ) ( #29070 )
...
This check rejects load request if running out the pool workers, but
small segment would be loaded soon, another segments would been loading
again after a check interval, which leads to slow loading for collection
Block the request by go pool
pr: #28518
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-12-08 15:58:37 +08:00
0a996f777e
fix: clean snapshots meta ( #28502 ) ( #28998 )
...
issue: https://github.com/milvus-io/milvus/issues/28496 /kind bug
pr: #28502
The input parameters collection.partitions and collection.Field are both
nil, so these two metas have not been cleared.
Signed-off-by: xige-16 <xi.ge@zilliz.com>
Co-authored-by: xige-16 <xi.ge@zilliz.com>
2023-12-06 09:51:05 +08:00
3deca87023
enhance: Remove partition load infos with prefix ( #28969 ) ( #28972 )
...
Removing partition load infos with prefix instead of removing them one
by one. This enhance can save transactions.
issue: https://github.com/milvus-io/milvus/issues/28343
pr: https://github.com/milvus-io/milvus/pull/28969
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-12-05 19:46:35 +08:00
f726650a6d
fix: [cherry-pick]Fix failed to release collection with more than 128 partitions ( #28939 )
...
pr: #28446
Signed-off-by: wayblink <anyang.wang@zilliz.com>
2023-12-05 10:14:33 +08:00
a0c5b67d18
enhance: [hotfix] Reject compaction task with growing segments ( #28953 )
...
Cherry-pick from 2.3
pr: #28927
See also #28924
The compaction task generated before datanode finish SaveBinlogPath grpc
call contains segments which are still in Growing state
DataNode shall verify each non-levelzero segments before submit
compaction task to executor
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-12-05 09:56:33 +08:00
cacc018ece
enhance: [hotfix] Resolve libunwind requirement conflict using 1.7.2 ( #28936 )
...
Cherry-pick from master
pr: #28929
Try to resolve libunwind dependency requirement conflict between glog &
folly
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-12-04 17:30:32 +08:00
628df9741c
fix: [cherry-pick] azure ListObjects ( #27931 ) ( #28913 )
...
issue: #27932
master pr: #27931
2.3 pr: #28894
Signed-off-by: PowderLi <min.li@zilliz.com>
2023-12-03 19:00:32 +08:00
0b6235a1f5
Hide the password info when failing to authorize ( #28428 ) ( #28480 )
...
/kind improvement
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-11-16 15:49:35 +08:00
f2e972abb3
Fix external grpc server doesn't graceful stop ( #28310 )
...
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-11-09 22:12:22 +08:00
c8cab4dfc3
No hit when the index exceeds the array length ( #28302 )
...
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2023-11-09 22:10:32 +08:00
83b7f05fe1
Sync proto generated files to avoid --dirty tag ( #28316 )
...
The proto generated files is out of sync for image build env
This will cause --dirty="-dev" tag in Milvus build version
Sync changed files to avoid this case
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-11-09 21:22:21 +08:00
0497bbf05d
Handle exception while loading ( #28306 )
...
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-11-09 19:28:35 +08:00
44063be0a6
Fix buffer FieldData has no `ElementType` and array logsize always zero ( #28296 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-11-09 14:22:36 +08:00
fe7c87592c
Fix auto balance block channel reassign after datanode restart ( #28276 )
...
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-11-09 13:52:19 +08:00
16dc26833b
Fix retry when proxy stopped ( #28263 )
...
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-11-09 10:10:19 +08:00
39c24fe07b
print azure sdk log ( #28241 )
...
Signed-off-by: PowderLi <min.li@zilliz.com>
2023-11-09 10:08:20 +08:00
d3f149c403
fix unstable auto balance config ut ( #28289 )
...
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-11-09 10:02:19 +08:00
e51ceaae3a
Not convert legacy error code to new merr ( #28232 ) ( #28274 )
...
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-11-08 19:18:20 +08:00
385507ce47
Fix the target updated before version updated to cause data missing ( #28257 )
...
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-11-08 18:54:18 +08:00
12a09231f1
fix datacoord unstable ut ( #28282 )
...
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-11-08 18:44:58 +08:00
99350b1be4
Force set aliyun use_virtual_host to true for all ( #28237 )
...
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2023-11-08 11:10:21 +08:00
1944e955cb
publish cardinal for milvus 2.3.3 ( #28254 )
...
Signed-off-by: xianliang <xianliang.li@zilliz.com>
2023-11-08 10:56:20 +08:00
29e66ed46b
Fix bulkinsert bug that segments are compacted after import ( #28227 )
...
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2023-11-08 10:18:20 +08:00
cc3dd6eda4
Set Knowhere version to v2.2.2 ( #28252 )
...
Signed-off-by: chasingegg <chao.gao@zilliz.com>
2023-11-08 08:46:19 +08:00
918333817e
Disable auto balance when old node exists ( #28191 ) ( #28224 )
...
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-11-08 07:10:17 +08:00
d76c744646
Fix proxy not healthy before graceful stop ( #28255 )
...
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-11-08 02:22:26 +08:00
598788e6b8
Delay the cancellation of ctx when stopping the node ( #28249 )
...
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-11-08 01:46:20 +08:00
19230db7f5
Fix bug for constructing ArrayView with fixed-length type ( #28186 )
...
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2023-11-07 23:36:21 +08:00
304f232a02
Fix bug for setting index state when indexnode connecting failed ( #28221 )
...
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2023-11-07 23:34:26 +08:00
d0bea0eb0a
Get vector concurrently ( #28119 )
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-11-07 19:38:21 +08:00
62e585d8db
fix check grpc error logic ( #28182 ) ( #28218 )
...
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-11-07 14:22:16 +08:00
d10a82dba4
Fix getting incorrect CPU num ( #28178 )
...
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-11-07 11:52:22 +08:00
bbcaf7a703
Fix coordinator fast restart ( #28205 )
...
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2023-11-07 10:04:16 +08:00
e8950a529e
fix grpc client retry on node server not match error ( #28169 ) ( #28177 )
...
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-11-07 00:44:17 +08:00
300c1f5529
reduce useless ObjectExists ( #28157 )
...
replace ListBlobs() with GetProperties()
unified style std::string& / char*
config azure requestTimeoutMs
Signed-off-by: PowderLi <min.li@zilliz.com>
2023-11-06 22:14:21 +08:00
43db873770
add a custom http header: Accept-Type-Allow-Int64 ( #28125 )
...
Signed-off-by: PowderLi <min.li@zilliz.com>
2023-11-06 16:02:18 +08:00
87e8d04ed7
fix sync distribution with wrong version ( #28130 ) ( #28170 )
...
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-11-06 11:38:18 +08:00
foxspy
SimFG
aoiasd
yah01
Jiquan Long
congqixia
wei liu
yah01
zhenshan.cao
yihao.dai
wayblink
PowderLi
cai.zhang
zhagnlu
groot
Gao
Xiaofan