42e538b683
enhance: enable balance channel in querycoord ( #28469 )
...
issue: #23726
/kind improvement
1. enable auto balance channel between nodes in querycoord
2. make `genSegmentPlan` reuse the `AssignSegment` logic
3. make `genChannelPlan` reuse the `AssignChannel` logic
---------
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-12-11 14:18:37 +08:00
d2207cac27
fix: Fix rustup update ( #29093 )
...
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2023-12-11 11:36:37 +08:00
02621e131c
enhance: [skip-e2e] enlarge disk space of action runner ( #29077 )
...
/kind improvement
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2023-12-08 16:14:07 +08:00
2b05460ef9
enhance: Make import-related error message clearer ( #28978 )
...
issue: #28976
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2023-12-08 10:12:38 +08:00
464bc9e8f4
fix: fix reduce precision for search( #27325 ) ( #29031 )
...
related: #27325
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
2023-12-08 10:04:37 +08:00
c4dda3c50f
fix: panic caused by type assert LocalSegment on Segment ( #29018 )
...
related #29017
---------
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-12-08 01:50:38 +08:00
43abe9ce31
enhance: Delete datanode metric after flowgraph released ( #29003 )
...
`DataNodeFlowGraphBufferDataSize` monitoring metric is leaked. This PR
resolve it.
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-12-07 17:00:35 +08:00
9b3e06ae86
enhance: add more metrics for level zero segments ( #29029 )
...
- Add SegmentNum metric for level zero segments
- Add level zero segments size metirc
---------
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-12-07 14:34:35 +08:00
52594c8596
enhance: add libasan to amazon linux runner ( #29039 )
...
fix : #29038
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2023-12-07 14:14:42 +08:00
dcb662d9ed
enhance: Refine C.NewSegment response and handle exception ( #28952 )
...
See also #28795
Orignal `C.NewSegment` may panic if some condition is not met, this pr
changes response struct to `CNewSegmentResult`, which contains
`C.CStatus` and may return catched exception
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-12-07 13:34:35 +08:00
cb43647b9e
enhance: Log channel checkpoint source info in writebuffer ( #28993 )
...
See also #27675
Print channel checkpoint source with rated log will help debugging
system behavior
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-12-07 11:50:36 +08:00
6736f65345
feat: skip some empty ttMsg in Datanode flowgraph ( #28756 )
...
/kind feature
Signed-off-by: wayblink <anyang.wang@zilliz.com>
2023-12-07 01:00:37 +08:00
fb089cda8b
enhance: Load raw data while scalar index doesn't have raw data ( #28888 )
...
issue: #28886
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2023-12-06 20:36:36 +08:00
6a86ac0ac6
fix: Align minio object storage ut to new minio server behavior ( #29014 )
...
See also #29013
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-12-06 15:42:43 +08:00
4a067a4c8c
enhance: Add proxy.ginLogSkipPaths ( #28945 )
...
Fix #28944
/cc @xiaofan-luan
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
2023-12-06 10:26:35 +08:00
5bac7f7897
fix: Fix L0 compaction in datacoord ( #28814 )
...
See also: #27606
---------
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2023-12-05 18:44:37 +08:00
cb31016640
enhance: Write buffer time range when syncing logs ( #28970 )
...
Related to #27675
The timestamp from, to field is not field for new implementation of
writebuffer & sync manager
This pr fills these field for better log information
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-12-05 17:36:36 +08:00
fab52d167b
fix: may miss stream delta while loading ( #28871 )
...
we consume the delta data from the lastest channel checkpoint while
loading segment,
this works well without level 0 segments, but now it may lead to miss
some delta data,
so we have to consume from the current target's channel checkpoint
related: #27349
---------
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-12-05 17:34:45 +08:00
bb82074937
fix: Fix missing target db name for RenameCollection ( #28909 )
...
issue: #28908
Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-12-05 17:12:37 +08:00
1037b16471
[automated] Update Builder image changes ( #28982 )
...
/kind improvement
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2023-12-05 16:52:36 +08:00
3cc4209d26
enhance: pack proxy connection code and support accesslog print SDK_Version ( #28835 )
...
relate: https://github.com/milvus-io/milvus/issues/28086
https://github.com/milvus-io/milvus/issues/28940
---------
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2023-12-05 16:50:47 +08:00
36f69ea031
feat: integrate storagev2 in building index of segcore ( #28768 )
...
issue: https://github.com/milvus-io/milvus/issues/28655
---------
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2023-12-05 16:48:54 +08:00
ad1daebc8e
fix: Change ranges of HNSW parameters according to Knowhere ( #28875 )
...
fix: https://github.com/milvus-io/milvus/issues/28860
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2023-12-05 16:20:42 +08:00
89b965e2b1
fix: clean snapshots meta ( #28502 )
...
issue: https://github.com/milvus-io/milvus/issues/28496
/kind bug
The input parameters collection.partitions and collection.Field are both
nil, so these two metas have not been cleared.
Signed-off-by: xige-16 <xi.ge@zilliz.com>
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2023-12-05 16:04:36 +08:00
f01e507b15
[automated] Update Knowhere Commit ( #28965 )
...
Update Knowhere Commit
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2023-12-05 15:56:35 +08:00
11687dce60
enhance: [skip e2e]upgrade minio version ( #28971 )
...
Signed-off-by: Sammy Huang <sammy.huang@zilliz.com>
2023-12-05 14:14:34 +08:00
7f819ac594
enhance: Remove partition load infos with prefix ( #28969 )
...
Removing partition load infos with prefix instead of removing them one
by one. This enhance can save transactions.
issue: https://github.com/milvus-io/milvus/issues/28343
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-12-05 12:52:34 +08:00
e2d0333831
[automated] Update Builder gpu image changes ( #28963 )
...
/kind improvement
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2023-12-05 11:58:33 +08:00
10bb2431d8
test: add checker unittests ( #28954 )
...
issue: https://github.com/milvus-io/milvus/issues/28610
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2023-12-05 10:56:33 +08:00
91e92ee8d4
[automated] Update Pytest image changes ( #28938 )
...
Update Pytest image changes
See changes:
8740adbe28sre-ci-robot@users.noreply.github.com
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2023-12-05 10:06:35 +08:00
ab630548f4
Update all contributors
...
Signed-off-by: sre-ci-robot <sre-ci-robot@zilliz.com>
2023-12-04 12:48:33 +00:00
d26b563a8b
feat: Define import API and metadata ( #28731 )
...
Define the new rpc and metadata for ImportV2.
see also: https://github.com/milvus-io/milvus/issues/28521
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-12-04 19:56:35 +08:00
393b1f943c
fix: Reject compaction task with growing segments ( #28925 )
...
See also #28924
The compaction task generated before datanode finish `SaveBinlogPath`
grpc call contains segments which are still in Growing state DataNode
shall verify each non-levelzero segments before submit compaction task
to executor
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-12-04 19:06:40 +08:00
e79c3eaa90
enhance: [skip e2e] Update install_deps.sh cmake version ( #28946 )
...
cmake version needs to be greater or equal than 3.26.4, sync
`install_deps.sh` scripts version to 3.26.5
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-12-04 18:26:34 +08:00
b4af6f8c40
fix: sync action load segment with lack collection index info list ( #28788 )
...
relate: https://github.com/milvus-io/milvus/issues/28779
https://github.com/milvus-io/milvus/issues/28637
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2023-12-04 18:14:34 +08:00
45e6801ce4
feat: Add checker activation service interfaces ( #28850 )
...
issue: #28610
---------
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2023-12-04 17:38:37 +08:00
9b6cbe956a
[automated] Update Knowhere Commit ( #28917 )
...
Update Knowhere Commit
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2023-12-04 15:42:34 +08:00
9723e16058
enhance: Update pymilvus ( #28922 )
...
issue: https://github.com/milvus-io/milvus/issues/17614
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-12-04 15:36:35 +08:00
c8b1a4618a
enhance: Resolve libunwind requirement conflict using 1.7.2 ( #28929 )
...
Try to resolve libunwind dependency requirement conflict between glog &
folly
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-12-04 14:06:40 +08:00
cbd3b3c836
test: modify test cases for error msg update ( #28846 )
...
#23075
Signed-off-by: nico <cheng.yuan@zilliz.com>
2023-12-04 11:22:37 +08:00
8740adbe28
test: update pymilvus version ( #28832 )
...
#17614
Signed-off-by: nico <cheng.yuan@zilliz.com>
2023-12-04 11:20:47 +08:00
d081fd5481
enhance: Change some frequency log to rated level ( #28897 )
...
This pr change some frequency log's level to rated.
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-12-04 10:38:35 +08:00
fccec125b2
enhance: [skip-e2e] add rust to builder image ( #28838 )
...
https://github.com/milvus-io/milvus/issues/27704
---------
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2023-12-04 10:10:35 +08:00
6718986a5b
enhance: Remove too frequency read ddl log from rootcoord ( #28891 )
...
This PR removed too frequency log for such
`DescribeCollection/ShowPartition` operation from root coord
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-12-04 09:42:33 +08:00
20fc90c591
enhance: find collection schema from cache ( #28782 )
...
issue: #28781 #28329
1. There is no need to call `DescribeCollection`, if the collection's
schema is found in the globalMetaCache
2. did `GetProperties` to check the access to Azure Blob Service while
construct the ChunkManager
Signed-off-by: PowderLi <min.li@zilliz.com>
2023-12-03 19:22:33 +08:00
342635ed61
enhance: enable assert method to format arguments ( #28812 )
...
for now the assert method in segcore could accept a string information,
too many codes don't print the value they assert.
make it happy
related #28811
---------
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-12-01 18:04:33 +08:00
24c565e37b
fix: accesslog method status not return failed when error in response ( #28824 )
...
relate: https://github.com/milvus-io/milvus/issues/28086
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2023-12-01 15:30:38 +08:00
1174a7527a
enhance: [skip e2e]delete unsed arch for gpu builder ( #28887 )
...
1. disable arm64 arch
2. only use docker cli instead of docker buildx cli
---------
Signed-off-by: Sammy Huang <sammy.huang@zilliz.com>
2023-12-01 14:48:32 +08:00
f5856812a2
fix: Fix get binary vector from chunk cache ( #28866 )
...
The way of getting binary vector size is wrong. This PR will fix it.
issue: https://github.com/milvus-io/milvus/issues/28865
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-12-01 14:40:32 +08:00
7206795e91
fix: update folly to resolve simd issue ( #28878 )
...
related #27552 , after this, milvus could run successfully on sse4.2
only machine
Signed-off-by: chasingegg <chao.gao@zilliz.com>
2023-12-01 13:50:32 +08:00
wei liu
XuanYang-cn
Jiquan Long
cai.zhang
MrPresent-Han
yah01
yihao.dai
congqixia
wayblink
shaoyue
jaime
aoiasd
Bingyi Sun
groot
xige-16
sre-ci-robot
sammy.huang
sre-ci-robot
nico
PowderLi
Gao