5bac7f7897
fix: Fix L0 compaction in datacoord ( #28814 )
...
See also: #27606
---------
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2023-12-05 18:44:37 +08:00
cb31016640
enhance: Write buffer time range when syncing logs ( #28970 )
...
Related to #27675
The timestamp from, to field is not field for new implementation of
writebuffer & sync manager
This pr fills these field for better log information
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-12-05 17:36:36 +08:00
fab52d167b
fix: may miss stream delta while loading ( #28871 )
...
we consume the delta data from the lastest channel checkpoint while
loading segment,
this works well without level 0 segments, but now it may lead to miss
some delta data,
so we have to consume from the current target's channel checkpoint
related: #27349
---------
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-12-05 17:34:45 +08:00
bb82074937
fix: Fix missing target db name for RenameCollection ( #28909 )
...
issue: #28908
Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-12-05 17:12:37 +08:00
1037b16471
[automated] Update Builder image changes ( #28982 )
...
/kind improvement
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2023-12-05 16:52:36 +08:00
3cc4209d26
enhance: pack proxy connection code and support accesslog print SDK_Version ( #28835 )
...
relate: https://github.com/milvus-io/milvus/issues/28086
https://github.com/milvus-io/milvus/issues/28940
---------
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2023-12-05 16:50:47 +08:00
36f69ea031
feat: integrate storagev2 in building index of segcore ( #28768 )
...
issue: https://github.com/milvus-io/milvus/issues/28655
---------
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2023-12-05 16:48:54 +08:00
ad1daebc8e
fix: Change ranges of HNSW parameters according to Knowhere ( #28875 )
...
fix: https://github.com/milvus-io/milvus/issues/28860
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2023-12-05 16:20:42 +08:00
89b965e2b1
fix: clean snapshots meta ( #28502 )
...
issue: https://github.com/milvus-io/milvus/issues/28496
/kind bug
The input parameters collection.partitions and collection.Field are both
nil, so these two metas have not been cleared.
Signed-off-by: xige-16 <xi.ge@zilliz.com>
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2023-12-05 16:04:36 +08:00
f01e507b15
[automated] Update Knowhere Commit ( #28965 )
...
Update Knowhere Commit
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2023-12-05 15:56:35 +08:00
11687dce60
enhance: [skip e2e]upgrade minio version ( #28971 )
...
Signed-off-by: Sammy Huang <sammy.huang@zilliz.com>
2023-12-05 14:14:34 +08:00
7f819ac594
enhance: Remove partition load infos with prefix ( #28969 )
...
Removing partition load infos with prefix instead of removing them one
by one. This enhance can save transactions.
issue: https://github.com/milvus-io/milvus/issues/28343
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-12-05 12:52:34 +08:00
e2d0333831
[automated] Update Builder gpu image changes ( #28963 )
...
/kind improvement
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2023-12-05 11:58:33 +08:00
10bb2431d8
test: add checker unittests ( #28954 )
...
issue: https://github.com/milvus-io/milvus/issues/28610
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2023-12-05 10:56:33 +08:00
91e92ee8d4
[automated] Update Pytest image changes ( #28938 )
...
Update Pytest image changes
See changes:
8740adbe28sre-ci-robot@users.noreply.github.com
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2023-12-05 10:06:35 +08:00
ab630548f4
Update all contributors
...
Signed-off-by: sre-ci-robot <sre-ci-robot@zilliz.com>
2023-12-04 12:48:33 +00:00
d26b563a8b
feat: Define import API and metadata ( #28731 )
...
Define the new rpc and metadata for ImportV2.
see also: https://github.com/milvus-io/milvus/issues/28521
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-12-04 19:56:35 +08:00
393b1f943c
fix: Reject compaction task with growing segments ( #28925 )
...
See also #28924
The compaction task generated before datanode finish `SaveBinlogPath`
grpc call contains segments which are still in Growing state DataNode
shall verify each non-levelzero segments before submit compaction task
to executor
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-12-04 19:06:40 +08:00
e79c3eaa90
enhance: [skip e2e] Update install_deps.sh cmake version ( #28946 )
...
cmake version needs to be greater or equal than 3.26.4, sync
`install_deps.sh` scripts version to 3.26.5
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-12-04 18:26:34 +08:00
b4af6f8c40
fix: sync action load segment with lack collection index info list ( #28788 )
...
relate: https://github.com/milvus-io/milvus/issues/28779
https://github.com/milvus-io/milvus/issues/28637
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2023-12-04 18:14:34 +08:00
45e6801ce4
feat: Add checker activation service interfaces ( #28850 )
...
issue: #28610
---------
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2023-12-04 17:38:37 +08:00
9b6cbe956a
[automated] Update Knowhere Commit ( #28917 )
...
Update Knowhere Commit
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2023-12-04 15:42:34 +08:00
9723e16058
enhance: Update pymilvus ( #28922 )
...
issue: https://github.com/milvus-io/milvus/issues/17614
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-12-04 15:36:35 +08:00
c8b1a4618a
enhance: Resolve libunwind requirement conflict using 1.7.2 ( #28929 )
...
Try to resolve libunwind dependency requirement conflict between glog &
folly
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-12-04 14:06:40 +08:00
cbd3b3c836
test: modify test cases for error msg update ( #28846 )
...
#23075
Signed-off-by: nico <cheng.yuan@zilliz.com>
2023-12-04 11:22:37 +08:00
8740adbe28
test: update pymilvus version ( #28832 )
...
#17614
Signed-off-by: nico <cheng.yuan@zilliz.com>
2023-12-04 11:20:47 +08:00
d081fd5481
enhance: Change some frequency log to rated level ( #28897 )
...
This pr change some frequency log's level to rated.
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-12-04 10:38:35 +08:00
fccec125b2
enhance: [skip-e2e] add rust to builder image ( #28838 )
...
https://github.com/milvus-io/milvus/issues/27704
---------
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2023-12-04 10:10:35 +08:00
6718986a5b
enhance: Remove too frequency read ddl log from rootcoord ( #28891 )
...
This PR removed too frequency log for such
`DescribeCollection/ShowPartition` operation from root coord
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-12-04 09:42:33 +08:00
20fc90c591
enhance: find collection schema from cache ( #28782 )
...
issue: #28781 #28329
1. There is no need to call `DescribeCollection`, if the collection's
schema is found in the globalMetaCache
2. did `GetProperties` to check the access to Azure Blob Service while
construct the ChunkManager
Signed-off-by: PowderLi <min.li@zilliz.com>
2023-12-03 19:22:33 +08:00
342635ed61
enhance: enable assert method to format arguments ( #28812 )
...
for now the assert method in segcore could accept a string information,
too many codes don't print the value they assert.
make it happy
related #28811
---------
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-12-01 18:04:33 +08:00
24c565e37b
fix: accesslog method status not return failed when error in response ( #28824 )
...
relate: https://github.com/milvus-io/milvus/issues/28086
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2023-12-01 15:30:38 +08:00
1174a7527a
enhance: [skip e2e]delete unsed arch for gpu builder ( #28887 )
...
1. disable arm64 arch
2. only use docker cli instead of docker buildx cli
---------
Signed-off-by: Sammy Huang <sammy.huang@zilliz.com>
2023-12-01 14:48:32 +08:00
f5856812a2
fix: Fix get binary vector from chunk cache ( #28866 )
...
The way of getting binary vector size is wrong. This PR will fix it.
issue: https://github.com/milvus-io/milvus/issues/28865
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-12-01 14:40:32 +08:00
7206795e91
fix: update folly to resolve simd issue ( #28878 )
...
related #27552 , after this, milvus could run successfully on sse4.2
only machine
Signed-off-by: chasingegg <chao.gao@zilliz.com>
2023-12-01 13:50:32 +08:00
038eebba4d
enhance: Move datacoord broker into separate package ( #28876 )
...
See also #28861
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-12-01 10:22:34 +08:00
8036ee13fa
feat: avoid dereferencing nullptr ( #28862 )
...
issue: #28793
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2023-12-01 10:20:32 +08:00
043ac87be0
fix: Balance channel may cause channel not availble error ( #28829 )
...
issue: #28831
release old delegator before new delegator update it's distribution may
cause `channel not availble` error
This PR will block release old delgator before new delegator finish
`syncDistribution`
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-12-01 10:08:34 +08:00
af98d1cb64
[automated] Update Builder image changes ( #28874 )
...
Update Builder image changes
See changes:
cac802ef7fsre-ci-robot@users.noreply.github.com
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2023-12-01 10:04:32 +08:00
ecc3ca374c
[automated] Update Knowhere Commit ( #28882 )
...
Update Knowhere Commit
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2023-12-01 02:28:31 +08:00
e62edb991a
enhance: Add FlowgraphManager interface ( #28852 )
...
- Change flowgraphManager to fgManagerImpl
- Change close to stop
- change execute to controlMemWaterLevel
- Change method name of fgManager for readability
- Add mockery for fgmanager
Issue: #28853
---------
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2023-11-30 18:42:32 +08:00
bf633bb5d7
enhance: refine the retry error ( #28573 )
...
return the last error but not combining all errors, to improve
readability and erorr handling
resolve : #28572
---------
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-11-30 18:34:32 +08:00
b4353ca4ce
enhance: Remove vector chunk manager ( #28569 )
...
We have implemented the chunkcache (in cpp) to retrieve vectors, hence
rendering the vectorchunkcache (in golang) obsolete.
issue: https://github.com/milvus-io/milvus/issues/28568
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-11-30 18:00:33 +08:00
adce6aab4f
fix:handle err when ManualCompaction ( #28804 )
...
handle err when ManualCompaction
#28644
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2023-11-30 17:06:31 +08:00
5d0a9f9344
fix: Forget to set EntriesNum for deltalogs ( #28858 )
...
See also: #28520
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2023-11-30 16:52:31 +08:00
cac802ef7f
enhance: use already installed vcpkg ( #28703 )
...
issue #28686
1. Update Builder gpu image changes, see changes #28505
2. update azure-identity-cpp from beta to release
Signed-off-by: PowderLi <min.li@zilliz.com>
2023-11-30 15:58:32 +08:00
aae7e62729
feat: Add levelzero compaction in DN ( #28470 )
...
See also: #27606
---------
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2023-11-30 14:30:28 +08:00
d69440524b
fix: bypass growing index if no index meta ( #28791 )
...
we shouldn't panic if no index meta, just skip building it
fix #28022
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-11-30 14:10:27 +08:00
43680043e3
auto close Update Knowhere Commit PR as long as ci-passed
...
Assign the 'lgtm' and 'approved' labels following the successful testing of the 'Update Knowhere Commit'
then prow/tide bot will auto merge the PR to master branch.
Signed-off-by: Sammy Huang <sammy.huang@zilliz.com>
2023-11-30 12:03:55 +08:00
8b2b0d412c
enhance: storeageType default value change to remote ( #28792 )
...
/kind enhancement
/cc @PowderLi
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
2023-11-30 11:34:27 +08:00
XuanYang-cn
congqixia
yah01
jaime
Jiquan Long
aoiasd
Bingyi Sun
groot
xige-16
sre-ci-robot
sammy.huang
yihao.dai
sre-ci-robot
nico
wei liu
PowderLi
Gao
smellthemoon
shaoyue