7de9311fe4
fix:[2.4] Indexing.Iterator ut: build index with all data at once ( #32928 )
...
pr: #32844
issue: #32843
Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com>
2024-05-11 10:59:32 +08:00
7b7b56b4e1
enhance: [2.4] Utilize coll2replica mapping when getting rg by collection ( #32892 ) ( #32922 )
...
Cherry-pick from master
pr: #32892
See also #32165
In old `GetResourceGroupByCollection` implementation, it iterates all
replicas to match collection id, which is slow and CPU time consuming.
This PR make it utilize the coll2Replicas mapping by calling
`GetByCollection` and mapping replicas into resource group.
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-05-10 14:35:30 +08:00
3acac728dd
fix: [2.4]Make SyncManager callback func ignore nil error ( #32891 ) ( #32904 )
...
Cherry-pick from master
pr: #32891
introduced by #32865
sync manager callback handler panicked when error is nil
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-05-10 10:13:31 +08:00
263d3cc997
enhance: [2.4] Avoid iteration over channel results when update leaderview( #32887 ) ( #32893 )
...
Cherry-pick from master
pr: #32887
See also #32165
Cache channel name to channel info to avoid iteration over channel
results when updating leader view version.
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-05-09 17:45:30 +08:00
843d2c0da1
fix: [skip-e2e][2.4] Fix unstable policy_test ( #32884 )
...
See also: #32849
pr: #32850
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-05-09 15:39:40 +08:00
1f11bc484d
enhance: [2.4] Improve datacoord segment filtering with collection ( #32831 ) ( #32874 )
...
Cherry-pick from master
pr: #32831
See also #32165
This PR modify the `SelectSegments` interface to utilizing collection id
information when selecting segment with provided collection
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-05-09 15:37:30 +08:00
7a00bcee65
fix: make sure inverted index has only one segment ( #32858 ) ( #32877 )
...
issue: #32717
pr: #32858
---------
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2024-05-09 14:39:30 +08:00
4b020dd4c2
fix: [2.4] Make syncmgr lock key before returning future ( #32865 ) ( #32882 )
...
Cherry-pick from master
pr: #32865
See also #32860
SyncMgr did not ensure task key is locked before `SyncData` returning
which may cause concurrent problem during sync wich multiple policies.
This PR change sync mgr implementation to make sure the key is locked
before returning task result `*conc.Future`
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-05-09 14:13:30 +08:00
c0d9efa67b
enhance: [2.4]trigger compaction by size and fix ( #32326 ) ( #32846 )
...
fix: Compaction trigger choose 2 same segments (#32800 )
DataNode would stuck at compactor try to lock the
same segmentID
See: #32328 , #32765
pr: #32800 #32326
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
Co-authored-by: Ted Xu <ted.xu@zilliz.com>
2024-05-09 10:27:31 +08:00
68771fa495
feat: [2.4]datacoord/node watch based on rpc ( #32558 )
...
issue: https://github.com/milvus-io/milvus/issues/25309
Signed-off-by: yiwangdr <yiwangdr@gmail.com>
2024-05-09 04:17:31 +08:00
19f09206de
enhance: [2.4] Remove legacy L0 segment if watch failed ( #32725 ) ( #32852 )
...
Cherry-pick from master
pr: #32725
Like growing segments, legacy l0 segments shall be removed if watch dml
channel execution fails as well.
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-05-08 19:13:30 +08:00
738026007f
fix: [2.4]Make compactor able to clear empty segments ( #32823 )
...
See also: #32553
pr: #32821
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-05-08 19:09:35 +08:00
5023f1a446
enhance: [2.4] add the cost response for the rest api ( #32841 )
...
issue: #30436
pr: #32620
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-05-08 14:19:29 +08:00
3818c166c5
enhance: Change ddl log to info ( #32838 ) ( #32839 )
...
/kind improvement
pr: https://github.com/milvus-io/milvus/pull/32838
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-05-08 14:12:10 +08:00
94bd2f0d19
fix: Fix cannot specify partition name in binlog import ( #32730 ) ( #32829 )
...
issue: https://github.com/milvus-io/milvus/issues/32807
pr: https://github.com/milvus-io/milvus/pull/32730
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-05-08 11:59:30 +08:00
879733b47a
enhance: Remove deprecated EnableIndex ( #32704 ) ( #32828 )
...
/kind improvement
pr: https://github.com/milvus-io/milvus/pull/32704
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-05-08 11:57:29 +08:00
311d66ca02
fix: [2.4] Unify querynode metrics cleanup in collection release ( #32805 ) ( #32825 )
...
Cherry-pick from master
pr: #32805
Related to #32803
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-05-08 10:11:30 +08:00
49b760f8ea
fix: [2.4]Validate num of rows for insert field data with schema ( #32770 ) ( #32799 )
...
Cherry-pick from master
pr: #32770
See also #32769
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-05-07 19:13:35 +08:00
54e00db012
fix: Make the dynamic column optional in parquet import ( #32738 ) ( #32802 )
...
issue: https://github.com/milvus-io/milvus/issues/32729
pr: https://github.com/milvus-io/milvus/pull/32738
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-05-07 17:07:31 +08:00
46acc07cef
fix: [2.4] cherry-pick three litter pr ( #32798 )
...
/kind improvement
issue: #32591
pr: #32775 #32670 #32680
fix: skip to check auto id when inserting data (#32775 )
enhance: check the partition num when creating collection with partition
key (#32670 )
enhance: add the config to control the way when fail to init plugin
(#32680 )
---------
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-05-07 14:37:29 +08:00
2a4a5693c1
enhance: check field data type of input ( #32777 ) ( #32789 )
...
issue: #32769
pr: #32777
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2024-05-07 14:07:29 +08:00
71b4ca367d
fix: [Pick]Fix scalar auto index config incorrect version ( #32796 )
...
issue: #29309
master pr: #32795
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-05-07 10:13:10 +08:00
def4e14175
fix: remove flaky sparse integration test ( #32794 )
...
issue: #32766
pr: #32767
Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com>
2024-05-07 09:57:56 +08:00
0fc6b2246a
fix: [2.4] Refine index param check for Create/Alter Index ( #32712 ) ( #32784 )
...
Cherry-pick from master
pr: #32712
See also #32711
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-05-06 20:49:29 +08:00
e9f41c5d91
[automated] Bump milvus version to v2.4.1 ( #32774 )
...
Bump milvus version to v2.4.1
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-05-06 14:23:28 +08:00
fe7a1777d8
enhance: resource group unittest refactory ( #32740 )
...
issue: #30647
pr: #32739
Signed-off-by: chyezh <chyezh@outlook.com>
2024-05-06 10:17:34 +08:00
15b78749a6
Revert "enhance: trigger compaction by size ( #32326 )" ( #32745 )
...
This reverts commit fef7812254
2024-04-30 20:41:27 +08:00
6003963958
enhance: update nightly script ( #32731 )
...
Signed-off-by: nico <cheng.yuan@zilliz.com>
2024-04-30 17:13:31 +08:00
77f4f0c5a8
fix: gc only do once ( #32722 )
...
issue: #29655
produced by pr: #31740
Signed-off-by: chyezh <chyezh@outlook.com>
2024-04-30 14:19:27 +08:00
b6c06ff2c3
fix: Removed tests on non existent directories and solved name resolution errors ( #32266 )
...
issue: #32254
Signed-off-by: Ald392 <imagesai32@gmail.com>
2024-04-30 11:33:26 +08:00
42d0412e93
enhance: Add channelCPs in FlushResponse ( #32044 )
...
#32609
Signed-off-by: wayblink <anyang.wang@zilliz.com>
2024-04-30 09:45:27 +08:00
ac82cef04d
enhance: disable reload partstats by config ( #32702 )
...
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
2024-04-29 19:11:26 +08:00
0359139299
enhance: return the auth collection list when using the `show collection` api ( #32576 )
...
issue: #32550
/kind improvement
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-04-29 19:09:34 +08:00
ecd8e52b53
fix: Use default integration case timeout for `TestBinlogImport` ( #32701 )
...
See also #32700
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-04-29 19:07:27 +08:00
a43baa0323
enhance: Upgrade knowhere to 2.3.1 ( #32699 )
...
/kind improvement
Signed-off-by: Li Liu <li.liu@zilliz.com>
2024-04-29 17:21:26 +08:00
6289f3a9eb
fix: build milvus in rockylinux8 ( #32619 )
...
issue: #32299
1. xz utils recovers
2. forget to install ninja
Signed-off-by: PowderLi <min.li@zilliz.com>
2024-04-29 14:53:26 +08:00
f07e78ec91
enhance: Bump Milvus & proto version to v2.4.1 ( #32693 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-04-29 14:37:26 +08:00
858599d831
enhance: sparse float vector to support brute force iterator and range search ( #32635 )
...
issue: #29419
Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com>
2024-04-29 14:35:26 +08:00
083bd38c77
test: add test cases for code change ( #32289 )
...
issue: #31368
Signed-off-by: binbin lv <binbin.lv@zilliz.com>
2024-04-29 14:29:26 +08:00
4e01591158
enhance: use soft node affinity instead of hard nodeSelector ( #32677 )
...
issue: #32627
Signed-off-by: Liang Huang <sammy.huang@zilliz.com>
2024-04-29 14:25:36 +08:00
1a6293df09
test: update some test cases ( #32687 )
...
Signed-off-by: nico <cheng.yuan@zilliz.com>
2024-04-29 14:23:26 +08:00
d900e68440
fix: fix GetShardLeaders return empty node list ( #32685 )
...
issue: #32449
to avoid GetShardLeaders return empty node list, this PR add node list
check in both client side and server side.
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-04-29 14:19:26 +08:00
ef4c875d4c
fix: resource group ut may failure ( #32688 )
...
issue: https://github.com/milvus-io/milvus/issues/30647
Signed-off-by: chyezh <chyezh@outlook.com>
2024-04-29 14:17:26 +08:00
29d3c576a3
enhance: [skip e2e] Enlarge integration test timeout to 90min ( #32686 )
...
The running time of integration is above 50 minutes and may surpass 60
and timeout in some cases. This PR enlarge timeout first.
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-04-29 11:33:25 +08:00
2c1e8f4774
enhance: Use `struct{}` for sync task future result ( #32673 )
...
Related to #27675
Use `struct{}` instead `error` for sync task future result type to
reduce result size and preventing logci error.
Also change some unused parameter to `_` to suppress lint warning
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-04-29 10:59:26 +08:00
09cd56d44f
enhance: add the skip auto id and partition key check config ( #32592 )
...
/kind improvement
issue: #32591
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-04-29 10:29:26 +08:00
84a14fa8dc
Update all contributors
...
Signed-off-by: sre-ci-robot <sre-ci-robot@zilliz.com>
2024-04-29 00:01:15 +00:00
c0555d4b45
fix: Remove read only node from replica immedaitely after node down ( #32666 )
...
issue: #32665
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-04-28 20:25:25 +08:00
cb1dbf20c7
enhance:remove openblas from conan ( #32002 )
...
issue: #30683
Signed-off-by: yusheng.ma <yusheng.ma@zilliz.com>
2024-04-28 20:13:25 +08:00
1594122c0a
enhance: Make the dynamic field file optional during numpy import ( #32596 )
...
1. Make the dynamic field file optional during numpy import
2. Add integration importing test with dynamic
3. Disallow file of pk when autoID=true during numpy import
issue: https://github.com/milvus-io/milvus/issues/32542
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-04-28 19:39:25 +08:00
Buqian Zheng
congqixia
XuanYang-cn
Jiquan Long
yiwangdr
SimFG
yihao.dai
cai.zhang
sre-ci-robot
chyezh
Ted Xu
nico
Aldrin
wayblink
Chun Han
liliu-z
PowderLi
binbin
sammy.huang
wei liu
sre-ci-robot
presburger