Commit Graph

2825 Commits (4031abd2fa4d1b38ecc208f1e107c68a3d6fb619)

Author SHA1 Message Date
SimFG 4031abd2fa
enhance: change default partition num to 16 when using partition key (#32950)
/kind improvement

Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-05-13 14:19:31 +08:00
wei liu e2332bdc17
enhance: Enable channel exclusive balance policy (#32911)
issue: #32910  
* split replica's node list to channels when create replicas
 * balance nodes among channels when node change happens
 * implement channel level balance, let balance happens in channel level

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-05-10 17:27:31 +08:00
Cai Yudong dc89c6f810
enhance: remove duplicated data generation APIs for bulk insert test (#32889)
Issue: #22837

including following changes:
1. Add API CreateInsertData() and BuildArrayData() in
internal/util/testutil
2. Remove duplicated test APIs from importutilv2 unittest and bulk
insert integration test

Signed-off-by: Cai Yudong <yudong.cai@zilliz.com>
2024-05-10 15:27:31 +08:00
yanliang567 82a1e1ebe1
test: Update insert data tests and refactor some data gen functions (#32924)
related issue: #32653

Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
2024-05-10 14:57:32 +08:00
zhuwenxing e776c807c4
test: add more json datatype in restful test cases (#32930)
add more json datatype in restful test cases

related to https://github.com/milvus-io/milvus/issues/31051

Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2024-05-10 14:11:30 +08:00
zhuwenxing 9a269f1489
test: add import checker to chaos test (#32908)
add import checker to chaos test

Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2024-05-10 11:43:30 +08:00
Cai Yudong bcdbd1966e
feat: Support sparse float vector bulk insert for binlog/json/parquet (#32649)
Issue: #22837

Signed-off-by: Cai Yudong <yudong.cai@zilliz.com>
2024-05-07 18:43:30 +08:00
yihao.dai 53874ce245
fix: Fix cannot specify partition name in binlog import (#32730)
issue: https://github.com/milvus-io/milvus/issues/32807

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-05-07 17:19:30 +08:00
yiwangdr b1eacb2ae8
feat: datacoord/node watch based on rpc (#32036)
issue: https://github.com/milvus-io/milvus/issues/25309

Signed-off-by: yiwangdr <yiwangdr@gmail.com>
2024-05-07 15:49:30 +08:00
zhuwenxing 29f3cb692b
test:fix retry for network error (#32804)
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2024-05-07 14:01:29 +08:00
zhuwenxing e1eb03ad0d
test: add testcase for import without meta when enabled dynamic (#32706)
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2024-05-07 13:59:36 +08:00
yihao.dai 4de063ae14
fix: Make the dynamic column optional in parquet import (#32738)
issue: https://github.com/milvus-io/milvus/issues/32729

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-05-07 11:21:29 +08:00
SimFG 7da1ca9efb
enhance: check the partition num when creating collection with partition key (#32670)
issue: #30577

Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-05-07 10:43:29 +08:00
sre-ci-robot c5191e9b28
[automated] Update Pytest image changes (#32697)
Update Pytest image changes
See changes:
1a6293df09
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-05-06 21:03:29 +08:00
Buqian Zheng 37a99ca23e
fix: remove flaky sparse integration test (#32767)
issue: https://github.com/milvus-io/milvus/issues/32766

this test is outdated, thus removing it instead of fixing it.

Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com>
2024-05-06 19:19:29 +08:00
nico b0ebc50097
test: update test cases (#32780)
Signed-off-by: nico <cheng.yuan@zilliz.com>
2024-05-06 17:41:29 +08:00
yanliang567 b9713f96fd
test: Fix a deep copy issue when getting search params (#32781)
related issue: #32653

---------

Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
2024-05-06 17:11:29 +08:00
yanliang567 5bb672d70d
test: Add a new range search test for all indexes and align some index params (#32724)
related issue: https://github.com/milvus-io/milvus/issues/32653

1. align some default index params
2. add a new range search tests for all indexes and float vectors

---------

Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
2024-04-30 17:37:27 +08:00
nico c70c21e70b
test: modify test cases (#32727)
Signed-off-by: nico <cheng.yuan@zilliz.com>
2024-04-30 17:13:31 +08:00
zhuwenxing e346714bf2
test: add reruns params in pytest command (#32720)
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2024-04-30 17:11:26 +08:00
congqixia ecd8e52b53
fix: Use default integration case timeout for `TestBinlogImport` (#32701)
See also #32700

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-04-29 19:07:27 +08:00
binbin 083bd38c77
test: add test cases for code change (#32289)
issue: #31368

Signed-off-by: binbin lv <binbin.lv@zilliz.com>
2024-04-29 14:29:26 +08:00
sammy.huang 4e01591158
enhance: use soft node affinity instead of hard nodeSelector (#32677)
issue: #32627

Signed-off-by: Liang Huang <sammy.huang@zilliz.com>
2024-04-29 14:25:36 +08:00
nico 1a6293df09
test: update some test cases (#32687)
Signed-off-by: nico <cheng.yuan@zilliz.com>
2024-04-29 14:23:26 +08:00
yihao.dai 1594122c0a
enhance: Make the dynamic field file optional during numpy import (#32596)
1. Make the dynamic field file optional during numpy import
2. Add integration importing test with dynamic
3. Disallow file of pk when autoID=true during numpy import

issue: https://github.com/milvus-io/milvus/issues/32542

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-04-28 19:39:25 +08:00
congqixia 8c4fc1e61c
enhance: Close singleton etcd client in integration teardown (#32664)
Found lots of `failed to updateTimeTick` with error `skip
ChannelTimeTickMsg from un-recognized session 1` The reason was etcd
client became singleton and used last root path in multiple cases are
run in one suite.

This PR add close singleton client invocation to fix this problem.

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-04-28 18:17:26 +08:00
yanliang567 c7807afe71
test: Remove some lower priority e2e test params to reduce tests (#32654)
related issue: #32653
1. reduce test params in search.py
2. skip all the xfail test to save running time

Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
2024-04-28 16:13:25 +08:00
zhuwenxing 2ab57ca10c
test: remove re-building index (#32534)
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2024-04-28 14:19:25 +08:00
nico 96efa18cf4
test: update test cases (#32568)
Signed-off-by: nico <cheng.yuan@zilliz.com>
2024-04-28 11:29:28 +08:00
elstic 11bde38063
test: [cherry-pick]add test cases for sparse vector (#32155)
issue : https://github.com/milvus-io/milvus/issues/31483

Signed-off-by: elstic <hao.wang@zilliz.com>
2024-04-28 11:27:28 +08:00
sammy.huang 9083f6999d
enhance: bind dependency of miluvs during e2e to dedicated nodes (#32600)
#32627

Signed-off-by: Liang Huang <sammy.huang@zilliz.com>
2024-04-26 11:43:25 +08:00
congqixia 0ff7a46e95
fix: [skip e2e] Disable compaction for balance integration test (#32603)
See also #31468

Balance test suite may assert segment number based on test setup.
However the compaction may reduce the number and cause test cases
unstable.

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-04-25 16:55:23 +08:00
Buqian Zheng 8a1017a152
enhance: add helpers to parse sparse float vector in JSON (#32543)
issue: #29419

added helper functions to parse JSON representation of sparse float
vectors, will be used by both the restful server and the import utils.

Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com>
2024-04-25 14:47:24 +08:00
binbin 5119292411
test: update pymilvus version and modify test cases (#32572)
issue: #32376

Signed-off-by: binbin lv <binbin.lv@zilliz.com>
2024-04-25 08:57:31 +08:00
sammy.huang 4c7d3dade3
enhance: bind e2e to dedicated nodes (#32577)
Signed-off-by: Liang Huang <sammy.huang@zilliz.com>
2024-04-24 19:27:24 +08:00
PowderLi 29922829da
fix: [restful v2] parameters of create collection (#32485)
issue: #31176

Signed-off-by: PowderLi <min.li@zilliz.com>
2024-04-24 09:43:29 +08:00
Cai Yudong 16b8b7b35d
enhance: Add get_vector unittest for float16 & bfloat16 (#32153)
Issue: #22837

Signed-off-by: Cai Yudong <yudong.cai@zilliz.com>
2024-04-23 16:15:23 +08:00
zhuwenxing a5f0fc4373
test: add bf/f16 bulk insert testcase (#32506)
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2024-04-23 14:05:23 +08:00
zhuwenxing 8442098457
test: skip index if it already created (#32402)
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2024-04-22 10:37:22 +08:00
Cai Yudong 5fc439c600
feat: Bulk insert support fp16/bf16 (#32157)
Issue: #22837

Signed-off-by: Cai Yudong <yudong.cai@zilliz.com>
2024-04-22 10:05:22 +08:00
zhuwenxing 109cd0402f
test: fix delete expr (#32377)
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2024-04-18 17:25:20 +08:00
zhuwenxing 2cff65511c
test: use float32 to do import (#32276)
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2024-04-18 17:23:27 +08:00
nico e83651def4
test: update test cases (#32396)
Signed-off-by: nico <cheng.yuan@zilliz.com>
2024-04-18 16:15:25 +08:00
zhuwenxing 3e2314ebee
test: add negative import testcase (#32163)
add negative import testcase

---------

Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2024-04-17 10:59:31 +08:00
smellthemoon d1ef0a81ee
enhance: [skip e2e] change some long logs (#32309)
<img width="1042" alt="image"
src="https://github.com/milvus-io/milvus/assets/64083300/8daa9ab9-1988-4398-a92a-7d2dac2cd8cd">
change this log

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-04-17 10:25:19 +08:00
zhuwenxing 84c69f59a3
test: add retry for restful testcases (#32318)
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2024-04-16 23:03:21 +08:00
zhuwenxing ae5e3a4fe3
test:[skip e2e]update script to get image tag (#32334)
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2024-04-16 20:29:19 +08:00
nico f0caf3a2da
test: update some cases (#32323)
Signed-off-by: nico <cheng.yuan@zilliz.com>
2024-04-16 20:27:32 +08:00
zhuwenxing 520a302f3a
test:[skip e2e]fix get image tag (#32306)
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2024-04-16 15:53:19 +08:00
yihao.dai 558feed5ed
fix: Use pk from binlog during import (#32118)
During binlog import, even if the primary key's autoID is set to true,
the primary key from the binlog should be used instead of being
reassigned.

issue: https://github.com/milvus-io/milvus/discussions/31943,
https://github.com/milvus-io/milvus/issues/28521

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-04-16 14:51:20 +08:00