Commit Graph

20963 Commits (304098cd401208a9d426f94f6ab43726497ab361)

Author SHA1 Message Date
cai.zhang fc8b5ab791
enhance: Convert unincode to ascii to improving expression parsing efficiency (#36675)
issue: #36672

---------

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-10-09 09:23:24 +08:00
Zhen Ye 2ec6e602d6
enhance: add streaming client metrics (#36523)
issue: #33285

---------

Signed-off-by: chyezh <chyezh@outlook.com>
2024-10-08 21:25:19 +08:00
sre-ci-robot 8b867aae63 Update all contributors
Signed-off-by: sre-ci-robot <sre-ci-robot@zilliz.com>
2024-10-08 12:01:01 +00:00
jaime ef1832ff9c
enhance: enable manual compaction for collections without indexes (#36577)
issue: #36576

Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-10-08 19:57:18 +08:00
nico 9fdf0505a8
test: update test case for support dynamic load replica (#36589)
Signed-off-by: nico <cheng.yuan@zilliz.com>
2024-10-08 17:31:20 +08:00
binbin 0d57ff01a6
test: add null and default test cases (#36612)
issue: #36129

Signed-off-by: binbin lv <binbin.lv@zilliz.com>
2024-10-08 17:25:24 +08:00
jaime 1fded42277
enhance: decrease CPU overhead when calculating index file size (#36579)
issue: #36578

---------

Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-10-08 14:29:25 +08:00
XuanYang-cn 4e0ea39235
fix: Remove neighbors if compactTo is unindexed (#36503)
See also: #36360

---------

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-10-08 14:15:19 +08:00
wayblink c7a81d5de3
enhance:use proto.clone in compaction tasks to prevent omissions (#36624)
issue: #36623

Signed-off-by: wayblink <anyang.wang@zilliz.com>
2024-10-08 11:23:23 +08:00
yihao.dai 1f47d5510b
fix: Fix import segments leak in segment manager (#36602)
Directly add import segments from the meta, eliminating the dependency
on the segment manager.

issue: https://github.com/milvus-io/milvus/issues/34648

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-10-08 10:11:22 +08:00
sre-ci-robot ca9842e4d5 Update all contributors
Signed-off-by: sre-ci-robot <sre-ci-robot@zilliz.com>
2024-10-08 00:01:23 +00:00
sre-ci-robot 3936d12661
[automated] Update Knowhere Commit (#36634)
Update Knowhere Commit
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-10-01 01:05:15 +08:00
sre-ci-robot 3543d0c81d Update all contributors
Signed-off-by: sre-ci-robot <sre-ci-robot@zilliz.com>
2024-09-30 12:00:40 +00:00
congqixia ddc3e76803
fix: Add defer Unpin when error happens (#36620)
Resolves: #36619

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-09-30 19:49:17 +08:00
smellthemoon 2055df81aa
enhance: upgrade pulsar-client-go to 0.12.1 (#36615)
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-09-30 17:13:16 +08:00
wei liu 470bb0cc3f
enhance: Enable balance on querynode with different mem capacity (#36466)
issue: #36464
This PR enable balance on querynode with different mem capacity, for
query node which has more mem capactity will be assigned more records,
and query node with the largest difference between assignedScore and
currentScore will have a higher priority to carry the new segment.

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-09-30 16:15:17 +08:00
wayblink 00a5025949
enhance: support clustering compaction on null value (#36372)
issue: #36055

Signed-off-by: wayblink <anyang.wang@zilliz.com>
2024-09-30 14:33:17 +08:00
congqixia e4292569c7
enhance: Unify InboundLabel case using metrics.Label (#36613)
Previous label case broken by #36107, this PR make all inbound label
using label constants from metrics package.

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-09-30 14:15:17 +08:00
Rijin-N a05a37a583
enhance: GCS native support (GCS implemented using Google Cloud Storage libraries) (#36214)
Native support for Google cloud storage using the Google Cloud Storage
libraries. Authentication is performed using GCS service account
credentials JSON.

Currently, Milvus supports Google Cloud Storage using S3-compatible APIs
via the AWS SDK. This approach has the following limitations:

1. Overhead: Translating requests between S3-compatible APIs and GCS can
introduce additional overhead.
2. Compatibility Limitations: Some features of the original S3 API may
not fully translate or work as expected with GCS.

To address these limitations, This enhancement is needed.

Related Issue: #36212
2024-09-30 13:23:32 +08:00
jaime 5686a9a024
fix: unhandle error in upsert task (#36604)
issue: #36611

Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-09-30 12:59:16 +08:00
Chun Han a25dc98794
enhance: support group_size and hybridsearch+groupby on httpv2 side(#36386) (#36461)
related: #36386

Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
2024-09-30 11:09:16 +08:00
Buqian Zheng 94005b7198
fix: Sparse float vector incorrectly ExpandData at mmap mode (#36603)
issue: #36561

Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com>
2024-09-30 10:39:16 +08:00
wt 798ef223f1
test: add test cases for param checking (#36595)
Signed-off-by: wangting0128 <ting.wang@zilliz.com>
2024-09-30 10:33:16 +08:00
Chun Han a54bffd6cd
fix: refine test case for search_group_by(#36401) (#36511)
related: #36401

Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
2024-09-30 10:13:17 +08:00
cai.zhang ecb2b242e2
enhance: Add sorted for segment info (#36469)
issue: #33744

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-09-30 10:01:16 +08:00
smellthemoon a78a6b33ab
fix: not check json data when set nullable or default_value (#36575)
#36354

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-09-29 20:39:15 +08:00
SimFG 9c1772f659
enhance: avoid to create many timer object in the target (#36570)
/kind improvement

Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-09-29 19:23:16 +08:00
Zhen Ye a47abb2f2b
fix: stats log lost after disable stats log loading on flush (#36592)
issue: #36555

Signed-off-by: chyezh <chyezh@outlook.com>
2024-09-29 16:53:15 +08:00
Zhen Ye a6545b2e29
fix: refactor milvus config and change default txn timeout (#36522)
issue: #36498

Signed-off-by: chyezh <chyezh@outlook.com>
2024-09-29 11:01:15 +08:00
nico c43d94319f
enhance: [skip e2e]open querynode mmap for all pipeline (#36586)
Signed-off-by: nico <cheng.yuan@zilliz.com>
2024-09-29 09:41:15 +08:00
binbin d1d5a50014
test: add null and default test cases (#36539)
issue: #36129

Signed-off-by: binbin lv <binbin.lv@zilliz.com>
2024-09-29 09:39:22 +08:00
yihao.dai 8ed34dce84
enhance: Reopen chunk cache cpp ut (#33622)
issue: https://github.com/milvus-io/milvus/issues/33210

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-09-28 18:19:15 +08:00
zhagnlu 9e3efa06be
fix:fix empty search result bug (#36582)
#36450

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-09-28 17:45:16 +08:00
yihao.dai 80f25d497f
enhance: Add metrics to monitor import throughput and imported rows (#36519)
issue: https://github.com/milvus-io/milvus/issues/36518

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-09-28 17:31:15 +08:00
cai.zhang acc9b5ada5
fix: Regenerate partition stats after stats task for clustering compaction (#36408)
issue: #33744

---------

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-09-28 17:29:21 +08:00
cai.zhang 7bf40694fd
fix: Record the nodeID before assigning tasks (#36371)
issue: #33744

---------

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-09-28 17:21:15 +08:00
cai.zhang 2adca8b754
fix: Fix data race for cluerting compaction (#36440)
issue: #36438

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-09-28 17:19:21 +08:00
zhuwenxing 31353ae406
test: add restful cases for text match feature (#36405)
/kind improvement

Since creating a collection with text match is not yet implemented on
the RESTful interface, we will temporarily use pymilvus to create a
collection for now. This PR includes a case to test using text match
filters in search queries through the RESTful interface.

Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2024-09-28 16:51:15 +08:00
Zhen Ye bd7910632a
fix: etcd session discoverer cannot recover from ErrCompacted (#36568)
issue: #36554

Signed-off-by: chyezh <chyezh@outlook.com>
2024-09-28 10:51:14 +08:00
aoiasd ffc12fb5c4
fix: split delete task msg to MaxMessageSize to avoid mq message too large error (#36197)
relate: https://github.com/milvus-io/milvus/issues/36089

---------

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-09-27 18:15:19 +08:00
smellthemoon b1ac3f0df0
enhance: do not log out the full req (#36546)
![img_v3_02f3_61ac166c-df6f-4b5e-b06a-756e5f40a65g](https://github.com/user-attachments/assets/df668339-d0d8-4295-a2b6-52d89ad98766)

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-09-27 16:39:15 +08:00
wt 1f271e39c6
test: add partition key & bitmap test case (#36564)
Signed-off-by: wangting0128 <ting.wang@zilliz.com>
2024-09-27 14:17:14 +08:00
yanliang567 e6c7fd6605
test: Add group search test on sparse vectors (#36562)
related issue: #36295
add one test for alter alias

Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
2024-09-27 14:15:14 +08:00
zhuwenxing 7c2cb8c5d4
test: add bulk insert case for text match feature (#36398)
/kind improvement

---------

Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2024-09-27 10:01:14 +08:00
wt ff4c62e44f
test: add more scalar filter expressions (#36545)
Signed-off-by: wangting0128 <ting.wang@zilliz.com>
2024-09-27 09:59:14 +08:00
zhuwenxing 9444329da1
test: relax the checks on range search (#36542)
/kind improvement

---------

Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2024-09-27 09:57:13 +08:00
cai.zhang 50905e0b45
fix: Check string array max length after type matching (#36449)
issue: #36029

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-09-27 09:45:14 +08:00
Zhen Ye d29e01e284
fix: port listen racing in mix or standalone mode (#36442)
issue: #36441

---------

Signed-off-by: chyezh <chyezh@outlook.com>
2024-09-26 21:23:16 +08:00
wayblink 7ff41697f9
enhance: make slotUsage a field of CompactionTask (#36510)
#36509

Signed-off-by: wayblink <anyang.wang@zilliz.com>
2024-09-26 20:19:14 +08:00
wei liu c056620899
fix: Skip unnecessary query node health check in proxy (#36491)
issue: #36490
After the query node changes from a delegator to a worker, proxy should
skip this querynode's health check.

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-09-26 18:13:15 +08:00