Commit Graph

19443 Commits (d299fa502e53a97e005c6f6bdcbf26bde16afe80)

Author SHA1 Message Date
PowderLi d299fa502e
fix: use milvus-io/vcpkg (#31770)
issue: #31769
GitHub Disables The XZ Repository because of CVE-2024-3094

Signed-off-by: PowderLi <min.li@zilliz.com>
2024-04-01 15:01:13 +08:00
chyezh 5655ec4fc0
enhance: add mmap usage metrics (#31708)
issue: #31707

Signed-off-by: chyezh <chyezh@outlook.com>
2024-04-01 11:35:12 +08:00
zhuwenxing 571f0bc054
test: fix bulk insert case (#31738)
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2024-04-01 10:49:18 +08:00
wei liu bb500d66c7
fix: Remove segment from leader view can't be executed (#31663)
issue: #31664

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-04-01 10:39:12 +08:00
wei liu c311932d5f
fix: Update segment's version in leader task (#31643)
issue: #31468

1. when segment's version in leader view doesn't match segment's version
in dist, should update leader view
2. after call loadDeltalog, should update segment's load version with
latest ts
3. change leader task's priority from high to low, to avoid leader task
replace segment task and balance task

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-04-01 10:37:21 +08:00
sammy.huang f7a59766df
enhance: support rockylinux8 (#31751)
Signed-off-by: Sammy Huang <sammy.huang@zilliz.com>
Signed-off-by: Liang Huang <sammy.huang@zilliz.com>
2024-04-01 10:23:12 +08:00
chyezh 1ad5ccc50f
enhance: add rg and db interface for segment and db/rg metric label (#31715)
issue: #30931

Signed-off-by: chyezh <chyezh@outlook.com>
2024-04-01 10:21:21 +08:00
congqixia 3ffe126dc7
enhance: Refine error message when search vector type not matched (#31725)
Previously the error message only reports the case happened without
field name and vector type.

This PR add field name and vector type information in the error
messages.

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-04-01 10:11:12 +08:00
sammy.huang b0a941c2fb
enhance: to reduce container image size (#31576)
Signed-off-by: Liang Huang <sammy.huang@zilliz.com>
2024-04-01 09:39:12 +08:00
yihao.dai 417c678754
enhance: Retry on inconsistent requery result (#31713)
Make retry on ErrInConsistentRequery in proxy rather than in every SDK.

issue: https://github.com/milvus-io/milvus/issues/31642

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-03-30 20:53:19 +08:00
zhuwenxing 88d426f390
enhance:[skip e2e]add some custom deploy config back (#31716)
This reverts commit ede5269221.

Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2024-03-29 18:15:17 +08:00
Cai Yudong 675a5dc822
fix: Save traceID and spanID as std::vector into search config (#31278)
Issue: #30961

Signed-off-by: Yudong Cai <yudong.cai@zilliz.com>
2024-03-29 14:29:11 +08:00
congqixia 357fe814ce
fix: Remove unnecessary deleteSession operation (#31647)
See also #31628

The `Revoke` operation shall delete all keys related to the lease
attaching to. This `deleteSession` operation may also remove the session
key in next epoch by mistake and cause chaos session status

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-03-29 13:57:11 +08:00
binbin c15beecb96
test: modify the hybrid search test case (#31702)
issue: #31339

Signed-off-by: binbin lv <binbin.lv@zilliz.com>
2024-03-29 10:47:17 +08:00
congqixia fe2f34d76f
fix: Use server ctx instead of loopCtx for datacoord LivenessCheck (#31691)
See also #31689

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-03-29 10:37:11 +08:00
Jiquan Long 9750e78f1d
enhance: lock tantivy dependencies (#31688)
issue: https://github.com/milvus-io/milvus/issues/31681

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2024-03-29 10:15:17 +08:00
cai.zhang be6b468a3b
fix: Check whether the DataType of the column is legal (#31697)
issue: #31696

---------

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-03-29 09:33:16 +08:00
binbin 6c6764bcad
test: modify the order function in test cases (#31680)
issue: #30694

Signed-off-by: binbin lv <binbin.lv@zilliz.com>
2024-03-28 17:37:10 +08:00
Chun Han b99c46246c
enhance: ban groupby on binary vector(#31134) (#31659)
related: #31134

Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
2024-03-28 15:19:10 +08:00
Jiquan Long e33dba8afe
fix: [skip-e2e] use zstd-sys 2.0.9 (#31682)
fix: #31681 
/kind improvement

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2024-03-28 15:14:10 +08:00
cqy123456 976928ecd1
fix: fix fp16/bf16 some code missing and add more fp16/bf16 test (#31612)
issue: #31534

Signed-off-by: cqy123456 <qianya.cheng@zilliz.com>
2024-03-28 14:11:10 +08:00
Bingyi Sun 3d66670619
fix: add lazy load field to mark segment load type (#31591)
issue: https://github.com/milvus-io/milvus/issues/31673

---------

Signed-off-by: sunby <sunbingyi1992@gmail.com>
2024-03-28 11:23:10 +08:00
zhuwenxing c42492c0fd
test: enable import job test in ci (#31530)
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2024-03-28 10:31:10 +08:00
yihao.dai 78fbb87b3a
enhance: Release blobs in sync task once sync is completed (#31661)
Once the synchronization of the sync task is completed, it's necessary
to release the blob within the sync task, as the caller may continue to
reference it.

issue: https://github.com/milvus-io/milvus/issues/31545

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-03-28 10:23:11 +08:00
SimFG 8f3e0b6b41
enhance: the return result of list db api (#31544)
issue: #31543

Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-03-28 07:13:10 +08:00
SimFG b1a1cca10b
feat: add more operation detail info for better allocation (#30438)
issue: #30436

---------

Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-03-28 06:33:11 +08:00
Bingyi Sun fbff46a005
enhance: add lazyload global config (#31610)
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2024-03-27 20:23:10 +08:00
Jiquan Long 4eb4df1e81
fix: predict inverted index resource usage more reasonably (#31615)
/kind improvement
issue: #31617

---------

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2024-03-27 19:33:09 +08:00
congqixia 655097f171
fix: Verify PlaceHolderValue type before search (#31626)
See also #31625

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-03-27 17:49:10 +08:00
sammy.huang d88026bd73
enhance: enable docker buildkit (#31600)
Signed-off-by: Liang Huang <sammy.huang@zilliz.com>
2024-03-27 16:49:15 +08:00
Ted Xu efa645db6f
feat: adding the Dowait primitive in cache (#31578)
See: #31577

---------

Signed-off-by: Ted Xu <ted.xu@zilliz.com>
2024-03-27 16:17:15 +08:00
wei liu 92971707de
enhance: Add restful api for devops to execute rolling upgrade (#29998)
issue: #29261
This PR Add restful api for devops to execute rolling upgrade, including
suspend/resume balance and manual transfer segments/channels.

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-03-27 16:15:19 +08:00
Chun Han bd44bd5ae2
enhance: add default value config for segment prune filterRatio(#31003) (#31580)
related: #31003

Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
2024-03-27 16:13:10 +08:00
congqixia f399416b92
enhance: Use gotestsum to run go unit test (#31622)
See also #31490

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-03-27 15:29:10 +08:00
groot 5be395354c
fix: minio ssl compatible issue (#31607)
issue: https://github.com/milvus-io/milvus/issues/30709

Signed-off-by: yhmo <yihua.mo@zilliz.com>
2024-03-27 14:41:20 +08:00
wei liu 5d752498e7
fix: Skip release duplicate l0 segment (#31540)
issue: #31480 #31481

release duplicate l0 segment task, which execute on old delegator may
cause segment lack, and execute on new delegator may break new
delegator's leader view.

This PR skip release duplicate l0 segment by segment_checker, cause l0
segment will be released with unsub channel

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-03-27 12:53:10 +08:00
binbin d995b3f0fa
test: modify hybrid search cases (#31624)
issue: #31339

Signed-off-by: binbin lv <binbin.lv@zilliz.com>
2024-03-27 12:05:09 +08:00
nico c299bdec1b
enhance: update pymilvus version (#31529)
Signed-off-by: nico <cheng.yuan@zilliz.com>
2024-03-27 11:13:09 +08:00
sre-ci-robot 678cb187e8
[automated] Update Knowhere Commit (#31630)
Update Knowhere Commit
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-03-27 01:15:10 +08:00
congqixia 8e5865f630
enhance: Save collection targets by batches (#31616)
See also #28491 #31240

When colleciton number is large, querycoord saves collection target one
by one, which is slow and may block querycoord exits.

In local run, 500 collections scenario may lead to about 40 seconds
saving collection targets.

This PR changes the `SaveCollectionTarget` interface into batch one and
organizes the collection in 16 per bundle batches to accelerate this
procedure.

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-03-27 00:09:08 +08:00
jaime 248c923e59
fix: proxy ctx cancel before revoking session (#31596)
issue: #31219

Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-03-26 14:25:07 +08:00
zhagnlu 659ad81ab7
fix: remove deprecated ut test (#31499)
#31498

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-03-26 14:01:07 +08:00
congqixia 73858b23bc
fix: Make target observer auto/manual task mutual exclusive (#31584)
See also #30867

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-03-26 09:57:08 +08:00
sre-ci-robot 9243e6087e Update all contributors
Signed-off-by: sre-ci-robot <sre-ci-robot@zilliz.com>
2024-03-26 00:01:16 +00:00
Xiaofan 19ee8810e8
Update README.md 2024-03-25 12:35:11 -07:00
yihao.dai 31cf849f68
enhance: Support retriving file size from importutilv2.Reader (#31533)
To reduce the overhead caused by listing the S3 objects, add an
interface to importutil.Reader to retrieve file sizes.

issue: https://github.com/milvus-io/milvus/issues/31532,
https://github.com/milvus-io/milvus/issues/28521

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-03-25 20:29:07 +08:00
sre-ci-robot 6bb654f23f Update all contributors
Signed-off-by: sre-ci-robot <sre-ci-robot@zilliz.com>
2024-03-25 12:00:51 +00:00
cqy123456 e3d78f55b3
fix: add some check fieldata dim (#31564)
issue:https://github.com/milvus-io/milvus/issues/30138

Signed-off-by: cqy123456 <qianya.cheng@zilliz.com>
2024-03-25 19:19:07 +08:00
wei liu 6438d65459
fix: Grow task stuck at stopping node (#31487)
issue: #30816
this PR fix that grow task stuck at stopping node

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-03-25 18:57:07 +08:00
John ced7629d7d
doc: remove repetitive words (#31105)
Signed-off-by: hishope <csqiye@126.com>
2024-03-25 17:27:09 +08:00