3ffe126dc7
enhance: Refine error message when search vector type not matched ( #31725 )
...
Previously the error message only reports the case happened without
field name and vector type.
This PR add field name and vector type information in the error
messages.
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-04-01 10:11:12 +08:00
b0a941c2fb
enhance: to reduce container image size ( #31576 )
...
Signed-off-by: Liang Huang <sammy.huang@zilliz.com>
2024-04-01 09:39:12 +08:00
417c678754
enhance: Retry on inconsistent requery result ( #31713 )
...
Make retry on ErrInConsistentRequery in proxy rather than in every SDK.
issue: https://github.com/milvus-io/milvus/issues/31642
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-03-30 20:53:19 +08:00
88d426f390
enhance:[skip e2e]add some custom deploy config back ( #31716 )
...
This reverts commit ede5269221
2024-03-29 18:15:17 +08:00
675a5dc822
fix: Save traceID and spanID as std::vector into search config ( #31278 )
...
Issue: #30961
Signed-off-by: Yudong Cai <yudong.cai@zilliz.com>
2024-03-29 14:29:11 +08:00
357fe814ce
fix: Remove unnecessary deleteSession operation ( #31647 )
...
See also #31628
The `Revoke` operation shall delete all keys related to the lease
attaching to. This `deleteSession` operation may also remove the session
key in next epoch by mistake and cause chaos session status
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-03-29 13:57:11 +08:00
c15beecb96
test: modify the hybrid search test case ( #31702 )
...
issue: #31339
Signed-off-by: binbin lv <binbin.lv@zilliz.com>
2024-03-29 10:47:17 +08:00
fe2f34d76f
fix: Use server ctx instead of loopCtx for datacoord LivenessCheck ( #31691 )
...
See also #31689
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-03-29 10:37:11 +08:00
9750e78f1d
enhance: lock tantivy dependencies ( #31688 )
...
issue: https://github.com/milvus-io/milvus/issues/31681
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2024-03-29 10:15:17 +08:00
be6b468a3b
fix: Check whether the DataType of the column is legal ( #31697 )
...
issue: #31696
---------
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-03-29 09:33:16 +08:00
6c6764bcad
test: modify the order function in test cases ( #31680 )
...
issue: #30694
Signed-off-by: binbin lv <binbin.lv@zilliz.com>
2024-03-28 17:37:10 +08:00
b99c46246c
enhance: ban groupby on binary vector( #31134 ) ( #31659 )
...
related: #31134
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
2024-03-28 15:19:10 +08:00
e33dba8afe
fix: [skip-e2e] use zstd-sys 2.0.9 ( #31682 )
...
fix : #31681
/kind improvement
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2024-03-28 15:14:10 +08:00
976928ecd1
fix: fix fp16/bf16 some code missing and add more fp16/bf16 test ( #31612 )
...
issue: #31534
Signed-off-by: cqy123456 <qianya.cheng@zilliz.com>
2024-03-28 14:11:10 +08:00
3d66670619
fix: add lazy load field to mark segment load type ( #31591 )
...
issue: https://github.com/milvus-io/milvus/issues/31673
---------
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2024-03-28 11:23:10 +08:00
c42492c0fd
test: enable import job test in ci ( #31530 )
...
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2024-03-28 10:31:10 +08:00
78fbb87b3a
enhance: Release blobs in sync task once sync is completed ( #31661 )
...
Once the synchronization of the sync task is completed, it's necessary
to release the blob within the sync task, as the caller may continue to
reference it.
issue: https://github.com/milvus-io/milvus/issues/31545
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-03-28 10:23:11 +08:00
8f3e0b6b41
enhance: the return result of list db api ( #31544 )
...
issue: #31543
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-03-28 07:13:10 +08:00
b1a1cca10b
feat: add more operation detail info for better allocation ( #30438 )
...
issue: #30436
---------
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-03-28 06:33:11 +08:00
fbff46a005
enhance: add lazyload global config ( #31610 )
...
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2024-03-27 20:23:10 +08:00
4eb4df1e81
fix: predict inverted index resource usage more reasonably ( #31615 )
...
/kind improvement
issue: #31617
---------
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2024-03-27 19:33:09 +08:00
655097f171
fix: Verify PlaceHolderValue type before search ( #31626 )
...
See also #31625
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-03-27 17:49:10 +08:00
d88026bd73
enhance: enable docker buildkit ( #31600 )
...
Signed-off-by: Liang Huang <sammy.huang@zilliz.com>
2024-03-27 16:49:15 +08:00
efa645db6f
feat: adding the Dowait primitive in cache ( #31578 )
...
See: #31577
---------
Signed-off-by: Ted Xu <ted.xu@zilliz.com>
2024-03-27 16:17:15 +08:00
92971707de
enhance: Add restful api for devops to execute rolling upgrade ( #29998 )
...
issue: #29261
This PR Add restful api for devops to execute rolling upgrade, including
suspend/resume balance and manual transfer segments/channels.
---------
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-03-27 16:15:19 +08:00
bd44bd5ae2
enhance: add default value config for segment prune filterRatio( #31003 ) ( #31580 )
...
related: #31003
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
2024-03-27 16:13:10 +08:00
f399416b92
enhance: Use gotestsum to run go unit test ( #31622 )
...
See also #31490
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-03-27 15:29:10 +08:00
5be395354c
fix: minio ssl compatible issue ( #31607 )
...
issue: https://github.com/milvus-io/milvus/issues/30709
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2024-03-27 14:41:20 +08:00
5d752498e7
fix: Skip release duplicate l0 segment ( #31540 )
...
issue: #31480 #31481
release duplicate l0 segment task, which execute on old delegator may
cause segment lack, and execute on new delegator may break new
delegator's leader view.
This PR skip release duplicate l0 segment by segment_checker, cause l0
segment will be released with unsub channel
---------
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-03-27 12:53:10 +08:00
d995b3f0fa
test: modify hybrid search cases ( #31624 )
...
issue: #31339
Signed-off-by: binbin lv <binbin.lv@zilliz.com>
2024-03-27 12:05:09 +08:00
c299bdec1b
enhance: update pymilvus version ( #31529 )
...
Signed-off-by: nico <cheng.yuan@zilliz.com>
2024-03-27 11:13:09 +08:00
678cb187e8
[automated] Update Knowhere Commit ( #31630 )
...
Update Knowhere Commit
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-03-27 01:15:10 +08:00
8e5865f630
enhance: Save collection targets by batches ( #31616 )
...
See also #28491 #31240
When colleciton number is large, querycoord saves collection target one
by one, which is slow and may block querycoord exits.
In local run, 500 collections scenario may lead to about 40 seconds
saving collection targets.
This PR changes the `SaveCollectionTarget` interface into batch one and
organizes the collection in 16 per bundle batches to accelerate this
procedure.
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-03-27 00:09:08 +08:00
248c923e59
fix: proxy ctx cancel before revoking session ( #31596 )
...
issue: #31219
Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-03-26 14:25:07 +08:00
659ad81ab7
fix: remove deprecated ut test ( #31499 )
...
#31498
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-03-26 14:01:07 +08:00
73858b23bc
fix: Make target observer auto/manual task mutual exclusive ( #31584 )
...
See also #30867
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-03-26 09:57:08 +08:00
9243e6087e
Update all contributors
...
Signed-off-by: sre-ci-robot <sre-ci-robot@zilliz.com>
2024-03-26 00:01:16 +00:00
19ee8810e8
Update README.md
2024-03-25 12:35:11 -07:00
31cf849f68
enhance: Support retriving file size from importutilv2.Reader ( #31533 )
...
To reduce the overhead caused by listing the S3 objects, add an
interface to importutil.Reader to retrieve file sizes.
issue: https://github.com/milvus-io/milvus/issues/31532 ,
https://github.com/milvus-io/milvus/issues/28521
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-03-25 20:29:07 +08:00
6bb654f23f
Update all contributors
...
Signed-off-by: sre-ci-robot <sre-ci-robot@zilliz.com>
2024-03-25 12:00:51 +00:00
e3d78f55b3
fix: add some check fieldata dim ( #31564 )
...
issue:https://github.com/milvus-io/milvus/issues/30138
Signed-off-by: cqy123456 <qianya.cheng@zilliz.com>
2024-03-25 19:19:07 +08:00
6438d65459
fix: Grow task stuck at stopping node ( #31487 )
...
issue: #30816
this PR fix that grow task stuck at stopping node
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-03-25 18:57:07 +08:00
ced7629d7d
doc: remove repetitive words ( #31105 )
...
Signed-off-by: hishope <csqiye@126.com>
2024-03-25 17:27:09 +08:00
0ac9bb4a9c
enhance: add mmap migration tool ( #30909 )
...
issue: #30908
---------
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2024-03-25 15:51:09 +08:00
8e661f791a
fix: lazy load index data in cache ( #31094 )
...
issue: https://github.com/milvus-io/milvus/issues/31571
---------
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2024-03-25 15:43:07 +08:00
fe1961ff14
enhance: add comments for gpu mem pool setting ( #31231 )
...
Signed-off-by: yusheng.ma <yusheng.ma@zilliz.com>
2024-03-25 14:41:07 +08:00
0c868b654d
fix: fix mmap enabled check in resource estimation ( #31536 )
...
issue: https://github.com/milvus-io/milvus/issues/31569
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2024-03-25 14:29:07 +08:00
9a13b9822f
enhance: Return more fields in import progress response ( #31539 )
...
Return more fields in import progress response, include importedRows and
totalRows. Additionally, ensure compatibility with the old import
progress response by retaining fields of create timestamp and row count.
issue: https://github.com/milvus-io/milvus/issues/31448
https://github.com/milvus-io/milvus/issues/31237
https://github.com/milvus-io/milvus/issues/28521
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-03-24 21:57:06 +08:00
c4b37fb285
enhance: Custom bitset and bitsetview prototypes ( #30454 )
...
Issue: #31285
Basically, I've replaced `FixedVector<bool>` and `boost::dynamic_bitset`
with custom bitset and bitsetview in order to reduce the memory
bandwidth & increase performance for the filtering.
This PR is for internal use only.
Current progress (numbers are for GCC 9.5.0 on Ubuntu 22.04 LTS;
clang-17 produces better performance numbers):
Baseline:
```
[ RUN ] CApiTest.AssembeChunkPerfTest
start test
cost: 17903us
[ OK ] CApiTest.AssembeChunkPerfTest (183 ms)
[ RUN ] Expr.TestMultiLogicalExprsOptimization
cost: 1391us
cost: 5us
cost: 4us
cost: 4us
cost: 6us
cost: 4us
cost: 4us
cost: 4us
cost: 4us
cost: 4us
143
cost: 10us
cost: 8us
cost: 10us
cost: 8us
cost: 8us
cost: 8us
cost: 8us
cost: 8us
cost: 8us
cost: 9us
8
/home/ubuntu/zilliz/milvus4/milvus/internal/core/unittest/test_expr.cpp:1561: Failure
Expected: (cost_op) < (cost_no_op), actual: 143 vs 8
[ FAILED ] Expr.TestMultiLogicalExprsOptimization (7 ms)
[ RUN ] Expr.TestExprs
start test
3cost: 889us
start test
10cost: 2us
start test
20cost: 2us
start test
30cost: 2us
start test
50cost: 3us
start test
100cost: 7us
start test
200cost: 16us
[ OK ] Expr.TestExprs (9 ms)
[ RUN ] Expr.TestUnaryBenchTest
start test type:2
cost: 124.8us
start test type:3
cost: 163.1us
start test type:4
cost: 275.9us
start test type:5
cost: 590.9us
start test type:10
cost: 62.7us
start test type:11
cost: 65.9us
[ OK ] Expr.TestUnaryBenchTest (1153 ms)
[ RUN ] Expr.TestBinaryRangeBenchTest
start test type:2
cost: 151.4us
start test type:3
cost: 198.4us
start test type:4
cost: 361.9us
start test type:5
cost: 753.9us
start test type:10
cost: 64.6us
start test type:11
cost: 62.2us
[ OK ] Expr.TestBinaryRangeBenchTest (1151 ms)
[ RUN ] Expr.TestLogicalUnaryBenchTest
start test type:2
cost: 121.14us
start test type:3
cost: 156.84us
start test type:4
cost: 249.76us
start test type:5
cost: 534.44us
start test type:10
cost: 82.2us
start test type:11
cost: 83.52us
[ OK ] Expr.TestLogicalUnaryBenchTest (1202 ms)
[ RUN ] Expr.TestBinaryLogicalBenchTest
start test type:2
cost: 80.64us
start test type:3
cost: 78.22us
start test type:4
cost: 255.76us
start test type:5
cost: 532.04us
start test type:10
cost: 89.26us
start test type:11
cost: 90us
[ OK ] Expr.TestBinaryLogicalBenchTest (1198 ms)
[ RUN ] Expr.TestBinaryArithOpEvalRangeBenchExpr
start test type:2
cost: 401.7us
start test type:3
cost: 420.96us
start test type:4
cost: 418.04us
start test type:5
cost: 470.54us
start test type:10
cost: 250.32us
start test type:11
cost: 850.08us
[ OK ] Expr.TestBinaryArithOpEvalRangeBenchExpr (1273 ms)
[ RUN ] Expr.TestCompareExprBenchTest
start test type:2
cost: 162us
start test type:3
cost: 142us
start test type:4
cost: 374us
start test type:5
cost: 674us
start test type:10
cost: 366us
start test type:11
cost: 645us
[ OK ] Expr.TestCompareExprBenchTest (1214 ms)
[ RUN ] Expr.TestRefactorExprs
start test
3cost: 1253us
start test
10cost: 1060us
start test
20cost: 681us
start test
30cost: 522us
start test
50cost: 511us
start test
100cost: 506us
start test
200cost: 497us
[ OK ] Expr.TestRefactorExprs (1142 ms)
```
Candidate:
```
[ RUN ] CApiTest.AssembeChunkPerfTest
start test
cost: 6099us
[ OK ] CApiTest.AssembeChunkPerfTest (153 ms)
[ RUN ] Expr.TestMultiLogicalExprsOptimization
cost: 42us
cost: 15us
cost: 15us
cost: 14us
cost: 15us
cost: 15us
cost: 15us
cost: 15us
cost: 15us
cost: 15us
17
cost: 41us
cost: 39us
cost: 33us
cost: 33us
cost: 33us
cost: 33us
cost: 34us
cost: 41us
cost: 34us
cost: 34us
35
[ OK ] Expr.TestMultiLogicalExprsOptimization (6 ms)
[ RUN ] Expr.TestExprs
start test
3cost: 20us
start test
10cost: 2us
start test
20cost: 2us
start test
30cost: 2us
start test
50cost: 4us
start test
100cost: 8us
start test
200cost: 15us
[ OK ] Expr.TestExprs (8 ms)
[ RUN ] Expr.TestUnaryBenchTest
start test type:2
cost: 55.7us
start test type:3
cost: 79.8us
start test type:4
cost: 177.6us
start test type:5
cost: 337.2us
start test type:10
cost: 16.9us
start test type:11
cost: 15.7us
[ OK ] Expr.TestUnaryBenchTest (1140 ms)
[ RUN ] Expr.TestBinaryRangeBenchTest
start test type:2
cost: 57.1us
start test type:3
cost: 87us
start test type:4
cost: 177.5us
start test type:5
cost: 342.7us
start test type:10
cost: 17.9us
start test type:11
cost: 16.7us
[ OK ] Expr.TestBinaryRangeBenchTest (1152 ms)
[ RUN ] Expr.TestLogicalUnaryBenchTest
start test type:2
cost: 34.58us
start test type:3
cost: 68.86us
start test type:4
cost: 151.38us
start test type:5
cost: 286.8us
start test type:10
cost: 16.54us
start test type:11
cost: 16.7us
[ OK ] Expr.TestLogicalUnaryBenchTest (1165 ms)
[ RUN ] Expr.TestBinaryLogicalBenchTest
start test type:2
cost: 20us
start test type:3
cost: 17.1us
start test type:4
cost: 154.12us
start test type:5
cost: 286.1us
start test type:10
cost: 19.6us
start test type:11
cost: 19.24us
[ OK ] Expr.TestBinaryLogicalBenchTest (1188 ms)
[ RUN ] Expr.TestBinaryArithOpEvalRangeBenchExpr
start test type:2
cost: 125.7us
start test type:3
cost: 111.34us
start test type:4
cost: 148.02us
start test type:5
cost: 306.7us
start test type:10
cost: 149.3us
start test type:11
cost: 282.94us
[ OK ] Expr.TestBinaryArithOpEvalRangeBenchExpr (1221 ms)
[ RUN ] Expr.TestCompareExprBenchTest
start test type:2
cost: 89us
start test type:3
cost: 79us
start test type:4
cost: 323us
start test type:5
cost: 629us
start test type:10
cost: 313us
start test type:11
cost: 591us
[ OK ] Expr.TestCompareExprBenchTest (1228 ms)
[ RUN ] Expr.TestRefactorExprs
start test
3cost: 874us
start test
10cost: 611us
start test
20cost: 290us
start test
30cost: 294us
start test
50cost: 272us
start test
100cost: 278us
start test
200cost: 279us
[ OK ] Expr.TestRefactorExprs (1149 ms)
```
Signed-off-by: Alexandr Guzhva <alexanderguzhva@gmail.com>
2024-03-24 21:49:07 +08:00
42f2425822
Update all contributors
...
Signed-off-by: sre-ci-robot <sre-ci-robot@zilliz.com>
2024-03-24 00:01:15 +00:00
congqixia
sammy.huang
yihao.dai
zhuwenxing
Cai Yudong
binbin
Jiquan Long
cai.zhang
Chun Han
cqy123456
Bingyi Sun
SimFG
Ted Xu
wei liu
groot
nico
sre-ci-robot
jaime
zhagnlu
sre-ci-robot
Xiaofan
John
presburger
Alexander Guzhva