c8dd665bf6
test: supplementing case for text match ( #36693 )
...
/kind improvement
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2024-10-28 10:31:40 +08:00
d7b2ffe5aa
enhance: add an unify vector index config checker ( #36844 )
...
issue: #34298
Signed-off-by: xianliang.li <xianliang.li@zilliz.com>
2024-10-28 10:11:37 +08:00
ed37c27bda
fix: Fix collection leak in querynode ( #37061 )
...
Unref the removed L0 segment count.
issue: https://github.com/milvus-io/milvus/issues/36918
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-10-25 19:59:29 +08:00
3628593d20
feat: Implement custom function module in milvus expr ( #36560 )
...
OSPP 2024 project:
https://summer-ospp.ac.cn/org/prodetail/247410235?list=org&navpage=org
Solutions:
- parser (planparserv2)
- add CallExpr in planparserv2/Plan.g4
- update parser_visitor and show_visitor
- grpc protobuf
- add CallExpr in plan.proto
- execution (`core/src/exec`)
- add `CallExpr` `ValueExpr` and `ColumnExpr` (both logical and
physical) for function call and function parameters
- function factory (`core/src/exec/expression/function`)
- create a global hashmap when starting milvus (see server.go)
- the global hashmap stores function signatures and their function
pointers, the CallExpr in execution engine can get the function pointer
by function signature.
- custom functions
- empty(string)
- starts_with(string, string)
- add cpp/go unittests and E2E tests
closes : #36559
Signed-off-by: Yinzuo Jiang <jiangyinzuo@foxmail.com>
2024-10-25 15:25:30 +08:00
088d5d7d76
fix: optimize BM25 err message ( #37074 )
...
issue: https://github.com/milvus-io/milvus/issues/37022
Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com>
2024-10-25 14:35:45 +08:00
ac2858d418
test: add full text search checker in test ( #37122 )
...
/kind improvement
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2024-10-25 14:09:29 +08:00
80aa9ab4d6
test: Add insert and upsert related cases for null and default value support ( #37088 )
...
issue: #36129
---------
Signed-off-by: qixuan <673771573@qq.com>
2024-10-25 11:03:29 +08:00
c285853de8
test: Add test cases about expr for null and default support ( #37121 )
...
issue: #36129
Signed-off-by: binbin lv <binbin.lv@zilliz.com>
2024-10-25 11:01:30 +08:00
2b3f5bec07
fix: panic when create index on all none data ( #37046 )
...
#37045
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-10-24 17:09:28 +08:00
53836f320a
[automated] Update Pytest image changes ( #37067 )
...
Update Pytest image changes
See changes:
3b024f9b36sre-ci-robot@users.noreply.github.com
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-10-24 16:53:28 +08:00
0d376f1a8f
test: Add query iterator test and seperate a new test file ( #37085 )
...
related issue: #37084
Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
2024-10-24 11:47:29 +08:00
4746f47282
feat: management WebUI homepage ( #36822 )
...
issue: #36784
1. Implement an embedded web server for WebUI access.
2. Complete the homepage development.
Home page demo:
<img width="2177" alt="iShot_2024-10-10_17 57 34"
src="https://github.com/user-attachments/assets/38539917-ce09-4e54-a5b5-7f4f7eaac353 ">
Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-10-23 11:29:28 +08:00
3b024f9b36
test: Add full-text search test cases ( #36998 )
...
/kind improvement
---------
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2024-10-23 09:51:27 +08:00
80d48f1e53
test: add text match checker in test ( #37052 )
...
/kind improvement
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2024-10-23 09:49:27 +08:00
36147b10f5
test: Add insert and upsert related cases for null and default value support ( #36932 )
...
issue: #36129
Signed-off-by: qixuan <673771573@qq.com>
2024-10-22 10:13:27 +08:00
ef923958bb
test: add restful param check testcases ( #36931 )
...
/kind improvement
---------
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2024-10-19 19:13:25 +08:00
d17135d3bc
test: Add alias test with rename collection ( #36978 )
...
related issue: #36963
1. add alias tests with rename collection for issue #36963
2. update some test levels
3. fix some xfail tests
---------
Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
2024-10-18 16:27:24 +08:00
eb3e4583ec
enhance: all op(Null) is false in expr ( #35527 )
...
#31728
---------
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-10-17 21:14:30 +08:00
97ff012c67
test: check query response data ( #36952 )
...
Signed-off-by: wangting0128 <ting.wang@zilliz.com>
2024-10-17 19:13:25 +08:00
e5948bd039
enhance: [GoSDK] Use variadic params for options ( #36912 )
...
Use variadic parameter function for options make client options easier
to use.
Related to #31293
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-10-16 19:33:23 +08:00
44d80c1355
fix: not return err if consistencyLevel is not set to a valid value ( #36714 )
...
https://github.com/milvus-io/milvus/issues/36444
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-10-16 13:23:22 +08:00
2291d0c9f5
test: update test cases ( #36878 )
...
Signed-off-by: nico <cheng.yuan@zilliz.com>
2024-10-15 19:13:24 +08:00
0ec92513a5
test: Update Diskann supporting group by ( #36853 )
...
related issue: #36813
milvus diskann does not support group_search before, it supports now.
Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
2024-10-15 10:35:23 +08:00
d566b0ceff
test: add stats task feature cases and DefaultVectorSearchParams ( #36768 )
...
issue: #36767
---------
Signed-off-by: ThreadDao <yufen.zong@zilliz.com>
2024-10-14 19:15:23 +08:00
937ebec2ce
test: update test cases ( #36841 )
...
Signed-off-by: nico <cheng.yuan@zilliz.com>
2024-10-14 19:13:23 +08:00
af2da8d8d6
[automated] Update Pytest image changes ( #36845 )
...
Update Pytest image changes
See changes:
c96bbe19basre-ci-robot@users.noreply.github.com
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-10-14 19:11:23 +08:00
c96bbe19ba
test: Add upsert in rows tests ( #36820 )
...
related issue: #36710
---------
Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
2024-10-14 16:41:22 +08:00
621dbc9107
test: Add json key in [] test to verify issue 36718 ( #36738 )
...
related issue: #36718
---------
Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
2024-10-11 14:29:21 +08:00
db34572c56
feat: support load and query with bm25 metric ( #36071 )
...
relate: https://github.com/milvus-io/milvus/issues/35853
---------
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-10-11 10:23:20 +08:00
90285830de
[automated] Update Pytest image changes ( #36755 )
...
Update Pytest image changes
See changes:
f7b811450dsre-ci-robot@users.noreply.github.com
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-10-11 09:29:19 +08:00
f7b811450d
feat: add enable_tokenizer params to VarChar field ( #36480 )
...
issue: #35922
add an enable_tokenizer param to varchar field: must be set to true so
that a varchar field can enable_match or used as input of BM25 function
---------
Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com>
2024-10-10 20:33:21 +08:00
e1b511a219
enhance: open mmap on standalone-kafka in ci-e2e ( #36688 )
...
Signed-off-by: nico <cheng.yuan@zilliz.com>
2024-10-09 19:03:19 +08:00
ef1832ff9c
enhance: enable manual compaction for collections without indexes ( #36577 )
...
issue: #36576
Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-10-08 19:57:18 +08:00
9fdf0505a8
test: update test case for support dynamic load replica ( #36589 )
...
Signed-off-by: nico <cheng.yuan@zilliz.com>
2024-10-08 17:31:20 +08:00
0d57ff01a6
test: add null and default test cases ( #36612 )
...
issue: #36129
Signed-off-by: binbin lv <binbin.lv@zilliz.com>
2024-10-08 17:25:24 +08:00
00a5025949
enhance: support clustering compaction on null value ( #36372 )
...
issue: #36055
Signed-off-by: wayblink <anyang.wang@zilliz.com>
2024-09-30 14:33:17 +08:00
798ef223f1
test: add test cases for param checking ( #36595 )
...
Signed-off-by: wangting0128 <ting.wang@zilliz.com>
2024-09-30 10:33:16 +08:00
a54bffd6cd
fix: refine test case for search_group_by( #36401 ) ( #36511 )
...
related: #36401
Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
2024-09-30 10:13:17 +08:00
c43d94319f
enhance: [skip e2e]open querynode mmap for all pipeline ( #36586 )
...
Signed-off-by: nico <cheng.yuan@zilliz.com>
2024-09-29 09:41:15 +08:00
d1d5a50014
test: add null and default test cases ( #36539 )
...
issue: #36129
Signed-off-by: binbin lv <binbin.lv@zilliz.com>
2024-09-29 09:39:22 +08:00
31353ae406
test: add restful cases for text match feature ( #36405 )
...
/kind improvement
Since creating a collection with text match is not yet implemented on
the RESTful interface, we will temporarily use pymilvus to create a
collection for now. This PR includes a case to test using text match
filters in search queries through the RESTful interface.
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2024-09-28 16:51:15 +08:00
1f271e39c6
test: add partition key & bitmap test case ( #36564 )
...
Signed-off-by: wangting0128 <ting.wang@zilliz.com>
2024-09-27 14:17:14 +08:00
e6c7fd6605
test: Add group search test on sparse vectors ( #36562 )
...
related issue: #36295
add one test for alter alias
Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
2024-09-27 14:15:14 +08:00
7c2cb8c5d4
test: add bulk insert case for text match feature ( #36398 )
...
/kind improvement
---------
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2024-09-27 10:01:14 +08:00
ff4c62e44f
test: add more scalar filter expressions ( #36545 )
...
Signed-off-by: wangting0128 <ting.wang@zilliz.com>
2024-09-27 09:59:14 +08:00
9444329da1
test: relax the checks on range search ( #36542 )
...
/kind improvement
---------
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2024-09-27 09:57:13 +08:00
d29e01e284
fix: port listen racing in mix or standalone mode ( #36442 )
...
issue: #36441
---------
Signed-off-by: chyezh <chyezh@outlook.com>
2024-09-26 21:23:16 +08:00
0799d927c6
fix:fix term expr overflow bug ( #36525 )
...
#36520
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-09-26 15:01:14 +08:00
954d8a5f68
test: add search test cases with text match ( #36399 )
...
/kind improvement
---------
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2024-09-26 10:15:14 +08:00
1970ab4c34
[automated] Update Pytest image changes ( #36496 )
...
Update Pytest image changes
See changes:
58baeee8f1sre-ci-robot@users.noreply.github.com
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-09-26 10:05:13 +08:00
cfd636ed5b
test: add different language tests and modify some cases ( #36465 )
...
fix : #36396
Signed-off-by: nico <cheng.yuan@zilliz.com>
2024-09-26 09:21:13 +08:00
14be89df29
test: add case to enable offset cache and mmap at the same time ( #36505 )
...
Signed-off-by: wangting0128 <ting.wang@zilliz.com>
2024-09-25 19:59:13 +08:00
58baeee8f1
test: add query with text match filter ( #36381 )
...
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
Co-authored-by: yanliang567 <82361606+yanliang567@users.noreply.github.com>
2024-09-25 15:17:13 +08:00
aee046e52c
test: add some restful negative case to ci ( #36479 )
...
/kind improvement
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2024-09-25 10:19:13 +08:00
3cd0b26285
enhance: Enable dynamic update loaded collection's replica ( #35822 )
...
issue: #35821
After collection loaded, if we need to increase/decrease collection's
replica, we need to release and load it again.
milvus offers 4 solution to update loaded collection's replica, this PR
aims to dynamic change the replica number without release, and after
replica number changed, milvus will execute load replica or release
replica in async, and the replica loaded status can be checked by
getReplicas API.
Notice that if set too much replicas than querynode can afford，the new
replica won't be loaded successfully until enough querynode joins.
---------
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-09-25 10:13:18 +08:00
da2d3ce763
[automated] Update Pytest image changes ( #36463 )
...
Update Pytest image changes
See changes:
d55d9d6e1dsre-ci-robot@users.noreply.github.com
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-09-24 19:49:12 +08:00
d55d9d6e1d
fix: change pymilvus version for hybridsearch-groupby( #36407 ) ( #36451 )
...
related: #36407
---------
Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
2024-09-24 14:29:13 +08:00
4779c6cb8f
test: update restful v2 test cases ( #36448 )
...
/kind improvement
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2024-09-24 10:21:12 +08:00
51cdee1f84
test: Add query related test cases for default and null support ( #36277 )
...
issue: #36129
Signed-off-by: qixuan <673771573@qq.com>
2024-09-23 19:07:13 +08:00
701f3bf26e
test: add search group by test case for bitmap ( #36410 )
...
Signed-off-by: wangting0128 <ting.wang@zilliz.com>
2024-09-23 17:29:12 +08:00
6e880d19a8
enhance: add search params in search request in restful ( #36304 )
...
https://github.com/milvus-io/milvus/issues/36321
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-09-23 16:51:12 +08:00
1fb8b46db0
test: Share one collection for group search tests ( #36427 )
...
related issue: #36407
1. add partial load tests
2. use new test class to share one collection for all grouping search
tests
Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
2024-09-23 14:09:11 +08:00
89397d1e66
enhance: adjust parquet reader type check with null type ( #36266 )
...
#36252
remove no need type check. if users use null type writer to write
parquet, hope it successfully.
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-09-19 18:43:10 +08:00
c0317ce672
fix: Wait check node id goroutine in case of data race ( #36302 )
...
Resolves : #36301
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-09-19 10:41:10 +08:00
e013ef1908
test: Add tests for hybrid search group by ( #36326 )
...
related issue: #36295
---------
Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
2024-09-19 09:41:12 +08:00
5ca4d5977a
test: Add bulk insert related test cases for default and null support ( #36219 )
...
issue: #36129
Signed-off-by: binbin lv <binbin.lv@zilliz.com>
2024-09-18 19:33:17 +08:00
526a672bae
test: add more bitmap test cases ( #36290 )
...
Signed-off-by: wangting0128 <ting.wang@zilliz.com>
2024-09-16 11:03:09 +08:00
a61668c77e
feat: Introduce stats task for import ( #35868 )
...
This PR introduce stats task for import:
1. Define new `Stats` and `IndexBuilding` states for importJob
2. Add new stats step to the import process: trigger the stats task and
wait for its completion
3. Abort stats task if import job failed
issue: https://github.com/milvus-io/milvus/issues/33744
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-09-15 15:17:08 +08:00
5ce99e2535
[automated] Update Pytest image changes ( #36250 )
...
Update Pytest image changes
See changes:
2e434b2358sre-ci-robot@users.noreply.github.com
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-09-14 17:59:09 +08:00
e3e88ffbe6
test: update pymilvus version and skip some cases ( #36281 )
...
Signed-off-by: nico <cheng.yuan@zilliz.com>
2024-09-14 17:57:14 +08:00
2e434b2358
test: Add group size tests ( #36240 )
...
related issue: #36146
Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
2024-09-13 16:13:08 +08:00
b5ff348906
test: Add part of test cases for default and null support ( #36186 )
...
issue : #36129
Signed-off-by: binbin lv <binbin.lv@zilliz.com>
2024-09-12 17:29:13 +08:00
d9c8d1ea90
test: Add part of test cases for default and null support ( #36130 )
...
issue: #36129
Signed-off-by: binbin lv <binbin.lv@zilliz.com>
2024-09-11 10:59:07 +08:00
dbe03a6151
test: add check after altering cardinality limit ( #36158 )
...
Signed-off-by: wangting0128 <ting.wang@zilliz.com>
2024-09-11 10:49:06 +08:00
42eef4900d
test: update test cases and sdk version ( #36148 )
...
Signed-off-by: nico <cheng.yuan@zilliz.com>
2024-09-11 10:45:07 +08:00
b0939fd20e
enhance: Reset task check interval configuration ( #36121 )
...
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-09-10 14:35:08 +08:00
9d0aa5c202
fix: empty result when having only one subReq( #36098 ) ( #36128 )
...
related: #36098
Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
2024-09-10 14:25:07 +08:00
f4d0c589fa
test: set bulk insert test fail fast ( #36039 )
...
/kind improvement
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2024-09-10 11:43:06 +08:00
53a87825f3
test: add more bitmap test cases ( #36131 )
...
1. verified issues #36054 and #35971
2. add mix scenes test cases for BITMAP index
Signed-off-by: wangting0128 <ting.wang@zilliz.com>
2024-09-10 10:55:07 +08:00
c916407f37
test: add query expr test cases ( #36073 )
...
1. query with expr under different scalar index types
2. test framework supports preparing one piece of data and multiple
parameter queries
Signed-off-by: wangting0128 <ting.wang@zilliz.com>
2024-09-09 19:55:06 +08:00
3a381bc247
enhance: Bulkinsert supports null in csv formats ( #35912 )
...
see details in this issue
https://github.com/milvus-io/milvus/issues/35911
---------
Signed-off-by: OxalisCu <2127298698@qq.com>
2024-09-09 19:17:07 +08:00
208c8a2328
fix:support config index offsetcache and fix create same index again ( #35985 )
...
#35971
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-09-08 18:23:05 +08:00
21b135c7c2
fix: not append valid data when transfer to insert record ( #36027 )
...
fix not append valid data when transfer to insert record and add a tiny
check when in groupBy field.
#35924
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-09-06 14:53:04 +08:00
43a5523a8b
[automated] Update Pytest image changes ( #35951 )
...
Update Pytest image changes
See changes:
40d34f711esre-ci-robot@users.noreply.github.com
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-09-06 09:41:05 +08:00
11325d9ed5
fix: binary arith expression on inverted index ( #35945 )
...
issue: https://github.com/milvus-io/milvus/issues/35946
---------
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2024-09-05 20:01:05 +08:00
8b043f58dc
fix: Fix DB limiter nodes are mistakenly cleaned up ( #35991 )
...
This issue only occurs for a short time right after a table is created.
To avoid this, we simply reduce the frequency of cleaning up invalid
limiter nodes.
issue: https://github.com/milvus-io/milvus/issues/35933
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-09-05 17:35:04 +08:00
32e55a02ea
fix: Fix privilege group hasn't been register for validate ( #35937 )
...
issue: #35471
---------
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-09-05 15:35:04 +08:00
919e96ac22
enhance: add IT for rate limit using db properties ( #35930 )
...
issue: #35929
Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-09-04 14:37:04 +08:00
40d34f711e
test: update test cases ( #35867 )
...
Signed-off-by: nico <cheng.yuan@zilliz.com>
2024-09-04 10:37:03 +08:00
f068729a26
test: fix repeated random partition name ( #35805 )
...
/kind improvement
Signed-off-by: ThreadDao <yufen.zong@zilliz.com>
2024-09-03 17:13:12 +08:00
cb49b32358
test: add bitmap index cases ( #35909 )
...
Signed-off-by: wangting0128 <ting.wang@zilliz.com>
2024-09-03 16:49:03 +08:00
57422cb2ed
test: add array inverted index function test ( #35874 )
...
/kind improvement
---------
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2024-09-02 17:49:02 +08:00
2c9bb4dfa3
feat: Support stats task to sort segment by PK ( #35054 )
...
issue: #33744
This PR includes the following changes:
1. Added a new task type to the task scheduler in datacoord: stats task,
which sorts segments by primary key.
2. Implemented segment sorting in indexnode.
3. Added a new field `FieldStatsLog` to SegmentInfo to store token index
information.
---------
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-09-02 14:19:03 +08:00
1e3e34d149
enhance: [ci] organize values file in the form of milvus deployment options ( #35832 )
...
https://github.com/milvus-io/milvus/issues/35842
Signed-off-by: Yellow Shine <sammy.huang@zilliz.com>
2024-08-30 13:07:15 +08:00
a3f2f044d6
fix: not set nullable when stream writer write headers ( #35799 )
...
#35802
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-08-29 20:59:00 +08:00
99dff06391
enhance: using streaming service in insert/upsert/flush/delete/querynode ( #35406 )
...
issue: #33285
- using streaming service in insert/upsert/flush/delete/querynode
- fixup flusher bugs and refactor the flush operation
- enable streaming service for dml and ddl
- pass the e2e when enabling streaming service
- pass the integration tst when enabling streaming service
---------
Signed-off-by: chyezh <chyezh@outlook.com>
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-08-29 10:03:08 +08:00
b75b25794b
enhance: [skip e2e] Fix test script always skip pip install ( #35766 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-08-28 17:01:00 +08:00
bb9bed64a3
enhance: new nightly ci to support testing distributed-streaming ( #35672 )
...
issue: https://github.com/milvus-io/milvus/issues/35704
Signed-off-by: Yellow Shine <sammy.huang@zilliz.com>
2024-08-27 14:24:59 +08:00
0e7877d413
fix: [skip e2e]unstable l0 it ( #35612 )
...
See also: #35617
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-08-26 18:53:04 +08:00
a90133cdf5
test: Optimize sparse vector case ( #35661 )
...
/kind improvement
Signed-off-by: ThreadDao <yufen.zong@zilliz.com>
2024-08-26 14:54:57 +08:00
f756f01445
test: update test cases ( #35640 )
...
Signed-off-by: nico <cheng.yuan@zilliz.com>
2024-08-26 12:00:58 +08:00
5c245d51c4
enhance: Refresh proxy cache after restore rbac meta ( #35635 )
...
issue: #35443
---------
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-08-22 19:09:01 +08:00
570a8879e7
test: update gosdk case for valid empty sparse vector ( #35621 )
...
/kind improvement
Signed-off-by: ThreadDao <yufen.zong@zilliz.com>
2024-08-22 10:54:56 +08:00
ed4eaffc9d
enhance: add csv support for bulkinsert ( #34938 )
...
See this issue for details: #34937
---------
Signed-off-by: OxalisCu <2127298698@qq.com>
2024-08-21 17:47:01 +08:00
ba6db117e3
enhance: add some integration test in null ( #35599 )
...
#31728
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-08-21 17:44:56 +08:00
7ac339ac64
test: Update init collection method ( #35596 )
...
Related issue: #32653
Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
2024-08-21 09:22:56 +08:00
80a7c78f28
enhance: import supports null in parquet and json formats ( #35558 )
...
#31728
---------
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-08-20 16:50:55 +08:00
031ee6f155
enhance: support httpv1/v2 throttle and add it for httpV2( #35350 ) ( #35470 )
...
related: #35350
Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
2024-08-20 16:16:55 +08:00
249dc4d9eb
test: Add tests for upsert with auto id ( #35556 )
...
Related issue: #34668
---------
Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
2024-08-20 14:20:56 +08:00
e09dc3be58
enhance: Mark query node as read only after suspend ( #35492 )
...
issue: #34985 #35493
after querynode has been suspended, it's not allow to load
segment/channel on it, which means the node is read only. to be
compatible with resource group design, after query node has been
suspend, we remove it from it's original resource group, make it a read
only query node in replica. then two things will happens:
1. it's original resource group will be lacking of query nodes, query
coord will assign new node to it.
2. querycoord will try to move out all segments/channels after querynode
has been suspended
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-08-20 14:02:54 +08:00
22ced010cd
enhance: make configure load param feature be compatible with old sdk ( #35520 )
...
issue: #31570 #35521
---------
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-08-20 10:30:55 +08:00
6e29d713b3
enhance: [skip e2e]dockerfile for milvus-sdk-go test ( #35426 )
...
https://github.com/milvus-io/milvus/issues/35567
---------
Signed-off-by: Yellow Shine <sammy.huang@zilliz.com>
2024-08-20 09:48:55 +08:00
b4743b4ca8
enhance: [GoSDK] Sync latest milvus proto and mockery ( #35511 )
...
Related to #35443
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-08-19 16:22:55 +08:00
967f38672a
enhance: Add integration tests for l0 ( #35429 )
...
See also: #34796
---------
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-08-19 10:56:54 +08:00
635e9ddb0e
test: fix go sdk cases parallel context deadline ( #35519 )
...
Signed-off-by: ThreadDao <yufen.zong@zilliz.com>
2024-08-16 20:32:53 +08:00
f4a91e135b
enhance: Allow empty sparse row ( #34700 )
...
issue: #29419
* If a sparse vector with 0 non-zero value is inserted, no ANN search on
this sparse vector field will return it as a result. User may retrieve
this row via scalar query or ANN search on another vector field though.
* If the user uses an empty sparse vector as the query vector for a ANN
search, no neighbor will be returned.
Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com>
2024-08-16 14:14:54 +08:00
1d49358f82
enhance: Add BackupRBAC/RestoreRBAC API to enable rbac backup ( #35444 )
...
issue: #35443
---------
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-08-16 10:10:53 +08:00
344dc6a9f8
enhance: enable to set load config in cluster level ( #35169 )
...
issue: #35170
This PR enable to set load configs in cluster level, such as replicas
and resource groups. then when load collections will use the load
config.
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-08-07 12:38:21 +08:00
c19fe95154
fix: support string match for hybrid and bitmap index ( #35294 )
...
#34841
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-08-07 09:54:22 +08:00
a62118ca33
test: update cases for index describe and upsert with autoID ( #35191 )
...
issue: #33419
---------
Signed-off-by: ThreadDao <yufen.zong@zilliz.com>
2024-08-05 18:00:16 +08:00
d8668fe405
enhance: upgrade go version to 1.21.11 ( #35257 )
...
issue: #35142
Signed-off-by: shaoting-huang <shaoting.huang@zilliz.com>
2024-08-05 16:12:15 +08:00
4b553b0333
enhance: revert remove duplicated pk function ( #35103 )
...
issue: #34778
Revert "fix: fix query count(*) concurrently"
Revert "enhance: mark duplicated pk as deleted "
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-08-05 10:48:17 +08:00
eb5461905b
[automated] Update Pytest image changes ( #35244 )
...
Update Pytest image changes
See changes:
241c71fddesre-ci-robot@users.noreply.github.com
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-08-05 09:24:33 +08:00
241c71fdde
enhance: use docker compose instead of docker-compose ( #35208 )
...
https://github.com/milvus-io/milvus/issues/35209
---------
Signed-off-by: Yellow Shine <sammy.huang@zilliz.com>
2024-08-02 19:32:32 +08:00
db06b86594
enhance: Sync otlp dependency version to fix security issue ( #35192 )
...
Related to #34434
otelgrpc DoS vulnerability due to unbound cardinality metrics
https://github.com/milvus-io/milvus/security/dependabot/91
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-08-02 16:22:20 +08:00
196a7986b3
enhance: Change the fixed value to a ratio for clustering segment size ( #35076 )
...
issue: #34495
---------
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-08-01 22:04:14 +08:00
a642a26ed4
enhance: Resolve ChunkFileWriter lint issue ( #35166 )
...
See also #34483
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-08-01 16:52:13 +08:00
5bbb1c201c
enhance:support l2 single compaction ( #34935 )
...
#34928
Signed-off-by: wayblink <anyang.wang@zilliz.com>
2024-08-01 14:36:13 +08:00
dfda9f0478
enhance: Add depguard rules to ban deprecated proto lib ( #35140 )
...
See also #34394 #34252
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-08-01 10:01:49 +08:00
6106a48acb
fix: upsert result use the previous pk ( #34672 )
...
#34668
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-07-31 15:25:51 +08:00
de8a266d8a
enhance: Enable linux code checker ( #35084 )
...
See also #34483
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-07-30 15:53:51 +08:00
a8a4779749
fix: fix parse plan proto failed for search type ( #34944 )
...
#25848
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-07-29 21:19:49 +08:00
86322e0468
fix: fix query count(*) concurrently ( #35007 )
...
#34778
#34849
fix two problems:
1. count(*) incorrect, if growing insert duplicated (pk, timestamp)
pairs that pk and timestamp all same, need to keep just one pair.
2. count(*) may core dump, if get_real_count interface get snapshot and
do mvcc at not consistency status, mainly happens under concurrency.
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-07-29 19:53:50 +08:00
c45f38aa61
enhance: Update protobuf-go to protobuf-go v2 ( #34394 )
...
issue: #34252
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-07-29 11:31:51 +08:00
2372452fac
enhance: Optimized the GC logic to ensure that memory is released in time ( #34949 )
...
issue: #34703
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-07-28 23:53:47 +08:00
783f9d9c33
fix: Unify hook singleton implementation in proxy ( #34887 )
...
Related to #34885
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-07-26 18:07:53 +08:00
4ee6c69217
enhance: Add Segment Level in milvus segment info APIs ( #34763 )
...
See also #34746
This PR add segment level field in response of
`GetPersistentSegmentInfo` and `GetQuerySegmentInfo`
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-07-26 10:01:46 +08:00
482373b3d3
enhance: Sync go e2e package proto dependency ( #34913 )
...
Go e2e package point to a lower verion of milvus-proto and may cause
compile error locally.
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-07-23 16:09:43 +08:00
4c45bc412f
enhance: Add integration test for clustering compaction ( #34881 )
...
issue: #34792
---------
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-07-23 10:13:43 +08:00
c339df26fc
enhance: refine clustering compaction basic it ( #34793 )
...
#34792
Signed-off-by: wayblink <anyang.wang@zilliz.com>
2024-07-22 11:27:51 +08:00
b22e549844
enhance: Rename config of sealing by growing segmetns size ( #34787 )
...
/kind improvement
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-07-19 20:27:41 +08:00
4939f82d4f
enhance: Seal by total growing segments size ( #34692 )
...
Seals the largest growing segment if the total size of growing segments
of each shard exceeds the size threshold(default 4GB). Introducing this
policy can help keep the size of growing segments within a suitable
level, alleviating the pressure on the delegator.
issue: https://github.com/milvus-io/milvus/issues/34554
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-07-17 21:45:41 +08:00
acb33bba4d
enhance: Preserve fixed-size memory in delegator node for growing segment. ( #34596 )
...
issue: #34595
When consuming insert data on the delegator node, QueryCoord will move
out some sealed segments to manage its memory usage. After the growing
segment gets flushed, some sealed segments from other workers will be
moved back to the delegator node. To avoid the frequent movement of
segments, we estimate the maximum growing row count and preserve a
fixed-size memory in the delegator node.
---------
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-07-15 20:51:46 +08:00
eb472b7f08
enhance: [skip e2e]Enable compaction it test ( #34526 )
...
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-07-15 20:45:39 +08:00
2c462d387b
test: Fix index creation mismatch for sparse vector columns in multivector case ( #34618 )
...
If `is_all_data_type` is true, the case will add float32, f16, bf16, and
sparse vectors, but the created indexes are all `flat` indexes by
default. The sparse type cannot create a flat index. Fix the test code
to create a `SPARSE_INVERTED_INDEX` index for the sparse vector when
is_all_data_type is true
Signed-off-by: elstic <hao.wang@zilliz.com>
2024-07-13 10:45:37 +08:00
07b94b4615
enhance: support upsert autoid==true ( #30342 )
...
related with: #29258
---------
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-07-11 16:53:35 +08:00
e4115485b6
test: add cases for gosdk v2 partitions ( #34593 )
...
issue: #33419 #34426
Signed-off-by: ThreadDao <yufen.zong@zilliz.com>
2024-07-11 16:49:36 +08:00
9b37d3f517
enhance: Enable setting the replica number and resource group during collection creation ( #34403 )
...
issue: #30040
---------
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-07-10 10:20:13 +08:00
3333160b8d
enhance: Fix lint issues from recent PRs ( #34482 )
...
See also #34483
Some lint issues are introduced due to lack of static check run. This PR
fixes these problems.
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-07-09 10:06:24 +08:00
ae6d6f91e6
enhance: change autoindex default metric type ( #34261 )
...
issue: #34304
cosine is more widely used in float vectors, and cosine and hamming
distance are 'metrics' which have good geometric properties
Signed-off-by: chasingegg <chao.gao@zilliz.com>
2024-07-08 19:52:24 +08:00
f7898a3ad6
test: update test case ( #34108 )
...
Signed-off-by: elstic <hao.wang@zilliz.com>
2024-07-08 11:46:10 +08:00
67747245f4
test: add test cases for gosdk v2 index ( #34431 )
...
issue: #33419
- Add test cases for gosdk v2 index
- Add sparse index for go client
Signed-off-by: ThreadDao <yufen.zong@zilliz.com>
2024-07-05 09:10:09 +08:00
e4cece8de8
test: add cases for gosdk v2 upsert ( #34400 )
...
- test: add cases for gosdk v2 upsert
- test: update clientv2 to client reference
issue: #33419
Signed-off-by: ThreadDao <yufen.zong@zilliz.com>
2024-07-04 14:26:09 +08:00
0fd0fcfe1d
enhance: Fix lint issues & sdk testcase ( #34399 )
...
Some lint issue is not detect due to recent static check pipeline issue.
This PR fixes these problem and Go milvusclient testcases.
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-07-03 19:42:10 +08:00
d6afb31b94
enhance: make subfunctions of datanode component modular ( #33992 )
...
issue: #33994
also remove deprecated channel manager based on the etcd implementation
Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-07-01 14:46:07 +08:00
00e5a406d4
test: remove dup code ( #34242 )
...
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2024-06-28 10:46:04 +08:00
f2107ad263
test: update test cases about autoindex ( #34224 )
...
Signed-off-by: nico <cheng.yuan@zilliz.com>
2024-06-27 17:06:05 +08:00
be23495233
test: add cases got gosdk v2 search and databases ( #34204 )
...
issue: #33952 #33627 #33419
---------
Signed-off-by: ThreadDao <yufen.zong@zilliz.com>
2024-06-27 09:40:08 +08:00
a08000cfbf
enhance:[skip e2e]update one pod mode resource ( #34196 )
...
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2024-06-26 16:54:05 +08:00
a77e2951af
test: add cases for go milvusclient query ( #33884 )
...
issue: #33419
---------
Signed-off-by: ThreadDao <yufen.zong@zilliz.com>
2024-06-26 09:52:04 +08:00
2cc9df5c50
[automated] Update cpu Builder image changes ( #34078 )
...
Signed-off-by: Liang Huang <sammy.huang@zilliz.com>
2024-06-25 10:56:10 +08:00
4c6f6c54dc
test: refine restful testcases trace ( #34066 )
...
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2024-06-24 10:36:25 +08:00
c85644e1b3
test: modify test case about upsert partition ( #33919 )
...
issue: #33902
Signed-off-by: nico <cheng.yuan@zilliz.com>
2024-06-21 18:50:00 +08:00
f3f390ea43
test: Update tests for range search and add test for query with dup ids ( #34057 )
...
related issue: #33883
Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
2024-06-21 18:48:02 +08:00
03a5f7e6c0
test: update the lib of bf16 ( #34043 )
...
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2024-06-21 14:20:08 +08:00
35ea775c14
enhance: Add rules and fix for go_client e2e code style ( #34033 )
...
See also #31293
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-06-21 10:32:02 +08:00
4e414fb7fc
test: optimizing variable names ( #34035 )
...
Signed-off-by: elstic <hao.wang@zilliz.com>
2024-06-21 10:14:00 +08:00
ce3d7070ac
test: change the number of insertion vectors ( #34014 )
...
pr: https://github.com/milvus-io/milvus/pull/33916
1. Fix case: test_collection_multi_sparse_vectors
2. Reduce the amount of data generated
Signed-off-by: elstic <hao.wang@zilliz.com>
2024-06-20 13:58:05 +08:00
f3d902cf16
test: add test case for bulkwriter ( #33879 )
...
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2024-06-20 09:24:04 +08:00
0264588df9
enhance: update pymilvus version ( #33905 )
...
Signed-off-by: nico <cheng.yuan@zilliz.com>
2024-06-20 09:22:18 +08:00
cc77363b66
enhance: Set maxPartitionNum default value to 1024 ( #33949 )
...
See also #30059
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-06-20 09:11:59 +08:00
1216a4bcd8
test: Add more sparse test cases ( #33916 )
...
issue: https://github.com/milvus-io/milvus/issues/31483
Signed-off-by: elstic <hao.wang@zilliz.com>
2024-06-19 15:24:09 +08:00
b748d8af5d
test: add different datatype for some functions ( #33869 )
...
Signed-off-by: nico <cheng.yuan@zilliz.com>
2024-06-14 17:51:57 +08:00
a963afa0d3
test: refine load state check in restful test ( #33861 )
...
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2024-06-14 16:22:10 +08:00
ca1f7ab019
test: update import test case to support different dim ( #33709 )
...
add test case for https://github.com/milvus-io/milvus/issues/33681
---------
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2024-06-13 17:11:55 +08:00
144ee269f2
fix: [skip e2e] Skip unstable integration test for master ( #33824 )
...
See also #33716 #33823
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-06-13 16:53:55 +08:00
9e99f960ab
test: add cases for go milvusclient delete ( #33777 )
...
issue: #33419
Signed-off-by: ThreadDao <yufen.zong@zilliz.com>
2024-06-13 14:43:57 +08:00
9d4535ce0b
enhance: Handle Float16Vector/BFloat16Vector numpy bulk insert as same as BinaryVector ( #33760 )
...
Issue: #22837
Signed-off-by: Cai Yudong <yudong.cai@zilliz.com>
2024-06-12 17:17:55 +08:00
1697706ac0
test: [skip e2e] optimize get_image_tag_by_short_name function ( #33780 )
...
Refactor the function to improve performance and readability. Instead of
making API requests to Docker Hub, the function now retrieves tags from
the Harbor registry. It also filters the tags based on the provided
architecture and selects the latest tag that matches the prefix. This
change enhances the efficiency of retrieving image tags by short name.
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2024-06-12 14:27:56 +08:00
f7af323d1e
fix: sync partitiion stats blocking balance task( #33741 ) ( #33742 )
...
related: #33741
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
2024-06-11 14:21:56 +08:00
fa26953168
test: fix unstable import case ( #33726 )
...
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2024-06-11 14:03:54 +08:00
b1d46eb34b
fix: Fix multiple vector fields import ( #33723 )
...
1. Fix dim mismatch with multi-vector fields and JSON import
2. Enhance: do not display file ID in GetImportResponse.
issue: https://github.com/milvus-io/milvus/issues/33681 ,
https://github.com/milvus-io/milvus/issues/33682
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-06-10 21:57:54 +08:00
a1232fafda
feat: Major compaction ( #33620 )
...
#30633
Signed-off-by: wayblink <anyang.wang@zilliz.com>
Co-authored-by: MrPresent-Han <chun.han@zilliz.com>
2024-06-10 21:34:08 +08:00
3729d2c8e7
feat: add test case for go client insert ( #33603 )
...
issue: #33419
Signed-off-by: ThreadDao <yufen.zong@zilliz.com>
2024-06-07 14:47:52 +08:00
3540eee977
enhance: Support L0 import ( #33514 )
...
issue: https://github.com/milvus-io/milvus/issues/33157
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-06-07 14:17:20 +08:00
e3d50a192d
test: wait all tasks finished in import test ( #33674 )
...
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2024-06-07 10:35:12 +08:00
9c2e3259d1
test: add coo format sparse vector in restful test ( #33677 )
...
* add coo format sparse vector
* search data and insert data in the same sparse format or a different
format
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2024-06-06 18:05:51 +08:00
27cc9f2630
enhance: Support analyze data ( #33651 )
...
issue: #30633
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
Co-authored-by: chasingegg <chao.gao@zilliz.com>
2024-06-06 17:37:51 +08:00
86274f70bd
test: improve concurrency and reduce import test execution time ( #33356 )
...
improve concurrency and reduce import test execution time
---------
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2024-06-06 15:57:58 +08:00
4ec7cbcae7
test: remove some local config ( #33567 )
...
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2024-06-05 11:27:50 +08:00
35532a3e7d
fix: Fill stats log id and check validity ( #33477 )
...
1. Fill log ID of stats log from import
2. Add a check to validate the log ID before writing to meta
issue: https://github.com/milvus-io/milvus/issues/33476
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-06-05 11:17:56 +08:00
05a80f4def
test: refine file dir in import test ( #33600 )
...
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2024-06-05 10:29:51 +08:00
ac5e098e13
feat: Add e2e test cases for GoSDK ( #33378 )
...
- Add e2e test cases for Go `milvusclient`
- Fix client SparseEmbedding to vector
issue: #33419
Signed-off-by: ThreadDao <yufen.zong@zilliz.com>
2024-06-04 10:57:47 +08:00
c6a1c49e02
enhance: Use Blocked Bloom Filter instead of basic bloom fitler impl. ( #33405 )
...
issue: #32995
To speed up the construction and querying of Bloom filters, we chose a
blocked Bloom filter instead of a basic Bloom filter implementation.
WARN: This PR is compatible with old version bf impl, but if fall back
to old milvus version, it may causes bloom filter deserialize failed.
In single Bloom filter test cases with a capacity of 1,000,000 and a
false positive rate (FPR) of 0.001, the blocked Bloom filter is 5 times
faster than the basic Bloom filter in both querying and construction, at
the cost of a 30% increase in memory usage.
- Block BF construct time {"time": "54.128131ms"}
- Block BF size {"size": 3021578}
- Block BF Test cost {"time": "55.407352ms"}
- Basic BF construct time {"time": "210.262183ms"}
- Basic BF size {"size": 2396308}
- Basic BF Test cost {"time": "192.596229ms"}
In multi Bloom filter test cases with a capacity of 100,000, an FPR of
0.001, and 100 Bloom filters, we reuse the primary key locations for all
Bloom filters to avoid repeated hash computations. As a result, the
blocked Bloom filter is also 5 times faster than the basic Bloom filter
in querying.
- Block BF TestLocation cost {"time": "529.97183ms"}
- Basic BF TestLocation cost {"time": "3.197430181s"}
---------
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-05-31 17:49:45 +08:00
6b51bae5be
test: add different nq for restful v2 test ( #33456 )
...
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2024-05-31 13:57:45 +08:00
3336b91ce6
test: add channel exclusive balance test and resource group test ( #33093 )
...
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2024-05-31 13:55:52 +08:00
4159a4d5d7
test: update nightly ci ( #33430 )
...
update nightly ci
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2024-05-31 13:54:07 +08:00
0c5d8660aa
feat: support inverted index for array ( #33452 )
...
issue: https://github.com/milvus-io/milvus/issues/27704
---------
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2024-05-31 09:47:47 +08:00
b13932bb55
enhance: Enable database level replica num and resource groups for loading collection ( #33052 )
...
issue: #30040
This PR introduce two database level props:
1. database.replica.number
2. database.resource_groups
User can set those two database props by AlterDatabase API, then can
load collection without specified replica_num and resource groups. then
it will use database level load param when try to load collections.
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-05-29 10:59:43 +08:00
5b86226465
test: update test cases about connection ( #33257 )
...
Signed-off-by: nico <cheng.yuan@zilliz.com>
2024-05-27 14:59:42 +08:00
zhuwenxing
foxspy
yihao.dai
Yinzuo Jiang
Buqian Zheng
qixuan
binbin
smellthemoon
sre-ci-robot
yanliang567
jaime
wt
congqixia
nico
ThreadDao
aoiasd
wayblink
Chun Han
Zhen Ye
zhagnlu
wei liu
cai.zhang
OxalisCu
Jiquan Long
yellow-shine
XuanYang-cn
sthuang
elstic
Gao
Cai Yudong