Commit Graph

3094 Commits (update_knowhere_commit_83c902992c801d0579964518d2dec8805d4b5051)

Author SHA1 Message Date
nico 866d512f6f
test: update test cases and sdk version (#37774)
Signed-off-by: nico <cheng.yuan@zilliz.com>
2024-11-19 14:14:32 +08:00
cai.zhang dae4160466
enhance: Whether to enable mergeSort mode when performing mixCompaction (#37664)
issue: #37579

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-11-19 11:28:31 +08:00
sthuang 2d72ad33f2
enhance: RBAC built in privilege groups (#37720)
issue: #37031

Signed-off-by: shaoting-huang <shaoting.huang@zilliz.com>
2024-11-18 20:38:39 +08:00
zhuwenxing 3cdb485022
test: update timeout of load in ci cases (#37596)
action for
https://github.com/milvus-io/milvus/issues/37166#issuecomment-2469502955

Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2024-11-16 16:00:31 +08:00
zhuwenxing 3f7352f3cf
test: remove xfail of fts test cases after fix (#37724)
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2024-11-16 11:14:30 +08:00
wei liu a1b6be1253
fix: Delegator stuck at unserviceable status (#37694)
issue: #37679

pr #36549 introduce the logic error which update current target when
only parts of channel is ready.

This PR fix the logic error and let dist handler keep pull distribution
on querynode until all delegator becomes serviceable.

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-11-15 10:20:31 +08:00
foxspy 5ae347aba0
enhance: update knowhere version (#37688)
issue: #37665 #37631 #37620 #37587 #36906 
knowhere has add default nlist value, so some invalid param test ut with
no nlist param will be valid.

Signed-off-by: xianliang.li <xianliang.li@zilliz.com>
2024-11-15 10:10:31 +08:00
nico 405c12f7d1
test: update test cases (#37660)
Signed-off-by: nico <cheng.yuan@zilliz.com>
2024-11-14 17:46:39 +08:00
yanliang567 af433ffd68
test: Add expression template tests and use error msg only as assertion (#37618)
related issue: https://github.com/milvus-io/milvus/issues/37451

---------

Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
2024-11-13 17:14:30 +08:00
smellthemoon a654487995
fix: some error in restful (#37487)
#37370

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-11-13 17:12:39 +08:00
nico 2630717836
test: update sdk version and cases (#37507)
Signed-off-by: nico <cheng.yuan@zilliz.com>
2024-11-12 17:48:28 +08:00
jaime 1e8ea4a7e7
feat: add segment/channel/task/slow query render (#37561)
issue: #36621

Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-11-12 17:44:29 +08:00
Bingyi Sun 2742e9573f
enhance: Print results if a test fails (#37606)
test_hybrid_search_different_limit_round_decimal fails occasionally. 
Print more details for debug.

---------

Signed-off-by: sunby <sunbingyi1992@gmail.com>
2024-11-12 17:12:28 +08:00
wei liu 266f8ef1f5
fix: Search may return less result after qn recover (#36549)
issue: #36293 #36242
after qn recover, delegator may be loaded in new node, after all segment
has been loaded, delegator becomes serviceable. but delegator's target
version hasn't been synced, and if search/query comes, delegator will
use wrong target version to filter out a empty segment list, which
caused empty search result.

This pr will block delegator's serviceable status until target version
is synced

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-11-12 16:34:28 +08:00
Zhen Ye b5b003551e
enhance: use localhost for it and ut (#37529)
issue: #37528

Signed-off-by: chyezh <chyezh@outlook.com>
2024-11-12 11:36:27 +08:00
XuanYang-cn a45a288a25
fix: Separate L0 and Mix trigger interval (#37190)
See also: #37108

- Add MixCompactionTriggerInterval, default 60s
- Add L0CompactionTriggerInterval, default 10s
- Export Single related compaction configs
- Raise SingleCompactionDeltaLogMaxSize from 2MB to 16MB

---------

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-11-12 10:56:37 +08:00
Chun Han 2d29dcd30c
enhance:refine group_strict_size parameter(#37482) (#37483)
related: #37482

Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
2024-11-12 09:56:28 +08:00
binbin 21b68029a0
test: Add nullable test cases for bulk writer (#37572)
issue: #36129

Signed-off-by: binbin lv <binbin.lv@zilliz.com>
2024-11-12 09:46:28 +08:00
congqixia 83d2729f94
enhance: Generate milvusclient mockery with latest proto (#37567)
Related to #37558

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-11-11 14:28:30 +08:00
ThreadDao f42869c0ce
test: remove case skip label (#37539)
[test] remove case skip label
issue: #33952

Signed-off-by: ThreadDao <yufen.zong@zilliz.com>
2024-11-10 16:14:27 +08:00
aoiasd 12951f0abb
enhance: rename tokenizer to analyzer and check analyzer params (#37478)
relate: https://github.com/milvus-io/milvus/issues/35853

---------

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-11-10 16:12:26 +08:00
sthuang ff00a12805
enhance: RBAC custom privilege group ut coverage (#37558)
issue: https://github.com/milvus-io/milvus/issues/37031

Signed-off-by: shaoting-huang <shaoting.huang@zilliz.com>
2024-11-09 20:40:25 +08:00
zhuwenxing ffdde391e3
test: fix unstable testcase caused by server refactor (#37545)
issue: https://github.com/milvus-io/milvus/issues/37533

Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2024-11-09 09:58:26 +08:00
sthuang 70605cf5b3
enhance: Support custom privilege group for RBAC (#37087)
issue: #37031

---------

Signed-off-by: shaoting-huang <shaoting.huang@zilliz.com>
2024-11-09 08:44:28 +08:00
cai.zhang ae227e3934
enhance: Add integration test for stats task (#37506)
issue: #33744

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-11-08 15:48:26 +08:00
congqixia 11f4fe0177
enhance: [GoSDK] move client pkg go files to sub one (#37492)
Related to #31293

Client source files under client pkg cannot be evaluate correctly by
codecov. This PR moves them to `milvusclient` sub-package to fix this
issue and follow go major version best practice.

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-11-08 07:32:26 +08:00
smellthemoon 86fd3200be
enhance: refactor createIndex in RESTful API (#37235)
Make the parameter input method consistent with miluvs-client.

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-11-07 14:18:30 +08:00
Buqian Zheng 40b770cb7b
fix: add rescorer activation function for BM25 (#37481)
issue: #35853

Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com>
2024-11-07 12:08:25 +08:00
cai.zhang aed3b94b5d
enhance: Refine error message for contains array (#37383)
issue: #36221

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-11-07 10:44:25 +08:00
wei liu 7cfd609ebc
fix: [skip e2e]unstable integration test TestNodeDownOnSingleReplica (#37480)
issue: #37289
cause pr #37116 introduce retry on get shard leader, which make search
won't fail during query node down.

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-11-06 17:08:21 -08:00
nico cf66fbee8a
test: update test cases (#37215)
Signed-off-by: nico <cheng.yuan@zilliz.com>
2024-11-06 17:50:25 +08:00
aoiasd d67853fa89
feat: Tokenizer support build with params and clone for concurrency (#37048)
relate: https://github.com/milvus-io/milvus/issues/35853
https://github.com/milvus-io/milvus/issues/36751

---------

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-11-06 17:48:24 +08:00
congqixia c83b93946e
enhance: [GoSDK] Add alter collection API & expose options (#37365)
Related to #31293

This PR:

- Add `AlterCollection` API for collection property modification
- Expose hidden or missing option methods

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-11-06 14:36:25 +08:00
zhuwenxing 0fc6c634b0
test: fix tokenizer and monkey patch faker function (#37119)
/kind improvement

---------

Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
Signed-off-by: zhuwenxing <wxzhuyeah@gmail.com>
2024-11-05 08:42:23 +08:00
cai.zhang ba9f36ba33
enhance: Increase the concurrency of index tasks in CI (#37335)
#37096

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-11-04 12:04:23 +08:00
foxspy eaf86f7649
fix: optimize invalid datatype error msg (#37376)
issue: #37151

Signed-off-by: xianliang.li <xianliang.li@zilliz.com>
2024-11-04 11:46:28 +08:00
cai.zhang 50de122dc7
enhance: Rename textmatch to text_match (#37290)
issue: #36672

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-11-03 18:40:27 +08:00
congqixia be71b98146
enhance: Fix case error msg after knowhere upgrade (#37339)
Previous PR: #37315

Error message updated after knowhere behavior change, this PR update
corresponding test error message.

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-11-01 14:00:22 +08:00
wei liu f190e5d802
fix: [skip e2e] TestNodeDownOnSingleReplica has unstable result (#37288)
issue: #37289
those test case use search to verify replica's status, but if the search
gap is 1s, the node down's effect may be fixed up by balance.

This PR remove the 1 second gap between search operation.

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-11-01 10:50:21 +08:00
zhuwenxing 247f75180f
test: add restful cases for full text search and some minor fix (#37148)
/kind improvement

---------

Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2024-10-31 21:18:23 +08:00
zhuwenxing d24970c090
test: fix potential unstable cases of full-text search (#37328)
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2024-10-31 21:12:21 +08:00
zhenshan.cao 63843dce33
fix: Fix conan gdal building problem (#37338)
issue:https://github.com/milvus-io/milvus/issues/27576

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2024-10-31 21:04:16 +08:00
Hao Tan 67c4340565
feat: Geospatial Data Type and GIS Function Support for milvus server (#35990)
issue:https://github.com/milvus-io/milvus/issues/27576

# Main Goals
1. Create and describe collections with geospatial fields, enabling both
client and server to recognize and process geo fields.
2. Insert geospatial data as payload values in the insert binlog, and
print the values for verification.
3. Load segments containing geospatial data into memory.
4. Ensure query outputs can display geospatial data.
5. Support filtering on GIS functions for geospatial columns.

# Solution
1. **Add Type**: Modify the Milvus core by adding a Geospatial type in
both the C++ and Go code layers, defining the Geospatial data structure
and the corresponding interfaces.
2. **Dependency Libraries**: Introduce necessary geospatial data
processing libraries. In the C++ source code, use Conan package
management to include the GDAL library. In the Go source code, add the
go-geom library to the go.mod file.
3. **Protocol Interface**: Revise the Milvus protocol to provide
mechanisms for Geospatial message serialization and deserialization.
4. **Data Pipeline**: Facilitate interaction between the client and
proxy using the WKT format for geospatial data. The proxy will convert
all data into WKB format for downstream processing, providing column
data interfaces, segment encapsulation, segment loading, payload
writing, and cache block management.
5. **Query Operators**: Implement simple display and support for filter
queries. Initially, focus on filtering based on spatial relationships
for a single column of geospatial literal values, providing parsing and
execution for query expressions.
6. **Client Modification**: Enable the client to handle user input for
geospatial data and facilitate end-to-end testing.Check the modification
in pymilvus.

---------

Signed-off-by: tasty-gumi <1021989072@qq.com>
2024-10-31 20:58:20 +08:00
cai.zhang 2ef6cbbf59
feat: The expression supports filling elements through templates (#37033)
issue: #36672

The expression supports filling elements through templates, which helps
to reduce the overhead of parsing the elements.

---------

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-10-31 14:20:22 +08:00
zhuwenxing 6e37372619
test: update checker (#37275)
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2024-10-31 09:50:20 +08:00
yanliang567 3a3404658e
test: Add a test for iterator enhancement (#37296)
related issue: https://github.com/milvus-io/milvus/issues/37084

Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
2024-10-30 16:54:20 +08:00
Zhen Ye 889434691c
enhance: enable asan for e2e (#37149)
issue: #35854

---------

Signed-off-by: chyezh <chyezh@outlook.com>
2024-10-29 14:14:24 +08:00
zhuwenxing 4c108b1564
test: update jieba tokenizer in test (#37199)
/kind improvement

---------

Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2024-10-28 19:22:22 +08:00
zhuwenxing cdee149191
test: fix testcases for verification after chaos (#37153)
/kind improvement

Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2024-10-28 10:33:29 +08:00
zhuwenxing c8dd665bf6
test: supplementing case for text match (#36693)
/kind improvement

Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2024-10-28 10:31:40 +08:00