milvus-io
/
milvus
mirror of https://github.com/milvus-io/milvus.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
19,552 Commits
153 Branches
122 Tags
593 MiB
Commit Graph

19552 Commits (ab6ddf6929a18ee3dce3844521db8e55414fe812)

Author SHA1 Message Date
sammy.huang ab6ddf6929
enhance: [skip e2e] use pull_request_target event instead (#32236) 
issue: #32235

Signed-off-by: Liang Huang <sammy.huang@zilliz.com>
 2024-04-13 21:17:17 +08:00
congqixia 674991d07c
fix: Wait StandBy server ready for testcase (#32216) 
See also #32069

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-04-12 23:49:18 +08:00
edward.zeng b7ff85638d
fix: mvcc database space exceeded for embed etcd (#32048) 
Fix #30314

Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
 2024-04-12 21:39:19 +08:00
SimFG 2944971507
enhance: fix the dev docker yaml and use `go install` to install the gotestsum (#32197) 
/kind improvement

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2024-04-12 19:17:18 +08:00
Cai Yudong 06e0c8baac
fix: fix estimate float16 field data size wrong (#32193) 
Issue: #32192

Signed-off-by: Cai Yudong <yudong.cai@zilliz.com>
 2024-04-12 16:29:26 +08:00
zhenshan.cao 88c6828d6c
fix: failed to raise metric_type not match error (#32202) 
issue: https://github.com/milvus-io/milvus/issues/32176

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2024-04-12 16:19:18 +08:00
SimFG c012e6786f
feat: support rate limiter based on db and partition levels (#31070) 
issue: https://github.com/milvus-io/milvus/issues/30577
co-author: @jaime0815

---------

Signed-off-by: Patrick Weizhi Xu <weizhi.xu@zilliz.com>
Signed-off-by: SimFG <bang.fu@zilliz.com>
Co-authored-by: Patrick Weizhi Xu <weizhi.xu@zilliz.com>
 2024-04-12 16:01:19 +08:00
aoiasd fb376fd1e6
enhance: Refine varchar length exceeds max length error message (#31521) 
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2024-04-12 15:01:19 +08:00
XuanYang-cn 4617d22482
enhance: Use channel manager interface in server_test (#31621) 
Tidy the following test codes

    - Remove channel in newTestServer
    - Remove newTestServerWithMeta
    - Remove newTestServer2
    - Remove testDataCoordBase
    - Use the same func for handleTTmsg and handleRPCTTmsg

See also: #31620

---------

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2024-04-12 14:59:20 +08:00
Buqian Zheng 33801c32c4
fix: in milvus check sparse index to be less than uint32 max (#32199) 
issue: #29419

Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com>
 2024-04-12 14:25:19 +08:00
congqixia c0fa169d9a
enhance: Make write buffer memory check do until safe (#32172) 
See also #27675 #26177

Make memory check evict memory buffer until memory water level is safe.
Also make `EvictBuffer` wait until sync task done.

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-04-12 10:55:18 +08:00
wei liu 68dec7dcd4
fix: Use correct ts to avoid exclude segment list leak (#31991) 
issue: #31990

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-04-12 10:39:19 +08:00
aoiasd 43a9be2cce
enhance: support access log dynamic config and prepare for support resutful interface access log (#29638) 
relate: https://github.com/milvus-io/milvus/issues/29637
https://github.com/milvus-io/milvus/issues/31823

---------

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2024-04-12 10:37:27 +08:00
sre-ci-robot 454984aa4e
[automated] Update Knowhere Commit (#32181) 
Update Knowhere Commit
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 2024-04-12 01:25:19 +08:00
jaime 371e6d2c1a
enhance: refine sync memory watermark configuration (#32140) 
Signed-off-by: jaime <yun.zhang@zilliz.com>
 2024-04-11 20:07:24 +08:00
Alexander Guzhva b5455d176e
fix: dynamically resolve whether SVE is available for bitset (#32137) 
Issue: #32129 
This PR adds a dynamic SVE detection for ARM CPU families for the bitset
code.
Also, allows the code to be compiled if the compiler does not support
NEON (arm-v7).

Signed-off-by: Alexandr Guzhva <alexanderguzhva@gmail.com>
 2024-04-11 19:27:18 +08:00
Ted Xu 3d5fe7b45c
feat: adding cache stats support (#32068) 
See #32067

---------

Signed-off-by: Ted Xu <ted.xu@zilliz.com>
 2024-04-11 19:19:18 +08:00
sre-ci-robot 5e8c580fa1
[automated] Update Pytest image changes (#32148) 
Update Pytest image changes
See changes:
4bf32bbc87
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 2024-04-11 19:07:18 +08:00
congqixia b9a487608a
fix: Make `ResourceGroup.nodes` concurrent safe (#32159) 
See also #32158

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-04-11 17:53:18 +08:00
zhenshan.cao 0bfe130991
fix: fix metric about ProxySearchVectors (#32152) 
issue :https://github.com/milvus-io/milvus/issues/25639

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2024-04-11 16:13:17 +08:00
jaime 2c63f848bf
fix: upgrade nats server to fix security vulnerabilities (#32021) 
issue: #32022

Signed-off-by: jaime <yun.zhang@zilliz.com>
 2024-04-11 16:07:18 +08:00
Patrick Weizhi Xu 52ae47c850
enhance: gather materialized view search info once per request (#31996) 
issue: #29892 

This PR:
1. Move the process of gathering materialized search info to when the
search plan is created, before it goes to each segment, to avoid
repeated work and access the plan node under multi-threaded
circumstances.
2. Enforce the supported MV type to `VARCHAR`
3. Add integration test

Signed-off-by: Patrick Weizhi Xu <weizhi.xu@zilliz.com>
 2024-04-11 15:21:19 +08:00
XuanYang-cn 4bf32bbc87
test: Update pymilvus version to 2.5.0rc8 (#32111) 
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2024-04-11 11:03:18 +08:00
Ted Xu dc5ea6f17c
feat: adding binlog streaming writer (#31537) 
See #31679

---------

Signed-off-by: Ted Xu <ted.xu@zilliz.com>
 2024-04-11 10:33:20 +08:00
yihao.dai 273df98e20
enhance: Add binlog import intergration test (#32112) 
issue: https://github.com/milvus-io/milvus/issues/28521

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-04-11 10:31:18 +08:00
congqixia a938951eea
enhance: [skip e2e] fix upload code cov if syntax (#32128) 
The re-upload step is always true since the syntax is wrong

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-04-11 10:05:19 +08:00
SimFG 789e014c74
enhance: add the db id for the describe collection response (#32114) 
/kind improvement
issue: #32110

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2024-04-10 21:23:20 +08:00
jaime d4fd6c7283
enhance: add db label on binlog size metrics (#32003) 
Signed-off-by: jaime <yun.zhang@zilliz.com>
 2024-04-10 21:01:20 +08:00
wei liu 1a98ce39f5
enhance: Remove useless logic about FromShardLeader (#32029) 
issue: #32047

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-04-10 20:11:19 +08:00
Buqian Zheng 2fdf1a6e76
feat: [Sparse Float Vector] added some integration tests (#31062) 
add some integration tests for sparse float vector support

https://github.com/milvus-io/milvus/issues/29419

Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com>
 2024-04-10 19:57:18 +08:00
congqixia 25a1c9ecf0
fix: Make coordinator `Register` not blocked on ProcessActiveStandby (#32069) 
See also #32066

This PR make coordinator register successful and let
`ProcessActiveStandBy` run async. And roles may receive stop signal and
notify servers.

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-04-10 18:49:18 +08:00
zhuwenxing a697e80a3f
test: use strong as default consistency level (#32095) 
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
 2024-04-10 17:55:18 +08:00
zhuwenxing 5ddeec877a
test: fix vulnerability of python lib (#32109) 
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
 2024-04-10 17:53:18 +08:00
chyezh a3d6110957
fix: ut failure (#32120) 
issue: #30647

Signed-off-by: chyezh <chyezh@outlook.com>
 2024-04-10 17:30:48 +08:00
yihao.dai 1b5554c8cb
enhance: Support $meta key for json import (#32013) 
During JSON import:
1. Allow the specification of the $meta key
2. Prohibit duplicated keys within the $meta field, for instance,
`{"id": 1, "vector": [], "x": 6, "$meta": {"x": 8}}`

issue: https://github.com/milvus-io/milvus/issues/31835

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-04-10 17:27:17 +08:00
chyezh 0be67e7f99
fix: ut failure (#32119) 
issue: #30647

Signed-off-by: chyezh <chyezh@outlook.com>
 2024-04-10 17:23:27 +08:00
Xiaofan dbab9c5096
fix: reduce didn't handle offset without limit and reduceStopForBest correctly (#32089) 
fix https://github.com/milvus-io/milvus/issues/32059

this pr fix two issues:

offset is not handled correctly without specify a limit
reduceStopForBest doesn't guarantee to return limit result even if there
are more result when there is small segment

Signed-off-by: xiaofanluan <xiaofan.luan@zilliz.com>
 2024-04-10 16:01:18 +08:00
wei liu df208d538c
fix: Check exclude segment before add new growing segment (#31803) 
issue: #31479 #31797

milvus will add released segment to excluded info, and filter out it's
stream data in filter_node. but for data buffered in insert_node's
channel, if it belongs to growing segment which already be released,
then it will all the growing segment back again.

This PR maintain `excluded segments` in delegator, and check excluded
segment before new growing segment.

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-04-10 15:29:17 +08:00
Chun Han f3f2a5a7e9
fix: evicted segments in the serverlss mode(#31959) (#31961) 
related: #31959
1. reset segment index status after evicting to lazyload=true
2. reset num_rows to null_opt

Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
 2024-04-10 15:15:19 +08:00
wei liu c4806b69c4
enhance: Refactor leader view manager interface (#31133) 
issue: #31091
This PR add GetByFilter interface in leader view manager, instead of all
kind of get func

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-04-10 15:13:36 +08:00
XuanYang-cn aad3ed3835
fix: [cherry-pick]Skip changing meta if nodeID not match with channel (#31672) 
See also: #31648
pr: #31665, #31694

---------

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2024-04-10 15:09:18 +08:00
SimFG 90bed1caf9
enhance: add the related data size for the read apis (#31816) 
issue: #30436
origin pr: #30438
related pr: #31772

---------

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2024-04-10 15:07:17 +08:00
chyezh c9faa6d936
enhance: add more metrics for cache and search (#31777) 
issue: #30931

---------

Signed-off-by: chyezh <chyezh@outlook.com>
 2024-04-10 10:55:17 +08:00
congqixia 9d16aa0bd3
enhance: Bump jose2go for security alerts (#32040) 
See also #31986

- jose2go vulnerable to denial of service via large p2c value

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-04-10 10:05:19 +08:00
SimFG a63cf8625c
enhance: support to invalid the database meta cache (#32078) 
issue: #32077
/kind improvement

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2024-04-10 06:45:18 +08:00
wei liu c5a9cae44e
enhance: [skip e2e]remove useless suspend/resume gc operation in integration test (#31954) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-04-09 19:55:17 +08:00
zhuwenxing 36514d431a
test: fix some negative case for restful v2 (#32056) 
fix some negative case for restful v2
* fix some error code and message
* fix some payload

Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
 2024-04-09 18:03:23 +08:00
zhuwenxing 67160e77e0
test: increase the timeout of import (#32064) 
increase the timeout of import， since indexing time is also included

Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
 2024-04-09 17:55:16 +08:00
SimFG 420baacb6d
enhance: use the rate log in the privilege interceptor (#32037) 
/kind improvement

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2024-04-09 16:19:17 +08:00
Cai Yudong a0a4ec8b67
enhance: make range search param check message more meaningful (#32006) 
Issue: #31970

Signed-off-by: Cai Yudong <yudong.cai@zilliz.com>
 2024-04-09 16:17:26 +08:00