13fdaea9f0
fix: accesslog writer cache close cause deadlock ( #33261 )
...
relate: https://github.com/milvus-io/milvus/issues/33260
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-05-22 14:55:39 +08:00
f336b2d672
fix: Check schema without vector field in proxy ( #33211 )
...
Related to #33199
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-05-21 14:33:39 +08:00
9f81290c63
fix: try best to get enough query results ( #33178 )
...
issue: https://github.com/milvus-io/milvus/issues/33137
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2024-05-21 11:57:51 +08:00
ec98de3ad4
fix: reset the quota value when init the limiter ( #33111 )
...
issue: #33107
/kind improvement
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-05-20 10:35:38 +08:00
5c6de473bb
fix: check array field data is nil before getting the type ( #33114 )
...
issue: https://github.com/milvus-io/milvus/issues/33074
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2024-05-17 17:29:37 +08:00
1671c7898a
enhance: sync milvus proto version ( #33094 )
...
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-05-16 18:51:34 +08:00
b45798107a
enhance: add nullable in Field, check valid_data and fill data ( #32086 )
...
1. add nullable in model.Field
help to read nullable accurately.
2. check valid_data
a. if user pass default_value or the field is nullable, the length of
valid_data must be num_rows.
b. if passed valid_data, the length of passed field data must equal to
the number of 'true' in valid_data.
c. after fill default_value, only nullable field will still has
valid_data.
3. fill data in two situation
a. has no default_value, if nullable,
will fill nullValue when passed num_rows not equal to expected num_rows.
b. has default_value,
will fill default_value when passed num_rows not equal to expected
num_rows.
c. after fill data, the length of all field will equal to passed
num_rows.
#31728
---------
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-05-16 11:57:35 +08:00
875ad88d84
enhance: Support access log use stdout with write cache and auto flush as time ( #32213 )
...
relate: https://github.com/milvus-io/milvus/issues/28948
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-05-16 10:05:34 +08:00
6fc00e42d7
fix: Fix binlog import ( #33027 )
...
binlog import should only import into one partition.
issue: https://github.com/milvus-io/milvus/issues/32807
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-05-14 21:45:32 +08:00
4fc7915c70
enhance: unify data generation test APIs ( #32955 )
...
Issue: #22837
Signed-off-by: Cai Yudong <yudong.cai@zilliz.com>
2024-05-14 14:33:33 +08:00
dc058eaf61
fix: should init access logger before init server ( #32976 )
...
relate: https://github.com/milvus-io/milvus/issues/32968
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-05-14 10:47:32 +08:00
230c88b1de
enhance: improve reduce performance on very large topk ( #32871 )
...
fix #32870
reduce cpu usage for reduce on large k
Signed-off-by: xiaofanluan <xiaofan.luan@zilliz.com>
2024-05-13 11:55:31 +08:00
8a9a42198d
fix: Proxy crash due to shard leader cache data race ( #32971 )
...
issue: #32970
cause InvalidateShardLeaderCache use wrong lock, which may cause data
race in meta cache, then proxy may crash
This PR fixed that use leaderMut when try to access shard leader cache.
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-05-11 14:32:12 +08:00
c35797c399
enhance: expose DescribeDatabase api in proxy ( #32732 )
...
issue: #32707
---------
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-05-09 22:51:30 +08:00
8963615b33
enhance: add the cost response for the rest api ( #32620 )
...
issue: https://github.com/milvus-io/milvus/issues/30436
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-05-08 11:53:29 +08:00
ba02d54a30
enhance: update shard leader cache when leader location changed ( #32470 )
...
issue: #32466
this PR enhance that when shard location changed, update proxy's shard
leader cache. in case of query node failover case, proxy can find
replica recover
---------
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-05-08 10:05:29 +08:00
53874ce245
fix: Fix cannot specify partition name in binlog import ( #32730 )
...
issue: https://github.com/milvus-io/milvus/issues/32807
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-05-07 17:19:30 +08:00
1f58cda957
enhance: add more trace for search & query ( #32734 )
...
issue: https://github.com/milvus-io/milvus/issues/32728
---------
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2024-05-07 13:03:29 +08:00
7da1ca9efb
enhance: check the partition num when creating collection with partition key ( #32670 )
...
issue: #30577
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-05-07 10:43:29 +08:00
48fb8310ba
fix: skip to check auto id when inserting data ( #32775 )
...
/kind improvement
issue: https://github.com/milvus-io/milvus/issues/32591
related pr: #32592
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-05-07 10:17:30 +08:00
02bf47f295
fix: Validate num of rows for insert field data with schema ( #32770 )
...
See also #32769
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-05-07 10:07:29 +08:00
600db9d99e
enhance: check field data type of input ( #32777 )
...
issue: #32769
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2024-05-06 21:15:29 +08:00
fecd9c21ba
feat: LRU cache implementation ( #32567 )
...
issue: https://github.com/milvus-io/milvus/issues/32783
This pr is the implementation of lru cache on branch lru-dev.
Signed-off-by: sunby <sunbingyi1992@gmail.com>
Co-authored-by: chyezh <chyezh@outlook.com>
Co-authored-by: MrPresent-Han <chun.han@zilliz.com>
Co-authored-by: Ted Xu <ted.xu@zilliz.com>
Co-authored-by: jaime <yun.zhang@zilliz.com>
Co-authored-by: wayblink <anyang.wang@zilliz.com>
2024-05-06 20:29:30 +08:00
42d0412e93
enhance: Add channelCPs in FlushResponse ( #32044 )
...
#32609
Signed-off-by: wayblink <anyang.wang@zilliz.com>
2024-04-30 09:45:27 +08:00
0359139299
enhance: return the auth collection list when using the `show collection` api ( #32576 )
...
issue: #32550
/kind improvement
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-04-29 19:09:34 +08:00
d900e68440
fix: fix GetShardLeaders return empty node list ( #32685 )
...
issue: #32449
to avoid GetShardLeaders return empty node list, this PR add node list
check in both client side and server side.
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-04-29 14:19:26 +08:00
09cd56d44f
enhance: add the skip auto id and partition key check config ( #32592 )
...
/kind improvement
issue: #32591
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-04-29 10:29:26 +08:00
07720f1a95
enhance: expose alter database api in proxy ( #32639 )
...
issue: #30040
---------
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-04-28 18:15:31 +08:00
9a719ec89e
fix: skip the empty partition name in the rate limit interceptor ( #32647 )
...
issue: https://github.com/milvus-io/milvus/issues/30577
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-04-28 11:01:28 +08:00
3b5cbed86a
enhance: add db label for deletion and bulkinsert operation ( #32611 )
...
Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-04-26 15:53:29 +08:00
04f355a802
enhance: Enable alter database props in rootcoord ( #32458 )
...
issue: #30040
---------
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-04-25 10:53:25 +08:00
dcc15e3ed8
enhance: Remove `TestRateLimiter/tests_refresh_rate_by_config` case ( #32545 )
...
See also #32313
This case always fail since the watch config logic is removed
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-04-24 10:13:24 +08:00
365e50b63e
fix: revert add range search params check in proxy ( #32366 )
...
no need to check params in empty segment.
#30365
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-04-23 17:41:23 +08:00
3119a7b23f
fix: Remove watching config event in rate limiter ( #32313 )
...
Remove watching config event in rate limiter to prevent object leaks.
Instead, reset limiter rates periodically.
issue: https://github.com/milvus-io/milvus/issues/32312
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-04-23 16:19:24 +08:00
d0ffb71a17
enhance: forbid delete with always true expression ( #32472 )
...
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-04-22 11:09:23 +08:00
8594b55ad5
enhance: add `max insert request size` and `must use partition key` configs ( #32433 )
...
issue: https://github.com/milvus-io/milvus/issues/30577
/kind improvement
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-04-19 10:31:20 +08:00
31a29a2451
enhance: avoid the repeated metric info in the proxy ( #32380 )
...
issue: #30577
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-04-19 10:21:20 +08:00
02f17b842a
fix: fix incomplete hybrid search result when nq > 1 ( #32177 )
...
issue: https://github.com/milvus-io/milvus/issues/25639
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2024-04-17 17:09:32 +08:00
84f05ba66e
enhance: make the auth error message more suitable ( #32253 )
...
/issue: #32252
/kind improvement
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-04-16 16:47:19 +08:00
1af084ea6b
enhance: Make datanode exit and case `TestProxy` faster ( #32218 )
...
/kind improvement
issue: #32219
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-04-16 10:49:20 +08:00
48fe977a9d
enhance: declarative resource group api ( #31930 )
...
issue: #30647
- Add declarative resource group api
- Add config for resource group management
- Resource group recovery enhancement
---------
Signed-off-by: chyezh <chyezh@outlook.com>
2024-04-15 08:13:19 +08:00
88c6828d6c
fix: failed to raise metric_type not match error ( #32202 )
...
issue: https://github.com/milvus-io/milvus/issues/32176
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2024-04-12 16:19:18 +08:00
c012e6786f
feat: support rate limiter based on db and partition levels ( #31070 )
...
issue: https://github.com/milvus-io/milvus/issues/30577
co-author: @jaime0815
---------
Signed-off-by: Patrick Weizhi Xu <weizhi.xu@zilliz.com>
Signed-off-by: SimFG <bang.fu@zilliz.com>
Co-authored-by: Patrick Weizhi Xu <weizhi.xu@zilliz.com>
2024-04-12 16:01:19 +08:00
fb376fd1e6
enhance: Refine varchar length exceeds max length error message ( #31521 )
...
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-04-12 15:01:19 +08:00
43a9be2cce
enhance: support access log dynamic config and prepare for support resutful interface access log ( #29638 )
...
relate: https://github.com/milvus-io/milvus/issues/29637
https://github.com/milvus-io/milvus/issues/31823
---------
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-04-12 10:37:27 +08:00
0bfe130991
fix: fix metric about ProxySearchVectors ( #32152 )
...
issue :https://github.com/milvus-io/milvus/issues/25639
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2024-04-11 16:13:17 +08:00
52ae47c850
enhance: gather materialized view search info once per request ( #31996 )
...
issue: #29892
This PR:
1. Move the process of gathering materialized search info to when the
search plan is created, before it goes to each segment, to avoid
repeated work and access the plan node under multi-threaded
circumstances.
2. Enforce the supported MV type to `VARCHAR`
3. Add integration test
Signed-off-by: Patrick Weizhi Xu <weizhi.xu@zilliz.com>
2024-04-11 15:21:19 +08:00
2fdf1a6e76
feat: [Sparse Float Vector] added some integration tests ( #31062 )
...
add some integration tests for sparse float vector support
https://github.com/milvus-io/milvus/issues/29419
Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com>
2024-04-10 19:57:18 +08:00
dbab9c5096
fix: reduce didn't handle offset without limit and reduceStopForBest correctly ( #32089 )
...
fix https://github.com/milvus-io/milvus/issues/32059
this pr fix two issues:
offset is not handled correctly without specify a limit
reduceStopForBest doesn't guarantee to return limit result even if there
are more result when there is small segment
Signed-off-by: xiaofanluan <xiaofan.luan@zilliz.com>
2024-04-10 16:01:18 +08:00
90bed1caf9
enhance: add the related data size for the read apis ( #31816 )
...
issue: #30436
origin pr: #30438
related pr: #31772
---------
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-04-10 15:07:17 +08:00
aoiasd
congqixia
Jiquan Long
SimFG
Bingyi Sun
smellthemoon
yihao.dai
Cai Yudong
Xiaofan
wei liu
wayblink
jaime
zhenshan.cao
chyezh
Patrick Weizhi Xu
Buqian Zheng