b9bafc76b4
enhance: access log support print with consistency level ( #33503 )
...
relate: https://github.com/milvus-io/milvus/issues/33563
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-06-05 17:13:50 +08:00
25080bb455
enhance: Print `UseDefaultConsistency` param in read requests ( #33617 )
...
`UseDefaultConsistency` param is crucial for debugging slow query
problems. It could be confusing when guarantee timestamp is 1 while this
param is not logged
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-06-05 10:27:50 +08:00
0cf225fa75
enhance: access log support restful api ( #33155 )
...
relate: https://github.com/milvus-io/milvus/issues/31823
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-05-31 11:13:44 +08:00
416a2cf507
fix: query iterator lack results( #33137 ) ( #33422 )
...
related: #33137
adding has_more_result_tag for various level's reduce to rectify
reduce_stop_for_best
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
2024-05-30 17:51:44 +08:00
589d4dfd82
enhance: optimize bitmap index ( #33358 )
...
#32900
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-05-30 13:09:43 +08:00
b13932bb55
enhance: Enable database level replica num and resource groups for loading collection ( #33052 )
...
issue: #30040
This PR introduce two database level props:
1. database.replica.number
2. database.resource_groups
User can set those two database props by AlterDatabase API, then can
load collection without specified replica_num and resource groups. then
it will use database level load param when try to load collections.
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-05-29 10:59:43 +08:00
a59ed65c48
doc: [skip-e2e] add comments for task queue ( #33388 )
...
Add comments for task queue in internal/proxy/task_scheduler.go.
Signed-off-by: ArenaSu <704427617@qq.com>
2024-05-27 20:51:42 +08:00
af71116499
fix: Fix frequent 'failed to get request info' log ( #33334 )
...
issue: https://github.com/milvus-io/milvus/issues/33333
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-05-27 14:27:41 +08:00
cb99e3db34
enhance: add the includeCurrentMsg param for the Seek method ( #33326 )
...
/kind improvement
- issue: #33325
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-05-27 10:31:41 +08:00
2964f60edc
enhance: the proxy metric in the query request ( #33307 )
...
/kind improvement
issue: #33306
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-05-24 14:19:41 +08:00
592d701617
fix: Fix global rate limit is not working ( #33335 )
...
If the request is limited by rate limiter, limiter should not "Cancel".
This is because, if limited, tokens are not deducted; instead, "Cancel"
operation would increase the token count.
issue: https://github.com/milvus-io/milvus/issues/31705
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-05-24 09:05:47 +08:00
1b4e28b97f
enhance: Check by proxy rate limiter when delete get data by query. ( #30891 )
...
relate: https://github.com/milvus-io/milvus/issues/30927
---------
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-05-23 20:03:40 +08:00
155cb40fc9
fix: Fix printing type of request ( #33289 )
...
issue: https://github.com/milvus-io/milvus/issues/31705
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-05-23 14:19:40 +08:00
9ff023ee35
fix: Fix filtering by partition key fails for importing data ( #33274 )
...
Before executing the import, partition IDs should be reordered according
to partition names. Otherwise, the data might be hashed to the wrong
partition during import. This PR corrects this error.
issue: https://github.com/milvus-io/milvus/issues/33237
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-05-23 11:13:40 +08:00
dd0c6d6980
fix: the panic when db isn't existed in the rate limit interceptor ( #33244 )
...
issue: #33243
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-05-22 15:57:39 +08:00
13fdaea9f0
fix: accesslog writer cache close cause deadlock ( #33261 )
...
relate: https://github.com/milvus-io/milvus/issues/33260
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-05-22 14:55:39 +08:00
f336b2d672
fix: Check schema without vector field in proxy ( #33211 )
...
Related to #33199
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-05-21 14:33:39 +08:00
9f81290c63
fix: try best to get enough query results ( #33178 )
...
issue: https://github.com/milvus-io/milvus/issues/33137
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2024-05-21 11:57:51 +08:00
ec98de3ad4
fix: reset the quota value when init the limiter ( #33111 )
...
issue: #33107
/kind improvement
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-05-20 10:35:38 +08:00
5c6de473bb
fix: check array field data is nil before getting the type ( #33114 )
...
issue: https://github.com/milvus-io/milvus/issues/33074
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2024-05-17 17:29:37 +08:00
1671c7898a
enhance: sync milvus proto version ( #33094 )
...
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-05-16 18:51:34 +08:00
b45798107a
enhance: add nullable in Field, check valid_data and fill data ( #32086 )
...
1. add nullable in model.Field
help to read nullable accurately.
2. check valid_data
a. if user pass default_value or the field is nullable, the length of
valid_data must be num_rows.
b. if passed valid_data, the length of passed field data must equal to
the number of 'true' in valid_data.
c. after fill default_value, only nullable field will still has
valid_data.
3. fill data in two situation
a. has no default_value, if nullable,
will fill nullValue when passed num_rows not equal to expected num_rows.
b. has default_value,
will fill default_value when passed num_rows not equal to expected
num_rows.
c. after fill data, the length of all field will equal to passed
num_rows.
#31728
---------
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-05-16 11:57:35 +08:00
875ad88d84
enhance: Support access log use stdout with write cache and auto flush as time ( #32213 )
...
relate: https://github.com/milvus-io/milvus/issues/28948
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-05-16 10:05:34 +08:00
6fc00e42d7
fix: Fix binlog import ( #33027 )
...
binlog import should only import into one partition.
issue: https://github.com/milvus-io/milvus/issues/32807
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-05-14 21:45:32 +08:00
4fc7915c70
enhance: unify data generation test APIs ( #32955 )
...
Issue: #22837
Signed-off-by: Cai Yudong <yudong.cai@zilliz.com>
2024-05-14 14:33:33 +08:00
dc058eaf61
fix: should init access logger before init server ( #32976 )
...
relate: https://github.com/milvus-io/milvus/issues/32968
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-05-14 10:47:32 +08:00
230c88b1de
enhance: improve reduce performance on very large topk ( #32871 )
...
fix #32870
reduce cpu usage for reduce on large k
Signed-off-by: xiaofanluan <xiaofan.luan@zilliz.com>
2024-05-13 11:55:31 +08:00
8a9a42198d
fix: Proxy crash due to shard leader cache data race ( #32971 )
...
issue: #32970
cause InvalidateShardLeaderCache use wrong lock, which may cause data
race in meta cache, then proxy may crash
This PR fixed that use leaderMut when try to access shard leader cache.
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-05-11 14:32:12 +08:00
c35797c399
enhance: expose DescribeDatabase api in proxy ( #32732 )
...
issue: #32707
---------
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-05-09 22:51:30 +08:00
8963615b33
enhance: add the cost response for the rest api ( #32620 )
...
issue: https://github.com/milvus-io/milvus/issues/30436
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-05-08 11:53:29 +08:00
ba02d54a30
enhance: update shard leader cache when leader location changed ( #32470 )
...
issue: #32466
this PR enhance that when shard location changed, update proxy's shard
leader cache. in case of query node failover case, proxy can find
replica recover
---------
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-05-08 10:05:29 +08:00
53874ce245
fix: Fix cannot specify partition name in binlog import ( #32730 )
...
issue: https://github.com/milvus-io/milvus/issues/32807
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-05-07 17:19:30 +08:00
1f58cda957
enhance: add more trace for search & query ( #32734 )
...
issue: https://github.com/milvus-io/milvus/issues/32728
---------
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2024-05-07 13:03:29 +08:00
7da1ca9efb
enhance: check the partition num when creating collection with partition key ( #32670 )
...
issue: #30577
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-05-07 10:43:29 +08:00
48fb8310ba
fix: skip to check auto id when inserting data ( #32775 )
...
/kind improvement
issue: https://github.com/milvus-io/milvus/issues/32591
related pr: #32592
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-05-07 10:17:30 +08:00
02bf47f295
fix: Validate num of rows for insert field data with schema ( #32770 )
...
See also #32769
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-05-07 10:07:29 +08:00
600db9d99e
enhance: check field data type of input ( #32777 )
...
issue: #32769
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2024-05-06 21:15:29 +08:00
fecd9c21ba
feat: LRU cache implementation ( #32567 )
...
issue: https://github.com/milvus-io/milvus/issues/32783
This pr is the implementation of lru cache on branch lru-dev.
Signed-off-by: sunby <sunbingyi1992@gmail.com>
Co-authored-by: chyezh <chyezh@outlook.com>
Co-authored-by: MrPresent-Han <chun.han@zilliz.com>
Co-authored-by: Ted Xu <ted.xu@zilliz.com>
Co-authored-by: jaime <yun.zhang@zilliz.com>
Co-authored-by: wayblink <anyang.wang@zilliz.com>
2024-05-06 20:29:30 +08:00
42d0412e93
enhance: Add channelCPs in FlushResponse ( #32044 )
...
#32609
Signed-off-by: wayblink <anyang.wang@zilliz.com>
2024-04-30 09:45:27 +08:00
0359139299
enhance: return the auth collection list when using the `show collection` api ( #32576 )
...
issue: #32550
/kind improvement
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-04-29 19:09:34 +08:00
d900e68440
fix: fix GetShardLeaders return empty node list ( #32685 )
...
issue: #32449
to avoid GetShardLeaders return empty node list, this PR add node list
check in both client side and server side.
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-04-29 14:19:26 +08:00
09cd56d44f
enhance: add the skip auto id and partition key check config ( #32592 )
...
/kind improvement
issue: #32591
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-04-29 10:29:26 +08:00
07720f1a95
enhance: expose alter database api in proxy ( #32639 )
...
issue: #30040
---------
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-04-28 18:15:31 +08:00
9a719ec89e
fix: skip the empty partition name in the rate limit interceptor ( #32647 )
...
issue: https://github.com/milvus-io/milvus/issues/30577
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-04-28 11:01:28 +08:00
3b5cbed86a
enhance: add db label for deletion and bulkinsert operation ( #32611 )
...
Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-04-26 15:53:29 +08:00
04f355a802
enhance: Enable alter database props in rootcoord ( #32458 )
...
issue: #30040
---------
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-04-25 10:53:25 +08:00
dcc15e3ed8
enhance: Remove `TestRateLimiter/tests_refresh_rate_by_config` case ( #32545 )
...
See also #32313
This case always fail since the watch config logic is removed
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-04-24 10:13:24 +08:00
365e50b63e
fix: revert add range search params check in proxy ( #32366 )
...
no need to check params in empty segment.
#30365
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-04-23 17:41:23 +08:00
3119a7b23f
fix: Remove watching config event in rate limiter ( #32313 )
...
Remove watching config event in rate limiter to prevent object leaks.
Instead, reset limiter rates periodically.
issue: https://github.com/milvus-io/milvus/issues/32312
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-04-23 16:19:24 +08:00
d0ffb71a17
enhance: forbid delete with always true expression ( #32472 )
...
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-04-22 11:09:23 +08:00
aoiasd
congqixia
Chun Han
zhagnlu
wei liu
ArenaSu
yihao.dai
SimFG
Jiquan Long
Bingyi Sun
smellthemoon
Cai Yudong
Xiaofan
wayblink
jaime