1b4e28b97f
enhance: Check by proxy rate limiter when delete get data by query. ( #30891 )
...
relate: https://github.com/milvus-io/milvus/issues/30927
---------
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-05-23 20:03:40 +08:00
dc058eaf61
fix: should init access logger before init server ( #32976 )
...
relate: https://github.com/milvus-io/milvus/issues/32968
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-05-14 10:47:32 +08:00
c012e6786f
feat: support rate limiter based on db and partition levels ( #31070 )
...
issue: https://github.com/milvus-io/milvus/issues/30577
co-author: @jaime0815
---------
Signed-off-by: Patrick Weizhi Xu <weizhi.xu@zilliz.com>
Signed-off-by: SimFG <bang.fu@zilliz.com>
Co-authored-by: Patrick Weizhi Xu <weizhi.xu@zilliz.com>
2024-04-12 16:01:19 +08:00
43a9be2cce
enhance: support access log dynamic config and prepare for support resutful interface access log ( #29638 )
...
relate: https://github.com/milvus-io/milvus/issues/29637
https://github.com/milvus-io/milvus/issues/31823
---------
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-04-12 10:37:27 +08:00
ac26908cc4
enhance: Remove the storage info report ( #31772 )
...
issue: #30436
origin pr: #30438
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-04-01 20:50:59 -07:00
b1a1cca10b
feat: add more operation detail info for better allocation ( #30438 )
...
issue: #30436
---------
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-03-28 06:33:11 +08:00
248c923e59
fix: proxy ctx cancel before revoking session ( #31596 )
...
issue: #31219
Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-03-26 14:25:07 +08:00
982dd2834b
enhance: add materialized view search info ( #30888 )
...
issue: #29892
This PR
1. Pass Materialized View (MV) search information obtained from the
expression parsing planning procedure to Knowhere. It only performs when
MV is enabled and the partition key is involved in the expression. The
search information includes:
1. Touched field_id and the count of related categories in the
expression. E.g., `color == red && color == blue` yields `field_id ->
2`.
2. Whether the expression only includes AND (&&) logical operator,
default `true`.
3. Whether the expression has NOT (!) operator, default `false`.
4. Store if turning on MV on the proxy to eliminate reading from
paramtable for every search request.
5. Renames to MV.
## Rebuttals
1. Did not write in `ExtractInfoPlanNodeVisitor` since the new scalar
framework was introduced and this part might be removed in the future.
2. Currently only interested in `==` and `in` expression, `string` data
type, anything else is a bonus.
3. Leave handling expressions like `F == A || F == A` for future works
of the optimizer.
## Detailed MV Info
Signed-off-by: Patrick Weizhi Xu <weizhi.xu@zilliz.com>
2024-03-21 11:19:07 +08:00
0c7474d7e8
enhance: add graceful stop timeout to avoid node stop hang under extreme cases ( #30317 )
...
1. add coordinator graceful stop timeout to 5s
2. change the order of datacoord component while stop
3. change querynode grace stop timeout to 900s, and we should
potentially change this to 600s when graceful stop is smooth
issue: #30310
also see pr: #30306
---------
Signed-off-by: chyezh <chyezh@outlook.com>
2024-02-29 17:01:50 +08:00
e59bc1e526
enhance: use latest milvus proto, add unimplemented server ( #30889 )
...
issue: #30647
Signed-off-by: chyezh <chyezh@outlook.com>
2024-02-29 10:13:00 +08:00
fc56ed5c27
fix: Proxy restful api doesn't register ( #30072 )
...
issue: #30074
This PR fix that management restful api in proxy doesn't register to
http service
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-01-18 20:28:52 +08:00
dd9c61831d
enhance: Support to get the param value in the runtime ( #29297 )
...
/kind improvement
issue: #29299
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-12-22 18:36:44 +08:00
3cc4209d26
enhance: pack proxy connection code and support accesslog print SDK_Version ( #28835 )
...
relate: https://github.com/milvus-io/milvus/issues/28086
https://github.com/milvus-io/milvus/issues/28940
---------
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2023-12-05 16:50:47 +08:00
89d8ce2f73
enhance: refine access log to support format access log by yaml and print name info. ( #28319 )
...
relate: https://github.com/milvus-io/milvus/issues/28086
---------
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2023-11-28 15:32:31 +08:00
b1e0a27f31
enhance: Add logs for each step during service initialization ( #28624 )
...
/kind improvement
Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-11-27 16:30:26 +08:00
79c0edb1d8
enhance:Remove msgbase unnecessary assignments ( #28511 )
...
remove some unnecessary assignments, for the reason that
commonpbutil.NewMsgBase has default value.
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2023-11-24 15:02:39 +08:00
6b1a106a31
Moving etcd client into session ( #27069 )
...
Signed-off-by: Filip Haltmayer <filip.haltmayer@zilliz.com>
2023-10-27 07:36:12 +08:00
9b0ecbdca7
Support to replicate the mq message ( #27240 )
...
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-10-20 14:26:09 +08:00
ec1fe3549e
Add a stop hook to clean session ( #27564 )
...
Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-10-16 10:24:10 +08:00
3759857bc5
Refine Proxy errors ( #27499 )
...
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-10-09 10:09:33 +08:00
7f7c71ea7d
Decoupling client and server API in types interface ( #27186 )
...
Co-authored-by:: aoiasd <zhicheng.yue@zilliz.com>
Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-09-26 09:57:25 +08:00
26f06dd732
Format the code ( #27275 )
...
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-09-21 09:45:27 +08:00
168e82ee10
Fix panic while handling with the nil status ( #27040 )
...
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-09-15 10:09:21 +08:00
63b86b32a6
Add server id validation interceptor ( #26395 )
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-08-17 20:20:20 +08:00
84253f255e
Fix datanode graceful stop panic ( #25932 )
...
See also: #25925
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2023-07-28 10:11:08 +08:00
d63323d117
Add rate limit and deny write in upsert ( #25351 )
...
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2023-07-11 11:20:34 +08:00
17796743dd
add metrics for replica selection ( #25037 )
...
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-06-27 09:52:44 +08:00
33c2012675
Add more metrics ( #25081 )
...
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2023-06-26 17:52:44 +08:00
d143682d7d
Refactor logs in proxy package. ( #24936 )
...
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-06-19 13:28:41 +08:00
46f7d903a3
enable look aside balancer on replica selection ( #24791 )
...
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-06-16 18:38:39 +08:00
a3437e0ab5
refactor replica selection and retry policy on channel ( #24367 )
...
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-06-13 10:20:37 +08:00
41af0a98fa
Use go-api/v2 for milvus-proto ( #24770 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-06-09 01:28:37 +08:00
bd343550a5
Support to manage connections ( #24224 )
...
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2023-05-19 12:51:23 +08:00
43a9e175a3
Exit component process when session key is deleted ( #21658 ) ( #22164 )
...
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2023-04-12 20:12:28 +08:00
c9d0c157ec
Move some modules from internal to public package ( #22572 )
...
Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-04-06 19:14:32 +08:00
2c1c3aecfc
Fix metric ProxySyncTimeTickLag ( #23235 )
...
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2023-04-06 15:28:33 +08:00
e581fef115
Use the request's context to allocate ts ( #22633 )
...
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-03-09 18:51:52 +08:00
697dedac7e
Use cockroachdb/errors to replace other error pkg ( #22390 )
...
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-02-26 11:31:49 +08:00
e768437681
Correct usage of Timer and Ticker ( #22228 )
...
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2023-02-23 18:59:45 +08:00
90d9e165d4
Fix some configs not shown ( #21653 )
...
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-01-13 15:31:41 +08:00
6a722396bd
Integration test framework ( #21283 )
...
Signed-off-by: wayblink <anyang.wang@zilliz.com>
2023-01-12 19:49:40 +08:00
e5f408dceb
Merge IndexCoord and DataCoord ( #21267 )
...
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2023-01-04 19:37:36 +08:00
89b810a4db
Refactor all params into ParamItem ( #20987 )
...
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2022-12-07 18:01:19 +08:00
c05b9ad539
Add event dispatcher for config ( #20393 )
...
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2022-11-17 18:59:09 +08:00
19524a5344
Fix nodeID mismatch at standalone mode ( #20648 )
...
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2022-11-17 17:15:08 +08:00
a7ba416b5a
Remove invalid timestamp ( #20557 )
...
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2022-11-14 16:21:07 +08:00
fca2b71e28
add access log for proxy ( #19927 )
...
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2022-11-10 17:09:06 +08:00
7e56e7a976
Fix datarace at Setlogger ( #20376 )
...
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2022-11-08 14:33:03 +08:00
709a0a94e9
Add some msgstream metrics ( #20296 )
...
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
2022-11-07 10:15:02 +08:00
956c5e1b9d
Make Params singleton ( #20088 )
...
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2022-11-04 14:25:38 +08:00
aoiasd
SimFG
jaime
Patrick Weizhi Xu
chyezh
wei liu
smellthemoon
Filip Haltmayer
yah01
yihao.dai
XuanYang-cn
xige-16
Enwei Jiao
congqixia
Jiquan Long
cai.zhang
yah01
zhenshan.cao
wayblink