Commit Graph

180 Commits (88cca9d90f6f3b64d8eaa0a376d9c0042087ce46)

Author SHA1 Message Date
wei liu 0a3d456688
enhance: add restful api to trigger component stop (#32076)
issue: #32698
This PR add two rest api for component stop and status check:
1. `/management/stop?role=querynode` can stop the specified component
2. `/management/check/ready?role=rootcoord` can check whether the target
component is serviceable

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-06-07 10:35:54 +08:00
aoiasd 0cf225fa75
enhance: access log support restful api (#33155)
relate: https://github.com/milvus-io/milvus/issues/31823

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-05-31 11:13:44 +08:00
aoiasd dc058eaf61
fix: should init access logger before init server (#32976)
relate: https://github.com/milvus-io/milvus/issues/32968

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-05-14 10:47:32 +08:00
wei liu c35797c399
enhance: expose DescribeDatabase api in proxy (#32732)
issue: #32707

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-05-09 22:51:30 +08:00
wei liu ba02d54a30
enhance: update shard leader cache when leader location changed (#32470)
issue: #32466

this PR enhance that when shard location changed, update proxy's shard
leader cache. in case of query node failover case, proxy can find
replica recover

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-05-08 10:05:29 +08:00
wei liu 07720f1a95
enhance: expose alter database api in proxy (#32639)
issue: #30040

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-04-28 18:15:31 +08:00
chyezh 48fe977a9d
enhance: declarative resource group api (#31930)
issue: #30647

- Add declarative resource group api

- Add config for resource group management

- Resource group recovery enhancement

---------

Signed-off-by: chyezh <chyezh@outlook.com>
2024-04-15 08:13:19 +08:00
jaime db79be3ae0
fix: ctx cancel should be the last step while stopping server (#31220)
issue: #31219

Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-03-15 10:33:05 +08:00
yihao.dai a434d33e75
feat: Add import scheduler and manager (#29367)
This PR introduces novel managerial roles for importv2:
1. ImportMeta: To manage all the import tasks;
2. ImportScheduler: To process tasks and modify their states;
3. ImportChecker: To ascertain the completion of all tasks and instigate
relevant operations.

issue: https://github.com/milvus-io/milvus/issues/28521

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-03-01 18:31:02 +08:00
chyezh 0c7474d7e8
enhance: add graceful stop timeout to avoid node stop hang under extreme cases (#30317)
1. add coordinator graceful stop timeout to 5s
2. change the order of datacoord component while stop
3. change querynode grace stop timeout to 900s, and we should
potentially change this to 600s when graceful stop is smooth

issue: #30310
also see pr: #30306

---------

Signed-off-by: chyezh <chyezh@outlook.com>
2024-02-29 17:01:50 +08:00
chyezh e59bc1e526
enhance: use latest milvus proto, add unimplemented server (#30889)
issue: #30647

Signed-off-by: chyezh <chyezh@outlook.com>
2024-02-29 10:13:00 +08:00
PowderLi 50a78b682e
fix: set proxy.http.acceptTypeAllowInt64: true as default (#30720)
issue: #30680

also let the parameter item to be refreshable

Signed-off-by: PowderLi <min.li@zilliz.com>
2024-02-29 09:59:07 +08:00
PowderLi 5cf9bb236e
enhance: restful support import jobs (#30343)
issue: #28521 #29732

include
1. list collection's import jobs
2. create a new import job
3. get the progress of an import job

fix:
1. mix the order of dbName & collectionName #29728
2. trace log keep the same as v1
3. support traceID
4. azure precheck, blob name cannot end with / #29703

---------

Signed-off-by: PowderLi <min.li@zilliz.com>
2024-01-31 17:57:04 +08:00
yah01 2f778d9649
fix: proxy may never setup if the port binded (#30035)
the proxy miss-returned nil while failed to listen the port, then the
server continues to run but we can't connect to service
resolve #30034

Signed-off-by: yah01 <yah2er0ne@outlook.com>
2024-01-31 17:37:04 +08:00
PowderLi 6abbab12fa
feat: restful phase two (#29728)
issue: #29732

Signed-off-by: PowderLi <min.li@zilliz.com>
2024-01-28 16:03:01 +08:00
PowderLi 08ca0a2ca5
feat: support etcd authentication (#30226)
issue: #28895
add 3 configuration for ETCD config

Signed-off-by: PowderLi <min.li@zilliz.com>
2024-01-24 11:35:00 +08:00
wayblink 1df3f90696
feat: Implement DescribeAlias and ListAliases interfaces (#29641)
#22882
/kind feature

Signed-off-by: wayblink <anyang.wang@zilliz.com>
2024-01-11 19:12:51 +08:00
wayblink 05d735c322
enhance: Rename SearchV2 to HybridSearch (#29592)
related: https://github.com/milvus-io/milvus-proto/pull/233
issue: #29593
/kind enhancement

Signed-off-by: wayblink <anyang.wang@zilliz.com>
2024-01-04 19:22:46 +08:00
PowderLi 5f00bad4b8
fix: link with install path's libblob-chunk-manager (#29496)
issue: #29494

1. link with install path's libblob-chunk-manager
2. performance of `ShouldBindWith` is better than `ShouldBindBodyWith`
3. the middleware shouldn't read the unrefreshed parameter repeatly

Signed-off-by: PowderLi <min.li@zilliz.com>
2023-12-31 20:02:48 +08:00
yah01 a0e1a1eb31
feat: support enable/disable mmap for index (#29005)
support enable/disable mmap for index, the user could alter the index's
mode by `AlterIndex` method
related: https://github.com/milvus-io/milvus/issues/21866

---------

Signed-off-by: yah01 <yah2er0ne@outlook.com>
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-12-21 18:07:24 +08:00
PowderLi bcd6865b29
enhance: add 3 builtin roles (#28961)
issue: #28960 [milvus-proto
#212](https://github.com/milvus-io/milvus-proto/issues/212)

add new configuration: builtinRoles
user can define roles in config file: `milvus.yaml`

there is an example:
1. db_ro, only have read privileges, include load
2. db_rw, read and write privileges, include create/drop/rename
collection
3. db_admin, not only read and write privileges, but also user
administration

Signed-off-by: PowderLi <min.li@zilliz.com>
2023-12-18 14:28:41 +08:00
shaoyue 4a067a4c8c
enhance: Add proxy.ginLogSkipPaths (#28945)
Fix #28944

/cc @xiaofan-luan

Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
2023-12-06 10:26:35 +08:00
aoiasd 3cc4209d26
enhance: pack proxy connection code and support accesslog print SDK_Version (#28835)
relate: https://github.com/milvus-io/milvus/issues/28086
https://github.com/milvus-io/milvus/issues/28940

---------

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2023-12-05 16:50:47 +08:00
aoiasd 89d8ce2f73
enhance: refine access log to support format access log by yaml and print name info. (#28319)
relate: https://github.com/milvus-io/milvus/issues/28086

---------

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2023-11-28 15:32:31 +08:00
SimFG 9c46788d87
enhance: Support to trace restful request and request error (#28685)
issue: #28348

Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-11-27 20:14:26 +08:00
jaime c5f455dc6b
fix: cmux graceful shutdown on proxy service (#28383)
issue #28305

Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-11-27 16:28:34 +08:00
MrPresent-Han fc30d291be
fix createCollection failed occasionally (#28592) (#28712)
fix: create collection seldom failure #28592

Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
2023-11-27 11:10:25 +08:00
SimFG cfb6edea61
Support to trace the grpc request (#28349)
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-11-13 20:16:18 +08:00
wei liu 8a370f115a
Fix external grpc server doesn't graceful stop (#28309)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-11-10 00:14:18 +08:00
wei liu 64f6dbabdc
Fix proxy not healthy before graceful stop (#28223)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-11-08 02:22:26 +08:00
PowderLi 0c0f012e03
add a custom http header: Accept-Type-Allow-Int64 (#27901)
Signed-off-by: PowderLi <min.li@zilliz.com>
2023-11-01 11:42:16 +08:00
yah01 9658367a3c
Refine chunk manager errors (#27590)
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-10-31 12:18:15 +08:00
Filip Haltmayer 6b1a106a31
Moving etcd client into session (#27069)
Signed-off-by: Filip Haltmayer <filip.haltmayer@zilliz.com>
2023-10-27 07:36:12 +08:00
jaime bf7f32b025
Refine Proxy stop (#27919)
Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-10-26 10:24:09 +08:00
jaime 4640928280
Fix initialization and backward compatibility issue for GRPC compression (#27894)
Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-10-24 22:14:14 +08:00
SimFG 9b0ecbdca7
Support to replicate the mq message (#27240)
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-10-20 14:26:09 +08:00
SimFG 630636c4ec
Support the apikey authentication for the restful api (#27758)
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-10-18 16:36:12 +08:00
SimFG 84d05b939b
Support the apikey authentication (#27723)
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-10-17 21:00:11 +08:00
congqixia 2f201c25e2
Remove deprecated io/ioutil usage (#27747)
`io/ioutil` package is deprecated, use `io`,`os` package replacement
also added golangci-lint rule to block future reference

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
Co-authored-by: guoguangwu <guoguangwu@magic-shield.com>
2023-10-17 20:32:09 +08:00
PowderLi 09d8b76048
[restful] new context with grpc metadata (#27668)
Signed-off-by: PowderLi <min.li@zilliz.com>
2023-10-17 20:00:14 +08:00
yah01 be980fbc38
Refine state check (#27541)
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-10-11 21:01:35 +08:00
yah01 8394b3a1ec
Block creating new error from status reason (#27426)
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-10-07 11:29:32 +08:00
yah01 63ac43a3b8
Refine errors for import (#27379)
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-09-30 10:31:28 +08:00
wei liu 27046e242b
add timeout for graceful stop (#27326)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-09-26 10:15:25 +08:00
jaime 7f7c71ea7d
Decoupling client and server API in types interface (#27186)
Co-authored-by:: aoiasd <zhicheng.yue@zilliz.com>

Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-09-26 09:57:25 +08:00
SimFG 26f06dd732
Format the code (#27275)
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-09-21 09:45:27 +08:00
yihao.dai dd2cb1d44a
Flush by flush channels (#26859)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-09-12 21:07:19 +08:00
yah01 00c65fa0d7
Refine QueryNode errors (#27013)
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-09-12 16:07:18 +08:00
XuanYang-cn 7f1ae35e72
Add timeout in dispatcher, AsConsumer and Seek (#26686)
See also: #25309

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2023-09-08 09:51:17 +08:00
yah01 3349db4aa7
Refine errors to remove changes breaking design (#26521)
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-09-04 09:57:09 +08:00