Commit Graph

384 Commits (1db099973ff00f3b079ad739de010bb31c02b70f)

Author SHA1 Message Date
Ted Xu b4d0f4df0a
enhance: update param documents (#35152)
See: #35161

---------

Signed-off-by: Ted Xu <ted.xu@zilliz.com>
2024-08-01 12:38:13 +08:00
congqixia de8a266d8a
enhance: Enable linux code checker (#35084)
See also #34483

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-07-30 15:53:51 +08:00
wei liu 4bb969ef8f
enhance: Skip manual stopped component during health check (#34953)
after manual stop component by management restful api, `healthz` may
return unhealthy state. k8s may restart the pod to save the unhealthy
sate, and the manual stop operation will got unexpected result.

to solve this, we make `healthz` API skip the manual stopped component.

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-07-30 14:37:51 +08:00
wei liu c45f38aa61
enhance: Update protobuf-go to protobuf-go v2 (#34394)
issue: #34252

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-07-29 11:31:51 +08:00
cai.zhang 2372452fac
enhance: Optimized the GC logic to ensure that memory is released in time (#34949)
issue: #34703

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-07-28 23:53:47 +08:00
Ted Xu 63f0154dfb
fix: enable milvus.yaml check (#34567)
See #32168

---------

Signed-off-by: Ted Xu <ted.xu@zilliz.com>
2024-07-25 18:05:46 +08:00
chyezh bef06e5acf
enhance: add streaming coord and node grpc service register (#34655)
issue: #33285

- register streaming coord service into datacoord.
- add new streaming node role.
- add global static switch to enable streaming service or not.

Signed-off-by: chyezh <chyezh@outlook.com>
2024-07-25 12:19:44 +08:00
SimFG e56ab76d99
fix: make the log level can be dynamically changed at runtime (#34757)
- issue: #34756

Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-07-17 20:31:42 +08:00
jaime 9630974fbb
enhance: move rocksmq from internal to pkg module (#33881)
issue: #33956

Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-06-25 21:18:15 +08:00
wei liu 0a3d456688
enhance: add restful api to trigger component stop (#32076)
issue: #32698
This PR add two rest api for component stop and status check:
1. `/management/stop?role=querynode` can stop the specified component
2. `/management/check/ready?role=rootcoord` can check whether the target
component is serviceable

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-06-07 10:35:54 +08:00
aoiasd 2422084a29
fix: paramtable cache cause dynamic config non-dynamic (#33473)
relate: https://github.com/milvus-io/milvus/issues/33461

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-06-04 11:39:46 +08:00
Bingyi Sun 4724779b3b
enhance: remove fallback keys for config generator (#32946)
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2024-05-13 13:33:31 +08:00
aoiasd 54a51b1236
enhance: Support dynamic config for opentelemetry trace (#32169)
relate: https://github.com/milvus-io/milvus/issues/31940

---------

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-05-09 17:43:30 +08:00
Ted Xu 78d32bd8b2
enhance: update milvus.yaml (#31832)
See #32168

---------

Signed-off-by: Ted Xu <ted.xu@zilliz.com>
2024-04-16 16:17:19 +08:00
shaoyue be53e028a8
doc: [skip e2e] add config-docs-generator (#22134)
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>

Part of issue: #21856

Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
2024-04-03 14:13:13 +08:00
Xiaofan b6fefee0cf
fix: etcd not connectable when auth enabled (#31633)
Fix etcd config source didn't respect auth enabled
Also removed pulsar recoverable error when pulsar return ConsumerBusy.
It could happen that pulsar didn't find the original consumer is dead
and recover takes some time.
fix #31631

Signed-off-by: xiaofanluan <xiaofan.luan@zilliz.com>
2024-04-01 15:23:19 +08:00
Bingyi Sun 0ac9bb4a9c
enhance: add mmap migration tool (#30909)
issue: #30908

---------

Signed-off-by: sunby <sunbingyi1992@gmail.com>
2024-03-25 15:51:09 +08:00
chyezh 0c7474d7e8
enhance: add graceful stop timeout to avoid node stop hang under extreme cases (#30317)
1. add coordinator graceful stop timeout to 5s
2. change the order of datacoord component while stop
3. change querynode grace stop timeout to 900s, and we should
potentially change this to 600s when graceful stop is smooth

issue: #30310
also see pr: #30306

---------

Signed-off-by: chyezh <chyezh@outlook.com>
2024-02-29 17:01:50 +08:00
smellthemoon 9815cf50c9
fix: close rocksmq and natsmq when shutdown (#30706)
related: #30703 #30355

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-02-26 11:38:55 +08:00
chyezh 211143c5e6
enhance: add basic information of milvus into metrics (#29665)
add basic build information and runtime component dependency into
metrics.

issue: #29664

Signed-off-by: chyezh <ye.zhen@zilliz.com>
2024-01-29 15:47:02 +08:00
congqixia 20a3569c14
enhance: Set correct role for non-standalone deployment (#30303)
See also #30211

After fix initialization problem, distributed components do no have
their role set. This will cause logger & tracing miss component service
info when recording information.

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-01-26 14:05:01 +08:00
PowderLi 08ca0a2ca5
feat: support etcd authentication (#30226)
issue: #28895
add 3 configuration for ETCD config

Signed-off-by: PowderLi <min.li@zilliz.com>
2024-01-24 11:35:00 +08:00
congqixia c9860e3d2a
fix: Init paramtable after milvus role setup (#30211)
See also #30176

Move paramtable.Init after env setup in roles.Run. Also introduced a
flag for mixture run to set role correctly for mixture mode.

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-01-23 19:10:56 +08:00
Bingyi Sun dad27cbb40
fix: fix wrong log file name when running as mixture type (#29979)
See also: #25323, #29969
many users reported log file name is incorrect when starting in mixture
type.

---------

Signed-off-by: sunby <sunbingyi1992@gmail.com>
2024-01-19 11:48:54 +08:00
Jiquan Long a3bae80b59
enhance: print total memory when milvus starts (#29351)
fix: #29349

---------

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2024-01-07 20:09:49 +08:00
SimFG dd9c61831d
enhance: Support to get the param value in the runtime (#29297)
/kind improvement
issue: #29299

Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-12-22 18:36:44 +08:00
Bingyi Sun e8c5e75786
enhance: ignore proto generated files (#28537)
ignore proto generated files
issue: https://github.com/milvus-io/milvus/issues/28566

---------

Signed-off-by: sunby <sunbingyi1992@gmail.com>
2023-11-22 10:20:22 +08:00
jaime ec0d9d8fd4
Fix delete session key with prefix (#28260)
Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-11-08 15:22:21 +08:00
wei liu 0c69f48ba4
refine stop order (#28016)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-11-01 11:40:15 +08:00
yah01 9658367a3c
Refine chunk manager errors (#27590)
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-10-31 12:18:15 +08:00
Filip Haltmayer 6b1a106a31
Moving etcd client into session (#27069)
Signed-off-by: Filip Haltmayer <filip.haltmayer@zilliz.com>
2023-10-27 07:36:12 +08:00
wayblink aa1fba79b1
Register metrics before component start (#27744)
Signed-off-by: wayblink <anyang.wang@zilliz.com>
2023-10-19 17:14:10 +08:00
jaime ac2d1bb5c2
Support receive signals from parent process (#27756)
Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-10-18 20:20:11 +08:00
congqixia 2f201c25e2
Remove deprecated io/ioutil usage (#27747)
`io/ioutil` package is deprecated, use `io`,`os` package replacement
also added golangci-lint rule to block future reference

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
Co-authored-by: guoguangwu <guoguangwu@magic-shield.com>
2023-10-17 20:32:09 +08:00
jaime ec1fe3549e
Add a stop hook to clean session (#27564)
Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-10-16 10:24:10 +08:00
SimFG 26f06dd732
Format the code (#27275)
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-09-21 09:45:27 +08:00
yihao.dai bb6711f28c
Add ChunkCache: support get vector from storage (#26142)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-09-15 10:21:20 +08:00
congqixia f7b2ad6650
Fix component pointer to `component` interface never be nil caused panic (#27072)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-09-14 13:37:18 +08:00
cai.zhang 4fa878d03c
Remove unused log (#27032)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2023-09-13 16:07:18 +08:00
Enwei Jiao fb0705df1b
Decouple basetable and componentparam (#26725)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-09-05 10:31:48 +08:00
yah01 3349db4aa7
Refine errors to remove changes breaking design (#26521)
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-09-04 09:57:09 +08:00
congqixia 4bf26dd4ed
Refine standalone components stop order (#26742)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-08-31 16:51:02 +08:00
congqixia 073ac8350b
Refine signal handler for whole milvus role lifetime (#26642)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-08-29 19:28:28 +08:00
jaime c603f1c244
Remove mysql metastore (#26633)
Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-08-29 14:36:26 +08:00
Enwei Jiao 533f0ddf6d
Add amazonlinux image, prepare to remove Centos7 image, remove openblas image (#26368)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-08-18 18:32:19 +08:00
SimFG 8b54914287
Add go.mod file for the milvus/pkg package (#26359)
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-08-16 14:49:32 +08:00
Jiquan Long fb5d809599
Fix components' setup (#26182)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2023-08-09 14:33:15 +08:00
xige-16 33c2012675
Add more metrics (#25081)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2023-06-26 17:52:44 +08:00
jaime 18df2ba6fd
[Cherry-Pick] Support Database (#24769)
Support Database(#23742)
Fix db nonexists error for FlushAll (#24222)
Fix check collection limits fails (#24235)
backward compatibility with empty DB name (#24317)
Fix GetFlushAllState with DB (#24347)
Remove db from global meta cache after drop database (#24474)
Fix db name is empty for describe collection response (#24603)
Add RBAC for Database API (#24653)
Fix miss load the same name collection during recover stage (#24941)

RBAC supports Database validation (#23609)
Fix to list grant with db return empty (#23922)
Optimize PrivilegeAll permission check (#23972)
Add the default db value for the rbac request (#24307)

Signed-off-by: jaime <yun.zhang@zilliz.com>
Co-authored-by: SimFG <bang.fu@zilliz.com>
Co-authored-by: longjiquan <jiquan.long@zilliz.com>
2023-06-25 17:20:43 +08:00
SimFG 31c08a8135
Improve the panic code about the milvus component #24783 (#24919)
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-06-17 10:32:39 +08:00