Commit Graph

176 Commits (eb046863485fdf3e130fc60484485c901b81276b)

Author SHA1 Message Date
congqixia cb7f2fa6fd
enhance: Use v2 package name for pkg module (#39990)
Related to #39095

https://go.dev/doc/modules/version-numbers

Update pkg version according to golang dep version convention

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2025-02-22 23:15:58 +08:00
wei liu bfc802297e
enhance: Add management api to check querycoord balance status (#37784)
issue: #37783

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2025-02-14 18:00:14 +08:00
Zhen Ye a9e0e0a852
enhance: broadcast with event-based notification (#39522)
issue: #38399

- broadcast message can carry multi resource key now.
- implement event-based notification for broadcast messages
- broadcast message use broadcast id as a unique identifier in message
- broadcasted message on vchannels keep the broadcasted vchannel now.
- broadcasted message and broadcast message have a common broadcast
header now.

---------

Signed-off-by: chyezh <chyezh@outlook.com>
2025-02-07 11:14:43 +08:00
Zhen Ye bb8d1ab3bf
enhance: make new go package to manage proto (#39114)
issue: #39095

---------

Signed-off-by: chyezh <chyezh@outlook.com>
2025-01-10 10:49:01 +08:00
tinswzy 27229f7907
enhance: refine exists log print with ctx (#38080)
issue: #35917 
Refines exists log print with ctx

Signed-off-by: tinswzy <zhenyuan.wei@zilliz.com>
2024-12-14 22:36:44 +08:00
Zhen Ye d3ae8e9232
fix: delay the wait other coord logic in query coord after query coord change into standby state (#38259)
issue: https://github.com/milvus-io/milvus/issues/37764

- After removing rpc layer from mixcoord, the querycoord at standby mode
will be blocked forever of deployment rolling

---------

Signed-off-by: chyezh <chyezh@outlook.com>
2024-12-11 15:48:42 +08:00
shaoyue 1f66b9ebfb
feat: add config field to set internal tls sni (#38124)
/cc @xiaofan-luan @jaime0815 @nish112022

part of https://github.com/milvus-io/milvus/issues/36864

Signed-off-by: haorenfsa <haorenfsa@gmail.com>
2024-12-04 14:56:47 +08:00
Zhen Ye b9a10a2f68
enhance: remove the rpc layer of coordinator when enabling standalone or mixcoord (#37815)
issue: #37764

- add a local client to call local server directly for
querycoord/rootcoord/datacoord.
- enable local client if milvus is running mixcoord or standalone mode.

---------

Signed-off-by: chyezh <chyezh@outlook.com>
2024-11-23 21:36:33 +08:00
nish112022 484c6b5c44
feat: Added code for Internal-tls (#36865)
issue : https://github.com/milvus-io/milvus/issues/36864

I have a few questions regarding my approach.I will consolidate them
here for feedback and review.Thanks

---------

Signed-off-by: Nischay Yadav <nischay.yadav@ibm.com>
Signed-off-by: Nischay <Nischay.Yadav@ibm.com>
2024-11-20 06:00:32 +08:00
Zhen Ye d29e01e284
fix: port listen racing in mix or standalone mode (#36442)
issue: #36441

---------

Signed-off-by: chyezh <chyezh@outlook.com>
2024-09-26 21:23:16 +08:00
wei liu 3cd0b26285
enhance: Enable dynamic update loaded collection's replica (#35822)
issue: #35821
After collection loaded, if we need to increase/decrease collection's
replica, we need to release and load it again.

milvus offers 4 solution to update loaded collection's replica, this PR
aims to dynamic change the replica number without release, and after
replica number changed, milvus will execute load replica or release
replica in async, and the replica loaded status can be checked by
getReplicas API.

Notice that if set too much replicas than querynode can afford,the new
replica won't be loaded successfully until enough querynode joins.

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-09-25 10:13:18 +08:00
congqixia 9e96ed4873
fix: Fix tracing config update logic (#35928)
Related to #35927

There are serveral issue this PR addresses:
- Use `ResetTraceConfig` method instead init one in update event handler
- Implement dynamic stats.Handler to receive tracing config update event
- Update `enable_trace` flag when `ResetTraceConfig` is invoked
- Change `enable_trace` to `std::atomic<bool>` in case of data race

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-09-05 14:27:04 +08:00
chyezh 48fe977a9d
enhance: declarative resource group api (#31930)
issue: #30647

- Add declarative resource group api

- Add config for resource group management

- Resource group recovery enhancement

---------

Signed-off-by: chyezh <chyezh@outlook.com>
2024-04-15 08:13:19 +08:00
yiwangdr 1cd15d9322
test: support segment release in integration test (#31190)
issue: #29507

Notice that api_testonly.go files should be guarded by compiler tag
`test`, so that production build rules don't compile them and these APIs
don't get misused.

Signed-off-by: yiwangdr <yiwangdr@gmail.com>
2024-04-09 11:39:17 +08:00
wei liu 92971707de
enhance: Add restful api for devops to execute rolling upgrade (#29998)
issue: #29261
This PR Add restful api for devops to execute rolling upgrade, including
suspend/resume balance and manual transfer segments/channels.

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-03-27 16:15:19 +08:00
wei liu d79aa58b37
enhance: Speed up target recovery after query coord restart (#31240)
issue: #28491

after querycoord restart, it will pull a new target, which include
channel and segment list. when segments loaded on querynode has reached
the target, the collection could provide search/query. but if segment
list changes by time, ater querycoord pull a new target, it will takes a
few minutes to catch up the target's segment distribution. and before
that, query/search will fail due to lack of segments.

This PR save the current loaded target to meta storein querycoord's stop
progress, and recover it when query coord starts, to speed up the target
recovery time.

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-03-15 14:19:03 +08:00
jaime db79be3ae0
fix: ctx cancel should be the last step while stopping server (#31220)
issue: #31219

Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-03-15 10:33:05 +08:00
chyezh 0c7474d7e8
enhance: add graceful stop timeout to avoid node stop hang under extreme cases (#30317)
1. add coordinator graceful stop timeout to 5s
2. change the order of datacoord component while stop
3. change querynode grace stop timeout to 900s, and we should
potentially change this to 600s when graceful stop is smooth

issue: #30310
also see pr: #30306

---------

Signed-off-by: chyezh <chyezh@outlook.com>
2024-02-29 17:01:50 +08:00
chyezh 950624d8d3
fix: nil ptr is used as nil interface in grpc client (#30754)
issue: #30715

- Bug: Set nil struct pointer to describe nil interface.
Panic with segment violation when calling method on this nil struct
pointer.

Signed-off-by: chyezh <chyezh@outlook.com>
2024-02-23 10:26:53 +08:00
PowderLi 08ca0a2ca5
feat: support etcd authentication (#30226)
issue: #28895
add 3 configuration for ETCD config

Signed-off-by: PowderLi <min.li@zilliz.com>
2024-01-24 11:35:00 +08:00
wei liu fe1eeae2aa
enhance: Use mockery to replace manual mock code (#29074)
issue: #29043
This PR remove mannul mock code for proxy and data coord

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-12-13 10:46:44 +08:00
Bingyi Sun 10bb2431d8
test: add checker unittests (#28954)
issue: https://github.com/milvus-io/milvus/issues/28610

Signed-off-by: sunby <sunbingyi1992@gmail.com>
2023-12-05 10:56:33 +08:00
Bingyi Sun 45e6801ce4
feat: Add checker activation service interfaces (#28850)
issue: #28610

---------

Signed-off-by: sunby <sunbingyi1992@gmail.com>
2023-12-04 17:38:37 +08:00
smellthemoon 73f2bab454
enhance:add some log when create client and get component states (#28160)
/kind improvement

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2023-11-22 09:12:22 +08:00
Filip Haltmayer 6b1a106a31
Moving etcd client into session (#27069)
Signed-off-by: Filip Haltmayer <filip.haltmayer@zilliz.com>
2023-10-27 07:36:12 +08:00
jaime 4640928280
Fix initialization and backward compatibility issue for GRPC compression (#27894)
Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-10-24 22:14:14 +08:00
yah01 be980fbc38
Refine state check (#27541)
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-10-11 21:01:35 +08:00
wei liu 27046e242b
add timeout for graceful stop (#27326)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-09-26 10:15:25 +08:00
jaime 7f7c71ea7d
Decoupling client and server API in types interface (#27186)
Co-authored-by:: aoiasd <zhicheng.yue@zilliz.com>

Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-09-26 09:57:25 +08:00
SimFG 26f06dd732
Format the code (#27275)
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-09-21 09:45:27 +08:00
yah01 b4f86ea55e
Construct all success status with merr (#27226)
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-09-20 10:57:23 +08:00
yah01 168e82ee10
Fix panic while handling with the nil status (#27040)
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-09-15 10:09:21 +08:00
yiwangdr 337edc321b
tikv integration (#26246)
Signed-off-by: yiwangdr <yiwangdr@gmail.com>
2023-09-07 07:25:14 +08:00
Enwei Jiao fb0705df1b
Decouple basetable and componentparam (#26725)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-09-05 10:31:48 +08:00
congqixia e8f1b1736e
Remove log.Error(err.error())-style log (#26783)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-09-01 13:09:01 +08:00
yah01 dd4bc5b6a0
Fix data race in gRPC client (#26574)
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-08-28 18:26:28 +08:00
yihao.dai c3128aaef3
Accelerate server id interceptor validation (#26468)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-08-20 21:24:19 +08:00
wayblink 71133e5eaa
Check and reset if grpc client serverID mismatch with session (#26448)
Signed-off-by: wayblink <anyang.wang@zilliz.com>
2023-08-18 22:50:20 +08:00
yihao.dai 63b86b32a6
Add server id validation interceptor (#26395)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-08-17 20:20:20 +08:00
yiwangdr b9189b9f41
Organize mocks from types.go (#25466)
Signed-off-by: yiwangdr <yiwangdr@gmail.com>
2023-07-14 10:12:31 +08:00
yihao.dai 9f5756f16e
Add cluster validation interceptor to resolve the ip-reuse issue (#25001)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-06-21 11:36:46 +08:00
congqixia 41af0a98fa
Use go-api/v2 for milvus-proto (#24770)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-06-09 01:28:37 +08:00
yah01 ebd0279d3f
Check error by Error() and NoError() for better report message (#24736)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-06-08 15:36:36 +08:00
Preetham 4ea0830c25
Update: Wrap common calls querycoord rootcoord clients (#24360)
Signed-off-by: Preetham <kamidipreetham@gmail.com>
2023-05-24 19:25:26 +08:00
jaime c9d0c157ec
Move some modules from internal to public package (#22572)
Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-04-06 19:14:32 +08:00
congqixia 980aaeceb9
Fill coordinators client target node id (#22899)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-03-22 10:07:57 +08:00
yihao.dai 1f718118e9
Dynamic load/release partitions (#22655)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-03-20 14:55:57 +08:00
Enwei Jiao 697dedac7e
Use cockroachdb/errors to replace other error pkg (#22390)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-02-26 11:31:49 +08:00
Enwei Jiao 0a9a9058b9
Fix UT timeout (#22261)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-02-20 21:14:25 +08:00
wei liu 13b508103e
refine mock querycoord (#22198)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-02-16 15:38:34 +08:00