milvus-io
/
milvus
mirror of https://github.com/milvus-io/milvus.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
19,505 Commits
97 Branches
161 Tags
586 MiB
Commit Graph

937 Commits (2.4-hotfix)

Author SHA1 Message Date
wei liu fe1eeae2aa
enhance: Use mockery to replace manual mock code (#29074) 
issue: #29043
This PR remove mannul mock code for proxy and data coord

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-12-13 10:46:44 +08:00
shaoyue 4a067a4c8c
enhance: Add proxy.ginLogSkipPaths (#28945) 
Fix #28944

/cc @xiaofan-luan

Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
 2023-12-06 10:26:35 +08:00
aoiasd 3cc4209d26
enhance: pack proxy connection code and support accesslog print SDK_Version (#28835) 
relate: https://github.com/milvus-io/milvus/issues/28086
https://github.com/milvus-io/milvus/issues/28940

---------

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2023-12-05 16:50:47 +08:00
Bingyi Sun 10bb2431d8
test: add checker unittests (#28954) 
issue: https://github.com/milvus-io/milvus/issues/28610

Signed-off-by: sunby <sunbingyi1992@gmail.com>
 2023-12-05 10:56:33 +08:00
yihao.dai d26b563a8b
feat: Define import API and metadata (#28731) 
Define the new rpc and metadata for ImportV2.

see also: https://github.com/milvus-io/milvus/issues/28521

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-12-04 19:56:35 +08:00
Bingyi Sun 45e6801ce4
feat: Add checker activation service interfaces (#28850) 
issue: #28610

---------

Signed-off-by: sunby <sunbingyi1992@gmail.com>
 2023-12-04 17:38:37 +08:00
PowderLi 20fc90c591
enhance: find collection schema from cache (#28782) 
issue: #28781 #28329

1. There is no need to call `DescribeCollection`, if the collection's
schema is found in the globalMetaCache
2. did `GetProperties` to check the access to Azure Blob Service while
construct the ChunkManager

Signed-off-by: PowderLi <min.li@zilliz.com>
 2023-12-03 19:22:33 +08:00
aoiasd 89d8ce2f73
enhance: refine access log to support format access log by yaml and print name info. (#28319) 
relate: https://github.com/milvus-io/milvus/issues/28086

---------

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2023-11-28 15:32:31 +08:00
SimFG 9c46788d87
enhance: Support to trace restful request and request error (#28685) 
issue: #28348

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2023-11-27 20:14:26 +08:00
jaime c5f455dc6b
fix: cmux graceful shutdown on proxy service (#28383) 
issue #28305

Signed-off-by: jaime <yun.zhang@zilliz.com>
 2023-11-27 16:28:34 +08:00
MrPresent-Han fc30d291be
fix createCollection failed occasionally (#28592) (#28712) 
fix: create collection seldom failure #28592

Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
 2023-11-27 11:10:25 +08:00
smellthemoon 73f2bab454
enhance:add some log when create client and get component states (#28160) 
/kind improvement

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2023-11-22 09:12:22 +08:00
yah01 bfccfcd0ca
enhance: refine error messages (#28424) 
- Split the simple reason and full detail
- Refine existing error messages
related: #28422

---------

Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-11-21 17:02:24 +08:00
PowderLi c238bff9fb
fix: symbol 'GetStorageMetrics' and 'enableDynamicField' (#28580) 
/kind bug
to #28579 #28504

1. replace enableDynamic with enableDynamicField
2. cgo directly link to milvus_storage

Signed-off-by: PowderLi <min.li@zilliz.com>
 2023-11-21 10:20:22 +08:00
XuanYang-cn 40d5c902b6
Enable getting multiple segments in plan result (#28350) 
Compaction plan result contained one segment for one plan. For l0
compaction would write to multiple segments, this PR expand the segments
number in plan results and refactor some names for readibility.

- Name refactory: - CompactionStateResult -> CompactionPlanResult -
CompactionResult -> CompactionSegment

See also: #27606

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2023-11-14 15:56:19 +08:00
SimFG cfb6edea61
Support to trace the grpc request (#28349) 
Signed-off-by: SimFG <bang.fu@zilliz.com>
 2023-11-13 20:16:18 +08:00
wei liu 8a370f115a
Fix external grpc server doesn't graceful stop (#28309) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-11-10 00:14:18 +08:00
wei liu 64f6dbabdc
Fix proxy not healthy before graceful stop (#28223) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-11-08 02:22:26 +08:00
PowderLi 0c0f012e03
add a custom http header: Accept-Type-Allow-Int64 (#27901) 
Signed-off-by: PowderLi <min.li@zilliz.com>
 2023-11-01 11:42:16 +08:00
wei liu 0c69f48ba4
refine stop order (#28016) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-11-01 11:40:15 +08:00
yah01 9658367a3c
Refine chunk manager errors (#27590) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-10-31 12:18:15 +08:00
Filip Haltmayer 6b1a106a31
Moving etcd client into session (#27069) 
Signed-off-by: Filip Haltmayer <filip.haltmayer@zilliz.com>
 2023-10-27 07:36:12 +08:00
jaime bf7f32b025
Refine Proxy stop (#27919) 
Signed-off-by: jaime <yun.zhang@zilliz.com>
 2023-10-26 10:24:09 +08:00
jaime 4640928280
Fix initialization and backward compatibility issue for GRPC compression (#27894) 
Signed-off-by: jaime <yun.zhang@zilliz.com>
 2023-10-24 22:14:14 +08:00
SimFG 9b0ecbdca7
Support to replicate the mq message (#27240) 
Signed-off-by: SimFG <bang.fu@zilliz.com>
 2023-10-20 14:26:09 +08:00
PowderLi dcb7a89601
[restful] new interface: upsert (#27787) 
interface: delete support expression

Signed-off-by: PowderLi <min.li@zilliz.com>
 2023-10-20 09:54:11 +08:00
SimFG 630636c4ec
Support the apikey authentication for the restful api (#27758) 
Signed-off-by: SimFG <bang.fu@zilliz.com>
 2023-10-18 16:36:12 +08:00
SimFG 84d05b939b
Support the apikey authentication (#27723) 
Signed-off-by: SimFG <bang.fu@zilliz.com>
 2023-10-17 21:00:11 +08:00
congqixia 2f201c25e2
Remove deprecated io/ioutil usage (#27747) 
`io/ioutil` package is deprecated, use `io`,`os` package replacement
also added golangci-lint rule to block future reference

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
Co-authored-by: guoguangwu <guoguangwu@magic-shield.com>
 2023-10-17 20:32:09 +08:00
PowderLi 09d8b76048
[restful] new context with grpc metadata (#27668) 
Signed-off-by: PowderLi <min.li@zilliz.com>
 2023-10-17 20:00:14 +08:00
yah01 be980fbc38
Refine state check (#27541) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-10-11 21:01:35 +08:00
yah01 8394b3a1ec
Block creating new error from status reason (#27426) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-10-07 11:29:32 +08:00
Jiquan Long 0f14d18201
Optimize the codec code of session (#27360) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2023-10-01 10:33:30 +08:00
yah01 63ac43a3b8
Refine errors for import (#27379) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-09-30 10:31:28 +08:00
yah01 6539a5ae2c
Refine DataCoord status (#27262) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-09-26 17:15:27 +08:00
wei liu 27046e242b
add timeout for graceful stop (#27326) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-09-26 10:15:25 +08:00
jaime 7f7c71ea7d
Decoupling client and server API in types interface (#27186) 
Co-authored-by:: aoiasd <zhicheng.yue@zilliz.com>

Signed-off-by: jaime <yun.zhang@zilliz.com>
 2023-09-26 09:57:25 +08:00
XuanYang-cn 676024ff38
Add rpc to notify channel operation (#27172) 
- Add NotifyChannelOperation
- Add CheckChannelOperationProgress

See also: #25309

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2023-09-25 14:05:27 +08:00
SimFG 26f06dd732
Format the code (#27275) 
Signed-off-by: SimFG <bang.fu@zilliz.com>
 2023-09-21 09:45:27 +08:00
yah01 b4f86ea55e
Construct all success status with merr (#27226) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-09-20 10:57:23 +08:00
congqixia cc9974979f
Add staticcheck linter and fix existing problems (#27174) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-09-19 10:05:22 +08:00
yah01 168e82ee10
Fix panic while handling with the nil status (#27040) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-09-15 10:09:21 +08:00
congqixia f0d0651989
Do not reset connection immediately if grpc code is `Canceled` or `DeadlineExceeded` (#27014) 
We found lots of connection reset & canceled due to recent retry change
Current implementation resets connection no matter what the error code is
To sync behavior to previous retry, skip reset connection only if cancel error happens too much.

Also adds a config item for minResetInterval for grpc reset connection

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-09-13 15:01:18 +08:00
yihao.dai dd2cb1d44a
Flush by flush channels (#26859) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-09-12 21:07:19 +08:00
yah01 00c65fa0d7
Refine QueryNode errors (#27013) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-09-12 16:07:18 +08:00
yah01 45d9fb5929
Fix wrong error codes & names (#26904) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-09-12 14:41:18 +08:00
aoiasd e107d0794c
support complex delete expression (#25752) 
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2023-09-12 10:19:17 +08:00
XuanYang-cn 7f1ae35e72
Add timeout in dispatcher, AsConsumer and Seek (#26686) 
See also: #25309

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2023-09-08 09:51:17 +08:00
yiwangdr 337edc321b
tikv integration (#26246) 
Signed-off-by: yiwangdr <yiwangdr@gmail.com>
 2023-09-07 07:25:14 +08:00
Enwei Jiao fb0705df1b
Decouple basetable and componentparam (#26725) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-09-05 10:31:48 +08:00
yah01 3349db4aa7
Refine errors to remove changes breaking design (#26521) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-09-04 09:57:09 +08:00
congqixia e8f1b1736e
Remove log.Error(err.error())-style log (#26783) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-09-01 13:09:01 +08:00
smellthemoon e56b0018e4
Remove merr mask logic to make it ok to compare resp.status with origin error (#26773) 
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2023-09-01 11:09:00 +08:00
Xu Tong 5f08e3a72b
Fix field_name spell error (#26718) 
Signed-off-by: Writer-X <1256866856@qq.com>
 2023-08-31 11:39:02 +08:00
yah01 213db490bd
Use pointer receiver for large struct (#26668) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-08-30 10:24:29 +08:00
yah01 dd4bc5b6a0
Fix data race in gRPC client (#26574) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-08-28 18:26:28 +08:00
congqixia c2b53bb9fe
Return error response instead of panicking for unimplemented APIs (#26589) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-08-24 09:14:23 +08:00
yihao.dai c3128aaef3
Accelerate server id interceptor validation (#26468) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-08-20 21:24:19 +08:00
wayblink 71133e5eaa
Check and reset if grpc client serverID mismatch with session (#26448) 
Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2023-08-18 22:50:20 +08:00
yihao.dai 63b86b32a6
Add server id validation interceptor (#26395) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-08-17 20:20:20 +08:00
congqixia e13b6b88eb
Replace deprecated grpc WithInsecure option (#26226) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-08-09 13:37:16 +08:00
congqixia 2770ac4df5
Fix nilness linter errors (#26218) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-08-09 11:31:15 +08:00
congqixia b9850ce5c0
Fix copylocks linter errors (#26217) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-08-08 20:41:15 +08:00
PowderLi a7eecb1be0
support high-level RESTFUL API, listen on the same port as grpc. (#25108) 
Signed-off-by: PowderLi <min.li@zilliz.com>
 2023-08-08 10:15:07 +08:00
cai.zhang 760a2d9aa7
Support AllocTimestamp api for Milvus (#25784) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2023-07-25 10:05:00 +08:00
yiwangdr b9189b9f41
Organize mocks from types.go (#25466) 
Signed-off-by: yiwangdr <yiwangdr@gmail.com>
 2023-07-14 10:12:31 +08:00
jaime 18df2ba6fd
[Cherry-Pick] Support Database (#24769) 
Support Database(#23742)
Fix db nonexists error for FlushAll (#24222)
Fix check collection limits fails (#24235)
backward compatibility with empty DB name (#24317)
Fix GetFlushAllState with DB (#24347)
Remove db from global meta cache after drop database (#24474)
Fix db name is empty for describe collection response (#24603)
Add RBAC for Database API (#24653)
Fix miss load the same name collection during recover stage (#24941)

RBAC supports Database validation (#23609)
Fix to list grant with db return empty (#23922)
Optimize PrivilegeAll permission check (#23972)
Add the default db value for the rbac request (#24307)

Signed-off-by: jaime <yun.zhang@zilliz.com>
Co-authored-by: SimFG <bang.fu@zilliz.com>
Co-authored-by: longjiquan <jiquan.long@zilliz.com>
 2023-06-25 17:20:43 +08:00
yihao.dai 9f5756f16e
Add cluster validation interceptor to resolve the ip-reuse issue (#25001) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-06-21 11:36:46 +08:00
wayblink bfae6b49af
Remove datanode timetick mq, use rpc to report instead (#23156) 
Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2023-06-14 14:16:38 +08:00
congqixia 41af0a98fa
Use go-api/v2 for milvus-proto (#24770) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-06-09 01:28:37 +08:00
yihao.dai 2086b4f75d
Add RBAC for FlushAll (#24751) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-06-08 20:28:35 +08:00
yah01 ebd0279d3f
Check error by Error() and NoError() for better report message (#24736) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-06-08 15:36:36 +08:00
SimFG d246defa4d
Return error response when an unimplemented request is received (#24545) 
Signed-off-by: SimFG <bang.fu@zilliz.com>
 2023-05-31 20:24:31 +08:00
Preetham 0983d1fac7
Update: Wrap common calls datanode client (#24494) 
Signed-off-by: Preetham <kamidipreetham@gmail.com>
 2023-05-30 09:33:28 +08:00
Preetham 4ea0830c25
Update: Wrap common calls querycoord rootcoord clients (#24360) 
Signed-off-by: Preetham <kamidipreetham@gmail.com>
 2023-05-24 19:25:26 +08:00
Preetham 2f8032b55c
Update: Wrap common grpc calls indexnode and proxy clients (#24344) 
Signed-off-by: Preetham <kamidipreetham@gmail.com>
 2023-05-24 09:25:26 +08:00
wei liu ce03248b1a
simplify logic circle in search (#24110) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-05-23 16:01:26 +08:00
Preetham d8a3b9d07e
Update: Wrap common grpc calls datacoord client (#24311) 
Signed-off-by: Preetham <kamidipreetham@gmail.com>
 2023-05-23 11:43:25 +08:00
Jiquan Long bd343550a5
Support to manage connections (#24224) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2023-05-19 12:51:23 +08:00
Enwei Jiao 6865f93ad3
Refactor querynodeclient to remove duplicate code (#24195) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-05-18 11:35:23 +08:00
wayblink 899702f13c
Implement GetIndexStatistics interface (#23603) 
Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2023-05-06 10:34:39 +08:00
wayblink b7cb34b966
Add blank GetIndexStatistics interface (#23610) 
Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2023-04-23 14:50:32 +08:00
wei liu cbfe7a45ef
fix pull target (#23491) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-04-18 18:30:32 +08:00
jaime c9d0c157ec
Move some modules from internal to public package (#22572) 
Signed-off-by: jaime <yun.zhang@zilliz.com>
 2023-04-06 19:14:32 +08:00
yihao.dai a182594bf5
Add FlushAll support (#22942) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-03-30 11:28:25 +08:00
yah01 081572d31c
Refactor QueryNode (#21625) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
Co-authored-by: Congqi Xia <congqi.xia@zilliz.com>
Co-authored-by: aoiasd <zhicheng.yue@zilliz.com>
 2023-03-27 00:42:00 +08:00
congqixia 8f847884c2
Fix misc copylocks and remove exclude rule (#22962) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-03-24 11:17:58 +08:00
congqixia 980aaeceb9
Fill coordinators client target node id (#22899) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-03-22 10:07:57 +08:00
yihao.dai 1f718118e9
Dynamic load/release partitions (#22655) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-03-20 14:55:57 +08:00
congqixia 732986aa04
Remove fmt.Print from internal package (#22722) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-03-14 17:36:05 +08:00
Enwei Jiao 697dedac7e
Use cockroachdb/errors to replace other error pkg (#22390) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-02-26 11:31:49 +08:00
zhenshan.cao e768437681
Correct usage of Timer and Ticker (#22228) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2023-02-23 18:59:45 +08:00
Enwei Jiao 0a9a9058b9
Fix UT timeout (#22261) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-02-20 21:14:25 +08:00
wei liu 13b508103e
refine mock querycoord (#22198) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-02-16 15:38:34 +08:00
cai.zhang 66b3566ac1
Update component state to healthy after start (#22118) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2023-02-10 17:40:32 +08:00
wayblink 45ff568470
Fix probable ut fail caused by session key confict (#21883) 
Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2023-01-31 17:39:49 +08:00
congqixia d329cf33a0
Use embed etcd for ConnectManager unit test (#21884) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-01-31 13:35:49 +08:00
cai.zhang bcae97b125
Register after start to prevent there are tow coordinator at the same time (#21641) (#21707) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2023-01-30 11:11:50 +08:00
wei liu 73c44d4b29
resource group impl (#21609) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-01-30 10:19:48 +08:00
congqixia 5986106037
Make paramtable init only once (#21782) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-01-19 14:53:44 +08:00