0c83440f99
enhance: [Cherry-pick] cache collection schema attributes to reduce proxy cpu ( #29668 ) ( #29692 )
...
Cherry-pick from master
pr: #29668
See also #29113
The collection schema is crucial when performing search/query but some
of the information is calculated for every request.
This PR change schema field of cached collection info into a utility
`schemaInfo` type to store some stable result, say pk field,
partitionKeyEnabled, etc. And provided field name to id map for
search/query services.
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-01-07 22:36:48 +08:00
6eeb4b7f9a
enhance: [Cherry-Pick] Refine delete by expression for forbid proxy dml task scheduler hang ( #29359 )
...
relate: https://github.com/milvus-io/milvus/issues/29146
pr: https://github.com/milvus-io/milvus/pull/29340
---------
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2023-12-26 17:50:48 +08:00
15bad6485f
enhance: [cherry-pick] refine proxy meta cache partition logic ( #29315 ) ( #29378 )
...
Cherry-pick from master
pr: #29315
See also #29113
- Unify partition info refresh logic
- Prevent parse partition names for each partition key search request
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-12-21 14:16:42 +08:00
8ef0c571b3
enhance: make consistency level used in delete configurable ( #29280 ) ( #29284 )
...
pr: #29280
issue: #29279
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2023-12-20 14:14:34 +08:00
9af24da65a
fix: rootCoord decide the builtin role cannot be deleted ( #29248 )
...
issue: #29243
only rootCoord read the configuration item `builtinRoles`, so proxy
never know whether the role to be deleted is builtin.
Signed-off-by: PowderLi <min.li@zilliz.com>
2023-12-18 14:22:42 +08:00
10d3224638
enhance: add 3 builtin roles #28961 ( #29010 )
...
issue: #28960
master pr: #28961
add new configuration: builtinRoles
user can define roles in config file: milvus.yaml
there is an example:
db_ro, only have read privileges, include load
db_rw, read and write privileges, include create/drop/rename collection
db_admin, not only read and write privileges, but also user
administration
Signed-off-by: PowderLi <min.li@zilliz.com>
2023-12-08 10:52:36 +08:00
cdd1305f26
enhance: find collection schema from cache ( #28916 )
...
issue: #28781 #28329
master pr: #28782
1. There is no need to call `DescribeCollection`, if the collection's
schema is found in the globalMetaCache
2. did `GetProperties` to check the access to Azure Blob Service
Signed-off-by: PowderLi <min.li@zilliz.com>
2023-12-06 22:44:33 +08:00
c4be1ddf44
enhance: improve the error messages and logs ( #28684 ) ( #28726 )
...
- better name for log fields
- make the error and log consistent
pr: #28684
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-11-27 15:40:26 +08:00
9b0ecbdca7
Support to replicate the mq message ( #27240 )
...
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-10-20 14:26:09 +08:00
020ad9a6bc
Rectify wrong exception messages associated with Array datatype ( #27769 )
...
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2023-10-19 17:24:07 +08:00
630636c4ec
Support the apikey authentication for the restful api ( #27758 )
...
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-10-18 16:36:12 +08:00
09d8b76048
[restful] new context with grpc metadata ( #27668 )
...
Signed-off-by: PowderLi <min.li@zilliz.com>
2023-10-17 20:00:14 +08:00
6cbb67832f
Compatible with scalar index types marisa-trie and Ascending ( #27638 )
...
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2023-10-15 13:52:06 +08:00
3759857bc5
Refine Proxy errors ( #27499 )
...
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-10-09 10:09:33 +08:00
8394b3a1ec
Block creating new error from status reason ( #27426 )
...
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-10-07 11:29:32 +08:00
63ac43a3b8
Refine errors for import ( #27379 )
...
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-09-30 10:31:28 +08:00
6539a5ae2c
Refine DataCoord status ( #27262 )
...
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-09-26 17:15:27 +08:00
7f7c71ea7d
Decoupling client and server API in types interface ( #27186 )
...
Co-authored-by:: aoiasd <zhicheng.yue@zilliz.com>
Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-09-26 09:57:25 +08:00
26f06dd732
Format the code ( #27275 )
...
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-09-21 09:45:27 +08:00
a362bb1457
Support array datatype ( #26369 )
...
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2023-09-19 14:23:23 +08:00
cc9974979f
Add staticcheck linter and fix existing problems ( #27174 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-09-19 10:05:22 +08:00
168e82ee10
Fix panic while handling with the nil status ( #27040 )
...
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-09-15 10:09:21 +08:00
00c65fa0d7
Refine QueryNode errors ( #27013 )
...
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-09-12 16:07:18 +08:00
e107d0794c
support complex delete expression ( #25752 )
...
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2023-09-12 10:19:17 +08:00
9166011c4a
Add float16 vector ( #25852 )
...
Signed-off-by: Writer-X <1256866856@qq.com>
2023-09-08 10:03:16 +08:00
3349db4aa7
Refine errors to remove changes breaking design ( #26521 )
...
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-09-04 09:57:09 +08:00
8dc16b599b
Add binary metric types SUBSTRUCTURE/SUPERSTRUCTURE back ( #26766 )
...
Signed-off-by: Yudong Cai <yudong.cai@zilliz.com>
2023-08-31 20:07:00 +08:00
9311dc91ee
Clear error message in the delete request ( #26656 )
...
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-08-30 14:47:00 +08:00
87ecaac703
Add dynamic schema check in upsert ( #26644 )
...
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2023-08-30 10:52:26 +08:00
1e5836221a
Fix CollectionNotExists when search and retrieve vector ( #26524 )
...
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2023-08-22 17:06:22 +08:00
ca1349708b
Remove time travel ralted testcase ( #26119 )
...
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-08-10 18:53:17 +08:00
a7eecb1be0
support high-level RESTFUL API, listen on the same port as grpc. ( #25108 )
...
Signed-off-by: PowderLi <min.li@zilliz.com>
2023-08-08 10:15:07 +08:00
9a4761dcc7
Remove binary metrics TANIMOTO/SUPERSTRUCTURE/SUBSTRUCTURE ( #25708 )
...
Signed-off-by: Yudong Cai <yudong.cai@zilliz.com>
2023-07-19 16:16:58 +08:00
96c987ed62
Bulkinsert supports partition keys ( #25284 )
...
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2023-07-11 15:18:28 +08:00
8b9e3f1127
Fix max_length check ( #25207 )
...
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2023-06-30 11:50:26 +08:00
18df2ba6fd
[Cherry-Pick] Support Database ( #24769 )
...
Support Database(#23742 )
Fix db nonexists error for FlushAll (#24222 )
Fix check collection limits fails (#24235 )
backward compatibility with empty DB name (#24317 )
Fix GetFlushAllState with DB (#24347 )
Remove db from global meta cache after drop database (#24474 )
Fix db name is empty for describe collection response (#24603 )
Add RBAC for Database API (#24653 )
Fix miss load the same name collection during recover stage (#24941 )
RBAC supports Database validation (#23609 )
Fix to list grant with db return empty (#23922 )
Optimize PrivilegeAll permission check (#23972 )
Add the default db value for the rbac request (#24307 )
Signed-off-by: jaime <yun.zhang@zilliz.com>
Co-authored-by: SimFG <bang.fu@zilliz.com>
Co-authored-by: longjiquan <jiquan.long@zilliz.com>
2023-06-25 17:20:43 +08:00
ccf3f0066f
[Pick] Enable max result window limit ( #24986 )
...
Signed-off-by: chyezh <ye.zhen@zilliz.com>
2023-06-25 14:42:43 +08:00
3b5b50bda8
add autoID to varchar dataType ( #24907 )
...
Signed-off-by: shunjiezhao <939038111@qq.com>
2023-06-16 17:00:40 +08:00
59457eb75b
fix get partition progress return wrong error msg ( #24899 )
...
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-06-15 11:14:39 +08:00
41af0a98fa
Use go-api/v2 for milvus-proto ( #24770 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-06-09 01:28:37 +08:00
1593278f9d
add independent nq limit option ( #24632 )
...
Signed-off-by: chyezh <ye.zhen@zilliz.com>
2023-06-07 10:38:36 +08:00
732fe54775
Support partition Key ( #24047 )
...
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2023-06-06 10:24:34 +08:00
af42d7c2af
Fix crash when insert duplicate field datas ( #24559 )
...
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2023-05-31 20:32:31 +08:00
f7924724b0
Add consistency_level paramter in search/query request ( #24499 )
...
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2023-05-30 21:01:29 +08:00
7819a5733f
Hide dynamic field for describing collection ( #24312 )
...
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2023-05-23 14:27:25 +08:00
d16e18fd34
Add output fields for search/query results ( #24302 )
...
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2023-05-23 10:19:26 +08:00
98d86e2391
Return all dynamic field when retrieve json key ( #24205 )
...
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2023-05-19 09:41:25 +08:00
008285f849
Support dynamic schema for create collection ( #24176 )
...
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2023-05-18 09:33:24 +08:00
cb2a36ab52
Change output fields with star behavior ( #24162 )
...
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-05-17 12:41:22 +08:00
73a181d226
Fix get vector it timeout and improve some string const usage ( #24141 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-05-16 17:41:22 +08:00
congqixia
aoiasd
Jiquan Long
PowderLi
yah01
SimFG
zhenshan.cao
xige-16
jaime
cai.zhang
Xu Tong
Cai Yudong
smellthemoon
Enwei Jiao
groot
chyezh
Zhao Shunjie
wei liu
chyezh