7cf2be09b5
fix: Restore the MVCC functionality. ( #29749 ) ( #29802 )
...
When the TimeTravel functionality was previously removed, it
inadvertently affected the MVCC functionality within the system. This PR
aims to reintroduce the internal MVCC functionality as follows:
1. Add MvccTimestamp to the requests of Search/Query and the results of
Search internally.
2. When the delegator receives a Query/Search request and there is no
MVCC timestamp set in the request, set the delegator's current tsafe as
the MVCC timestamp of the request. If the request already has an MVCC
timestamp, do not modify it.
3. When the Proxy handles Search and triggers the second phase ReQuery,
divide the ReQuery into different shards and pass the MVCC timestamp to
the corresponding Query requests.
issue: #29656
pr: #29749
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2024-01-11 14:42:49 +08:00
0c83440f99
enhance: [Cherry-pick] cache collection schema attributes to reduce proxy cpu ( #29668 ) ( #29692 )
...
Cherry-pick from master
pr: #29668
See also #29113
The collection schema is crucial when performing search/query but some
of the information is calculated for every request.
This PR change schema field of cached collection info into a utility
`schemaInfo` type to store some stable result, say pk field,
partitionKeyEnabled, etc. And provided field name to id map for
search/query services.
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-01-07 22:36:48 +08:00
efef573c67
enhance: [cherry-pick] Add a config item for partition name as regexp feature ( #29154 ) ( #29183 )
...
Cherry pick from master
pr: #29154
See also #29177
Add a config item for partition name as regexp feature and disable it by
default
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-12-14 14:46:37 +08:00
96c8e8d587
fix: azure ListObjects ( #27931 ) ( #28894 )
...
issue: #27932
master pr: #27931
Signed-off-by: PowderLi <min.li@zilliz.com>
2023-12-01 18:34:31 +08:00
0fbfa5a9b0
Fix unstable search task ut ( #28101 ) ( #28120 )
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-11-03 00:34:15 +08:00
0f3e83a764
Construct plan directly when search with vector output ( #27963 )
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-10-27 10:32:13 +08:00
be980fbc38
Refine state check ( #27541 )
...
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-10-11 21:01:35 +08:00
7f7c71ea7d
Decoupling client and server API in types interface ( #27186 )
...
Co-authored-by:: aoiasd <zhicheng.yue@zilliz.com>
Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-09-26 09:57:25 +08:00
26f06dd732
Format the code ( #27275 )
...
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-09-21 09:45:27 +08:00
b4f86ea55e
Construct all success status with merr ( #27226 )
...
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-09-20 10:57:23 +08:00
168e82ee10
Fix panic while handling with the nil status ( #27040 )
...
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-09-15 10:09:21 +08:00
73512c72fd
Remove calc_distance ( #25663 )
...
Signed-off-by: Yudong Cai <yudong.cai@zilliz.com>
2023-07-18 14:23:20 +08:00
b9189b9f41
Organize mocks from types.go ( #25466 )
...
Signed-off-by: yiwangdr <yiwangdr@gmail.com>
2023-07-14 10:12:31 +08:00
4fdbc5af89
Support search with default params( #24516 ) ( #25255 )
...
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2023-07-03 17:00:24 +08:00
18df2ba6fd
[Cherry-Pick] Support Database ( #24769 )
...
Support Database(#23742 )
Fix db nonexists error for FlushAll (#24222 )
Fix check collection limits fails (#24235 )
backward compatibility with empty DB name (#24317 )
Fix GetFlushAllState with DB (#24347 )
Remove db from global meta cache after drop database (#24474 )
Fix db name is empty for describe collection response (#24603 )
Add RBAC for Database API (#24653 )
Fix miss load the same name collection during recover stage (#24941 )
RBAC supports Database validation (#23609 )
Fix to list grant with db return empty (#23922 )
Optimize PrivilegeAll permission check (#23972 )
Add the default db value for the rbac request (#24307 )
Signed-off-by: jaime <yun.zhang@zilliz.com>
Co-authored-by: SimFG <bang.fu@zilliz.com>
Co-authored-by: longjiquan <jiquan.long@zilliz.com>
2023-06-25 17:20:43 +08:00
3f4356df10
fix the spelling of `field` ( #25008 )
...
Signed-off-by: PowderLi <min.li@zilliz.com>
2023-06-21 14:00:42 +08:00
c9e456c6eb
Remove metric_type check and fix some minor bugs ( #24921 )
...
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
Co-authored-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2023-06-19 09:54:41 +08:00
46f7d903a3
enable look aside balancer on replica selection ( #24791 )
...
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-06-16 18:38:39 +08:00
a3437e0ab5
refactor replica selection and retry policy on channel ( #24367 )
...
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-06-13 10:20:37 +08:00
41af0a98fa
Use go-api/v2 for milvus-proto ( #24770 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-06-09 01:28:37 +08:00
ebd0279d3f
Check error by Error() and NoError() for better report message ( #24736 )
...
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-06-08 15:36:36 +08:00
ce03248b1a
simplify logic circle in search ( #24110 )
...
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-05-23 16:01:26 +08:00
ae6fccc3a3
Filter id field out if not specified ( #23982 )
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-05-09 17:26:41 +08:00
3827ac30bc
Remove load cache ( #23287 )
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-05-09 10:36:41 +08:00
092d743917
Add support for getting vectors by ids ( #23450 )
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-04-23 09:00:32 +08:00
03ec804e68
Change default shard number to 1 ( #23586 )
...
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2023-04-21 07:08:32 +08:00
c9d0c157ec
Move some modules from internal to public package ( #22572 )
...
Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-04-06 19:14:32 +08:00
bd5fab1e53
Remove merge policy of proxy RoundRobin policy ( #23021 )
...
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2023-03-31 14:36:22 +08:00
8b3e5189e1
Add querynode plugin ( #22379 )
...
Signed-off-by: chasingegg <chao.gao@zilliz.com>
2023-03-24 15:21:59 +08:00
872721e3ec
Refactor invalid unit test ( #22347 )
...
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2023-02-27 14:27:47 +08:00
697dedac7e
Use cockroachdb/errors to replace other error pkg ( #22390 )
...
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-02-26 11:31:49 +08:00
cc5d72a9f2
add search option to ignore ignore segment ( #21994 )
...
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2023-02-22 17:31:46 +08:00
13b508103e
refine mock querycoord ( #22198 )
...
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-02-16 15:38:34 +08:00
e3d1624e97
Handle distance Inf correctly ( #21828 )
...
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2023-02-01 14:59:51 +08:00
b01546edd3
Guard bad distance/score from knowhere ( #21034 )
...
/kind bug
issue: #21138
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-12-13 17:03:21 +08:00
89b810a4db
Refactor all params into ParamItem ( #20987 )
...
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2022-12-07 18:01:19 +08:00
956c5e1b9d
Make Params singleton ( #20088 )
...
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2022-11-04 14:25:38 +08:00
4c4db1f556
Display all shard error if search/query fails ( #20169 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-10-31 19:09:39 +08:00
a55f739608
Separate public proto files ( #19782 )
...
Signed-off-by: SimFG <bang.fu@zilliz.com>
Signed-off-by: SimFG <bang.fu@zilliz.com>
2022-10-16 20:49:27 +08:00
ae373d450f
Make NQ <= 16384 limit work ( #19726 )
...
https://milvus.io/docs/limitations.md
issue: #19682
/kind bug
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-10-12 18:37:23 +08:00
0ac4bc32a5
Add logic of autoindex ( #19570 )
...
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-10-08 15:38:58 +08:00
52c6a2706e
Fix binary results unstable ( #19401 )
...
See also: #19338 , #19366 , 19367
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-09-26 18:02:52 +08:00
7819297f68
Add more checks for search offset ( #19388 )
...
See also: #19250
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-09-26 18:00:57 +08:00
d7f38a803d
Separate some proto files ( #19218 )
...
Signed-off-by: SimFG <bang.fu@zilliz.com>
Signed-off-by: SimFG <bang.fu@zilliz.com>
2022-09-16 16:56:49 +08:00
200df76864
Enable search with pagination ( #19044 )
...
See also: #19043
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-09-14 20:36:32 +08:00
867ea63bdd
Move TopK check inside parseQueryInfo ( #18892 )
...
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-08-30 10:32:56 +08:00
dad060af2e
Fix func checkIfLoaded in proxy ( #18769 )
...
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2022-08-23 15:14:52 +08:00
0c9a10e8f8
1. refine logging interfaces ( #18692 )
...
2. adjust logs for query/search requests
Signed-off-by: Zach41 <zongmei.zhang@zilliz.com>
Signed-off-by: Zach41 <zongmei.zhang@zilliz.com>
2022-08-23 10:44:52 +08:00
e05e9cb9b4
Fix typo in error message ( #18688 )
...
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-08-19 10:48:50 +08:00
ff0200210a
Support Role-Based Access Control ( #18425 )
...
Signed-off-by: SimFG <bang.fu@zilliz.com>
2022-08-04 11:04:34 +08:00
zhenshan.cao
congqixia
PowderLi
yihao.dai
yah01
jaime
SimFG
Cai Yudong
yiwangdr
cai.zhang
wei liu
yah01
aoiasd
Gao
Enwei Jiao
Ten Thousand Leaves
XuanYang-cn
Zach