Commit Graph

1352 Commits (243e311515e06d50e7e37536cc9df061e9361f56)

Author SHA1 Message Date
XuanYang-cn b2e7cbdf4b
Remove TimeTravel in compactor ()
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2023-09-04 17:41:48 +08:00
yah01 3349db4aa7
Refine errors to remove changes breaking design ()
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-09-04 09:57:09 +08:00
yihao.dai 64cf5eab18
Accelerate flush in flushAll ()
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-09-01 15:23:01 +08:00
congqixia e8f1b1736e
Remove log.Error(err.error())-style log ()
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-09-01 13:09:01 +08:00
Cai Yudong 8dc16b599b
Add binary metric types SUBSTRUCTURE/SUPERSTRUCTURE back ()
Signed-off-by: Yudong Cai <yudong.cai@zilliz.com>
2023-08-31 20:07:00 +08:00
jaime d001133bd2
[skip e2e] Refine collection not exists error ()
Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-08-31 17:29:03 +08:00
Xu Tong 5f08e3a72b
Fix field_name spell error ()
Signed-off-by: Writer-X <1256866856@qq.com>
2023-08-31 11:39:02 +08:00
wei liu 2aab9b3234
fix proxy to querynode heartbeat failed counter logic ()
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-08-30 16:31:01 +08:00
SimFG 9311dc91ee
Clear error message in the delete request ()
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-08-30 14:47:00 +08:00
smellthemoon 87ecaac703
Add dynamic schema check in upsert ()
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2023-08-30 10:52:26 +08:00
yah01 213db490bd
Use pointer receiver for large struct ()
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-08-30 10:24:29 +08:00
wei liu 0bb68cac36
fix CollectionNotExist on describe rg ()
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-08-23 09:56:21 +08:00
xige-16 1e5836221a
Fix CollectionNotExists when search and retrieve vector ()
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2023-08-22 17:06:22 +08:00
yihao.dai 63b86b32a6
Add server id validation interceptor ()
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-08-17 20:20:20 +08:00
wei liu 4aac7a6642
refine error reason of query/search failed ()
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-08-16 19:38:18 +08:00
SimFG 19f210974f
Support to update the hook config in the runtime ()
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-08-16 16:03:33 +08:00
MrPresent-Han d30a920226
add log trace for segcore() ()
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
2023-08-16 11:41:33 +08:00
Enwei Jiao 78bc688d16
Remove QueryMsgStream in MqFactory interface ()
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-08-16 10:29:33 +08:00
congqixia f8bcf60e55
Adjust some confusing Warning log to INFO ()
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-08-15 17:49:33 +08:00
smellthemoon 248bbc749d
Fix GetCollectionID to return err ()
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2023-08-11 10:21:29 +08:00
wei liu 05c87af146
fix unexpected score in look aside balancer ()
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-08-10 19:45:16 +08:00
Enwei Jiao ca1349708b
Remove time travel ralted testcase ()
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-08-10 18:53:17 +08:00
wei liu f9823e039f
fix describe rg with non exist collection ()
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-08-10 18:43:16 +08:00
xige-16 1055c90456
Add default retrieve limit ()
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2023-08-10 14:11:15 +08:00
Jiquan Long 6578a27fe7
Support empty expression ()
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2023-08-10 08:37:15 +08:00
yah01 d267559bd0
Fix incorrect error message while validating insert data ()
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-08-09 20:33:15 +08:00
yihao.dai 6472a50349
Fix rate limit not woring for upsert ()
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-08-09 14:29:16 +08:00
congqixia b9850ce5c0
Fix copylocks linter errors ()
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-08-08 20:41:15 +08:00
PowderLi a7eecb1be0
support high-level RESTFUL API, listen on the same port as grpc. ()
Signed-off-by: PowderLi <min.li@zilliz.com>
2023-08-08 10:15:07 +08:00
SimFG 4e1b65d38f
Refine the rbac cache update process ()
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-08-07 11:59:07 +08:00
SimFG d2649b63db
Delete the user-role mapping info when deleting the user ()
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-08-04 18:37:08 +08:00
chyezh f915f1739a
[Fixup] Short-term fix metacache data race ()
Signed-off-by: chyezh <ye.zhen@zilliz.com>
2023-08-04 17:47:07 +08:00
wei liu 518b6310a2
refine retry times on replica ()
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-08-03 15:55:09 +08:00
jaime 6663e753e6
Check if database exists when Connect was called ()
Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-08-03 15:53:07 +08:00
Jiquan Long 517fb95207
Fix useless log ()
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2023-08-03 15:33:06 +08:00
Jiquan Long 5c1f79dc54
Push down the limit operator to segcore ()
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2023-08-01 20:29:05 +08:00
smellthemoon 9614e61f14
Fix collection and channel not match ()
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2023-08-01 17:33:06 +08:00
XuanYang-cn 84253f255e
Fix datanode graceful stop panic ()
See also: 

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2023-07-28 10:11:08 +08:00
MrPresent-Han b5e79e7f34
add metrics for assign-segment on proxy() ()
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
2023-07-26 14:15:01 +08:00
wei liu 302897f866
refine look aside balance logic ()
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-07-25 18:51:01 +08:00
cai.zhang 760a2d9aa7
Support AllocTimestamp api for Milvus ()
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2023-07-25 10:05:00 +08:00
congqixia 3c503afe7c
Use typeutil.ConcurrentMap instead of sync.Map ()
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-07-24 10:23:01 +08:00
smellthemoon a2f001e132
Remove max length check when autoid==true set by varchar primary field ()
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2023-07-21 11:30:59 +08:00
Cai Yudong 9a4761dcc7
Remove binary metrics TANIMOTO/SUPERSTRUCTURE/SUBSTRUCTURE ()
Signed-off-by: Yudong Cai <yudong.cai@zilliz.com>
2023-07-19 16:16:58 +08:00
Cai Yudong 73512c72fd
Remove calc_distance ()
Signed-off-by: Yudong Cai <yudong.cai@zilliz.com>
2023-07-18 14:23:20 +08:00
aoiasd 77a9553c3f
Add some log when node being reachable or unreachable ()
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2023-07-18 10:35:20 +08:00
yah01 948d1f1f4a
Handle errors by merr for QueryCoord ()
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-07-17 14:59:34 +08:00
wei liu 04a2ddda17
refine rated log of look aside balancer ()
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-07-17 14:49:34 +08:00
zhenshan.cao c4f009073c
Make query set guarantee ts based on default consistency level ()
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2023-07-17 09:58:38 +08:00
SimFG 69d274d233
Improve the operation log ()
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-07-14 16:08:31 +08:00
yihao.dai f19621fc8c
Support refund the tokens to limiter ()
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-07-14 15:58:32 +08:00
Enwei Jiao 66fdc71479
Refactor logs in DataCoord & DataNode ()
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-07-14 15:56:31 +08:00
yiwangdr b9189b9f41
Organize mocks from types.go ()
Signed-off-by: yiwangdr <yiwangdr@gmail.com>
2023-07-14 10:12:31 +08:00
wei liu fc9a6dc241
fix compute score on replica selection ()
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-07-14 10:02:31 +08:00
Jiquan Long a577cac46b
Fix incorrect IP distance ()
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2023-07-12 19:28:29 +08:00
groot 96c987ed62
Bulkinsert supports partition keys ()
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2023-07-11 15:18:28 +08:00
smellthemoon d63323d117
Add rate limit and deny write in upsert ()
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2023-07-11 11:20:34 +08:00
wei liu 986208bca0
fix set wrong consistency level in proxy ()
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-07-11 10:06:28 +08:00
wei liu b7b249770c
enable metrics expire for replica selection ()
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-07-06 19:04:25 +08:00
wei liu 951c312ef9
refine grafana metrics for replica selection ()
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-07-05 17:00:25 +08:00
Enwei Jiao 816158e4af
Remove outdated searchplan ()
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-07-04 18:30:25 +08:00
chyezh d7d61f529c
[Feature|Pick] enable scheduler policy and add user-task-polling policy ()
Signed-off-by: chyezh <ye.zhen@zilliz.com>
2023-07-03 18:24:25 +08:00
cai.zhang 4fdbc5af89
Support search with default params() ()
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2023-07-03 17:00:24 +08:00
smellthemoon 948d04cdd8
Check field type when create index ()
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2023-07-03 15:26:26 +08:00
xige-16 8b9e3f1127
Fix max_length check ()
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2023-06-30 11:50:26 +08:00
smellthemoon b30517d303
Enlarge timeout to prevent health check failure ()
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2023-06-29 17:10:23 +08:00
yihao.dai 37fe3393d1
Remove const to enable move semantics and improve schema util functions ()
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-06-29 11:04:22 +08:00
wei liu fe24228909
fix unstable ut in look aside balancer ()
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-06-28 10:52:45 +08:00
wei liu 9b91519c88
fix search cost metrics on replica selection ()
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-06-27 19:02:44 +08:00
congqixia 0fefc52aac
Make lookAsideBalancer checkHealth in parallel ()
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-06-27 15:18:45 +08:00
wei liu 17796743dd
add metrics for replica selection ()
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-06-27 09:52:44 +08:00
xige-16 33c2012675
Add more metrics ()
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2023-06-26 17:52:44 +08:00
wei liu 7b999b42bd
enable config policy on replica selection ()
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-06-25 19:46:44 +08:00
jaime 18df2ba6fd
[Cherry-Pick] Support Database ()
Support Database()
Fix db nonexists error for FlushAll ()
Fix check collection limits fails ()
backward compatibility with empty DB name ()
Fix GetFlushAllState with DB ()
Remove db from global meta cache after drop database ()
Fix db name is empty for describe collection response ()
Add RBAC for Database API ()
Fix miss load the same name collection during recover stage ()

RBAC supports Database validation ()
Fix to list grant with db return empty ()
Optimize PrivilegeAll permission check ()
Add the default db value for the rbac request ()

Signed-off-by: jaime <yun.zhang@zilliz.com>
Co-authored-by: SimFG <bang.fu@zilliz.com>
Co-authored-by: longjiquan <jiquan.long@zilliz.com>
2023-06-25 17:20:43 +08:00
chyezh ccf3f0066f
[Pick] Enable max result window limit ()
Signed-off-by: chyezh <ye.zhen@zilliz.com>
2023-06-25 14:42:43 +08:00
xige-16 04082b3de2
Migrate the ability to upload and download binlog to cpp ()
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2023-06-25 14:38:44 +08:00
PowderLi 3f4356df10
fix the spelling of `field` ()
Signed-off-by: PowderLi <min.li@zilliz.com>
2023-06-21 14:00:42 +08:00
Enwei Jiao d143682d7d
Refactor logs in proxy package. ()
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-06-19 13:28:41 +08:00
cai.zhang c9e456c6eb
Remove metric_type check and fix some minor bugs ()
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
Co-authored-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2023-06-19 09:54:41 +08:00
wei liu 46f7d903a3
enable look aside balancer on replica selection ()
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-06-16 18:38:39 +08:00
Zhao Shunjie 3b5b50bda8
add autoID to varchar dataType ()
Signed-off-by: shunjiezhao <939038111@qq.com>
2023-06-16 17:00:40 +08:00
yihao.dai b62429070c
Set pchannels before dml enqueue to prevent panic ()
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-06-16 16:36:40 +08:00
cai.zhang c81bdda55a
Hide autoindex params details () ()
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
Co-authored-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2023-06-16 14:30:40 +08:00
wei liu 59457eb75b
fix get partition progress return wrong error msg ()
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-06-15 11:14:39 +08:00
smellthemoon db31e88a73
Add length check when insert and upsert ()
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2023-06-15 10:24:38 +08:00
wei liu a3437e0ab5
refactor replica selection and retry policy on channel ()
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-06-13 10:20:37 +08:00
congqixia 864c1bbcee
Use collInfo instead of reading again ()
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-06-09 20:26:35 +08:00
congqixia 41af0a98fa
Use go-api/v2 for milvus-proto ()
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-06-09 01:28:37 +08:00
yihao.dai 2086b4f75d
Add RBAC for FlushAll ()
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-06-08 20:28:35 +08:00
yah01 ebd0279d3f
Check error by Error() and NoError() for better report message ()
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-06-08 15:36:36 +08:00
yah01 fa69c71d35
Fix data race in MetaCache ()
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-06-08 13:58:35 +08:00
yihao.dai d6dea8bcbb
Fix getting partitions in Proxy meta cache ()
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-06-08 11:14:35 +08:00
Enwei Jiao d3af451d92
Upgrade golangci-lint ()
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-06-07 19:34:36 +08:00
chyezh 1593278f9d
add independent nq limit option ()
Signed-off-by: chyezh <ye.zhen@zilliz.com>
2023-06-07 10:38:36 +08:00
xige-16 732fe54775
Support partition Key ()
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2023-06-06 10:24:34 +08:00
Jiquan Long 283bddec3b
Fix show loaded collections ()
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2023-06-02 18:04:35 +08:00
cai.zhang 93ea9c4925
Support return pending index rows when describe index ()
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2023-06-01 18:14:32 +08:00
xige-16 af42d7c2af
Fix crash when insert duplicate field datas ()
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2023-05-31 20:32:31 +08:00
SimFG d246defa4d
Return error response when an unimplemented request is received ()
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-05-31 20:24:31 +08:00
smellthemoon bb62cf4e60
Support upsert api rbac ()
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2023-05-31 19:01:09 +08:00
zhenshan.cao f7924724b0
Add consistency_level paramter in search/query request ()
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2023-05-30 21:01:29 +08:00
smellthemoon 708c724ccc
fix default value store unknown type ()
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2023-05-30 14:51:28 +08:00
yihao.dai 0fb9131973
Improve rate limiter log ()
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-05-30 11:23:28 +08:00
Jiquan Long 29ae1229b6
Support AutoIndex ()
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2023-05-29 20:35:28 +08:00
yihao.dai 5efa49f112
Fix ddl check in rate limiter ()
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-05-26 17:37:30 +08:00
cai.zhang 6209d5d717
Remove unused code for creating index ()
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2023-05-26 16:49:26 +08:00
yihao.dai 6e17aa2608
Improve rate limiter interceptor ()
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-05-25 14:55:32 +08:00
cai.zhang 59b0a61cb2
Fill outputFields for query/search result ()
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2023-05-25 11:39:26 +08:00
wei liu ce03248b1a
simplify logic circle in search ()
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-05-23 16:01:26 +08:00
cai.zhang 7819a5733f
Hide dynamic field for describing collection ()
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2023-05-23 14:27:25 +08:00
cai.zhang d16e18fd34
Add output fields for search/query results ()
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2023-05-23 10:19:26 +08:00
cai.zhang ce67d024ed
Fill is_dynamic flag for output field data ()
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2023-05-22 14:01:25 +08:00
yihao.dai 3f96c335bb
Add receivedNQ metric and change receivedBytes metric to collection level ()
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-05-21 22:23:24 +08:00
Jiquan Long bd343550a5
Support to manage connections ()
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2023-05-19 12:51:23 +08:00
cai.zhang 98d86e2391
Return all dynamic field when retrieve json key ()
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2023-05-19 09:41:25 +08:00
congqixia 4502a16819
Fix dynamic field attribute is not set when describe ()
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-05-18 12:55:23 +08:00
cai.zhang 008285f849
Support dynamic schema for create collection ()
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2023-05-18 09:33:24 +08:00
MrPresent-Han b74770b636
support try-best query reducer methods() ()
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
2023-05-17 16:23:23 +08:00
Enwei Jiao cb2a36ab52
Change output fields with star behavior ()
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-05-17 12:41:22 +08:00
Jiquan Long 6965495b9d
Check overflow for inserted integer ()
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2023-05-16 20:19:22 +08:00
congqixia 73a181d226
Fix get vector it timeout and improve some string const usage ()
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-05-16 17:41:22 +08:00
SimFG ef82e0f1a4
Fix the error when a role removes an already deleted user () ()
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-05-16 10:15:22 +08:00
smellthemoon 8a85dd6869
Support Default Value ()
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2023-05-15 16:15:21 +08:00
wei liu 4369915402
fix rate limit ut ()
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-05-15 11:13:21 +08:00
wei liu 4c956fab73
enable config collection level rate limit ()
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-05-12 18:13:26 +08:00
wei liu 4cd9f32dd6
enable global static rate limit ()
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-05-12 15:49:20 +08:00
yah01 7d0c47dd65
Check the length of JSON ()
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-05-12 11:33:20 +08:00
smellthemoon 146050db82
Fix some wrong ut ()
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2023-05-10 09:31:19 +08:00
yihao.dai ae6fccc3a3
Filter id field out if not specified ()
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-05-09 17:26:41 +08:00
wei liu d06ae53f79
make ddl rate limit be global level ()
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-05-09 16:56:40 +08:00
congqixia f6d3b4f7ac
Fix concurrent map read/write in rate limiter ()
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-05-09 11:00:40 +08:00
yihao.dai 3827ac30bc
Remove load cache ()
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-05-09 10:36:41 +08:00
Enwei Jiao d7edd1b99b
Fix parse token error ()
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-05-08 10:28:39 +08:00
wayblink e13d900398
Add some log in getIndexStatisticsTask ()
Signed-off-by: wayblink <anyang.wang@zilliz.com>
2023-05-08 10:02:39 +08:00
smellthemoon dcf55e77b5
Remove superfluous idempotent in proxy ()
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2023-05-07 21:08:39 +08:00
Jiquan Long 7be7e6f360
Refactor check logic of index parameters ()
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2023-05-06 10:40:39 +08:00
wayblink 899702f13c
Implement GetIndexStatistics interface ()
Signed-off-by: wayblink <anyang.wang@zilliz.com>
2023-05-06 10:34:39 +08:00
Gao c1e8406c44
Override index type as AUTOINDEX when autoindex is enabled ()
Signed-off-by: chasingegg <gaoc96@qq.com>
2023-04-30 11:50:40 +08:00
yihao.dai 6f94bfd26d
Add constraint of releasing partitions before dropping ()
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-04-28 11:12:36 +08:00
SimFG 5cd21893c8
Fix superusers' password verification problem ()
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-04-26 21:16:34 +08:00
yah01 f8ff97fe29
Fix travel timestamp set unique leads to disable task merge ()
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-04-25 19:04:34 +08:00
wei liu 4fb8919a97
support collection level rate limit ()
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-04-25 15:54:35 +08:00
wayblink b7cb34b966
Add blank GetIndexStatistics interface ()
Signed-off-by: wayblink <anyang.wang@zilliz.com>
2023-04-23 14:50:32 +08:00
yihao.dai 092d743917
Add support for getting vectors by ids ()
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-04-23 09:00:32 +08:00
zhenshan.cao 03ec804e68
Change default shard number to 1 ()
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2023-04-21 07:08:32 +08:00
Enwei Jiao 967a97b9bd
Support json & array types ()
Signed-off-by: yah01 <yang.cen@zilliz.com>
Co-authored-by: yah01 <yang.cen@zilliz.com>
2023-04-20 11:32:31 +08:00
zhenshan.cao a88178a5ea
Add slow log for read ()
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2023-04-14 15:30:29 +08:00
cai.zhang 43a9e175a3
Exit component process when session key is deleted () ()
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2023-04-12 20:12:28 +08:00
yah01 296380d6e6
Support async refresh ()
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-04-12 15:06:28 +08:00
Jiquan Long a940df94c7
Forbid counting entities with pagination ()
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2023-04-12 14:22:34 +08:00
congqixia f1f8ce01de
Remove querynode v1 package ()
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-04-11 11:28:31 +08:00
yihao.dai 0b354cbab1
Fix timetick block caused by dml task pop failed ()
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-04-11 11:00:31 +08:00
cai.zhang bd65ef8e95
Fix bug for type params when indexing () ()
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2023-04-09 16:18:29 +08:00
congqixia 132c010271
Fix proxy panicking when QueryCoord offline ()
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-04-07 17:30:28 +08:00
congqixia 32be374c33
Fix MetaCache ut internal not changed ()
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-04-07 13:20:28 +08:00
jaime c9d0c157ec
Move some modules from internal to public package ()
Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-04-06 19:14:32 +08:00
Jiquan Long 2c1c3aecfc
Fix metric ProxySyncTimeTickLag ()
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2023-04-06 15:28:33 +08:00
smellthemoon ce91f61e5a
Pre-create topics ()
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2023-04-03 16:44:23 +08:00
yihao.dai a0ca4d6108
Improve search/query logs in Proxy ()
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-04-03 10:54:23 +08:00
aoiasd bd5fab1e53
Remove merge policy of proxy RoundRobin policy ()
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2023-03-31 14:36:22 +08:00
zhenshan.cao 1287ca699a
Refine usage of TimeRecorder.Record ()
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2023-03-30 18:56:22 +08:00
yihao.dai a182594bf5
Add FlushAll support ()
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-03-30 11:28:25 +08:00
Jiquan Long 0f2c3ac832
Optimize log ()
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2023-03-30 10:44:22 +08:00
smellthemoon b1992e84cc
Delete dup check ()
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2023-03-29 10:06:01 +08:00
wei liu ed60b434fa
fix unstable ut ()
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-03-28 19:06:01 +08:00
yah01 081572d31c
Refactor QueryNode ()
Signed-off-by: yah01 <yang.cen@zilliz.com>
Co-authored-by: Congqi Xia <congqi.xia@zilliz.com>
Co-authored-by: aoiasd <zhicheng.yue@zilliz.com>
2023-03-27 00:42:00 +08:00
cai.zhang 977943463e
Check index params and type params ()
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2023-03-26 22:15:59 +08:00
Jiquan Long 0be96ab610
Check if all columns aligned with same num_rows ()
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2023-03-24 17:09:58 +08:00
smellthemoon 3dae84f065
Use merr in proxy ()
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2023-03-24 15:27:58 +08:00
Gao 8b3e5189e1
Add querynode plugin ()
Signed-off-by: chasingegg <chao.gao@zilliz.com>
2023-03-24 15:21:59 +08:00
SimFG e8f8c1b445
Check the collection num when creating the collection ()
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-03-23 16:47:57 +08:00
smellthemoon 4c603cd02c
Fix upsert msgid ()
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2023-03-21 14:06:01 +08:00
SimFG 8259ca6929
Add the user rpc counter ()
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-03-21 11:37:56 +08:00
yihao.dai 1f718118e9
Dynamic load/release partitions ()
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-03-20 14:55:57 +08:00
wei liu 5f571ea915
remove debug log ()
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-03-17 18:05:56 +08:00
yah01 99f31dc37a
Fix data after travel timestamp is still visible ()
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-03-17 18:03:55 +08:00
Jiquan Long dff15c3488
Check dimension of inserted records ()
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2023-03-17 17:33:58 +08:00
wei liu 6b5dfa6db2
deprecate shard cache immediately if query failed ()
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-03-17 16:25:55 +08:00
congqixia a085af931d
Fix metrics info panic when network error happens () ()
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-03-17 11:55:54 +08:00
Jiquan Long 8139106b51
Feat: count entities by expression ()
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2023-03-16 19:31:55 +08:00
wei liu 69cc68caff
support first or random strategy ()
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-03-16 14:25:55 +08:00
zhenshan.cao ecde42a493
Fix proxy's pchan statistic min ts out of order ()
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2023-03-15 11:17:55 +08:00
congqixia 732986aa04
Remove fmt.Print from internal package ()
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-03-14 17:36:05 +08:00
MrPresent-Han 89618b7e12
fix wrong sequence of timetick stats on proxy ()
Signed-off-by: MrPresent-Han <jamesharden11122@gmail.com>
2023-03-14 15:43:54 +08:00
groot 6f6bd98c27
Verify vector float data for bulkinsert and insert ()
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2023-03-14 14:03:58 +08:00
yah01 fa2c1c1f40
Fix RootCoord double updates TSO ()
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-03-13 18:01:53 +08:00
congqixia 3ffa6d5302
Mark cache deprecated instead of removing it ()
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-03-10 10:19:53 +08:00
yah01 e581fef115
Use the request's context to allocate ts ()
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-03-09 18:51:52 +08:00
congqixia d9908ff231
Fix TestMetaCache_ExpireShardLeaderCache unit test ()
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-03-09 18:25:52 +08:00
yah01 90a5aa6265
Refine errors, re-define error codes ()
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-03-09 15:47:52 +08:00
bigsheeper f994052099
Reduce printing frequency of rate limit log ()
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-03-09 14:15:55 +08:00
wei liu 1a1a5caafa
update shard leader cache periodically ()
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-03-08 15:13:51 +08:00
smellthemoon 29b3690c83
Upsert raise error ()
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2023-03-06 14:29:51 +08:00
jaime d126f06946
Decouple mq module from internal proto definition ()
Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-03-04 23:21:50 +08:00
aoiasd 872721e3ec
Refactor invalid unit test ()
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2023-02-27 14:27:47 +08:00
Xiaofan 3d6bded115
Fix Row Number Mismatch ()
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2023-02-27 10:41:46 +08:00
Enwei Jiao 697dedac7e
Use cockroachdb/errors to replace other error pkg ()
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-02-26 11:31:49 +08:00
zhenshan.cao e768437681
Correct usage of Timer and Ticker ()
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2023-02-23 18:59:45 +08:00
Enwei Jiao 0851e05014
Gernate milvus.yaml by code ()
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-02-23 11:37:46 +08:00
SimFG 7e19a924b7
Add the rbac for the GetLoadingProgress and GetLoadState api ()
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-02-22 20:37:45 +08:00
aoiasd cc5d72a9f2
add search option to ignore ignore segment ()
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2023-02-22 17:31:46 +08:00
congqixia abbf1efa0e
[Cherry-Pick] Fix read wrong variable caused nil error ()
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-02-21 18:38:26 +08:00
jaime 5936723904
Refine Read method of MinioChunkManager ()
Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-02-21 16:22:26 +08:00
Enwei Jiao 0a9a9058b9
Fix UT timeout ()
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-02-20 21:14:25 +08:00
wei liu 13b508103e
refine mock querycoord ()
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-02-16 15:38:34 +08:00
bigsheeper f4e7b246f2
Fix msgdispatcher concurrency issue and add prometheus metrics ()
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-02-16 14:58:34 +08:00
wei liu 87a4ddc7e2
fix rg e2e ()
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-02-16 10:48:34 +08:00
congqixia f2575e5fa8
Add unconvert & durationcheck linters and fix issues ()
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-02-15 17:22:34 +08:00
SimFG c2a49d5a0b
Add error log and metrics for the hook and privilege interceptor () ()
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-02-13 16:50:33 +08:00
bigsheeper d2667064bb
Add msgDispatcher to support sharing msgs for different vChannel ()
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-02-13 16:38:33 +08:00
wei liu d078441522
support rg api rbac ()
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-02-10 10:54:33 +08:00
wei liu d085abbd56
fix load collection with rg ()
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-02-09 16:24:31 +08:00
wei liu 6fb60629be
fix describe rg api ()
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-02-07 19:30:31 +08:00
wei liu aced41d5d8
add resource group name rule ()
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-02-07 17:52:31 +08:00
Ten Thousand Leaves 74e3cc64fb
Add refresh path to proxy ()
/kind improvement

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2023-02-02 19:29:51 +08:00
Enwei Jiao 53ae40b8c6
Make ratelimiter's config refreshable ()
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-02-01 16:03:51 +08:00
Cai Yudong e3d1624e97
Handle distance Inf correctly ()
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2023-02-01 14:59:51 +08:00
Xiaofan f1daef22a1
Fix failed bulkload segment marked as sealed ()
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2023-01-31 12:41:53 +08:00
smellthemoon bb086f47e4
Swap execution flow order in enqueue ()
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2023-01-30 10:27:50 +08:00
wei liu 73c44d4b29
resource group impl ()
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-01-30 10:19:48 +08:00
jaime d7bb908148
Fix rename collection data race ()
Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-01-24 10:01:46 +08:00
congqixia 5986106037
Make paramtable init only once ()
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-01-19 14:53:44 +08:00
jaime 1dcd9eeb79
Support rename collection ()
Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-01-19 14:13:43 +08:00
smellthemoon ad6cbc990f
Combine similar function ()
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2023-01-17 17:53:42 +08:00
Enwei Jiao 90d9e165d4
Fix some configs not shown ()
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-01-13 15:31:41 +08:00
smellthemoon 4620aebd46
Fix data race ()
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2023-01-13 14:29:41 +08:00
wayblink 6a722396bd
Integration test framework ()
Signed-off-by: wayblink <anyang.wang@zilliz.com>
2023-01-12 19:49:40 +08:00
wei liu c67070ab1f
add resource group interface ()
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-01-12 18:31:41 +08:00
Enwei Jiao fb42466c65
Use opentelemetry ()
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-01-12 16:09:39 +08:00
Jiquan Long 9fd9bed2b9
Remove collection meta after GC finished ()
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2023-01-12 09:55:42 +08:00
cai.zhang e127cf7b99
Reset indexpb for upgrade ()
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2023-01-11 14:35:40 +08:00
liliu-z cc5ecabac6
Skip search GRPC call for standalon ()
Signed-off-by: Li Liu <li.liu@zilliz.com>
2023-01-10 21:09:38 +08:00
bigsheeper 2146af1fb2
Return insufficient memory error when load failed ()
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-01-10 20:35:39 +08:00
bigsheeper 79535931ab
Return error code when deny to read/write ()
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-01-06 14:31:37 +08:00
cai.zhang aa203acfb3
Low IndexCoord weight ()
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2023-01-06 14:21:37 +08:00
Jiquan Long 6e521dd4b3
Optimize GetLoadingProgress to save RPC ()
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2023-01-04 20:33:35 +08:00
cai.zhang e5f408dceb
Merge IndexCoord and DataCoord ()
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2023-01-04 19:37:36 +08:00
yah01 7b39873ae0
limit the frequency of GetMetrics() log ()
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-01-04 17:39:35 +08:00
smellthemoon bf3c02155a
Support update ()
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2023-01-04 17:21:36 +08:00
Jiquan Long ff2a68e65a
Fix collection not exist when tried to do recovery ()
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2023-01-04 16:37:35 +08:00
bigsheeper c187de7754
Show trigger reason when deny to read/write ()
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-12-30 18:35:32 +08:00
Jiquan Long de9ed42623
Forbid deleting entities by non-pk field ()
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2022-12-30 14:17:32 +08:00
congqixia daa4492e95
Invalidate cache first when shard leader list contains error ()
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-12-29 23:29:32 +08:00
jaime a8a1b0b3e8
Remove logs that print insert results ()
Signed-off-by: jaime <yun.zhang@zilliz.com>
2022-12-27 12:49:29 +08:00
smellthemoon cded2ce77c
Fix offset panic ()
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2022-12-27 11:11:29 +08:00
congqixia 74db56dece
Fix cache refetch logic not applied to Query ()
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-12-16 16:49:23 +08:00
yah01 9fa58968b0
Fix proxy doesn't update shard leaders cache for some types of error ()
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-12-16 16:05:24 +08:00
Enwei Jiao 166e9f0da5
Refactor GrpcConfig ()
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>

Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2022-12-16 15:59:23 +08:00
SimFG 63cd4132a6
Implement the `GetLoadState` api ()
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
2022-12-16 14:39:24 +08:00
Enwei Jiao e6374901f9
Make TopK Limit configurable ()
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>

Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2022-12-15 14:03:22 +08:00