Commit Graph

1748 Commits (2.5)

Author SHA1 Message Date
Cai Yudong 8dc16b599b
Add binary metric types SUBSTRUCTURE/SUPERSTRUCTURE back (#26766)
Signed-off-by: Yudong Cai <yudong.cai@zilliz.com>
2023-08-31 20:07:00 +08:00
jaime d001133bd2
[skip e2e] Refine collection not exists error (#26707)
Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-08-31 17:29:03 +08:00
Xu Tong 5f08e3a72b
Fix field_name spell error (#26718)
Signed-off-by: Writer-X <1256866856@qq.com>
2023-08-31 11:39:02 +08:00
wei liu 2aab9b3234
fix proxy to querynode heartbeat failed counter logic (#26563)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-08-30 16:31:01 +08:00
SimFG 9311dc91ee
Clear error message in the delete request (#26656)
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-08-30 14:47:00 +08:00
smellthemoon 87ecaac703
Add dynamic schema check in upsert (#26644)
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2023-08-30 10:52:26 +08:00
yah01 213db490bd
Use pointer receiver for large struct (#26668)
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-08-30 10:24:29 +08:00
wei liu 0bb68cac36
fix CollectionNotExist on describe rg (#26541)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-08-23 09:56:21 +08:00
xige-16 1e5836221a
Fix CollectionNotExists when search and retrieve vector (#26524)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2023-08-22 17:06:22 +08:00
yihao.dai 63b86b32a6
Add server id validation interceptor (#26395)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-08-17 20:20:20 +08:00
wei liu 4aac7a6642
refine error reason of query/search failed (#26342)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-08-16 19:38:18 +08:00
SimFG 19f210974f
Support to update the hook config in the runtime (#26350)
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-08-16 16:03:33 +08:00
MrPresent-Han d30a920226
add log trace for segcore(#26277) (#26339)
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
2023-08-16 11:41:33 +08:00
Enwei Jiao 78bc688d16
Remove QueryMsgStream in MqFactory interface (#26374)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-08-16 10:29:33 +08:00
congqixia f8bcf60e55
Adjust some confusing Warning log to INFO (#26356)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-08-15 17:49:33 +08:00
smellthemoon 248bbc749d
Fix GetCollectionID to return err (#26239)
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2023-08-11 10:21:29 +08:00
wei liu 05c87af146
fix unexpected score in look aside balancer (#26213)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-08-10 19:45:16 +08:00
Enwei Jiao ca1349708b
Remove time travel ralted testcase (#26119)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-08-10 18:53:17 +08:00
wei liu f9823e039f
fix describe rg with non exist collection (#26227)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-08-10 18:43:16 +08:00
xige-16 1055c90456
Add default retrieve limit (#24782)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2023-08-10 14:11:15 +08:00
Jiquan Long 6578a27fe7
Support empty expression (#26153)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2023-08-10 08:37:15 +08:00
yah01 d267559bd0
Fix incorrect error message while validating insert data (#26187)
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-08-09 20:33:15 +08:00
yihao.dai 6472a50349
Fix rate limit not woring for upsert (#26127)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-08-09 14:29:16 +08:00
congqixia b9850ce5c0
Fix copylocks linter errors (#26217)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-08-08 20:41:15 +08:00
PowderLi a7eecb1be0
support high-level RESTFUL API, listen on the same port as grpc. (#25108)
Signed-off-by: PowderLi <min.li@zilliz.com>
2023-08-08 10:15:07 +08:00
SimFG 4e1b65d38f
Refine the rbac cache update process (#26150)
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-08-07 11:59:07 +08:00
SimFG d2649b63db
Delete the user-role mapping info when deleting the user (#25988)
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-08-04 18:37:08 +08:00
chyezh f915f1739a
[Fixup] Short-term fix metacache data race (#25802)
Signed-off-by: chyezh <ye.zhen@zilliz.com>
2023-08-04 17:47:07 +08:00
wei liu 518b6310a2
refine retry times on replica (#26043)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-08-03 15:55:09 +08:00
jaime 6663e753e6
Check if database exists when Connect was called (#26115)
Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-08-03 15:53:07 +08:00
Jiquan Long 517fb95207
Fix useless log (#26058)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2023-08-03 15:33:06 +08:00
Jiquan Long 5c1f79dc54
Push down the limit operator to segcore (#25959)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2023-08-01 20:29:05 +08:00
smellthemoon 9614e61f14
Fix collection and channel not match (#25859)
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2023-08-01 17:33:06 +08:00
XuanYang-cn 84253f255e
Fix datanode graceful stop panic (#25932)
See also: #25925

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2023-07-28 10:11:08 +08:00
MrPresent-Han b5e79e7f34
add metrics for assign-segment on proxy(#24570) (#25907)
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
2023-07-26 14:15:01 +08:00
wei liu 302897f866
refine look aside balance logic (#25837)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-07-25 18:51:01 +08:00
cai.zhang 760a2d9aa7
Support AllocTimestamp api for Milvus (#25784)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2023-07-25 10:05:00 +08:00
congqixia 3c503afe7c
Use typeutil.ConcurrentMap instead of sync.Map (#25846)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-07-24 10:23:01 +08:00
smellthemoon a2f001e132
Remove max length check when autoid==true set by varchar primary field (#25728)
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2023-07-21 11:30:59 +08:00
Cai Yudong 9a4761dcc7
Remove binary metrics TANIMOTO/SUPERSTRUCTURE/SUBSTRUCTURE (#25708)
Signed-off-by: Yudong Cai <yudong.cai@zilliz.com>
2023-07-19 16:16:58 +08:00
Cai Yudong 73512c72fd
Remove calc_distance (#25663)
Signed-off-by: Yudong Cai <yudong.cai@zilliz.com>
2023-07-18 14:23:20 +08:00
aoiasd 77a9553c3f
Add some log when node being reachable or unreachable (#25572)
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2023-07-18 10:35:20 +08:00
yah01 948d1f1f4a
Handle errors by merr for QueryCoord (#24926)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-07-17 14:59:34 +08:00
wei liu 04a2ddda17
refine rated log of look aside balancer (#25637)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-07-17 14:49:34 +08:00
zhenshan.cao c4f009073c
Make query set guarantee ts based on default consistency level (#25578)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2023-07-17 09:58:38 +08:00
SimFG 69d274d233
Improve the operation log (#25589)
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-07-14 16:08:31 +08:00
yihao.dai f19621fc8c
Support refund the tokens to limiter (#25598)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-07-14 15:58:32 +08:00
Enwei Jiao 66fdc71479
Refactor logs in DataCoord & DataNode (#25574)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-07-14 15:56:31 +08:00
yiwangdr b9189b9f41
Organize mocks from types.go (#25466)
Signed-off-by: yiwangdr <yiwangdr@gmail.com>
2023-07-14 10:12:31 +08:00
wei liu fc9a6dc241
fix compute score on replica selection (#25582)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-07-14 10:02:31 +08:00
Jiquan Long a577cac46b
Fix incorrect IP distance (#25527)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2023-07-12 19:28:29 +08:00
groot 96c987ed62
Bulkinsert supports partition keys (#25284)
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2023-07-11 15:18:28 +08:00
smellthemoon d63323d117
Add rate limit and deny write in upsert (#25351)
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2023-07-11 11:20:34 +08:00
wei liu 986208bca0
fix set wrong consistency level in proxy (#25458)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-07-11 10:06:28 +08:00
wei liu b7b249770c
enable metrics expire for replica selection (#25360)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-07-06 19:04:25 +08:00
wei liu 951c312ef9
refine grafana metrics for replica selection (#25319)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-07-05 17:00:25 +08:00
Enwei Jiao 816158e4af
Remove outdated searchplan (#25282)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-07-04 18:30:25 +08:00
chyezh d7d61f529c
[Feature|Pick] enable scheduler policy and add user-task-polling policy (#24839)
Signed-off-by: chyezh <ye.zhen@zilliz.com>
2023-07-03 18:24:25 +08:00
cai.zhang 4fdbc5af89
Support search with default params(#24516) (#25255)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2023-07-03 17:00:24 +08:00
smellthemoon 948d04cdd8
Check field type when create index (#25248)
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2023-07-03 15:26:26 +08:00
xige-16 8b9e3f1127
Fix max_length check (#25207)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2023-06-30 11:50:26 +08:00
smellthemoon b30517d303
Enlarge timeout to prevent health check failure (#25173)
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2023-06-29 17:10:23 +08:00
yihao.dai 37fe3393d1
Remove const to enable move semantics and improve schema util functions (#25193)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-06-29 11:04:22 +08:00
wei liu fe24228909
fix unstable ut in look aside balancer (#25184)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-06-28 10:52:45 +08:00
wei liu 9b91519c88
fix search cost metrics on replica selection (#25059)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-06-27 19:02:44 +08:00
congqixia 0fefc52aac
Make lookAsideBalancer checkHealth in parallel (#25155)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-06-27 15:18:45 +08:00
wei liu 17796743dd
add metrics for replica selection (#25037)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-06-27 09:52:44 +08:00
xige-16 33c2012675
Add more metrics (#25081)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2023-06-26 17:52:44 +08:00
wei liu 7b999b42bd
enable config policy on replica selection (#25067)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-06-25 19:46:44 +08:00
jaime 18df2ba6fd
[Cherry-Pick] Support Database (#24769)
Support Database(#23742)
Fix db nonexists error for FlushAll (#24222)
Fix check collection limits fails (#24235)
backward compatibility with empty DB name (#24317)
Fix GetFlushAllState with DB (#24347)
Remove db from global meta cache after drop database (#24474)
Fix db name is empty for describe collection response (#24603)
Add RBAC for Database API (#24653)
Fix miss load the same name collection during recover stage (#24941)

RBAC supports Database validation (#23609)
Fix to list grant with db return empty (#23922)
Optimize PrivilegeAll permission check (#23972)
Add the default db value for the rbac request (#24307)

Signed-off-by: jaime <yun.zhang@zilliz.com>
Co-authored-by: SimFG <bang.fu@zilliz.com>
Co-authored-by: longjiquan <jiquan.long@zilliz.com>
2023-06-25 17:20:43 +08:00
chyezh ccf3f0066f
[Pick] Enable max result window limit (#24986)
Signed-off-by: chyezh <ye.zhen@zilliz.com>
2023-06-25 14:42:43 +08:00
xige-16 04082b3de2
Migrate the ability to upload and download binlog to cpp (#22984)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2023-06-25 14:38:44 +08:00
PowderLi 3f4356df10
fix the spelling of `field` (#25008)
Signed-off-by: PowderLi <min.li@zilliz.com>
2023-06-21 14:00:42 +08:00
Enwei Jiao d143682d7d
Refactor logs in proxy package. (#24936)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-06-19 13:28:41 +08:00
cai.zhang c9e456c6eb
Remove metric_type check and fix some minor bugs (#24921)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
Co-authored-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2023-06-19 09:54:41 +08:00
wei liu 46f7d903a3
enable look aside balancer on replica selection (#24791)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-06-16 18:38:39 +08:00
Zhao Shunjie 3b5b50bda8
add autoID to varchar dataType (#24907)
Signed-off-by: shunjiezhao <939038111@qq.com>
2023-06-16 17:00:40 +08:00
yihao.dai b62429070c
Set pchannels before dml enqueue to prevent panic (#24828)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-06-16 16:36:40 +08:00
cai.zhang c81bdda55a
Hide autoindex params details (#24851) (#24935)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
Co-authored-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2023-06-16 14:30:40 +08:00
wei liu 59457eb75b
fix get partition progress return wrong error msg (#24899)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-06-15 11:14:39 +08:00
smellthemoon db31e88a73
Add length check when insert and upsert (#24759)
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2023-06-15 10:24:38 +08:00
wei liu a3437e0ab5
refactor replica selection and retry policy on channel (#24367)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-06-13 10:20:37 +08:00
congqixia 864c1bbcee
Use collInfo instead of reading again (#24784)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-06-09 20:26:35 +08:00
congqixia 41af0a98fa
Use go-api/v2 for milvus-proto (#24770)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-06-09 01:28:37 +08:00
yihao.dai 2086b4f75d
Add RBAC for FlushAll (#24751)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-06-08 20:28:35 +08:00
yah01 ebd0279d3f
Check error by Error() and NoError() for better report message (#24736)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-06-08 15:36:36 +08:00
yah01 fa69c71d35
Fix data race in MetaCache (#24739)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-06-08 13:58:35 +08:00
yihao.dai d6dea8bcbb
Fix getting partitions in Proxy meta cache (#24719)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-06-08 11:14:35 +08:00
Enwei Jiao d3af451d92
Upgrade golangci-lint (#24707)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-06-07 19:34:36 +08:00
chyezh 1593278f9d
add independent nq limit option (#24632)
Signed-off-by: chyezh <ye.zhen@zilliz.com>
2023-06-07 10:38:36 +08:00
xige-16 732fe54775
Support partition Key (#24047)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2023-06-06 10:24:34 +08:00
Jiquan Long 283bddec3b
Fix show loaded collections (#24628)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2023-06-02 18:04:35 +08:00
cai.zhang 93ea9c4925
Support return pending index rows when describe index (#24588)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2023-06-01 18:14:32 +08:00
xige-16 af42d7c2af
Fix crash when insert duplicate field datas (#24559)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2023-05-31 20:32:31 +08:00
SimFG d246defa4d
Return error response when an unimplemented request is received (#24545)
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-05-31 20:24:31 +08:00
smellthemoon bb62cf4e60
Support upsert api rbac (#24550)
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2023-05-31 19:01:09 +08:00
zhenshan.cao f7924724b0
Add consistency_level paramter in search/query request (#24499)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2023-05-30 21:01:29 +08:00
smellthemoon 708c724ccc
fix default value store unknown type (#24477)
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2023-05-30 14:51:28 +08:00
yihao.dai 0fb9131973
Improve rate limiter log (#24459)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-05-30 11:23:28 +08:00
Jiquan Long 29ae1229b6
Support AutoIndex (#24387)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2023-05-29 20:35:28 +08:00
yihao.dai 5efa49f112
Fix ddl check in rate limiter (#24436)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-05-26 17:37:30 +08:00
cai.zhang 6209d5d717
Remove unused code for creating index (#24442)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2023-05-26 16:49:26 +08:00
yihao.dai 6e17aa2608
Improve rate limiter interceptor (#24389)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-05-25 14:55:32 +08:00
cai.zhang 59b0a61cb2
Fill outputFields for query/search result (#24352)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2023-05-25 11:39:26 +08:00
wei liu ce03248b1a
simplify logic circle in search (#24110)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-05-23 16:01:26 +08:00
cai.zhang 7819a5733f
Hide dynamic field for describing collection (#24312)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2023-05-23 14:27:25 +08:00
cai.zhang d16e18fd34
Add output fields for search/query results (#24302)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2023-05-23 10:19:26 +08:00
cai.zhang ce67d024ed
Fill is_dynamic flag for output field data (#24283)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2023-05-22 14:01:25 +08:00
yihao.dai 3f96c335bb
Add receivedNQ metric and change receivedBytes metric to collection level (#24199)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-05-21 22:23:24 +08:00
Jiquan Long bd343550a5
Support to manage connections (#24224)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2023-05-19 12:51:23 +08:00
cai.zhang 98d86e2391
Return all dynamic field when retrieve json key (#24205)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2023-05-19 09:41:25 +08:00
congqixia 4502a16819
Fix dynamic field attribute is not set when describe (#24208)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-05-18 12:55:23 +08:00
cai.zhang 008285f849
Support dynamic schema for create collection (#24176)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2023-05-18 09:33:24 +08:00
MrPresent-Han b74770b636
support try-best query reducer methods(#24075) (#24103)
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
2023-05-17 16:23:23 +08:00
Enwei Jiao cb2a36ab52
Change output fields with star behavior (#24162)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-05-17 12:41:22 +08:00
Jiquan Long 6965495b9d
Check overflow for inserted integer (#24142)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2023-05-16 20:19:22 +08:00
congqixia 73a181d226
Fix get vector it timeout and improve some string const usage (#24141)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-05-16 17:41:22 +08:00
SimFG ef82e0f1a4
Fix the error when a role removes an already deleted user (#24049) (#24109)
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-05-16 10:15:22 +08:00
smellthemoon 8a85dd6869
Support Default Value (#23448)
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2023-05-15 16:15:21 +08:00
wei liu 4369915402
fix rate limit ut (#24096)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-05-15 11:13:21 +08:00
wei liu 4c956fab73
enable config collection level rate limit (#24012)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-05-12 18:13:26 +08:00
wei liu 4cd9f32dd6
enable global static rate limit (#24009)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-05-12 15:49:20 +08:00
yah01 7d0c47dd65
Check the length of JSON (#24035)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-05-12 11:33:20 +08:00
smellthemoon 146050db82
Fix some wrong ut (#23990)
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2023-05-10 09:31:19 +08:00
yihao.dai ae6fccc3a3
Filter id field out if not specified (#23982)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-05-09 17:26:41 +08:00
wei liu d06ae53f79
make ddl rate limit be global level (#23962)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-05-09 16:56:40 +08:00
congqixia f6d3b4f7ac
Fix concurrent map read/write in rate limiter (#23958)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-05-09 11:00:40 +08:00
yihao.dai 3827ac30bc
Remove load cache (#23287)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-05-09 10:36:41 +08:00
Enwei Jiao d7edd1b99b
Fix parse token error (#23909)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-05-08 10:28:39 +08:00
wayblink e13d900398
Add some log in getIndexStatisticsTask (#23897)
Signed-off-by: wayblink <anyang.wang@zilliz.com>
2023-05-08 10:02:39 +08:00
smellthemoon dcf55e77b5
Remove superfluous idempotent in proxy (#23894)
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2023-05-07 21:08:39 +08:00
Jiquan Long 7be7e6f360
Refactor check logic of index parameters (#23856)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2023-05-06 10:40:39 +08:00
wayblink 899702f13c
Implement GetIndexStatistics interface (#23603)
Signed-off-by: wayblink <anyang.wang@zilliz.com>
2023-05-06 10:34:39 +08:00
Gao c1e8406c44
Override index type as AUTOINDEX when autoindex is enabled (#23744)
Signed-off-by: chasingegg <gaoc96@qq.com>
2023-04-30 11:50:40 +08:00
yihao.dai 6f94bfd26d
Add constraint of releasing partitions before dropping (#23732)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-04-28 11:12:36 +08:00
SimFG 5cd21893c8
Fix superusers' password verification problem (#23733)
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-04-26 21:16:34 +08:00
yah01 f8ff97fe29
Fix travel timestamp set unique leads to disable task merge (#23605)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-04-25 19:04:34 +08:00
wei liu 4fb8919a97
support collection level rate limit (#22767)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-04-25 15:54:35 +08:00
wayblink b7cb34b966
Add blank GetIndexStatistics interface (#23610)
Signed-off-by: wayblink <anyang.wang@zilliz.com>
2023-04-23 14:50:32 +08:00
yihao.dai 092d743917
Add support for getting vectors by ids (#23450)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-04-23 09:00:32 +08:00
zhenshan.cao 03ec804e68
Change default shard number to 1 (#23586)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2023-04-21 07:08:32 +08:00
Enwei Jiao 967a97b9bd
Support json & array types (#23408)
Signed-off-by: yah01 <yang.cen@zilliz.com>
Co-authored-by: yah01 <yang.cen@zilliz.com>
2023-04-20 11:32:31 +08:00
zhenshan.cao a88178a5ea
Add slow log for read (#23272)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2023-04-14 15:30:29 +08:00
cai.zhang 43a9e175a3
Exit component process when session key is deleted (#21658) (#22164)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2023-04-12 20:12:28 +08:00
yah01 296380d6e6
Support async refresh (#23107)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-04-12 15:06:28 +08:00
Jiquan Long a940df94c7
Forbid counting entities with pagination (#23375)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2023-04-12 14:22:34 +08:00
congqixia f1f8ce01de
Remove querynode v1 package (#23289)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-04-11 11:28:31 +08:00
yihao.dai 0b354cbab1
Fix timetick block caused by dml task pop failed (#23277)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-04-11 11:00:31 +08:00
cai.zhang bd65ef8e95
Fix bug for type params when indexing (#23242) (#23250)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2023-04-09 16:18:29 +08:00
congqixia 132c010271
Fix proxy panicking when QueryCoord offline (#23254)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-04-07 17:30:28 +08:00
congqixia 32be374c33
Fix MetaCache ut internal not changed (#23276)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-04-07 13:20:28 +08:00
jaime c9d0c157ec
Move some modules from internal to public package (#22572)
Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-04-06 19:14:32 +08:00
Jiquan Long 2c1c3aecfc
Fix metric ProxySyncTimeTickLag (#23235)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2023-04-06 15:28:33 +08:00
smellthemoon ce91f61e5a
Pre-create topics (#21907)
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2023-04-03 16:44:23 +08:00
yihao.dai a0ca4d6108
Improve search/query logs in Proxy (#23144)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-04-03 10:54:23 +08:00
aoiasd bd5fab1e53
Remove merge policy of proxy RoundRobin policy (#23021)
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2023-03-31 14:36:22 +08:00
zhenshan.cao 1287ca699a
Refine usage of TimeRecorder.Record (#23142)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2023-03-30 18:56:22 +08:00
yihao.dai a182594bf5
Add FlushAll support (#22942)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-03-30 11:28:25 +08:00
Jiquan Long 0f2c3ac832
Optimize log (#23121)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2023-03-30 10:44:22 +08:00
smellthemoon b1992e84cc
Delete dup check (#23076)
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2023-03-29 10:06:01 +08:00
wei liu ed60b434fa
fix unstable ut (#23070)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-03-28 19:06:01 +08:00
yah01 081572d31c
Refactor QueryNode (#21625)
Signed-off-by: yah01 <yang.cen@zilliz.com>
Co-authored-by: Congqi Xia <congqi.xia@zilliz.com>
Co-authored-by: aoiasd <zhicheng.yue@zilliz.com>
2023-03-27 00:42:00 +08:00
cai.zhang 977943463e
Check index params and type params (#22988)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2023-03-26 22:15:59 +08:00
Jiquan Long 0be96ab610
Check if all columns aligned with same num_rows (#22968)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2023-03-24 17:09:58 +08:00
smellthemoon 3dae84f065
Use merr in proxy (#22904)
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2023-03-24 15:27:58 +08:00
Gao 8b3e5189e1
Add querynode plugin (#22379)
Signed-off-by: chasingegg <chao.gao@zilliz.com>
2023-03-24 15:21:59 +08:00
SimFG e8f8c1b445
Check the collection num when creating the collection (#22946)
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-03-23 16:47:57 +08:00
smellthemoon 4c603cd02c
Fix upsert msgid (#22839)
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2023-03-21 14:06:01 +08:00
SimFG 8259ca6929
Add the user rpc counter (#22872)
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-03-21 11:37:56 +08:00
yihao.dai 1f718118e9
Dynamic load/release partitions (#22655)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-03-20 14:55:57 +08:00
wei liu 5f571ea915
remove debug log (#22828)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-03-17 18:05:56 +08:00
yah01 99f31dc37a
Fix data after travel timestamp is still visible (#22813)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-03-17 18:03:55 +08:00
Jiquan Long dff15c3488
Check dimension of inserted records (#22819)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2023-03-17 17:33:58 +08:00
wei liu 6b5dfa6db2
deprecate shard cache immediately if query failed (#22779)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-03-17 16:25:55 +08:00
congqixia a085af931d
Fix metrics info panic when network error happens (#22802) (#22809)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-03-17 11:55:54 +08:00
Jiquan Long 8139106b51
Feat: count entities by expression (#22765)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2023-03-16 19:31:55 +08:00
wei liu 69cc68caff
support first or random strategy (#22766)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-03-16 14:25:55 +08:00
zhenshan.cao ecde42a493
Fix proxy's pchan statistic min ts out of order (#22757)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2023-03-15 11:17:55 +08:00
congqixia 732986aa04
Remove fmt.Print from internal package (#22722)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-03-14 17:36:05 +08:00
MrPresent-Han 89618b7e12
fix wrong sequence of timetick stats on proxy #21855 (#22685)
Signed-off-by: MrPresent-Han <jamesharden11122@gmail.com>
2023-03-14 15:43:54 +08:00
groot 6f6bd98c27
Verify vector float data for bulkinsert and insert (#22728)
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2023-03-14 14:03:58 +08:00
yah01 fa2c1c1f40
Fix RootCoord double updates TSO (#22715)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-03-13 18:01:53 +08:00
congqixia 3ffa6d5302
Mark cache deprecated instead of removing it (#22674)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-03-10 10:19:53 +08:00
yah01 e581fef115
Use the request's context to allocate ts (#22633)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-03-09 18:51:52 +08:00
congqixia d9908ff231
Fix TestMetaCache_ExpireShardLeaderCache unit test (#22651)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-03-09 18:25:52 +08:00
yah01 90a5aa6265
Refine errors, re-define error codes (#22501)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-03-09 15:47:52 +08:00
bigsheeper f994052099
Reduce printing frequency of rate limit log (#22636)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-03-09 14:15:55 +08:00
wei liu 1a1a5caafa
update shard leader cache periodically (#22595)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-03-08 15:13:51 +08:00
smellthemoon 29b3690c83
Upsert raise error (#22533)
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2023-03-06 14:29:51 +08:00
jaime d126f06946
Decouple mq module from internal proto definition (#22536)
Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-03-04 23:21:50 +08:00
aoiasd 872721e3ec
Refactor invalid unit test (#22347)
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2023-02-27 14:27:47 +08:00
Xiaofan 3d6bded115
Fix Row Number Mismatch (#22307)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2023-02-27 10:41:46 +08:00
Enwei Jiao 697dedac7e
Use cockroachdb/errors to replace other error pkg (#22390)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-02-26 11:31:49 +08:00
zhenshan.cao e768437681
Correct usage of Timer and Ticker (#22228)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2023-02-23 18:59:45 +08:00
Enwei Jiao 0851e05014
Gernate milvus.yaml by code (#22003)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-02-23 11:37:46 +08:00
SimFG 7e19a924b7
Add the rbac for the GetLoadingProgress and GetLoadState api (#22320)
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-02-22 20:37:45 +08:00
aoiasd cc5d72a9f2
add search option to ignore ignore segment (#21994)
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2023-02-22 17:31:46 +08:00
congqixia abbf1efa0e
[Cherry-Pick] Fix read wrong variable caused nil error (#22297)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-02-21 18:38:26 +08:00
jaime 5936723904
Refine Read method of MinioChunkManager (#22235)
Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-02-21 16:22:26 +08:00
Enwei Jiao 0a9a9058b9
Fix UT timeout (#22261)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-02-20 21:14:25 +08:00
wei liu 13b508103e
refine mock querycoord (#22198)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-02-16 15:38:34 +08:00
bigsheeper f4e7b246f2
Fix msgdispatcher concurrency issue and add prometheus metrics (#22174)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-02-16 14:58:34 +08:00
wei liu 87a4ddc7e2
fix rg e2e (#22187)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-02-16 10:48:34 +08:00
congqixia f2575e5fa8
Add unconvert & durationcheck linters and fix issues (#22161)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-02-15 17:22:34 +08:00
SimFG c2a49d5a0b
Add error log and metrics for the hook and privilege interceptor (#22111) (#22137)
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-02-13 16:50:33 +08:00
bigsheeper d2667064bb
Add msgDispatcher to support sharing msgs for different vChannel (#21917)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-02-13 16:38:33 +08:00
wei liu d078441522
support rg api rbac (#22097)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-02-10 10:54:33 +08:00
wei liu d085abbd56
fix load collection with rg (#22083)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-02-09 16:24:31 +08:00
wei liu 6fb60629be
fix describe rg api (#21998)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-02-07 19:30:31 +08:00
wei liu aced41d5d8
add resource group name rule (#21992)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-02-07 17:52:31 +08:00
Ten Thousand Leaves 74e3cc64fb
Add refresh path to proxy (#21933)
/kind improvement

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2023-02-02 19:29:51 +08:00
Enwei Jiao 53ae40b8c6
Make ratelimiter's config refreshable (#21757)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-02-01 16:03:51 +08:00
Cai Yudong e3d1624e97
Handle distance Inf correctly (#21828)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2023-02-01 14:59:51 +08:00
Xiaofan f1daef22a1
Fix failed bulkload segment marked as sealed (#21799)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2023-01-31 12:41:53 +08:00
smellthemoon bb086f47e4
Swap execution flow order in enqueue (#21845)
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2023-01-30 10:27:50 +08:00
wei liu 73c44d4b29
resource group impl (#21609)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-01-30 10:19:48 +08:00
jaime d7bb908148
Fix rename collection data race (#21827)
Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-01-24 10:01:46 +08:00
congqixia 5986106037
Make paramtable init only once (#21782)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-01-19 14:53:44 +08:00
jaime 1dcd9eeb79
Support rename collection (#21693)
Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-01-19 14:13:43 +08:00
smellthemoon ad6cbc990f
Combine similar function (#21752)
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2023-01-17 17:53:42 +08:00
Enwei Jiao 90d9e165d4
Fix some configs not shown (#21653)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-01-13 15:31:41 +08:00
smellthemoon 4620aebd46
Fix data race (#21706)
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2023-01-13 14:29:41 +08:00
wayblink 6a722396bd
Integration test framework (#21283)
Signed-off-by: wayblink <anyang.wang@zilliz.com>
2023-01-12 19:49:40 +08:00
wei liu c67070ab1f
add resource group interface (#21676)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-01-12 18:31:41 +08:00
Enwei Jiao fb42466c65
Use opentelemetry (#21509)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-01-12 16:09:39 +08:00
Jiquan Long 9fd9bed2b9
Remove collection meta after GC finished (#21595)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2023-01-12 09:55:42 +08:00
cai.zhang e127cf7b99
Reset indexpb for upgrade (#21620)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2023-01-11 14:35:40 +08:00
liliu-z cc5ecabac6
Skip search GRPC call for standalon (#21383)
Signed-off-by: Li Liu <li.liu@zilliz.com>
2023-01-10 21:09:38 +08:00
bigsheeper 2146af1fb2
Return insufficient memory error when load failed (#21574)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-01-10 20:35:39 +08:00
bigsheeper 79535931ab
Return error code when deny to read/write (#21561)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-01-06 14:31:37 +08:00
cai.zhang aa203acfb3
Low IndexCoord weight (#21548)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2023-01-06 14:21:37 +08:00
Jiquan Long 6e521dd4b3
Optimize GetLoadingProgress to save RPC (#21493)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2023-01-04 20:33:35 +08:00
cai.zhang e5f408dceb
Merge IndexCoord and DataCoord (#21267)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2023-01-04 19:37:36 +08:00
yah01 7b39873ae0
limit the frequency of GetMetrics() log (#21514)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-01-04 17:39:35 +08:00
smellthemoon bf3c02155a
Support update (#20875)
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2023-01-04 17:21:36 +08:00
Jiquan Long ff2a68e65a
Fix collection not exist when tried to do recovery (#21471)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2023-01-04 16:37:35 +08:00
bigsheeper c187de7754
Show trigger reason when deny to read/write (#21368)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-12-30 18:35:32 +08:00
Jiquan Long de9ed42623
Forbid deleting entities by non-pk field (#21459)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2022-12-30 14:17:32 +08:00
congqixia daa4492e95
Invalidate cache first when shard leader list contains error (#21451)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-12-29 23:29:32 +08:00
jaime a8a1b0b3e8
Remove logs that print insert results (#21394)
Signed-off-by: jaime <yun.zhang@zilliz.com>
2022-12-27 12:49:29 +08:00
smellthemoon cded2ce77c
Fix offset panic (#21292)
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2022-12-27 11:11:29 +08:00
congqixia 74db56dece
Fix cache refetch logic not applied to Query (#21281)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-12-16 16:49:23 +08:00
yah01 9fa58968b0
Fix proxy doesn't update shard leaders cache for some types of error (#21185)
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-12-16 16:05:24 +08:00
Enwei Jiao 166e9f0da5
Refactor GrpcConfig (#21142)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>

Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2022-12-16 15:59:23 +08:00
SimFG 63cd4132a6
Implement the `GetLoadState` api (#21257)
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
2022-12-16 14:39:24 +08:00
Enwei Jiao e6374901f9
Make TopK Limit configurable (#21151)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>

Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2022-12-15 14:03:22 +08:00
zhenshan.cao ce3e0fcf9e
Fixbug: proxy meta_cache partially update (#21232) (#21242)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-12-14 20:45:23 +08:00
SimFG f31d5facff
Fix the unsafe casbin `Model` (#21129)
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
2022-12-14 10:29:22 +08:00
SimFG 655db658c8
Empty implement of the `GetLoadState` api (#21205)
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
2022-12-14 10:23:22 +08:00
Ten Thousand Leaves b01546edd3
Guard bad distance/score from knowhere (#21034)
/kind bug

issue: #21138
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-12-13 17:03:21 +08:00
congqixia b0afdb2180
Fix access log panicking for nil response (#21149)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-12-12 17:23:22 +08:00
aoiasd 9079b1d5a4
Rename default accesslog dir name (#21086)
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2022-12-09 18:09:19 +08:00
SimFG 4a5c282b62
Add the superuser config (#21090)
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
2022-12-09 16:13:20 +08:00
SimFG d67e878f96
Fix the concurrent write issue when init privilege policy (#21071)
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
2022-12-08 19:43:23 +08:00
aoiasd 60abe08f85
Add retention for access log on minio (#20681)
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2022-12-08 19:39:19 +08:00
smellthemoon 18cad3a1fb
Optimization of delete and insert (#20990)
Signed-off-by: lixinguo <xinguo.li@zilliz.com>

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2022-12-08 18:37:19 +08:00
Jiquan Long 6709ccf690
Fix empty retrieve result (#21032)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2022-12-08 10:15:19 +08:00
Enwei Jiao 89b810a4db
Refactor all params into ParamItem (#20987)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>

Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2022-12-07 18:01:19 +08:00
Xiaofan e131915207
Support 64M Rpc Limit (#20845)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>

Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-12-07 10:07:18 +08:00
jaime 039e9ce4bb
Refine msgstream interface (#20832)
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
2022-12-05 20:55:17 +08:00
yah01 25a3b9ae19
Check target node ID for query/search (#20967)
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-12-04 20:05:17 +08:00
Enwei Jiao 2ecdb4ba4a
Etcd config source support TLS (#20874)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>

Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2022-11-30 18:23:15 +08:00
Jiquan Long 5a6a92d603
Refine log level (#20896)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2022-11-30 14:11:15 +08:00
Jiquan Long 7f76e798e9
Fix metric ProxyApplyPrimaryKeyLatency (#20486)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2022-11-29 15:23:14 +08:00
smellthemoon f5ab719f21
timestamp decided if the pks were the same (#20166)
Signed-off-by: lixinguo <xinguo.li@zilliz.com>

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2022-11-21 10:55:10 +08:00
jaime 0a5270e8c0
Fix alter collection hang (#20694)
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
2022-11-21 10:09:10 +08:00
Enwei Jiao c05b9ad539
Add event dispatcher for config (#20393)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>

Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2022-11-17 18:59:09 +08:00
Enwei Jiao 19524a5344
Fix nodeID mismatch at standalone mode (#20648)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>

Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2022-11-17 17:15:08 +08:00
yah01 bec1d4c50f
Revert "Make proxy process data definition requests concurrently (#20503)" (#20566)
This reverts commit 911921d5a7.

Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-11-15 00:41:06 +08:00
smellthemoon a7ba416b5a
Remove invalid timestamp (#20557)
Signed-off-by: lixinguo <xinguo.li@zilliz.com>

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2022-11-14 16:21:07 +08:00
yah01 911921d5a7
Make proxy process data definition requests concurrently (#20503)
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-11-14 16:03:07 +08:00
smellthemoon 7325b3e1c3
Substitute traceid for msgid in rpc (#20450)
Signed-off-by: lixinguo <xinguo.li@zilliz.com>

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2022-11-14 15:29:06 +08:00
yah01 1a7912e300
Fix checkFullLoaded always returns all partitions as unloaded (#20514)
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-11-11 16:17:04 +08:00
bigsheeper cd19d99ad7
Add channel level checkpoint (#20350)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-11-10 22:13:04 +08:00
aoiasd fca2b71e28
add access log for proxy (#19927)
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2022-11-10 17:09:06 +08:00
Jiquan Long 449e4f9153
Fix delete log (#20407)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2022-11-09 19:17:04 +08:00
Jiquan Long 2189530f5b
Fix some unreasonable logs in Proxy (#20411)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2022-11-09 15:05:08 +08:00
Jiquan Long 6886cc3fd8
Fix rpc log of Proxy (#20409)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2022-11-08 21:59:02 +08:00
Enwei Jiao 7e56e7a976
Fix datarace at Setlogger (#20376)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>

Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2022-11-08 14:33:03 +08:00
congqixia 14c1182418
Add healthy check for InvalidateCollectionMetaCache (#20368)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-11-07 16:55:01 +08:00
jaime 709a0a94e9
Add some msgstream metrics (#20296)
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
2022-11-07 10:15:02 +08:00
Enwei Jiao 956c5e1b9d
Make Params singleton (#20088)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>

Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2022-11-04 14:25:38 +08:00
yah01 5143df1809
Fix Proxy never updates the shard leaders cache (#20286)
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-11-03 12:27:36 +08:00
groot cd3b3d3da6
Only allow one file for row-based import (#20234)
Signed-off-by: yhmo <yihua.mo@zilliz.com>

Signed-off-by: yhmo <yihua.mo@zilliz.com>
2022-11-02 10:23:35 +08:00
bigsheeper a2d5947fe4
Change limiter burst type to float64 (#20215)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-10-31 20:59:34 +08:00
congqixia 4c4db1f556
Display all shard error if search/query fails (#20169)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-10-31 19:09:39 +08:00
Xiaofan 2bfecf5b4e
Refine bloomfilter and memory usage (#20168)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>

Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-10-31 17:41:34 +08:00
cai.zhang bcffb3c389
Check indexes in indexcoord instead of proxy (#20193)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>

Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2022-10-31 11:39:33 +08:00
jaime 19baecdc4d
[skip e2e]Refine error msg for authentication checking (#20117)
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
2022-10-28 19:25:34 +08:00
bigsheeper bddd47c60f
Prevent limiter burst overflow (#20153)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-10-28 19:05:32 +08:00
cai.zhang 53289a63e6
Specification log (#20119)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>

Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2022-10-28 17:01:32 +08:00
cai.zhang 26d26aa8ab
Set index name in proxy when drop index (#20160)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>

Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2022-10-28 16:41:32 +08:00
cai.zhang 4848edda8d
Append field indexID for index name (#20046)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>

Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2022-10-27 20:23:38 +08:00
bigsheeper b949fa0ea9
Increase default ttProtection max delay (#20125)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-10-27 17:37:32 +08:00
groot bee66631e3
Refine bulkload (#19671)
Signed-off-by: yhmo <yihua.mo@zilliz.com>

Signed-off-by: yhmo <yihua.mo@zilliz.com>
2022-10-27 16:21:34 +08:00
bigsheeper b074f530e6
Forbid createIndex if collection loaded before (#20100)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-10-27 13:05:31 +08:00
bigsheeper f0c616050a
Reduce and improve quota center log (#20058)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-10-25 19:39:30 +08:00
Xiaofan 1f170ce1ba
Refine kafka consumer (#19846)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>

Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-10-25 13:23:30 +08:00
cai.zhang c551de8f72
Catch errors on gloabl meta cache (#20023)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>

Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2022-10-25 11:29:30 +08:00
Xiaofan 7d2ba60412
If autoID enabled, check to make sure user didn't specify autoID field (#19943)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>

Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-10-21 19:03:27 +08:00
smellthemoon 581e1d8c35
Fill MsgBase ALL (#19912)
Signed-off-by: lixinguo <xinguo.li@zilliz.com>

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2022-10-21 15:57:28 +08:00
cai.zhang 94c15a49e9
Can't drop loaded partition (#19938)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>

Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2022-10-21 14:41:28 +08:00
Xiaofan 3d3289ce3e
Fix collection metrics leakage (#19944)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>

Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-10-21 10:13:28 +08:00
xige-16 36accde034
Fix wrong msg when load failed (#19884)
Signed-off-by: xige-16 <xi.ge@zilliz.com>

Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-10-20 19:55:27 +08:00
jaime cf1491e41a
Add status for GetVersionResponse and CheckHealthResponse (#19895)
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
2022-10-20 19:47:33 +08:00
congqixia e8cf049392
Fix proxy trace operation name (#19932)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-10-20 18:35:28 +08:00
congqixia b15e97a61a
Improve proxy leader cache lock logic (#19917)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-10-20 16:11:27 +08:00
wayblink dbd60757c9
Support pass start_ts and end_ts through Import API (#19810)
Signed-off-by: wayblink <anyang.wang@zilliz.com>

Signed-off-by: wayblink <anyang.wang@zilliz.com>
2022-10-20 12:15:27 +08:00
cai.zhang 836081c8c7
Dropping index must happen without collection being loaded (#19886)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>

Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2022-10-19 23:17:44 +08:00
bigsheeper db0486c870
Fix missing import limit and rename functions in quotaCenter (#19907)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-10-19 21:43:30 +08:00
smellthemoon 108e51b2f0
[test]Create index before load and fix error message (#19874) (#19857)
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>

Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>

Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
Co-authored-by: zhuwenxing <wenxing.zhu@zilliz.com>
2022-10-19 10:01:26 +08:00
Xiaofan 5b9988b627
Support collection level monitoring metrics (#19686)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>

Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-10-18 19:17:27 +08:00
bigsheeper 15f7592739
Intercept import request in Proxy interceptor (#19856)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-10-18 16:53:26 +08:00
jaime 59bcbf0cf6
Support check health for the service level (#19779)
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
2022-10-18 13:39:26 +08:00
Ten Thousand Leaves db33f401fd
Fix proxy crash issue when meta cache miss (#19844)
issue: #19557

/kind bug

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-10-18 13:11:26 +08:00
Ten Thousand Leaves a4a6f60279
Refine NQ check logic (#19819)
issue: #19682

/kind improvement

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-10-18 13:07:34 +08:00
cai.zhang 9d43947f1c
Must create index before load (#19516)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>

Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2022-10-17 18:01:25 +08:00
cai.zhang 9e8a59ac4c
Add nameing rules for index name (#19803)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>

Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2022-10-16 21:05:25 +08:00
SimFG a55f739608
Separate public proto files (#19782)
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
2022-10-16 20:49:27 +08:00
SimFG 3d12325796
Gracefully handle the hook plugin loading failure error (#19789)
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
2022-10-14 19:59:24 +08:00
xige-16 a1db9038fb
Move disk index params to config file (#19714)
Signed-off-by: xige-16 <xi.ge@zilliz.com>

Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-10-14 17:51:24 +08:00
jaime 8e7a4733ec
Add GetVersion API for the client (#19694)
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
2022-10-14 09:59:24 +08:00
bigsheeper f1e330a997
Add read result rate protection (#19728)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-10-13 14:57:24 +08:00
Ten Thousand Leaves ae373d450f
Make NQ <= 16384 limit work (#19726)
https://milvus.io/docs/limitations.md

issue: #19682

/kind bug

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-10-12 18:37:23 +08:00
XuanYang-cn d73186b2eb
Fix timetravel error msg (#19722)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-10-12 15:17:23 +08:00
jaime 10c03de628
Refine method name from BroadCastXx to BroadcastXx (#19700)
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
2022-10-11 21:07:23 +08:00
Eng Zer Jun 438790744d
test: use `T.Setenv` to set env vars in tests (#19649)
This commit replaces `os.Setenv` with `t.Setenv` in tests. The
environment variable is automatically restored to its original value
when the test and all its subtests complete.

Reference: https://pkg.go.dev/testing#T.Setenv
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2022-10-11 18:43:23 +08:00
yah01 2925d8a1ac
Remove QueryCoordV1 (#19681)
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-10-11 11:11:23 +08:00
jaime 31db01b6ae
Support TTL for collection level (#19383)
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
2022-10-10 20:31:22 +08:00
wayblink 85e04d845c
Move GetComponentStates to milvus.proto, make it an external interface (#19594)
Signed-off-by: wayblink <anyang.wang@zilliz.com>

Signed-off-by: wayblink <anyang.wang@zilliz.com>
2022-10-10 15:55:22 +08:00
XuanYang-cn b6ba226059
Fix pagination check in QueryNode (#19497)
See also: #19482

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-10-09 16:26:57 +08:00
bigsheeper f98dbcf5be
Update quota params and disable limit for showCollection (#19641)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-10-09 14:30:58 +08:00
Cai Yudong 246693acb7
Fix panic when expr invalid (#19642)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-10-09 11:48:57 +08:00
Xiaofan 6826964e60
Rename some interface and files in id allocator (#19605)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>

Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-10-09 10:06:58 +08:00
zhenshan.cao 0ac4bc32a5
Add logic of autoindex (#19570)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-10-08 15:38:58 +08:00
wayblink e7c2500c71
Implement GetPersistentSegmentInfo through datacoord (#19205)
Signed-off-by: wayblink <anyang.wang@zilliz.com>

Signed-off-by: wayblink <anyang.wang@zilliz.com>
2022-10-08 11:51:02 +08:00
SimFG 7d54e35fa5
Add the `GetLoadingProgress` RPC (#19491)
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
2022-10-01 09:02:55 +08:00
cai.zhang 2f07344d75
Support default index name for each field (#19512)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>

Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2022-09-29 17:14:54 +08:00
cai.zhang 3a770f7dd3
Suport to create flat index (#19398)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>

Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2022-09-29 09:32:55 +08:00
bigsheeper 5405dedd2e
Fix DDL rate limit unit, and improve log (#19504)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-09-28 20:58:54 +08:00
SimFG 9d40be7e67
Support to modify the `context` param in the hook interceptor (#19495)
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
2022-09-28 13:26:54 +08:00
Jiquan Long 73463d030d
Fix dml stream leakage in Proxy (#19450)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2022-09-27 19:18:54 +08:00
SimFG 13ca8936d1
Deprecate `GetIndexState` and `GetIndexBuildProgress` (#19451)
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
2022-09-27 19:12:54 +08:00
SimFG 09255e5507
Add the `indexName` param for the `indexpb.DescribeIndexRequest` (#19440)
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
2022-09-26 19:10:53 +08:00
Ten Thousand Leaves b30c9d4f60
Support Bulk Load in Milvus 2.2 (#18982)
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-09-26 18:06:54 +08:00
XuanYang-cn 52c6a2706e
Fix binary results unstable (#19401)
See also: #19338, #19366, 19367

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-09-26 18:02:52 +08:00
XuanYang-cn 7819297f68
Add more checks for search offset (#19388)
See also: #19250

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-09-26 18:00:57 +08:00
bigsheeper 804c18df68
Update quota params (#19351)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-09-26 16:48:53 +08:00
SimFG 662c654fe3
Add the `context` param for the `hook` interceptor (#19405)
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
2022-09-24 09:56:51 +08:00
SimFG 68a257458b
Implement the `hook` interceptor (#19294)
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
2022-09-23 09:50:52 +08:00
cai.zhang 17f5e3c7fe
Drop segment indexes when drop partition (#19022)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>

Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2022-09-23 09:36:51 +08:00
Bingyi Sun 72346f31b0
Add collectionID when checking collection load (#19322)
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@zilliz.com>
2022-09-22 15:08:50 +08:00
xige-16 428840178c
Support diskann index for vector field (#19093)
Signed-off-by: xige-16 <xi.ge@zilliz.com>

Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-09-21 20:16:51 +08:00
XuanYang-cn 297d75fc92
Enable query pagination (#19231)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-09-20 20:48:50 +08:00
bigsheeper 5143e2a75e
Disable quota and limits (#19242)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-09-20 09:34:50 +08:00
SimFG d7f38a803d
Separate some proto files (#19218)
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
2022-09-16 16:56:49 +08:00
Bingyi Sun 3998752db3
Fix log print (#19204)
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@zilliz.com>
2022-09-16 11:12:47 +08:00
bigsheeper 5cdd6ca251
Add quotaCenter, rateLimiter and rateCollector (#18636)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-09-16 09:56:47 +08:00
Bingyi Sun 626854cf0c
Refactor QueryCoord (#18836)
Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: yah01 <yang.cen@zilliz.com>
Co-authored-by: Wei Liu <wei.liu@zilliz.com>
Co-authored-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: yah01 <yang.cen@zilliz.com>
Co-authored-by: Wei Liu <wei.liu@zilliz.com>
Co-authored-by: Congqi Xia <congqi.xia@zilliz.com>
2022-09-15 18:48:32 +08:00
XuanYang-cn 200df76864
Enable search with pagination (#19044)
See also: #19043

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-09-14 20:36:32 +08:00
wayblink 3927ae9952
Return all flushed segments and time-of-seal in Flush interface (#18882)
Signed-off-by: wayblink <anyang.wang@zilliz.com>

Signed-off-by: wayblink <anyang.wang@zilliz.com>
2022-09-09 09:58:37 +08:00
groot ebc263f440
Not allow '$' in collection/partition/field name (#19082)
Signed-off-by: yhmo <yihua.mo@zilliz.com>

Signed-off-by: yhmo <yihua.mo@zilliz.com>
2022-09-08 10:16:34 +08:00
SimFG 2bafdf8c53
Fix the `Flush` privilege can't be granted (#19020)
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
2022-09-05 21:05:11 +08:00
Jiquan Long a5e2d6b6fb
Refactor RootCoord (#18930)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
Co-authored-by: xaxys <tpnnghd@163.com>

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
Co-authored-by: xaxys <tpnnghd@163.com>
2022-09-05 13:29:11 +08:00
SimFG 9cd19f5396
Return nil error when handling grpc request (#18955)
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
2022-09-02 21:12:59 +08:00
bigsheeper 8a45381e53
Make DescribeCollection return collectionName (#19000)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-09-02 19:20:59 +08:00
XuanYang-cn c054873908
Add limit to proto (#18950)
See also: #18893

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-09-01 18:54:58 +08:00
XuanYang-cn 867ea63bdd
Move TopK check inside parseQueryInfo (#18892)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-08-30 10:32:56 +08:00
SimFG 69c0b2fb49
Improve the storage method of the grant info in the metastore (#18817)
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
2022-08-26 19:22:56 +08:00
cai.zhang c924f73105
Refactor for IndexCoord to support cloud (#18643)
Co-authored-by: Zach41 <zongmei.zhang@zilliz.com>
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>

Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
Co-authored-by: Zach41 <zongmei.zhang@zilliz.com>
2022-08-25 15:48:54 +08:00
aoiasd dad060af2e
Fix func checkIfLoaded in proxy (#18769)
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2022-08-23 15:14:52 +08:00
Zach 0c9a10e8f8
1. refine logging interfaces (#18692)
2. adjust logs for query/search requests

Signed-off-by: Zach41 <zongmei.zhang@zilliz.com>

Signed-off-by: Zach41 <zongmei.zhang@zilliz.com>
2022-08-23 10:44:52 +08:00
SimFG ce434b496e
Implement the mysql metastore of the rbac (#18704)
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
2022-08-23 10:26:53 +08:00
jaime 516fd928f9
Fix password verification miss cache (#18730)
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
2022-08-19 19:42:50 +08:00
XuanYang-cn e05e9cb9b4
Fix typo in error message (#18688)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-08-19 10:48:50 +08:00
congqixia d4c54d96b0
Dedup output fields for task query (#18673)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-08-16 18:16:48 +08:00
SimFG 447ce8ee48
Handle some apis about the default role (#18647)
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
2022-08-15 16:40:48 +08:00
aoiasd 8a70f2badb
Add ShowConfigurations for all remaining session (#18594)
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2022-08-12 13:20:39 +08:00
SimFG 1366f130a2
Add a specific `KeyNotExistError` err for the metastore `load` interface (#18582)
Signed-off-by: SimFG <bang.fu@zilliz.com>
2022-08-10 17:20:41 +08:00
Ten Thousand Leaves 12023e8569
Fix a potential nullptr issue with metric (#18540)
/kind improvement

related to: #18530

but not the root cause

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-08-09 21:12:42 +08:00
SimFG c267ad612d
Support Role-Based Access Control II (#18528)
Signed-off-by: SimFG <bang.fu@zilliz.com>
2022-08-05 16:28:35 +08:00
Xiaofan b427450593
Remove Load Delta Channel Request (#18497)
Fix #18393

remove watch delta channel task related logic

Co-authored-by: xiaofan-luan <xiaofan.luan@zilliz.com>
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Co-authored-by: Congqi Xia <congqi.xia@zilliz.com>
2022-08-04 17:50:37 +08:00
SimFG ff0200210a
Support Role-Based Access Control (#18425)
Signed-off-by: SimFG <bang.fu@zilliz.com>
2022-08-04 11:04:34 +08:00
aoiasd a7ce0eb6f6
Add ShowConfigurations to QueryNode (#18426)
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2022-08-03 01:12:33 +08:00
Jeng.Gwan 638f6c36e9
Support to get real row count of segment (#18115)
Signed-off-by: xaxys <zheng.guan@zilliz.com>
2022-07-18 09:58:28 +08:00
congqixia a9aff3a678
Allocate ID for each insertMsg (#18217)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-07-11 14:38:24 +08:00
bigsheeper 0fb5e9c1a9
Remove load collection check in Proxy (#18166)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-07-08 16:12:25 +08:00
codeman 5fda8261d0
Fix: Unable to list newly created credential users (#18117) (#18153)
Signed-off-by: kejiang <ke.jiang@zilliz.com>

Co-authored-by: kejiang <ke.jiang@zilliz.com>
2022-07-07 21:16:22 +08:00
Zach 1c9647ff31
Merging multiple shard query/search requests into one rpc request if QueryNode is the leader of multiple shards of Collection (#17588)
Signed-off-by: Zach41 <zongmei.zhang@zilliz.com>
2022-07-06 15:06:21 +08:00
bigsheeper 696d747afa
Check if loaded in Proxy when LoadCollection (#18074)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-07-05 21:16:20 +08:00
zhenshan.cao 9bd35dd84e
Refine logs to reduce print (#17759) (#18028)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-07-04 11:10:20 +08:00
zhenshan.cao f33b090819
Fix bug: datarace in proxy unittest (#17990)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-07-01 17:56:21 +08:00
codeman 6ab850bedf
Fix update credential failure (#17963) (#17970)
Signed-off-by: kejiang <ke.jiang@zilliz.com>

Co-authored-by: kejiang <ke.jiang@zilliz.com>
2022-06-30 21:34:18 +08:00
codeman 95a3830dc6
fix password comparison too slow (#17874) (#17897)
Signed-off-by: kejiang <ke.jiang@zilliz.com>

Co-authored-by: kejiang <ke.jiang@zilliz.com>
2022-06-29 20:02:18 +08:00
XuanYang-cn 6b72f9013f
Fix message type of show partitions (#17821)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-06-28 10:06:17 +08:00
xige-16 8ff1a2c4c2
Print expr when proxy receive qyery req (#17662)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-06-23 13:42:13 +08:00
Jiquan Long 216e2f80aa
Add msg id to log in query path (#17677)
/kind improvement

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2022-06-23 10:46:13 +08:00
SimFG 3fadf4c5d7
Return the unavailable error to make the client's retry policy take effect (#17719)
Signed-off-by: SimFG <bang.fu@zilliz.com>
2022-06-22 21:56:13 +08:00
XuanYang-cn e43f487f02
Make LoadCollection compatible (#17669)
See also: #17656

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-06-21 17:26:15 +08:00
Jiquan Long 0282f69eb4
Add msg id to log in search path. (#17654)
issue: #17513
Add msg id to log in search path,
which should help us to trace a detailed request more easier.

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2022-06-21 13:30:12 +08:00
shaoyue 32a3ed0791
Add e2e test for RESTful APIs, fix json encoding for on proto_oneof fields (#17362)
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
2022-06-20 22:06:12 +08:00
cai.zhang f123d657ed
Fix bug for dropping index (#17635)
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
2022-06-20 15:22:13 +08:00
cai.zhang 141a3bbe00
Mark deleted before drop index (#17490)
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
2022-06-17 18:08:12 +08:00
cai.zhang 53a49a9255
GetIndexState only get the state generated call CreateIndex (#17549)
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
2022-06-16 20:12:11 +08:00
Xiaofan 1f6fbf91b2
Fix pulsar unsubsribe issue (#17562)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-06-16 17:28:11 +08:00
Jiquan Long 2f66531fdf
Fix wrong IP distances (#17590)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2022-06-16 16:34:11 +08:00
SimFG e825215c44
Remove the redundant 'for' in the TaskCondition (#17558)
Signed-off-by: SimFG <bang.fu@zilliz.com>
2022-06-15 16:32:10 +08:00
xing.zhao e9dcda161f
Add aliases return value in describeCollection task (#17516)
Signed-off-by: xingzhao <xing.zhao@zilliz.com>

Co-authored-by: 赵星 <zhaoxing@zhaoxings-MacBook-Pro.local>
2022-06-13 19:22:10 +08:00
Letian Jiang dfaed5acdd
Add QueryServiceAvailable field in ShowCollections (#17456)
Signed-off-by: Letian Jiang <letian.jiang@zilliz.com>
2022-06-09 18:20:07 +08:00
Jiquan Long 2ca81620ea
Reduce lock operations when get dml stream (#17468)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2022-06-09 17:34:09 +08:00
xige-16 b5c11a216d
Alter varChar type params's name to max_length (#17409)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-06-07 15:58:06 +08:00
congqixia de8f2f76c5
Fix shardLeader cache with ErrConnect check (#17402)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-06-07 12:20:06 +08:00
zhenshan.cao 98e95275fe
Fix several code issues caused by carelessness (#17385)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-06-06 18:58:05 +08:00
Ten Thousand Leaves f5e63177d2
Fix an issue with bulk load with multiple DataNodes (#17324)
issue: #17300
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-06-02 18:54:04 +08:00
zhenshan.cao 5f44e45480
Add metrics on QueryNode and Proxy (#17328)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-06-02 16:06:03 +08:00
Jiquan Long adf3b14027
Refine channels management in Proxy. (#17334)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2022-06-02 15:34:04 +08:00
zhenshan.cao 2763efc9b0
Add cache of grpc client of ShardLeader in proxy (#17301)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-06-02 12:16:03 +08:00
Jiquan Long 72f454e054
Fix 16961, make creating dml stream idempotent (#17302)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2022-05-31 22:58:02 +08:00
cai.zhang bcf3b7426a
Add distributed lock for segment refgerence (#16782)
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
2022-05-31 16:36:03 +08:00
congqixia c88514bc49
Remove not used QueryChannel in Proxy and Query Cluster (#16856)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-05-30 19:50:04 +08:00
SimFG 731870211a
Support role-based access control (only pb) (#16987)
Signed-off-by: SimFG <bang.fu@zilliz.com>
2022-05-28 00:04:01 +08:00
zhenshan.cao ad9276a440
Add metrics for proxy and querynode (#17205)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-05-27 14:12:01 +08:00
Letian Jiang f2a27e0e64
Retry GetShardLeaders until service available or timeout (#17183)
Signed-off-by: Letian Jiang <letian.jiang@zilliz.com>
2022-05-26 20:28:02 +08:00
groot 3928da6493
Refine calc distance function (#17161)
Signed-off-by: groot <yihua.mo@zilliz.com>
2022-05-26 14:34:01 +08:00
congqixia e06d6d7cc6
Returns collection instead of copy to make leaderMutex work (#17206)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-05-25 15:20:00 +08:00
yah01 de0ba6d495
Fix GetQuerySegmentInfo() returns incorrect result after LoadBalance (#17190)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-05-25 15:17:59 +08:00
Jiquan Long 4293b640b2
Remove confusing log in segment id assigner (#17195)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2022-05-25 10:26:00 +08:00
zhenshan.cao ec1103ca27
Simplify the merge logic of searchTask (#17194)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-05-24 21:27:59 +08:00
zhenshan.cao 2512e668f7
Update timestmap according to gracefultime for Bounded Consistency (#17171)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-05-24 12:05:59 +08:00
zhenshan.cao ea8e1623cd
Acquire nq from placeholdergroup incase of nq is zero (#17168)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-05-24 10:05:58 +08:00
zhenshan.cao 08a4dff41d
Add logic for search merging and a simple task scheduler for read tasks (#17022)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>

Co-authored-by: cai.zhang <cai.zhang@zilliz.com>
Co-authored-by: bigsheeper <yihao.dai@zilliz.com>

Co-authored-by: cai.zhang <cai.zhang@zilliz.com>
Co-authored-by: bigsheeper <yihao.dai@zilliz.com>
2022-05-23 16:41:58 +08:00
yah01 7746a5b742
Add NodeIds field for QuerySegmentInfo (#17121)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-05-20 18:03:58 +08:00
congqixia 90ee23df22
Fix shardLeader cache concurrent access (#17120)
Fix  write map without mutex control
Also GetShards returns a copy of leader list instead of original one

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-05-20 14:33:57 +08:00
Jiquan Long 276ba69819
Replace plan parser with antlr (#17090)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2022-05-19 17:15:57 +08:00
bigsheeper 9eeec4a2d5
Add collection load cache and InvalidateCollMetaCache by collID (#16882)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-05-19 10:13:56 +08:00
XuanYang-cn 127dd34b37
Make proxy use roundrobin to choose replica (#17063)
Fixes: #17055

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-05-17 22:35:57 +08:00
XuanYang-cn a1480aaf76
Clear shard leaders cache when releasing (#16942)
See also: #16926

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-05-17 11:11:56 +08:00
SimFG ea2c395a38
Fix the comment info in task_test.go (#16982)
Signed-off-by: SimFG <bang.fu@zilliz.com>
2022-05-15 09:03:54 +08:00
Xiaofan 52fc9a558c
Fix valid error message (#16953)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-05-13 18:19:53 +08:00
XuanYang-cn 43b36b4e5f
Make proxy update cache with NotShardLeader status code (#16940)
See also: #16926

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-05-13 11:35:53 +08:00
xige-16 9e6d9fb7d8
Print collectionName when load failed (#16878)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-05-11 09:47:53 +08:00
xige-16 8fcf349609
Fix create index failed after drop index (#16836)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-05-09 11:35:53 +08:00
Jiquan Long 98ceb162aa
Support antlr as plan parser (#16696)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>

Co-authored-by: xaxys <tpnnghd@163.com>

Co-authored-by: xaxys <tpnnghd@163.com>
2022-05-06 17:43:51 +08:00
Letian Jiang b0053b7a45
Add isShardLeader field in search/query request (#16798)
Signed-off-by: Letian Jiang <letian.jiang@zilliz.com>
2022-05-06 16:55:51 +08:00
xige-16 009b773b6e
Delete set default type param max_len for varchar field (#16794)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-05-06 15:59:51 +08:00
shaoyue fa97f86cda
Clean up not used id_cache related code (#16655)
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
2022-05-05 09:59:51 +08:00
Jiquan Long 2fe8677cbf
Enable dimension check in Proxy when create index request received (#16718)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2022-04-29 18:01:49 +08:00
zhenshan.cao 97757405de
Fix bug: wrong metrics name and forget to register metrics (#16714)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-04-29 15:35:47 +08:00
xige-16 515d0369de
Support string type in segcore (#16546)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
Co-authored-by: dragondriver <jiquan.long@zilliz.com>

Co-authored-by: dragondriver <jiquan.long@zilliz.com>
2022-04-29 13:35:49 +08:00
groot 21b5973253
Refine list import task api (#16690)
Signed-off-by: groot <yihua.mo@zilliz.com>
2022-04-28 17:21:47 +08:00
zhenshan.cao 0a953948af
Simplify monitoring metrics (#16687)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-04-27 23:03:47 +08:00
XuanYang-cn 35b7267edb
Add ut for task_policies (#16663)
See also: #16652

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-04-27 16:55:46 +08:00
congqixia 0bf176a7e1
Add SyncReplicaSegments interface for QueryNode (#16672)
Add SyncReplicaSegments for LoadCollection/LoadPartition child task
This API notifies related ShardCluster to record segment Loaded
Preventing the Etcd events is delayed, which will cause some segment not search

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-04-27 10:41:46 +08:00
zhenshan.cao a3f7bb5f2d
Remove the collectionID label from metrics (#16571)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-04-26 17:29:45 +08:00
XuanYang-cn fbc7fe1cdc
Fix proxy unable to update cache (#16646)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-04-26 11:27:53 +08:00
groot bc5e9ec929
Add list import tasks api (#16605)
Signed-off-by: groot <yihua.mo@zilliz.com>
2022-04-25 17:37:46 +08:00
Xiaofan 89b4a34892
Fix Data race in NodeID (#16603)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-04-24 22:03:44 +08:00
Aivin V. Solatorio 04fffb08ef
Support arithmetic operations on numerical fields for scalar filtering (#16520)
Signed-off-by: Aivin V. Solatorio <avsolatorio@gmail.com>
2022-04-24 16:43:45 +08:00
yah01 804fbb6e27
Fix panic: Set Base of GetReplicas request (#16578)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-04-22 14:47:42 +08:00
Ten Thousand Leaves 6f75d02c65
Disable knowhere logging for embedded Milvus and some other tweaks (#16496)
/kind enhancement

issue: #15711
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-04-20 17:23:46 +08:00
congqixia 69252f812d
Implement memory replica in Proxy, QueryNode and QueryCoord (#16470)
Related to #16298 #16291 #16154
Co-authored-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: yangxuan <xuan.yang@zilliz.com>
Co-authored-by: yah01 <yang.cen@zilliz.com>
Co-authored-by: Letian Jiang <letian.jiang@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-04-20 16:15:41 +08:00
Ten Thousand Leaves 289e468a7a
Make bulk load fully work (#16512)
issue: #15604

/kind enhancement

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-04-20 14:03:40 +08:00
codeman 3a1b2cedd2
create default root user for authentication (#16545) (#16549)
Signed-off-by: kejiang <ke.jiang@zilliz.com>

Co-authored-by: kejiang <ke.jiang@zilliz.com>
2022-04-20 13:03:40 +08:00
codeman f5e1ded854
fix #16504 (#16523)
Signed-off-by: kejiang <ke.jiang@zilliz.com>

Co-authored-by: kejiang <ke.jiang@zilliz.com>
2022-04-19 16:35:39 +08:00
Ten Thousand Leaves cf9b9aba79
Fix a bug where RPC calls before proxy finishes initialization crashes Milvus (#16474)
/kind bug

issue: #16473
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-04-14 20:01:34 +08:00
Ten Thousand Leaves 442c73a544
Support passing channel names to DataNode (#16424)
/kind feature

issue: #15604
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-04-12 13:25:34 +08:00
codeman 4e12271d79
Support login with username and password (#15656) (#16341)
Signed-off-by: kejiang <ke.jiang@zilliz.com>

Co-authored-by: kejiang <ke.jiang@zilliz.com>
2022-04-11 19:49:34 +08:00
godchen bb7a0766fe
Add dependency factory (#16204)
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
2022-04-07 22:05:32 +08:00
Ten Thousand Leaves 1e6f6bd072
Implement set segment state call from RootCoord to DataCoord. (#16374)
/kind feature

issue: #15604
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-04-06 15:33:32 +08:00
XuanYang-cn 78200009a3
Impl GetReplicas in Proxy (#16390)
See also: #16298

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-04-06 14:57:31 +08:00
bigsheeper 7161a7760c
Fix segIDAssigner spelling mistake (#16370)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-04-06 14:11:31 +08:00
xige-16 99984b88e1
Support delete varChar value (#16229)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-04-02 17:43:29 +08:00
XuanYang-cn e9090a62ab
Rearrange search/queryTask for readability (#16325)
`searchTask` has 683 lines of code, `queryTask` has 485 lines of code.
`task.go` contains 4k+ codes including `searchTask` and `queryTask`.

It was so much pain navigating codes of searchTask and queryTask though task.go,
task.go and task_test.go are literaly unreadable.

This PR moves
1. 650+ lines of code of `searchTask` from `task.go` to `task_search.go`.
2. 1.4k+ lines of test code of `searchTask` from `task_test.go` to
   `task_search_test.go`.
3. 450+ lines of code of `queryTask` from `task.go` to `task_query.go`.
4. 200+ lines of test code of `queryTask from `task_test.go to
   `task_query_test.go`.

This PR also rearrange methods positions of `searchTask` and
`queryTask`:
-  Putting the most important methods `PreExecute`, `Execute`, and
   `PosExecute` at the beginning of the file.
-  Moves interface methods `ID`, `SetID`, `Type`, `BeginTs`, etc.
   that nobody cares about to the bottom of the file.

See also: #16298

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-04-01 18:59:29 +08:00
yah01 c005f07ccc
Modify proto to add GetReplicas() interface (#16328)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-04-01 16:15:29 +08:00
Ten Thousand Leaves 45be3deb3f
Implement RC, DC, DN calling path for import. (#16321)
/kind feature

issue: #15604
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-04-01 11:33:28 +08:00
XuanYang-cn 58addbf9f4
Add GetShardLeaders rpc in querycoord proto (#16299)
See also: #16298

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-03-31 16:39:29 +08:00
XuanYang-cn a0bb758be5
Refine log in proxy searchTask (#16263)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-03-30 22:09:28 +08:00
Jiquan Long ba37531456
Add support for loading multiple indexes (#16138)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2022-03-30 21:11:28 +08:00
Jiquan Long cd0f3cbcd3
Refine log (#16269)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2022-03-29 22:43:28 +08:00
zhenshan.cao 11cea5aad0
Refine proxy log (#16246)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-03-29 02:29:27 +08:00
congqixia 2c03287e79
Improve log level in proxy QueryTask (#16234)
Use warn instead of debug in error case

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-03-28 23:11:26 +08:00
Jiquan Long 973ea406ca
Improve log of show collections (#16233)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2022-03-28 21:47:27 +08:00
Ten Thousand Leaves 40c703dacd
Support check index state in root coord, for bulk load feature. (#16198)
issue: #15604

/kind feature

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-03-28 16:41:28 +08:00
xige-16 205c92e54b
Support insert string data (#15993)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-03-25 14:27:25 +08:00
xiyichan ca129d4308
Support configurable msgstream (#16131)
Signed-off-by: xiyichan <2863768433@qq.com>
2022-03-24 10:15:25 +08:00
groot 1c4b949a1d
modify import rpc interface (#16129)
Signed-off-by: groot <yihua.mo@zilliz.com>
2022-03-22 15:11:24 +08:00
groot 2078b24dca
Prepare for import (#16083)
Signed-off-by: groot <yihua.mo@zilliz.com>
2022-03-21 15:47:23 +08:00
Jiquan Long f8d9bc919d
Unify interface of vector index & scalar index. (#15959)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2022-03-21 14:23:24 +08:00
Xiaofan 787d4f8bfd
Refine prometheus metrics and etcd log (#16084)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-03-18 15:51:22 +08:00
Ji Bin 3cd28420f1
Support compile under windows (#15786)
This patch makes compile milvus under windows(MSYS), including:
- some cpp adaptation for compile under msys/gcc-10.3
- install toolchain scripts for setup from MinGW/MSYS `scripts/install_deps_msys.sh`
- adaptation for POSIX API use in golang
  * using gofrs/flock instead of syscall.Flock
  * using x/exp/mmap instead of syscall.Mmap
- introducing github actions for build milvus.exe under windows/MSYS
- rocksdb's patch for MSYS
- adaptation for compile knowhere under windows
- a windows package script for pack zip file, `scripts/package_windows.sh`

issue #7706

Signed-off-by: Ji Bin <matrixji@live.com>
2022-03-17 17:17:22 +08:00
jaime cf4c014b2e
Remove collect query result by mq (#15988)
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
2022-03-16 10:19:21 +08:00
zhenshan.cao f3eeecf146
Reduce the number of metrics (#16030)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-03-15 21:51:21 +08:00
jaime 97b1ed7bca
Remove reader API from Mqstream and replace it with consumer API (#15971)
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
2022-03-15 14:45:22 +08:00
xige-16 09b2c46b61
Support create collection with VarChar field (#15904)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-03-14 23:20:02 +08:00
jaime 29975a7a26
Remove Consume() API from mqstream interface (#15886)
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
2022-03-11 20:09:59 +08:00
groot 89356ebcb9
Modify import rpc interfaces (#16006)
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2022-03-11 17:13:59 +08:00
groot 5fdef607d5
Add rpc interfaces for import (#15930)
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2022-03-09 18:33:59 +08:00
cai.zhang 2512dff261
Fix misuse of timerecord (#15868)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2022-03-04 15:45:56 +08:00
Jiquan Long f71651e294
Support column-based insert data in message stream (#15802)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2022-03-04 15:09:56 +08:00
Cai Yudong 3e862ecdfb
Merge msgChannelConfig and knowhereConfig into commonConfig (#15843)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-03-04 11:17:56 +08:00
Cai Yudong 57b97ccb9c
Let search fail as if one query_node return error result (#15855)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-03-04 10:43:56 +08:00
jaime 7a3a721380
Reconstruct mqstream module (#15784)
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
2022-03-03 21:57:56 +08:00
congqixia 3f34bc1ab7
Ban DataType_String in CreateCollection (#15838)
Fix #15836
/kind enhancement

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-03-03 16:57:56 +08:00
朱亚光 27a7875a15
add array hashKeys empty check (#15796)
fix #15791 https://github.com/milvus-io/milvus/issues/15791
Signed-off-by: zhuyaguang <zhuyaguang1368@163.com>
2022-03-03 00:33:56 +08:00
cai.zhang d183748484
Add some prometheus metrics for proxy (#15582)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2022-03-02 16:23:55 +08:00
Nemo 3eb074a9b0
Update some comments for data coord with some minor code refactoring. (#15686)
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-02-22 13:15:51 +08:00
xige-16 782f350da5
Modify parameter spelling of task queue (#15581)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-02-22 10:25:51 +08:00
Xiaofan be42627ab2
Move Logutil to Util dir (#15447)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-02-09 18:55:46 +08:00
Cai Yudong 8255b713ac
Move ProxySubName/QueryNodeSubName/DataNodeSubName to msgChannelConfig (#15501)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-02-09 14:41:45 +08:00
guoxiangzhou a157487f8c
[skip e2e] Modify code comments according to the naming conventions (#15172)
Signed-off-by: guoxiangzhou <xiangzhou.guo@zilliz.com>
2022-02-09 10:18:02 +08:00
guoxiangzhou c7e93f95aa
[skip e2e] Modify code comments according to the naming conventions (#15171)
Signed-off-by: guoxiangzhou <xiangzhou.guo@zilliz.com>
2022-02-09 10:14:05 +08:00
Cai Yudong 1f337891e1
Rename base_param to service_param, rename global_param to component_param (#15463)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-02-08 20:57:47 +08:00
Cai Yudong f1ea9613ae
Add EtcdConfig and move PulsarConfig/RocksdbConfig/MinioConfig into BaseParamTable (#15434)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-02-07 10:09:45 +08:00
Filipe Caixeta dee1961b4d
Make SearchResults and QueryResults return collectionID (#15377)
* Make SearchResults and QueryResults return collectionID

Signed-off-by: Filipe Caixeta <filipecaixeta@gmail.com>

* add return collection name from QueryRequest and SearchRequest

Signed-off-by: Filipe Caixeta <filipecaixeta@gmail.com>

* remove comment

Signed-off-by: Filipe Caixeta <filipecaixeta@gmail.com>
2022-01-27 22:55:41 +08:00
xige-16 408a6e9e58
Fix collectResultLoop hang after search timeout (#15250)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-01-17 18:11:36 +08:00
Jiquan Long 675e6d352b
QueryNodes send search/query results by rpc (#15223)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2022-01-17 14:41:35 +08:00
Jiquan Long 3bf34f6326
[skip e2e] Add comment for baseTaskQueue (#15170)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2022-01-12 10:59:37 +08:00
Jiquan Long 5d91fb3c70
[skip e2e] Add comment for MetaCache (#15169)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2022-01-12 10:57:41 +08:00
Jiquan Long 7e74db0d65
[skip e2e] Add comment for insertRepackFunc (#15115)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2022-01-10 22:41:46 +08:00
Jiquan Long 5ab8d99ee9
[skip e2e] Add comment for lastTick (#15114)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2022-01-10 22:39:54 +08:00
Jiquan Long ff60bb8bfc
[skip e2e] Add comment for GetCollectionSchema (#15113)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2022-01-10 22:37:58 +08:00
Cai Yudong 6323907566
Use CommonConfig in GlobalParams for all components (#15106)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-01-10 19:03:35 +08:00
shaoyue 2e8dd74c7e
[skip e2e] Add comment for proxy task (#15089)
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
2022-01-10 18:33:46 +08:00
Cai Yudong 7efb02a4c5
Use PulsarConfig in GlobalParams for all components (#15046)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-01-09 23:45:33 +08:00
guoxiangzhou 1807ba534d
[skip e2e] Modify log text according to the naming conventions (#14969)
Signed-off-by: guoxiangzhou <xiangzhou.guo@zilliz.com>
2022-01-07 18:41:23 +08:00
guoxiangzhou 8ec3c69447
[skip e2e] Modify log text according to the naming conventions (#14970)
Signed-off-by: guoxiangzhou <xiangzhou.guo@zilliz.com>
2022-01-07 18:39:22 +08:00
Jiquan Long 92b466800f
[skip e2e] Add implementation assertion for taskQueue (#15023)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2022-01-07 17:53:28 +08:00
Jiquan Long 149e049f04
[skip e2e] Add implementation assertion for Cache (#15022)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2022-01-07 17:51:33 +08:00
Jiquan Long 72e3352366
[skip e2e] Add comment for GetPartitionInfo (#15021)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2022-01-07 17:49:43 +08:00
Cai Yudong f8f61fb116
Use BaseParams etcd configuration for all components (#15012)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-01-07 16:15:26 +08:00
Jiquan Long 6a8d8f43c4
[skip e2e] Add comment for GetPartitions (#14954)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2022-01-06 23:32:00 +08:00
Jiquan Long 5eb346fd9e
[skip e2e] Add license for id_cache (#14953)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2022-01-06 23:30:10 +08:00
Jiquan Long 691e4d954d
[skip e2e] Add implementation assertion for channelsTimeTicker (#14952)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2022-01-06 23:24:16 +08:00
Jiquan Long a18744346a
[skip e2e] Add comment for GetPartitionID (#14872)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2022-01-05 23:19:21 +08:00
Jiquan Long 95fd0d1bf4
[skip e2e] Add comment for newChannelsTimeTicker (#14871)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2022-01-05 21:01:30 +08:00
Jiquan Long 3e9340e3f6
[skip e2e] Add comment for channelsTimeTicker (#14870)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2022-01-05 19:07:20 +08:00
congqixia 36a43e647d
[skip e2e] Add comment for GetCollectionInfo of proxy meta cache (#14824)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-01-05 14:27:42 +08:00
congqixia 6400263f3c
[skip e2e] Improve SearchTask log format (#14753)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-01-04 14:15:39 +08:00
Jiquan Long 77ef791b6e
[skip e2e] Add comment for minFloat32 (#14752)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2022-01-04 14:13:44 +08:00
Jiquan Long b2fbf66f2f
[skip e2e] Add comment for getMinTick (#14751)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2022-01-04 14:11:51 +08:00
Jiquan Long deff7de3ac
[skip e2e] Add comment for newChannelsMgrImpl (#14750)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2022-01-04 14:09:52 +08:00
congqixia cd43b12a77
[skip e2e] Add comment for meta_cache.go (#14746)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-01-04 14:07:56 +08:00
guoxiangzhou 5fccbe9764
[skip e2e] Modify log text according to the naming conventions (#14289)
Signed-off-by: guoxiangzhou <xiangzhou.guo@zilliz.com>
2021-12-31 11:39:31 +08:00
guoxiangzhou a87a79a65a
[skip e2e] Modify log text according to the naming conventions (#14287)
Signed-off-by: guoxiangzhou <xiangzhou.guo@zilliz.com>
2021-12-31 11:37:35 +08:00
guoxiangzhou 451178203e
[skip e2e] Modify log text according to the naming conventions (#14283)
Signed-off-by: guoxiangzhou <xiangzhou.guo@zilliz.com>
2021-12-31 11:33:41 +08:00
guoxiangzhou 4f1ee13795
[skip e2e] Modify log text according to the naming conventions (#14280)
Signed-off-by: guoxiangzhou <xiangzhou.guo@zilliz.com>
2021-12-31 11:31:22 +08:00
Jiquan Long 4fe0e5e457
[skip e2e] Add comment for definitionLoop (#14637)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-12-30 20:45:21 +08:00
Jiquan Long 700546708c
[skip e2e] Add comment for GetCollectionInfo (#14636)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-12-30 20:43:26 +08:00
Jiquan Long 2cf0a551b4
[skip e2e] Add comment for getMinTsStatistics (#14635)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-12-30 20:41:33 +08:00
Xiaofan 98e4ff33a8
Fix RocksDB Slow (#14614) 2021-12-30 20:08:15 +08:00
congqixia b4d995ceb3
[skip e2e] Add comment for proxy ManualCompaction (#14619)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-12-30 19:15:22 +08:00
guoxiangzhou 62f002f9df
[skip e2e] Modify log text according to the naming conventions (#14278)
Signed-off-by: guoxiangzhou <xiangzhou.guo@zilliz.com>
2021-12-30 16:47:01 +08:00
guoxiangzhou 437b4c0f29
[skip e2e] Modify log text according to the naming conventions (#14277)
Signed-off-by: guoxiangzhou <xiangzhou.guo@zilliz.com>
2021-12-30 16:44:09 +08:00
guoxiangzhou 0769c1f757
[skip e2e] Modify log text according to the naming conventions (#14275)
Signed-off-by: guoxiangzhou <xiangzhou.guo@zilliz.com>
2021-12-30 16:41:22 +08:00
Jiquan Long c0b7f52036
Make timeTickInterval of Proxy configurable (#14515)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-12-30 10:29:47 +08:00
congqixia 7bb1d6267d
[skip e2e] Add comment for CreateAliasTask methods (#14539)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-12-29 22:40:27 +08:00
zhenshan.cao 32a5126705
Add the limitation of the traceable interval to timeTravel (#14498)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-12-29 21:29:45 +08:00
Jiquan Long 028fcdc35a
[skip e2e] Add comment for channelsMgrImpl (#14530)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-12-29 17:44:34 +08:00
Jiquan Long b358e9c88d
[skip e2e] Add comment for taskScheduler (#14528)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-12-29 17:42:45 +08:00
Jiquan Long 3baa698e9f
[skip e2e] Add comment for getLastTick (#14529)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-12-29 17:25:37 +08:00
Xiaofan 69087ff8dd
Support embedded etcd (#14333)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2021-12-29 14:35:21 +08:00
Jiquan Long 673806b1c1
[skip e2e] Add comment for baseTaskQueue (#14460)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-12-29 14:31:21 +08:00
Jiquan Long 9ca90841a7
[skip e2e] Add comment for initSession (#14459)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-12-29 14:29:37 +08:00
guoxiangzhou 4abc4c7e1a
[skip e2e] Modify log text according to the naming conventions (#14274)
Signed-off-by: guoxiangzhou <xiangzhou.guo@zilliz.com>
2021-12-29 14:09:21 +08:00
guoxiangzhou 3705ac7142
[skip e2e] Modify log text according to the naming conventions (#14292)
Signed-off-by: guoxiangzhou <xiangzhou.guo@zilliz.com>
2021-12-29 14:07:22 +08:00
congqixia 53784b5237
[skip e2e] Add comment for NewMetaCache (#14477)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-12-29 11:55:02 +08:00
congqixia ffe44ae1ea
[skip e2e] Add comment for CreateAliasTask (#14463)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-12-28 22:22:46 +08:00
congqixia 307b07363b
[skip e2e] Add comment for IDs2Expr (#14352)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-12-28 09:48:15 +08:00
cxytz01 7a009ee182
[skip e2e] Grammar correction (#14344)
Signed-off-by: cxytz01 <xueyou.chen@zilliz.com>
2021-12-27 19:00:08 +08:00
cxytz01 4c95803148
[skip e2e] Grammar correction (#14343)
Signed-off-by: cxytz01 <xueyou.chen@zilliz.com>
2021-12-27 18:58:20 +08:00
cxytz01 bc05f3e8b0
[skip e2e] Grammar correction (#14342)
Signed-off-by: cxytz01 <xueyou.chen@zilliz.com>
2021-12-27 18:56:32 +08:00
guoxiangzhou 4671a88fbb
[skip e2e] Modify log text according to the naming conventions (#14291)
Signed-off-by: guoxiangzhou <xiangzhou.guo@zilliz.com>
2021-12-27 15:21:13 +08:00
guoxiangzhou 049894fc14
[skip e2e] Modify log text according to the naming conventions (#14288)
Signed-off-by: guoxiangzhou <xiangzhou.guo@zilliz.com>
2021-12-27 15:19:27 +08:00
guoxiangzhou 703eabc2bf
[skip e2e] Modify log text according to the naming conventions (#14282)
Signed-off-by: guoxiangzhou <xiangzhou.guo@zilliz.com>
2021-12-27 15:10:49 +08:00
guoxiangzhou a578382d49
[skip e2e] Modify log text according to the naming conventions (#14279)
Signed-off-by: guoxiangzhou <xiangzhou.guo@zilliz.com>
2021-12-27 15:06:43 +08:00
Jiquan Long 8a488f1f20
Refine startup workflow of proxy (#13459)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-12-25 16:10:17 +08:00
zhenshan.cao 9a2c91d326
Fix bug: refresh config after session init (#14194)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-12-24 22:22:17 +08:00
Jiquan Long dc3c72b9b6
[skip e2e] Add comment for AllocOne (#14202)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-12-24 19:29:33 +08:00
Jiquan Long 3e2332bb3c
[skip e2e] Add comment for isAlpha (#14201)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-12-24 19:27:38 +08:00
Jiquan Long fb3b419c48
[skip e2e] Add comment for enableMultipleVectorFields (#14200)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-12-24 19:25:20 +08:00
cxytz01 7d5ff4f771
[skip e2e] Grammar correction (#14128)
Signed-off-by: cxytz01 <xueyou.chen@zilliz.com>
2021-12-24 17:36:19 +08:00
Xiaofan 7a4de9897d
Fix Rocksmq ignored error (#14011)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2021-12-24 14:14:15 +08:00
shaoyue 26ed17a905
[skip e2e] Add comment for AlterAliasTask (#14140)
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
2021-12-24 13:33:45 +08:00
guoxiangzhou de97236410
[skip e2e] Modify code comments according to naming conventions (#14135)
Signed-off-by: guoxiangzhou <xiangzhou.guo@zilliz.com>
2021-12-24 13:21:00 +08:00
guoxiangzhou 474b00c4d3
[skip e2e] Modify code comments according to naming conventions (#14134)
Signed-off-by: guoxiangzhou <xiangzhou.guo@zilliz.com>
2021-12-24 13:18:00 +08:00
guoxiangzhou 5c90b7d576
[skip e2e] Modify code comments according to naming conventions (#14131)
Signed-off-by: guoxiangzhou <xiangzhou.guo@zilliz.com>
2021-12-24 13:04:23 +08:00
cxytz01 b5fe5fc10b
[skip e2e] Grammar correction (#14129)
Signed-off-by: cxytz01 <xueyou.chen@zilliz.com>
2021-12-24 12:57:36 +08:00
Xiaofan 24e2faf110
Fix has collection err (#14089)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2021-12-23 21:46:10 +08:00
Jiquan Long b5f0dee1dd
[skip e2e] Add comment for pChan (#14098)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-12-23 21:44:12 +08:00
Jiquan Long 330c6eccb1
[skip e2e] Add comment for defaultInsertRepackFunc (#14097)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-12-23 21:42:22 +08:00
Cai Yudong 891c202b73
Use GlobalParamTable for all components (#13885)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-12-23 18:39:11 +08:00
congqixia 12bd3ade4c
[skip e2e] Add comment for proxy Cache and MetaCache struct (#14076)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-12-23 18:13:30 +08:00
edward.zeng 21ad57c3da
[skip e2e] Add comment to proxy meta cache (#14021)
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
2021-12-23 15:08:45 +08:00
Jiquan Long bca2e1604d
[skip e2e] Add comment for timestampAllocator (#14009)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-12-23 15:06:48 +08:00
edward.zeng 39fcbaac47
[skip e2e] Refine proxy comment (#14044)
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
2021-12-23 15:01:53 +08:00
shaoyue e8ebd926e0
[skip e2e] Add comment for internal/proxy/task.go (#14031)
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
2021-12-23 11:05:09 +08:00
Jiquan Long d6e26859a1
[skip e2e] Add comment for vChan (#14008)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-12-22 21:07:10 +08:00
edward.zeng 8eb4d9a2b4
[skip e2e] Add comment to proxy task (#14003)
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
2021-12-22 20:29:29 +08:00
Jiquan Long f6e23458aa
Refine log of ready to reduce (#13448)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-12-22 10:23:00 +08:00
Jiquan Long bdb001b20d
Add log of query to record time spent in mq (#13526)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-12-22 10:11:02 +08:00
Jiquan Long 9a7428243f
[skip e2e] Add comment for channelsTimeTickerImpl (#13897)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-12-21 21:01:36 +08:00
Jiquan Long 35021d6cc3
[skip e2e] Add comment for repackFuncType (#13895)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-12-21 20:59:11 +08:00
Jiquan Long be34b736b0
[skip e2e] Add comment for streamType (#13896)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-12-21 20:57:10 +08:00
Jiquan Long 8a63dba149
Add consistency level field in collection meta (#13766)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-12-21 19:49:02 +08:00
guoxiangzhou e46556c4e3
[skip e2e] Modify log text according to naming conventions (#13834)
Signed-off-by: guoxiangzhou <xiangzhou.guo@zilliz.com>
2021-12-21 15:26:59 +08:00
guoxiangzhou 5e034d7879
[skip e2e] Modify log text according to naming conventions (#13833)
Signed-off-by: guoxiangzhou <xiangzhou.guo@zilliz.com>
2021-12-21 15:15:02 +08:00
guoxiangzhou 8004292420
[skip e2e] Modify log text according to naming conventions (#13832)
Signed-off-by: guoxiangzhou <xiangzhou.guo@zilliz.com>
2021-12-21 14:40:54 +08:00
xige-16 dc7fb594af
[skip e2e]Improve log for QueryNode spell (#13809)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-12-20 23:10:51 +08:00
Jiquan Long 3edfafbfe0
[skip e2e] Add comment for getChannelsFuncType (#13781)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-12-20 17:55:16 +08:00
Jiquan Long df6333447c
[skip e2e] Add comment for getDqlChannelsFunc (#13783)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-12-20 17:53:17 +08:00
Jiquan Long fa37e6f954
[skip e2e] Add comment for getDmlChannelsFunc (#13782)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-12-20 17:52:00 +08:00
Jiquan Long d7a0857dc4
[skip e2e] Add comment for validateMultipleVectorFields (#13652)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-12-17 22:19:03 +08:00
Jiquan Long d5821ad18a
[skip e2e] Add comment for isVector (#13654)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-12-17 21:54:42 +08:00
Jiquan Long b0baff2249
[skip e2e] Add comment for isNumber (#13653)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-12-17 21:52:46 +08:00
Bingyi Sun 37705dfbbb
Add module name to logger in proxy (#13580)
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
2021-12-17 19:07:39 +08:00
shaoyue 22e0fc8701
[skip e2e] Add comment for DropAliasTask (#13626)
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
2021-12-17 18:46:38 +08:00
zhenshan.cao bfdf6eb402
[skip e2e]Fix datarace in proxy (#13607)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-12-17 16:32:52 +08:00
Jiquan Long 00c9797d91
Add log of insert to record time spent in mq (#13527)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-12-16 22:27:42 +08:00
Jiquan Long cb51df5985
Add log of search to record time spent in mq (#13525)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-12-16 22:22:06 +08:00
Jiquan Long c26341c0cd
Refine log of CreateCollection (#13449)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-12-15 22:43:10 +08:00
Jiquan Long 1a0449f4f9
Refine log of ReleaseCollection (#13450)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-12-15 22:13:10 +08:00
xige-16 42ac5f76ed
Add constraints to the combination of load and release (#13110)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-12-15 22:11:09 +08:00
shaoyue 0d88568957
[skip ci] Add comment for proxy (#13463)
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
2021-12-15 21:11:35 +08:00
xige-16 97d5231a0a
Reorganize querycoord proto (#13181)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-12-15 16:53:12 +08:00
cai.zhang 9f23fc7f2a
Register the service when the component state is healthy (#13248)
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
2021-12-15 11:47:10 +08:00
dragondriver 22d77b6d83
Refine log of DropAlias (#13297)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-12-15 10:31:52 +08:00
dragondriver fafcd1e7ef
Add ut for AlterAlias with a short ctx (#13355)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-12-15 10:29:57 +08:00
dragondriver 543cea2c62
Add ut for AlterAlias when dd queue is full (#13354)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-12-15 10:28:15 +08:00
dragondriver f7ccdd2cab
Refine log of AlterAlias (#13353)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-12-15 10:25:33 +08:00
dragondriver ee7e400640
Refine log of Query (#13295)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-12-15 10:19:10 +08:00
dragondriver 276b9a6cf5
Refine log of CreateAlias (#13296)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-12-15 07:03:10 +08:00
Xiangyu Wang 405b3cd932
Use the same lint rules with golint in revive (#13324)
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-12-14 15:31:07 +08:00
Cai Yudong b5d639a1ab
Use typeutil.ProxyRole instead of Params.RoleName in proxy (#13217)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-12-13 10:01:18 +08:00
jingkl 052fb7afbe
[skip ci]Add comments to impl (#13206)
Signed-off-by: jingkl <jingjing.jia@zilliz.com>
2021-12-10 22:53:27 +08:00
dragondriver 175f3147d0
Proxy exit when disconnected to etcd (#13180)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-12-10 22:09:17 +08:00
jingkl 3f0970153c
[skip ci]Add notes to impl (#13205)
Signed-off-by: jingkl <jingjing.jia@zilliz.com>
2021-12-10 22:01:17 +08:00
jingkl 602b3747ee
[skip ci]Add notes to impl (#13176)
Signed-off-by: jingkl <jingjing.jia@zilliz.com>
2021-12-10 21:10:42 +08:00
jingkl dd0ec0a64c
[skip ci]Add comments to impl (#13170)
Signed-off-by: jingkl <jingjing.jia@zilliz.com>
2021-12-10 21:06:40 +08:00
dragondriver 85c93b8844
Parallel when proxy get metrics from other components (#12980)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-12-10 17:03:58 +08:00
dragondriver 89d712ebc3
Refine log of Flush (#13165)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-12-10 17:01:42 +08:00
dragondriver f107f9d03e
Refine log of GetIndexState (#13164)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-12-10 16:59:54 +08:00
dragondriver 786444e322
Refine log of DescribeIndex (#13032)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-12-10 16:52:37 +08:00
dragondriver 86d69ef48c
Refine log of CreateIndex (#13031)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-12-10 16:48:50 +08:00
dragondriver 6571f811c8
Refine log of DropIndex (#13033)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-12-10 16:46:51 +08:00
dragondriver 8ad390c38a
Refine log of GetIndexBuildProgress (#13163)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-12-10 16:42:09 +08:00
jingkl e3f905350d
[skip ci] Add notes to impl (#13076)
Signed-off-by: jingkl <jingjing.jia@zilliz.com>
2021-12-09 16:09:32 +08:00
jaime f479381c39
[skip ci] Add a comment for describeCollection function in meta_cache (#13018)
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Co-authored-by: yun.zhang <yun.zhang@zilliz.com>
2021-12-09 10:51:19 +08:00
dragondriver 59c93eb351
Refine log of GetPartitionStatistics (#12940)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-12-08 23:05:07 +08:00
dragondriver 19ad282810
Add ut for GetComponentStates (#12968)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-12-08 15:17:31 +08:00
dragondriver 8eda489c5b
Refine log of ShowPartitions (#12941)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-12-08 14:57:48 +08:00