428840178c
Support diskann index for vector field ( #19093 )
...
Signed-off-by: xige-16 <xi.ge@zilliz.com>
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-09-21 20:16:51 +08:00
297d75fc92
Enable query pagination ( #19231 )
...
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-09-20 20:48:50 +08:00
5143e2a75e
Disable quota and limits ( #19242 )
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-09-20 09:34:50 +08:00
d7f38a803d
Separate some proto files ( #19218 )
...
Signed-off-by: SimFG <bang.fu@zilliz.com>
Signed-off-by: SimFG <bang.fu@zilliz.com>
2022-09-16 16:56:49 +08:00
3998752db3
Fix log print ( #19204 )
...
Signed-off-by: sunby <bingyi.sun@zilliz.com>
Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@zilliz.com>
2022-09-16 11:12:47 +08:00
5cdd6ca251
Add quotaCenter, rateLimiter and rateCollector ( #18636 )
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-09-16 09:56:47 +08:00
626854cf0c
Refactor QueryCoord ( #18836 )
...
Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: yah01 <yang.cen@zilliz.com>
Co-authored-by: Wei Liu <wei.liu@zilliz.com>
Co-authored-by: Congqi Xia <congqi.xia@zilliz.com>
Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: yah01 <yang.cen@zilliz.com>
Co-authored-by: Wei Liu <wei.liu@zilliz.com>
Co-authored-by: Congqi Xia <congqi.xia@zilliz.com>
2022-09-15 18:48:32 +08:00
200df76864
Enable search with pagination ( #19044 )
...
See also: #19043
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-09-14 20:36:32 +08:00
3927ae9952
Return all flushed segments and time-of-seal in Flush interface ( #18882 )
...
Signed-off-by: wayblink <anyang.wang@zilliz.com>
Signed-off-by: wayblink <anyang.wang@zilliz.com>
2022-09-09 09:58:37 +08:00
ebc263f440
Not allow '$' in collection/partition/field name ( #19082 )
...
Signed-off-by: yhmo <yihua.mo@zilliz.com>
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2022-09-08 10:16:34 +08:00
2bafdf8c53
Fix the `Flush` privilege can't be granted ( #19020 )
...
Signed-off-by: SimFG <bang.fu@zilliz.com>
Signed-off-by: SimFG <bang.fu@zilliz.com>
2022-09-05 21:05:11 +08:00
a5e2d6b6fb
Refactor RootCoord ( #18930 )
...
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
Co-authored-by: xaxys <tpnnghd@163.com>
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
Co-authored-by: xaxys <tpnnghd@163.com>
2022-09-05 13:29:11 +08:00
9cd19f5396
Return nil error when handling grpc request ( #18955 )
...
Signed-off-by: SimFG <bang.fu@zilliz.com>
Signed-off-by: SimFG <bang.fu@zilliz.com>
2022-09-02 21:12:59 +08:00
8a45381e53
Make DescribeCollection return collectionName ( #19000 )
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-09-02 19:20:59 +08:00
c054873908
Add limit to proto ( #18950 )
...
See also: #18893
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-09-01 18:54:58 +08:00
867ea63bdd
Move TopK check inside parseQueryInfo ( #18892 )
...
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-08-30 10:32:56 +08:00
69c0b2fb49
Improve the storage method of the grant info in the metastore ( #18817 )
...
Signed-off-by: SimFG <bang.fu@zilliz.com>
Signed-off-by: SimFG <bang.fu@zilliz.com>
2022-08-26 19:22:56 +08:00
c924f73105
Refactor for IndexCoord to support cloud ( #18643 )
...
Co-authored-by: Zach41 <zongmei.zhang@zilliz.com>
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
Co-authored-by: Zach41 <zongmei.zhang@zilliz.com>
2022-08-25 15:48:54 +08:00
dad060af2e
Fix func checkIfLoaded in proxy ( #18769 )
...
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2022-08-23 15:14:52 +08:00
0c9a10e8f8
1. refine logging interfaces ( #18692 )
...
2. adjust logs for query/search requests
Signed-off-by: Zach41 <zongmei.zhang@zilliz.com>
Signed-off-by: Zach41 <zongmei.zhang@zilliz.com>
2022-08-23 10:44:52 +08:00
ce434b496e
Implement the mysql metastore of the rbac ( #18704 )
...
Signed-off-by: SimFG <bang.fu@zilliz.com>
Signed-off-by: SimFG <bang.fu@zilliz.com>
2022-08-23 10:26:53 +08:00
516fd928f9
Fix password verification miss cache ( #18730 )
...
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
2022-08-19 19:42:50 +08:00
e05e9cb9b4
Fix typo in error message ( #18688 )
...
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-08-19 10:48:50 +08:00
d4c54d96b0
Dedup output fields for task query ( #18673 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-08-16 18:16:48 +08:00
447ce8ee48
Handle some apis about the default role ( #18647 )
...
Signed-off-by: SimFG <bang.fu@zilliz.com>
Signed-off-by: SimFG <bang.fu@zilliz.com>
2022-08-15 16:40:48 +08:00
8a70f2badb
Add ShowConfigurations for all remaining session ( #18594 )
...
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2022-08-12 13:20:39 +08:00
1366f130a2
Add a specific `KeyNotExistError` err for the metastore `load` interface ( #18582 )
...
Signed-off-by: SimFG <bang.fu@zilliz.com>
2022-08-10 17:20:41 +08:00
12023e8569
Fix a potential nullptr issue with metric ( #18540 )
...
/kind improvement
related to: #18530
but not the root cause
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-08-09 21:12:42 +08:00
c267ad612d
Support Role-Based Access Control II ( #18528 )
...
Signed-off-by: SimFG <bang.fu@zilliz.com>
2022-08-05 16:28:35 +08:00
b427450593
Remove Load Delta Channel Request ( #18497 )
...
Fix #18393
remove watch delta channel task related logic
Co-authored-by: xiaofan-luan <xiaofan.luan@zilliz.com>
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
Co-authored-by: Congqi Xia <congqi.xia@zilliz.com>
2022-08-04 17:50:37 +08:00
ff0200210a
Support Role-Based Access Control ( #18425 )
...
Signed-off-by: SimFG <bang.fu@zilliz.com>
2022-08-04 11:04:34 +08:00
a7ce0eb6f6
Add ShowConfigurations to QueryNode ( #18426 )
...
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2022-08-03 01:12:33 +08:00
638f6c36e9
Support to get real row count of segment ( #18115 )
...
Signed-off-by: xaxys <zheng.guan@zilliz.com>
2022-07-18 09:58:28 +08:00
a9aff3a678
Allocate ID for each insertMsg ( #18217 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-07-11 14:38:24 +08:00
0fb5e9c1a9
Remove load collection check in Proxy ( #18166 )
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-07-08 16:12:25 +08:00
5fda8261d0
Fix: Unable to list newly created credential users ( #18117 ) ( #18153 )
...
Signed-off-by: kejiang <ke.jiang@zilliz.com>
Co-authored-by: kejiang <ke.jiang@zilliz.com>
2022-07-07 21:16:22 +08:00
1c9647ff31
Merging multiple shard query/search requests into one rpc request if QueryNode is the leader of multiple shards of Collection ( #17588 )
...
Signed-off-by: Zach41 <zongmei.zhang@zilliz.com>
2022-07-06 15:06:21 +08:00
696d747afa
Check if loaded in Proxy when LoadCollection ( #18074 )
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-07-05 21:16:20 +08:00
9bd35dd84e
Refine logs to reduce print ( #17759 ) ( #18028 )
...
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-07-04 11:10:20 +08:00
f33b090819
Fix bug: datarace in proxy unittest ( #17990 )
...
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-07-01 17:56:21 +08:00
6ab850bedf
Fix update credential failure ( #17963 ) ( #17970 )
...
Signed-off-by: kejiang <ke.jiang@zilliz.com>
Co-authored-by: kejiang <ke.jiang@zilliz.com>
2022-06-30 21:34:18 +08:00
95a3830dc6
fix password comparison too slow ( #17874 ) ( #17897 )
...
Signed-off-by: kejiang <ke.jiang@zilliz.com>
Co-authored-by: kejiang <ke.jiang@zilliz.com>
2022-06-29 20:02:18 +08:00
6b72f9013f
Fix message type of show partitions ( #17821 )
...
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-06-28 10:06:17 +08:00
8ff1a2c4c2
Print expr when proxy receive qyery req ( #17662 )
...
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-06-23 13:42:13 +08:00
216e2f80aa
Add msg id to log in query path ( #17677 )
...
/kind improvement
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2022-06-23 10:46:13 +08:00
3fadf4c5d7
Return the unavailable error to make the client's retry policy take effect ( #17719 )
...
Signed-off-by: SimFG <bang.fu@zilliz.com>
2022-06-22 21:56:13 +08:00
e43f487f02
Make LoadCollection compatible ( #17669 )
...
See also: #17656
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-06-21 17:26:15 +08:00
0282f69eb4
Add msg id to log in search path. ( #17654 )
...
issue: #17513
Add msg id to log in search path,
which should help us to trace a detailed request more easier.
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2022-06-21 13:30:12 +08:00
32a3ed0791
Add e2e test for RESTful APIs, fix json encoding for on proto_oneof fields ( #17362 )
...
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
2022-06-20 22:06:12 +08:00
f123d657ed
Fix bug for dropping index ( #17635 )
...
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
2022-06-20 15:22:13 +08:00
141a3bbe00
Mark deleted before drop index ( #17490 )
...
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
2022-06-17 18:08:12 +08:00
53a49a9255
GetIndexState only get the state generated call CreateIndex ( #17549 )
...
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
2022-06-16 20:12:11 +08:00
1f6fbf91b2
Fix pulsar unsubsribe issue ( #17562 )
...
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-06-16 17:28:11 +08:00
2f66531fdf
Fix wrong IP distances ( #17590 )
...
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2022-06-16 16:34:11 +08:00
e825215c44
Remove the redundant 'for' in the TaskCondition ( #17558 )
...
Signed-off-by: SimFG <bang.fu@zilliz.com>
2022-06-15 16:32:10 +08:00
e9dcda161f
Add aliases return value in describeCollection task ( #17516 )
...
Signed-off-by: xingzhao <xing.zhao@zilliz.com>
Co-authored-by: 赵星 <zhaoxing@zhaoxings-MacBook-Pro.local>
2022-06-13 19:22:10 +08:00
dfaed5acdd
Add QueryServiceAvailable field in ShowCollections ( #17456 )
...
Signed-off-by: Letian Jiang <letian.jiang@zilliz.com>
2022-06-09 18:20:07 +08:00
2ca81620ea
Reduce lock operations when get dml stream ( #17468 )
...
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2022-06-09 17:34:09 +08:00
b5c11a216d
Alter varChar type params's name to max_length ( #17409 )
...
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-06-07 15:58:06 +08:00
de8f2f76c5
Fix shardLeader cache with ErrConnect check ( #17402 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-06-07 12:20:06 +08:00
98e95275fe
Fix several code issues caused by carelessness ( #17385 )
...
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-06-06 18:58:05 +08:00
f5e63177d2
Fix an issue with bulk load with multiple DataNodes ( #17324 )
...
issue: #17300
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-06-02 18:54:04 +08:00
5f44e45480
Add metrics on QueryNode and Proxy ( #17328 )
...
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-06-02 16:06:03 +08:00
adf3b14027
Refine channels management in Proxy. ( #17334 )
...
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2022-06-02 15:34:04 +08:00
2763efc9b0
Add cache of grpc client of ShardLeader in proxy ( #17301 )
...
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-06-02 12:16:03 +08:00
72f454e054
Fix 16961, make creating dml stream idempotent ( #17302 )
...
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2022-05-31 22:58:02 +08:00
bcf3b7426a
Add distributed lock for segment refgerence ( #16782 )
...
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
2022-05-31 16:36:03 +08:00
c88514bc49
Remove not used QueryChannel in Proxy and Query Cluster ( #16856 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-05-30 19:50:04 +08:00
731870211a
Support role-based access control (only pb) ( #16987 )
...
Signed-off-by: SimFG <bang.fu@zilliz.com>
2022-05-28 00:04:01 +08:00
ad9276a440
Add metrics for proxy and querynode ( #17205 )
...
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-05-27 14:12:01 +08:00
f2a27e0e64
Retry GetShardLeaders until service available or timeout ( #17183 )
...
Signed-off-by: Letian Jiang <letian.jiang@zilliz.com>
2022-05-26 20:28:02 +08:00
3928da6493
Refine calc distance function ( #17161 )
...
Signed-off-by: groot <yihua.mo@zilliz.com>
2022-05-26 14:34:01 +08:00
e06d6d7cc6
Returns collection instead of copy to make leaderMutex work ( #17206 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-05-25 15:20:00 +08:00
de0ba6d495
Fix GetQuerySegmentInfo() returns incorrect result after LoadBalance ( #17190 )
...
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-05-25 15:17:59 +08:00
4293b640b2
Remove confusing log in segment id assigner ( #17195 )
...
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2022-05-25 10:26:00 +08:00
ec1103ca27
Simplify the merge logic of searchTask ( #17194 )
...
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-05-24 21:27:59 +08:00
2512e668f7
Update timestmap according to gracefultime for Bounded Consistency ( #17171 )
...
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-05-24 12:05:59 +08:00
ea8e1623cd
Acquire nq from placeholdergroup incase of nq is zero ( #17168 )
...
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-05-24 10:05:58 +08:00
08a4dff41d
Add logic for search merging and a simple task scheduler for read tasks ( #17022 )
...
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
Co-authored-by: cai.zhang <cai.zhang@zilliz.com>
Co-authored-by: bigsheeper <yihao.dai@zilliz.com>
Co-authored-by: cai.zhang <cai.zhang@zilliz.com>
Co-authored-by: bigsheeper <yihao.dai@zilliz.com>
2022-05-23 16:41:58 +08:00
7746a5b742
Add NodeIds field for QuerySegmentInfo ( #17121 )
...
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-05-20 18:03:58 +08:00
90ee23df22
Fix shardLeader cache concurrent access ( #17120 )
...
Fix write map without mutex control
Also GetShards returns a copy of leader list instead of original one
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-05-20 14:33:57 +08:00
276ba69819
Replace plan parser with antlr ( #17090 )
...
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2022-05-19 17:15:57 +08:00
9eeec4a2d5
Add collection load cache and InvalidateCollMetaCache by collID ( #16882 )
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-05-19 10:13:56 +08:00
127dd34b37
Make proxy use roundrobin to choose replica ( #17063 )
...
Fixes : #17055
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-05-17 22:35:57 +08:00
a1480aaf76
Clear shard leaders cache when releasing ( #16942 )
...
See also: #16926
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-05-17 11:11:56 +08:00
ea2c395a38
Fix the comment info in task_test.go ( #16982 )
...
Signed-off-by: SimFG <bang.fu@zilliz.com>
2022-05-15 09:03:54 +08:00
52fc9a558c
Fix valid error message ( #16953 )
...
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-05-13 18:19:53 +08:00
43b36b4e5f
Make proxy update cache with NotShardLeader status code ( #16940 )
...
See also: #16926
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-05-13 11:35:53 +08:00
9e6d9fb7d8
Print collectionName when load failed ( #16878 )
...
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-05-11 09:47:53 +08:00
8fcf349609
Fix create index failed after drop index ( #16836 )
...
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-05-09 11:35:53 +08:00
98ceb162aa
Support antlr as plan parser ( #16696 )
...
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
Co-authored-by: xaxys <tpnnghd@163.com>
Co-authored-by: xaxys <tpnnghd@163.com>
2022-05-06 17:43:51 +08:00
b0053b7a45
Add isShardLeader field in search/query request ( #16798 )
...
Signed-off-by: Letian Jiang <letian.jiang@zilliz.com>
2022-05-06 16:55:51 +08:00
009b773b6e
Delete set default type param max_len for varchar field ( #16794 )
...
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-05-06 15:59:51 +08:00
fa97f86cda
Clean up not used id_cache related code ( #16655 )
...
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
2022-05-05 09:59:51 +08:00
2fe8677cbf
Enable dimension check in Proxy when create index request received ( #16718 )
...
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2022-04-29 18:01:49 +08:00
97757405de
Fix bug: wrong metrics name and forget to register metrics ( #16714 )
...
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-04-29 15:35:47 +08:00
515d0369de
Support string type in segcore ( #16546 )
...
Signed-off-by: xige-16 <xi.ge@zilliz.com>
Co-authored-by: dragondriver <jiquan.long@zilliz.com>
Co-authored-by: dragondriver <jiquan.long@zilliz.com>
2022-04-29 13:35:49 +08:00
21b5973253
Refine list import task api ( #16690 )
...
Signed-off-by: groot <yihua.mo@zilliz.com>
2022-04-28 17:21:47 +08:00
0a953948af
Simplify monitoring metrics ( #16687 )
...
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-04-27 23:03:47 +08:00
35b7267edb
Add ut for task_policies ( #16663 )
...
See also: #16652
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-04-27 16:55:46 +08:00
xige-16
XuanYang-cn
bigsheeper
SimFG
Bingyi Sun
wayblink
groot
Jiquan Long
cai.zhang
aoiasd
Zach
jaime
congqixia
Ten Thousand Leaves
Xiaofan
Jeng.Gwan
codeman
zhenshan.cao
shaoyue
xing.zhao
Letian Jiang
yah01
SimFG