c012e6786f
feat: support rate limiter based on db and partition levels ( #31070 )
...
issue: https://github.com/milvus-io/milvus/issues/30577
co-author: @jaime0815
---------
Signed-off-by: Patrick Weizhi Xu <weizhi.xu@zilliz.com>
Signed-off-by: SimFG <bang.fu@zilliz.com>
Co-authored-by: Patrick Weizhi Xu <weizhi.xu@zilliz.com>
2024-04-12 16:01:19 +08:00
90bed1caf9
enhance: add the related data size for the read apis ( #31816 )
...
issue: #30436
origin pr: #30438
related pr: #31772
---------
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-04-10 15:07:17 +08:00
089c805e0a
enhance:Refactor hybrid search ( #32020 )
...
issue: https://github.com/milvus-io/milvus/issues/25639
https://github.com/milvus-io/milvus/issues/31368
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2024-04-09 14:21:18 +08:00
b1a1cca10b
feat: add more operation detail info for better allocation ( #30438 )
...
issue: #30436
---------
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-03-28 06:33:11 +08:00
9a13b9822f
enhance: Return more fields in import progress response ( #31539 )
...
Return more fields in import progress response, include importedRows and
totalRows. Additionally, ensure compatibility with the old import
progress response by retaining fields of create timestamp and row count.
issue: https://github.com/milvus-io/milvus/issues/31448
https://github.com/milvus-io/milvus/issues/31237
https://github.com/milvus-io/milvus/issues/28521
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-03-24 21:57:06 +08:00
b5c67948b7
enhance: Enhance and modify the return content of ImportV2 ( #31192 )
...
1. The Import APIs now provide detailed progress information for each
imported file, including details such as file name, file size, progress,
and more.
2. The APIs now return the collection name and the completion time.
3. Other modifications include changing jobID to jobId and other similar
adjustments.
issue: https://github.com/milvus-io/milvus/issues/28521
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-03-13 19:51:03 +08:00
a434d33e75
feat: Add import scheduler and manager ( #29367 )
...
This PR introduces novel managerial roles for importv2:
1. ImportMeta: To manage all the import tasks;
2. ImportScheduler: To process tasks and modify their states;
3. ImportChecker: To ascertain the completion of all tasks and instigate
relevant operations.
issue: https://github.com/milvus-io/milvus/issues/28521
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-03-01 18:31:02 +08:00
060c8603a3
fix: Support mvcc with hybrid serach ( #30114 )
...
issue: https://github.com/milvus-io/milvus/issues/29656
/kind bug
Signed-off-by: xige-16 <xi.ge@zilliz.com>
---------
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2024-02-01 16:03:03 +08:00
c5918290e6
feat: Add import executor and manager for datanode ( #29438 )
...
This PR introduces novel importv2 roles for datanode:
1. Executor: To execute tasks, a import task will be divided into the
following steps: read data -> hash data -> sync data;
2. Manager: To manage all the tasks;
issue: https://github.com/milvus-io/milvus/issues/28521
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-01-31 20:45:04 +08:00
60e88fb833
fix: Restore the MVCC functionality. ( #29749 )
...
When the TimeTravel functionality was previously removed, it
inadvertently affected the MVCC functionality within the system. This PR
aims to reintroduce the internal MVCC functionality as follows:
1. Add MvccTimestamp to the requests of Search/Query and the results of
Search internally.
2. When the delegator receives a Query/Search request and there is no
MVCC timestamp set in the request, set the delegator's current tsafe as
the MVCC timestamp of the request. If the request already has an MVCC
timestamp, do not modify it.
3. When the Proxy handles Search and triggers the second phase ReQuery,
divide the ReQuery into different shards and pass the MVCC timestamp to
the corresponding Query requests.
issue: #29656
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2024-01-09 11:38:48 +08:00
7939f0e7d5
enable ctx traceId for assignsegment on dc( #26972 ) ( #27108 ) ( #27030 )
...
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
2023-09-18 11:39:20 +08:00
ca1349708b
Remove time travel ralted testcase ( #26119 )
...
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-08-10 18:53:17 +08:00
d63323d117
Add rate limit and deny write in upsert ( #25351 )
...
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2023-07-11 11:20:34 +08:00
d7d61f529c
[Feature|Pick] enable scheduler policy and add user-task-polling policy ( #24839 )
...
Signed-off-by: chyezh <ye.zhen@zilliz.com>
2023-07-03 18:24:25 +08:00
e40d95e093
add search/query request cost metrics on qn ( #24413 )
...
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-06-15 14:24:38 +08:00
b74770b636
support try-best query reducer methods( #24075 ) ( #24103 )
...
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
2023-05-17 16:23:23 +08:00
8139106b51
Feat: count entities by expression ( #22765 )
...
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2023-03-16 19:31:55 +08:00
d126f06946
Decouple mq module from internal proto definition ( #22536 )
...
Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-03-04 23:21:50 +08:00
cc5d72a9f2
add search option to ignore ignore segment ( #21994 )
...
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2023-02-22 17:31:46 +08:00
85e04d845c
Move GetComponentStates to milvus.proto, make it an external interface ( #19594 )
...
Signed-off-by: wayblink <anyang.wang@zilliz.com>
Signed-off-by: wayblink <anyang.wang@zilliz.com>
2022-10-10 15:55:22 +08:00
54e2c079c4
MEP-30:Support basic coordinators active-standby mechanism #16901 ( #19166 )
...
Signed-off-by: wayblink <anyang.wang@zilliz.com>
Signed-off-by: wayblink <anyang.wang@zilliz.com>
2022-09-29 18:35:02 +08:00
5cdd6ca251
Add quotaCenter, rateLimiter and rateCollector ( #18636 )
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-09-16 09:56:47 +08:00
c054873908
Add limit to proto ( #18950 )
...
See also: #18893
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-09-01 18:54:58 +08:00
ff0200210a
Support Role-Based Access Control ( #18425 )
...
Signed-off-by: SimFG <bang.fu@zilliz.com>
2022-08-04 11:04:34 +08:00
45746bb736
Add ShowConfiguration proto definition ( #18364 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-07-23 17:14:28 +08:00
638f6c36e9
Support to get real row count of segment ( #18115 )
...
Signed-off-by: xaxys <zheng.guan@zilliz.com>
2022-07-18 09:58:28 +08:00
1da970af18
Remove useless statsService in QueryNode ( #17395 )
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-07-11 15:12:25 +08:00
5fda8261d0
Fix: Unable to list newly created credential users ( #18117 ) ( #18153 )
...
Signed-off-by: kejiang <ke.jiang@zilliz.com>
Co-authored-by: kejiang <ke.jiang@zilliz.com>
2022-07-07 21:16:22 +08:00
95a3830dc6
fix password comparison too slow ( #17874 ) ( #17897 )
...
Signed-off-by: kejiang <ke.jiang@zilliz.com>
Co-authored-by: kejiang <ke.jiang@zilliz.com>
2022-06-29 20:02:18 +08:00
731870211a
Support role-based access control (only pb) ( #16987 )
...
Signed-off-by: SimFG <bang.fu@zilliz.com>
2022-05-28 00:04:01 +08:00
08a4dff41d
Add logic for search merging and a simple task scheduler for read tasks ( #17022 )
...
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
Co-authored-by: cai.zhang <cai.zhang@zilliz.com>
Co-authored-by: bigsheeper <yihao.dai@zilliz.com>
Co-authored-by: cai.zhang <cai.zhang@zilliz.com>
Co-authored-by: bigsheeper <yihao.dai@zilliz.com>
2022-05-23 16:41:58 +08:00
4e12271d79
Support login with username and password ( #15656 ) ( #16341 )
...
Signed-off-by: kejiang <ke.jiang@zilliz.com>
Co-authored-by: kejiang <ke.jiang@zilliz.com>
2022-04-11 19:49:34 +08:00
99984b88e1
Support delete varChar value ( #16229 )
...
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-04-02 17:43:29 +08:00
f71651e294
Support column-based insert data in message stream ( #15802 )
...
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2022-03-04 15:09:56 +08:00
0c39e0370c
Move segment statistics update to datanode tt loop ( #13233 )
...
Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@zilliz.com>
2021-12-15 10:53:16 +08:00
214f40b209
Add timeout ts for search and query ( #12890 )
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-12-08 14:15:03 +08:00
38375ee1d3
Add LoadBalance ( #9945 )
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-11-06 16:54:59 +08:00
92815f4542
Change timestamp to timestamps in internalproto.DeleteRequest ( #9802 )
...
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
2021-10-14 14:48:34 +08:00
d18c3bc5fd
Add db_name for alias proto ( #9010 )
...
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-10-01 21:40:21 +08:00
6ab00add27
Change expr_plan to primary_keys in internal proto DeleteRequest ( #8655 )
...
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-09-27 14:17:58 +08:00
b71bffe26f
Replace ChannelID with ShardName in DML Request ( #8539 )
...
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-09-27 10:01:59 +08:00
4fba738bec
Update internal proto DeleteRequest ( #8466 )
...
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-09-24 15:21:55 +08:00
ccb57ba4b2
add collection alias ( #7255 )
...
Signed-off-by: Yusup <yusup@lsgrep.com>
2021-09-18 11:13:51 +08:00
94272bba87
Support query by expression ( #7386 )
...
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
2021-09-03 17:12:55 +08:00
0ab70271a6
correct empty search result handle ( #7244 )
...
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-08-24 11:13:52 +08:00
1c29b42434
change output_fields to out_fields_id for RetrieveRequest ( #6506 )
...
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-07-14 13:11:54 +08:00
21a9bcd844
Create default partition when receive CreateCollectionMsg ( #6305 )
...
* create default partition when create collection
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* fix unittest
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* adjust sleep time for unittest
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-07-06 09:16:03 +08:00
07b0989628
Add GetFlushedSegments in data coordinator ( #6253 )
...
Send segment flush message by grpc and add a new interface
`GetFlushedSegments`
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-07-02 11:16:20 +08:00
ff4c6c7ff4
Fix travel timestamp and guarantee timestamp ( #6234 )
...
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-06-30 21:02:13 +08:00
b2eb596c23
Change search result to columnBased and return target_entry ( #5896 )
...
* Change search result to columnBased and return target_entry
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
Co-authored-by: fluorinedog <fluorinedog@gmail.com>
* synchronize the latest pymilvus-orm version
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
* Change pymilvus-orm version to 40
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
Co-authored-by: fluorinedog <fluorinedog@gmail.com>
2021-06-21 20:18:13 +08:00
SimFG
zhenshan.cao
yihao.dai
xige-16
MrPresent-Han
Enwei Jiao
smellthemoon
chyezh
wei liu
Jiquan Long
jaime
aoiasd
wayblink
XuanYang-cn
congqixia
Jeng.Gwan
codeman
Bingyi Sun
yukun
Cai Yudong
Yusup
sunby