f1a108c97b
enhance: Add max file num limit and max file size limit for import ( #31497 ) ( #31542 )
...
The max number of import files per request should not exceed 1024 by
default (configurable).
The import file size allowed for importing should not exceed 16GB by
default (configurable).
issue: https://github.com/milvus-io/milvus/issues/28521
pr: https://github.com/milvus-io/milvus/pull/31497
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-03-25 14:33:07 +08:00
8858f8d612
enhance: Return more fields in import progress response ( #31539 ) ( #31561 )
...
Return more fields in import progress response, include importedRows and
totalRows. Additionally, ensure compatibility with the old import
progress response by retaining fields of create timestamp and row count.
issue: https://github.com/milvus-io/milvus/issues/31448
https://github.com/milvus-io/milvus/issues/31237
https://github.com/milvus-io/milvus/issues/28521
pr: https://github.com/milvus-io/milvus/pull/31539
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-03-25 14:27:07 +08:00
1e0bf5acd2
enhance: Remove import v1 ( #31403 ) ( #31535 )
...
Remove all code and logic related to import v1.
issue: https://github.com/milvus-io/milvus/issues/28521
pr: https://github.com/milvus-io/milvus/pull/31403
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-03-24 21:51:07 +08:00
b5c67948b7
enhance: Enhance and modify the return content of ImportV2 ( #31192 )
...
1. The Import APIs now provide detailed progress information for each
imported file, including details such as file name, file size, progress,
and more.
2. The APIs now return the collection name and the completion time.
3. Other modifications include changing jobID to jobId and other similar
adjustments.
issue: https://github.com/milvus-io/milvus/issues/28521
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-03-13 19:51:03 +08:00
87b3c25b15
fix: Fix binlog import ( #31205 )
...
1. File type validation is omitted during binlog import.
2. System fields are appended to the schema during binlog import.
issue: https://github.com/milvus-io/milvus/issues/28521
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-03-13 10:35:04 +08:00
8cb06acfed
feat: Replacing the current import API with the v2 implementation ( #31046 )
...
Replacing the current import API v1 implementation with the v2
implementation.
issue: https://github.com/milvus-io/milvus/issues/28521
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-03-10 12:23:02 +08:00
a434d33e75
feat: Add import scheduler and manager ( #29367 )
...
This PR introduces novel managerial roles for importv2:
1. ImportMeta: To manage all the import tasks;
2. ImportScheduler: To process tasks and modify their states;
3. ImportChecker: To ascertain the completion of all tasks and instigate
relevant operations.
issue: https://github.com/milvus-io/milvus/issues/28521
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-03-01 18:31:02 +08:00
1c1dd48b71
fix: Deprecated shard leader cache while deprecating collection ( #30446 )
...
Resolves : #30445
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-02-02 09:11:05 +08:00
e22e8b30d4
enhance: check load state before altering collection ( #30399 )
...
/kind improvement
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2024-02-01 18:03:04 +08:00
060c8603a3
fix: Support mvcc with hybrid serach ( #30114 )
...
issue: https://github.com/milvus-io/milvus/issues/29656
/kind bug
Signed-off-by: xige-16 <xi.ge@zilliz.com>
---------
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2024-02-01 16:03:03 +08:00
f619d792c0
enhance: Break down the granularity of collection info cache expired ( #29977 )
...
issue: #29772
1. `DropPartition` only invalidates the cache related to the partition.
2. `CreateAlias` does not invalidate the cache.
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-01-30 16:45:02 +08:00
8fc4ebfa11
fix: empty MetricType ( #30216 )
...
issue: #30102 #30225
we should read MetricType from SearchResult,
because query node never
1. read metricType from LoadMeta
2. store to collection
3. set SearchRequest.MetricType
Signed-off-by: PowderLi <min.li@zilliz.com>
2024-01-28 15:33:02 +08:00
463765922e
enhance: support related privilege for grant api ( #30153 )
...
/kind improvement
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-01-23 15:56:54 +08:00
04e21564b6
enhance: delete should inc ProxyFunctionCall metric failed label when run failed instead abandon ( #29621 )
...
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-01-17 19:16:55 +08:00
80465a01b5
enhance: change some important request log level to Info ( #30062 )
...
Some important request log level shall be at least Info level
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-01-17 17:48:53 +08:00
1df3f90696
feat: Implement DescribeAlias and ListAliases interfaces ( #29641 )
...
#22882
/kind feature
Signed-off-by: wayblink <anyang.wang@zilliz.com>
2024-01-11 19:12:51 +08:00
9702cef2b5
feat: Support multiple vector search ( #29433 )
...
issue #25639
Signed-off-by: xige-16 <xi.ge@zilliz.com>
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2024-01-08 15:34:48 +08:00
05d735c322
enhance: Rename SearchV2 to HybridSearch ( #29592 )
...
related: https://github.com/milvus-io/milvus-proto/pull/233
issue: #29593
/kind enhancement
Signed-off-by: wayblink <anyang.wang@zilliz.com>
2024-01-04 19:22:46 +08:00
b12c90af72
enhance:Add upsert vector metrics ( #29226 )
...
add some metrics.
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-01-03 10:06:48 +08:00
a76e3b2813
Refine delete by expression for forbid proxy dml task scheduler hang ( #29340 )
...
relate: https://github.com/milvus-io/milvus/issues/29146
---------
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2023-12-26 19:52:48 +08:00
a0e1a1eb31
feat: support enable/disable mmap for index ( #29005 )
...
support enable/disable mmap for index, the user could alter the index's
mode by `AlterIndex` method
related: https://github.com/milvus-io/milvus/issues/21866
---------
Signed-off-by: yah01 <yah2er0ne@outlook.com>
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-12-21 18:07:24 +08:00
bcd6865b29
enhance: add 3 builtin roles ( #28961 )
...
issue: #28960 [milvus-proto
#212 ](https://github.com/milvus-io/milvus-proto/issues/212 )
add new configuration: builtinRoles
user can define roles in config file: `milvus.yaml`
there is an example:
1. db_ro, only have read privileges, include load
2. db_rw, read and write privileges, include create/drop/rename
collection
3. db_admin, not only read and write privileges, but also user
administration
Signed-off-by: PowderLi <min.li@zilliz.com>
2023-12-18 14:28:41 +08:00
3cc4209d26
enhance: pack proxy connection code and support accesslog print SDK_Version ( #28835 )
...
relate: https://github.com/milvus-io/milvus/issues/28086
https://github.com/milvus-io/milvus/issues/28940
---------
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2023-12-05 16:50:47 +08:00
4b8bc2798e
enhance: Print nq ( #28507 )
...
Log nq in search path.
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-11-30 10:42:27 +08:00
79c0edb1d8
enhance:Remove msgbase unnecessary assignments ( #28511 )
...
remove some unnecessary assignments, for the reason that
commonpbutil.NewMsgBase has default value.
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2023-11-24 15:02:39 +08:00
de13865769
enhance: Add load/release partitions to replicate msg stream ( #28399 )
...
/kind improvement
issue: #25655
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-11-23 15:38:24 +08:00
73f2bab454
enhance:add some log when create client and get component states ( #28160 )
...
/kind improvement
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2023-11-22 09:12:22 +08:00
e47fe4e4d1
enhance: Reduce ListImportTasks log content ( #28605 )
...
In proxy `ListImportTasks` may print all task information from
rootcoord, this may cause log content too large to process
This PR make this log print taskID and importState only
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-11-21 20:58:23 +08:00
f9c630247d
Construct plan directly when search with vector output ( #27928 )
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-10-26 19:30:10 +08:00
9b0ecbdca7
Support to replicate the mq message ( #27240 )
...
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-10-20 14:26:09 +08:00
e98e56f75d
Fix SIGSEGV if run within gdb ( #27736 )
...
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-10-18 02:16:11 +08:00
599012a340
Implement SearchByPks path for Search ( #25882 )
...
Signed-off-by: unfode <forrest.futao.wei@gmail.com>
2023-10-16 15:34:08 +08:00
310739d115
remove flush from ddl queue in proxy ( #27688 )
...
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-10-13 18:33:36 +08:00
be980fbc38
Refine state check ( #27541 )
...
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-10-11 21:01:35 +08:00
3759857bc5
Refine Proxy errors ( #27499 )
...
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-10-09 10:09:33 +08:00
241f2d77fb
Fix miss to handle error from GetCollectionID ( #26712 )
...
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2023-10-08 11:53:32 +08:00
63ac43a3b8
Refine errors for import ( #27379 )
...
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-09-30 10:31:28 +08:00
6539a5ae2c
Refine DataCoord status ( #27262 )
...
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-09-26 17:15:27 +08:00
7f7c71ea7d
Decoupling client and server API in types interface ( #27186 )
...
Co-authored-by:: aoiasd <zhicheng.yue@zilliz.com>
Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-09-26 09:57:25 +08:00
fc9a9a7926
fix panic in getFlushState ( #27237 )
...
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-09-21 14:53:25 +08:00
26f06dd732
Format the code ( #27275 )
...
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-09-21 09:45:27 +08:00
a6b98740b7
Replace all status with only error string ( #27125 )
...
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-09-18 14:05:28 +08:00
168e82ee10
Fix panic while handling with the nil status ( #27040 )
...
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-09-15 10:09:21 +08:00
fe5781e2b8
Maintain compatibility of GetFlushState with old client ( #27077 )
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-09-14 19:57:20 +08:00
dd2cb1d44a
Flush by flush channels ( #26859 )
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-09-12 21:07:19 +08:00
00c65fa0d7
Refine QueryNode errors ( #27013 )
...
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-09-12 16:07:18 +08:00
e107d0794c
support complex delete expression ( #25752 )
...
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2023-09-12 10:19:17 +08:00
ad1cc00a18
Fix typo in comment ( #26951 )
...
Signed-off-by: unfode <forrest.futao.wei@gmail.com>
2023-09-09 20:13:16 +08:00
d7b6f3a9e8
Enhance FlushAll/GetFlushAllState API ( #26802 )
...
Signed-off-by: wayblink <anyang.wang@zilliz.com>
2023-09-06 10:35:48 +08:00
3349db4aa7
Refine errors to remove changes breaking design ( #26521 )
...
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-09-04 09:57:09 +08:00
yihao.dai
congqixia
Bingyi Sun
xige-16
cai.zhang
PowderLi
SimFG
aoiasd
wayblink
smellthemoon
yah01
Enwei Jiao
Futao Wei
wei liu
yah01
jaime