Commit Graph

372 Commits (3c4df81261ee5e21e8c2f680eaea17fec996a66d)

Author SHA1 Message Date
congqixia f336b2d672
fix: Check schema without vector field in proxy (#33211)
Related to #33199

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-05-21 14:33:39 +08:00
smellthemoon b45798107a
enhance: add nullable in Field, check valid_data and fill data (#32086)
1. add nullable in model.Field
   help to read nullable accurately.
2. check valid_data
a. if user pass default_value or the field is nullable, the length of
valid_data must be num_rows.
b. if passed valid_data, the length of passed field data must equal to
the number of 'true' in valid_data.
c. after fill default_value, only nullable field will still has
valid_data.
3. fill data in two situation
    a. has no default_value, if nullable,
will fill nullValue when passed num_rows not equal to expected num_rows.
    b. has default_value,
will fill default_value when passed num_rows not equal to expected
num_rows.
c. after fill data, the length of all field will equal to passed
num_rows.
#31728

---------

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-05-16 11:57:35 +08:00
wei liu c35797c399
enhance: expose DescribeDatabase api in proxy (#32732)
issue: #32707

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-05-09 22:51:30 +08:00
SimFG 7da1ca9efb
enhance: check the partition num when creating collection with partition key (#32670)
issue: #30577

Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-05-07 10:43:29 +08:00
wayblink 42d0412e93
enhance: Add channelCPs in FlushResponse (#32044)
#32609

Signed-off-by: wayblink <anyang.wang@zilliz.com>
2024-04-30 09:45:27 +08:00
SimFG 0359139299
enhance: return the auth collection list when using the `show collection` api (#32576)
issue: #32550
/kind improvement

Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-04-29 19:09:34 +08:00
wei liu 07720f1a95
enhance: expose alter database api in proxy (#32639)
issue: #30040

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-04-28 18:15:31 +08:00
SimFG 8594b55ad5
enhance: add `max insert request size` and `must use partition key` configs (#32433)
issue: https://github.com/milvus-io/milvus/issues/30577
/kind improvement

Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-04-19 10:31:20 +08:00
chyezh 48fe977a9d
enhance: declarative resource group api (#31930)
issue: #30647

- Add declarative resource group api

- Add config for resource group management

- Resource group recovery enhancement

---------

Signed-off-by: chyezh <chyezh@outlook.com>
2024-04-15 08:13:19 +08:00
Cai Yudong 00438f408f
enhance: Unify data type check APIs for go (#31887)
Issue: #22837

Signed-off-by: Cai Yudong <yudong.cai@zilliz.com>
2024-04-07 14:27:22 +08:00
yihao.dai 0fe5e90e8b
enhance: Remove import v1 (#31403)
Remove all code and logic related to import v1.

issue: https://github.com/milvus-io/milvus/issues/28521

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-03-22 15:29:09 +08:00
Buqian Zheng 3c80083f51
feat: [Sparse Float Vector] add sparse vector support to milvus components (#30630)
add sparse float vector support to different milvus components,
including proxy, data node to receive and write sparse float vectors to
binlog, query node to handle search requests, index node to build index
for sparse float column, etc.

https://github.com/milvus-io/milvus/issues/29419

---------

Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com>
2024-03-13 14:32:54 -07:00
Chun Han 3574bdf858
enhance: ban range-search iteration for search-group-by (#30824)
related: #30033

Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
2024-03-08 14:17:00 +08:00
Bingyi Sun eb0cfe3e84
fix: set CollectionID field for AlterCollection request (#30814)
issue: https://github.com/milvus-io/milvus/issues/30800

---------

Signed-off-by: sunby <sunbingyi1992@gmail.com>
2024-03-07 14:49:02 +08:00
Bingyi Sun 7783098ddd
feat: support lazy load on querycoord (#30372)
https://github.com/milvus-io/milvus/issues/30361

Signed-off-by: sunby <sunbingyi1992@gmail.com>
2024-03-01 18:15:29 +08:00
aoiasd 6f184d354f
enhance: Refine describe collection error message when collection not found (#30766)
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-03-01 12:53:00 +08:00
cai.zhang 40ca98f57f
enhance: Skip timestamp allocation when search/query consistency level is eventually (#29773)
issue: #29772 
1. Skip timestamp allocation when search/query consistency level is
eventually.

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-02-21 09:52:59 +08:00
Bingyi Sun e22e8b30d4
enhance: check load state before altering collection (#30399)
/kind improvement

Signed-off-by: sunby <sunbingyi1992@gmail.com>
2024-02-01 18:03:04 +08:00
wayblink 1df3f90696
feat: Implement DescribeAlias and ListAliases interfaces (#29641)
#22882
/kind feature

Signed-off-by: wayblink <anyang.wang@zilliz.com>
2024-01-11 19:12:51 +08:00
Xu Tong e429965f32
Add float16 approve for multi-type part (#28427)
issue:https://github.com/milvus-io/milvus/issues/22837

Add bfloat16 vector, add the index part of float16 vector.

Signed-off-by: Writer-X <1256866856@qq.com>
2024-01-11 15:48:51 +08:00
aoiasd 73cfdab776
fix: Release collection delete proxy collection meta (#29854)
pr: https://github.com/milvus-io/milvus/issues/29675

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-01-10 21:54:49 +08:00
xige-16 9702cef2b5
feat: Support multiple vector search (#29433)
issue #25639 

Signed-off-by: xige-16 <xi.ge@zilliz.com>

Signed-off-by: xige-16 <xi.ge@zilliz.com>
2024-01-08 15:34:48 +08:00
wayblink 635a7f777c
feat: add clustering key in create/describe collection (#29506)
#28410
/kind feature

Signed-off-by: wayblink <anyang.wang@zilliz.com>
2024-01-07 19:56:48 +08:00
smellthemoon 1c1f2a1371
enhance:change some logs (#29579)
related #29588

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-01-05 16:12:48 +08:00
MrPresent-Han 9e2e7157e9
feat: support search_group_by for milvus(#25324) (#28983)
related: #25324

Search GroupBy function, used to aggregate result entities based on a
specific scalar column.
several points to mention:

1. Temporarliy, the whole groupby is implemented separated from
iterative expr framework **for the first period**
2. In the long term, the groupBy operation will be incorporated into the
iterative expr framework:https://github.com/milvus-io/milvus/pull/28166
3. This pr includes some unrelated mocked interface regarding alterIndex
due to some unworth-to-mention reasons. All these un-associated content
will be removed before the final pr is merged. This version of pr is
only for review
4. All other related details were commented in the files comparison

Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
2024-01-05 15:50:47 +08:00
congqixia 4f8c540c77
enhance: cache collection schema attributes to reduce proxy cpu (#29668)
See also #29113

The collection schema is crucial when performing search/query but some
of the information is calculated for every request.

This PR change schema field of cached collection info into a utility
`schemaInfo` type to store some stable result, say pk field,
partitionKeyEnabled, etc. And provided field name to id map for
search/query services.

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-01-04 17:28:46 +08:00
xige-16 0a70e8b601
enhance: Remove multiple vector field limit (#27827)
issue: https://github.com/milvus-io/milvus/issues/25639

/kind improvement
Signed-off-by: xige-16 <xi.ge@zilliz.com>

Signed-off-by: xige-16 <xi.ge@zilliz.com>
2023-12-28 16:40:46 +08:00
SimFG de13865769
enhance: Add load/release partitions to replicate msg stream (#28399)
/kind improvement
issue: #25655

Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-11-23 15:38:24 +08:00
SimFG 9b0ecbdca7
Support to replicate the mq message (#27240)
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-10-20 14:26:09 +08:00
yah01 be980fbc38
Refine state check (#27541)
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-10-11 21:01:35 +08:00
yah01 41495ed266
Improve the error message for getting all indexes of collection (#27389)
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-10-08 21:23:32 +08:00
yah01 8394b3a1ec
Block creating new error from status reason (#27426)
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-10-07 11:29:32 +08:00
cai.zhang 91244f0498
Return element type for describing collection (#27363)
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2023-09-28 14:55:26 +08:00
yah01 6539a5ae2c
Refine DataCoord status (#27262)
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-09-26 17:15:27 +08:00
jaime 7f7c71ea7d
Decoupling client and server API in types interface (#27186)
Co-authored-by:: aoiasd <zhicheng.yue@zilliz.com>

Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-09-26 09:57:25 +08:00
cai.zhang a362bb1457
Support array datatype (#26369)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2023-09-19 14:23:23 +08:00
MrPresent-Han 7939f0e7d5
enable ctx traceId for assignsegment on dc(#26972) (#27108) (#27030)
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
2023-09-18 11:39:20 +08:00
yah01 168e82ee10
Fix panic while handling with the nil status (#27040)
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-09-15 10:09:21 +08:00
yihao.dai dd2cb1d44a
Flush by flush channels (#26859)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-09-12 21:07:19 +08:00
yah01 00c65fa0d7
Refine QueryNode errors (#27013)
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-09-12 16:07:18 +08:00
Xu Tong 9166011c4a
Add float16 vector (#25852)
Signed-off-by: Writer-X <1256866856@qq.com>
2023-09-08 10:03:16 +08:00
yah01 3349db4aa7
Refine errors to remove changes breaking design (#26521)
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-09-04 09:57:09 +08:00
wei liu 0bb68cac36
fix CollectionNotExist on describe rg (#26541)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-08-23 09:56:21 +08:00
wei liu f9823e039f
fix describe rg with non exist collection (#26227)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-08-10 18:43:16 +08:00
chyezh f915f1739a
[Fixup] Short-term fix metacache data race (#25802)
Signed-off-by: chyezh <ye.zhen@zilliz.com>
2023-08-04 17:47:07 +08:00
smellthemoon 9614e61f14
Fix collection and channel not match (#25859)
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2023-08-01 17:33:06 +08:00
jaime 18df2ba6fd
[Cherry-Pick] Support Database (#24769)
Support Database(#23742)
Fix db nonexists error for FlushAll (#24222)
Fix check collection limits fails (#24235)
backward compatibility with empty DB name (#24317)
Fix GetFlushAllState with DB (#24347)
Remove db from global meta cache after drop database (#24474)
Fix db name is empty for describe collection response (#24603)
Add RBAC for Database API (#24653)
Fix miss load the same name collection during recover stage (#24941)

RBAC supports Database validation (#23609)
Fix to list grant with db return empty (#23922)
Optimize PrivilegeAll permission check (#23972)
Add the default db value for the rbac request (#24307)

Signed-off-by: jaime <yun.zhang@zilliz.com>
Co-authored-by: SimFG <bang.fu@zilliz.com>
Co-authored-by: longjiquan <jiquan.long@zilliz.com>
2023-06-25 17:20:43 +08:00
yihao.dai b62429070c
Set pchannels before dml enqueue to prevent panic (#24828)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-06-16 16:36:40 +08:00
congqixia 41af0a98fa
Use go-api/v2 for milvus-proto (#24770)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-06-09 01:28:37 +08:00
xige-16 732fe54775
Support partition Key (#24047)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2023-06-06 10:24:34 +08:00