Commit Graph

48 Commits (eb046863485fdf3e130fc60484485c901b81276b)

Author SHA1 Message Date
congqixia cb7f2fa6fd
enhance: Use v2 package name for pkg module (#39990)
Related to #39095

https://go.dev/doc/modules/version-numbers

Update pkg version according to golang dep version convention

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2025-02-22 23:15:58 +08:00
Cai Yudong f32830e016
feat: Support restful & go sdk for Int8Vector (#39278)
Issue: #38666

Signed-off-by: Cai Yudong <yudong.cai@zilliz.com>
2025-01-24 17:07:07 +08:00
smellthemoon 513b489c85
enhance: add some apis in Restful (#39105)
- drop/alter database properties
- simplify the structure of search_params
- flush
- compact
- get_compact_status
issue: #38709

---------

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2025-01-20 18:15:22 +08:00
SimFG c22e457c59
feat: root privileges can be customized (#39191)
- issue: #39184

Signed-off-by: SimFG <bang.fu@zilliz.com>
2025-01-17 16:25:04 +08:00
Zhen Ye bb8d1ab3bf
enhance: make new go package to manage proto (#39114)
issue: #39095

---------

Signed-off-by: chyezh <chyezh@outlook.com>
2025-01-10 10:49:01 +08:00
smellthemoon 6b105837b4
enhance: add some apis in Restful (#38733)
add some apis in Restful #38709 

- alter/drop collection properties
- alter/drop index properties
- alter collection field properties
- refresh load

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-12-31 11:28:51 +08:00
smellthemoon e359725530
enhance: support db request in Restful api (#38140)
#38077

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-12-03 20:04:41 +08:00
smellthemoon db453c063d
enhance: no need database check in restful sdk (#38078)
#38077 
remove the check for two reason
1. server will do the same to make sure use the correct database;
2. each req has an additional overhead of calling the proxy to check
database.

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-12-02 17:04:38 +08:00
smellthemoon cb926688f7
enhance: support templates for expression in Restful api (#38040)
#36672

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-12-02 16:54:38 +08:00
Yinzuo Jiang 5a06faca39
feat: fp32 vector to fp16/bf16 vector conversion for RESTful API (#37556)
RESTful API. The influenced API are as follows:

- Handler. insert
- HandlerV1. insert/upsert
- HandlerV2. insert/upsert/search

We do not modify search API in Handler/HandlerV1 because they do not
support fp16/bf16 vectors.

module github.com/milvus-io/milvus/pkg:

Add `Float32ArrayToBFloat16Bytes()`, `Float32ArrayToFloat16Bytes()` and
`Float32ArrayToBytes()`. These method will be used in GoSDK in the
future.

issue: #37448

Signed-off-by: Yinzuo Jiang <yinzuo.jiang@zilliz.com>
Signed-off-by: Yinzuo Jiang <jiangyinzuo@foxmail.com>
2024-11-24 17:46:33 +08:00
sthuang bd0811434b
fix: fix grant v2 service (#37945)
issue: https://github.com/milvus-io/milvus/issues/37031

Signed-off-by: shaoting-huang <shaoting.huang@zilliz.com>
2024-11-22 19:42:32 +08:00
sthuang 19572f5b06
enhance: RBAC new grant/revoke privilege (#37785)
issue: https://github.com/milvus-io/milvus/issues/37031
also fix issues: https://github.com/milvus-io/milvus/issues/37843,
https://github.com/milvus-io/milvus/issues/37842,
https://github.com/milvus-io/milvus/issues/37887

Signed-off-by: shaoting-huang <shaoting.huang@zilliz.com>
2024-11-21 22:20:34 +08:00
yihao.dai b3fc53015d
fix: Avoid memory copy in RESTful search and query (#37674)
Custom `jsonRender` that encodes JSON data directly into the response
stream, it uses less memory since it does not buffer the entire JSON
structure before sending it, unlike `c.JSON` in `HTTPReturn`, which
serializes the JSON fully in memory before writing it to the response.
Benchmark testing shows that, using the custom render incurs no
performance loss and reduces memory consumption by nearly 50%.

issue: https://github.com/milvus-io/milvus/issues/37671

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-11-19 17:24:31 +08:00
congqixia b0bd290a6e
enhance: Use internal json(sonic) to replace std json lib (#37708)
Related to #35020

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-11-18 10:46:31 +08:00
smellthemoon a654487995
fix: some error in restful (#37487)
#37370

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-11-13 17:12:39 +08:00
sthuang ff00a12805
enhance: RBAC custom privilege group ut coverage (#37558)
issue: https://github.com/milvus-io/milvus/issues/37031

Signed-off-by: shaoting-huang <shaoting.huang@zilliz.com>
2024-11-09 20:40:25 +08:00
smellthemoon 86fd3200be
enhance: refactor createIndex in RESTful API (#37235)
Make the parameter input method consistent with miluvs-client.

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-11-07 14:18:30 +08:00
smellthemoon 84d48b498b
enhance: support upsert autoid==true in Restful API (#37072)
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-10-25 14:33:39 +08:00
jaime 4746f47282
feat: management WebUI homepage (#36822)
issue: #36784
1. Implement an embedded web server for WebUI access.  
2. Complete the homepage development.

Home page demo:
<img width="2177" alt="iShot_2024-10-10_17 57 34"
src="https://github.com/user-attachments/assets/38539917-ce09-4e54-a5b5-7f4f7eaac353">

Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-10-23 11:29:28 +08:00
smellthemoon 44d80c1355
fix: not return err if consistencyLevel is not set to a valid value (#36714)
https://github.com/milvus-io/milvus/issues/36444

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-10-16 13:23:22 +08:00
smellthemoon c9752bd2e6
enhance: refactor createCollection in RESTful API (#36790)
1.  support isClusteringKey in restful api;
2. throw err if passed invalid 'enableDynamicField' params
3. parameters in indexparams are not processed properly, related with
#36365

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-10-15 10:29:22 +08:00
Buqian Zheng 16b533cbf0
feat: Restful support for BM25 function (#36713)
issue: https://github.com/milvus-io/milvus/issues/35853

Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com>
2024-10-13 17:41:21 +08:00
smellthemoon 5e1c392243
enhance: support null and default value in restful api (#35825)
#31728

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-10-11 21:23:20 +08:00
yihao.dai 0751c508de
enhance: Support new RESTful URL to describe import job (#36753)
Support the new RESTful URL for retrieving/describing import progress:
`/v2/vectordb/jobs/import/describe`.
Deprecate the old URL: `/v2/vectordb/jobs/import/get_progress`.

issue: https://github.com/milvus-io/milvus/issues/36752

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-10-11 16:07:21 +08:00
smellthemoon 6e880d19a8
enhance: add search params in search request in restful (#36304)
https://github.com/milvus-io/milvus/issues/36321

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-09-23 16:51:12 +08:00
Yinzuo Jiang 407fc933e7
fix: bump bytedance/sonic to v1.12.2 to fix compilation error with go 1.23.0 (#35879)
fixes: #35878

Signed-off-by: Yinzuo Jiang <jiangyinzuo@foxmail.com>
2024-09-01 13:43:01 +08:00
Chun Han 031ee6f155
enhance: support httpv1/v2 throttle and add it for httpV2(#35350) (#35470)
related: #35350

Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
2024-08-20 16:16:55 +08:00
congqixia de8a266d8a
enhance: Enable linux code checker (#35084)
See also #34483

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-07-30 15:53:51 +08:00
Patrick Weizhi Xu 80c0ae3519
feat: [restful v2] add partition key isolation prop (#34710)
issue: https://github.com/milvus-io/milvus/issues/34332

Signed-off-by: Patrick Weizhi Xu <weizhi.xu@zilliz.com>
2024-07-22 11:29:59 +08:00
SimFG 203fb554a4
enhance: support to config root user's password (#34752)
- issue: #33058

Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-07-17 20:19:42 +08:00
aoiasd 5bb0d21e32
fix: Restful API use deprecate error code cause access log panic. (#34576)
relate: https://github.com/milvus-io/milvus/issues/34578

---------

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-07-12 10:13:35 +08:00
PowderLi c16a704934
fix: [restful v2] count(*) & hook (#34369)
issue:  #31224 #34374

for query api:
1. param filter is not requried
2. param limit is useless while outputFields = [count(*)]

add hook about grpc call

---------

Signed-off-by: PowderLi <min.li@zilliz.com>
2024-07-04 19:38:09 +08:00
PowderLi ee73e62202
fix: [restful v2] search result be cut while nq > 1 (#33339)
issue: #33099 #32837 #32419

1. len(search result) may be nq * topk, we need return all rather than
topk
2. the  in restful response payload keep the same with milvus error code

Signed-off-by: PowderLi <min.li@zilliz.com>
2024-05-27 14:25:41 +08:00
Buqian Zheng 1b67cecd65
enhance: add sparse float vector support to restful v2 (#33231)
issue: #29419
also re-enabled an e2e test using restful api, which is previously
disabled due to https://github.com/milvus-io/milvus/issues/32214.

In restful api, the accepted json formats of sparse float vector are:

* `{"indices": [1, 100, 1000], "values": [0.1, 0.2, 0.3]}`
* {"1": 0.1, "100": 0.2, "1000": 0.3}

for accepted indice and value range, see
https://milvus.io/docs/sparse_vector.md#FAQ

Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com>
2024-05-27 00:47:40 +08:00
PowderLi 29922829da
fix: [restful v2] parameters of create collection (#32485)
issue: #31176

Signed-off-by: PowderLi <min.li@zilliz.com>
2024-04-24 09:43:29 +08:00
PowderLi a0a23d0666
fix: [restful]use default search parameter `nq: 0` (#32355)
issue: #32225 #31978 #32360

1. v1 can only accept one vector, but v2 accept list of vectors #32225
2. cannot get dbName from AliasReq #31978
3. enhance: [restful v2]support config DB-Name in the http header #32360

---------

Signed-off-by: PowderLi <min.li@zilliz.com>
2024-04-18 14:41:20 +08:00
PowderLi 610a65af14
fix: restful v2 (#32144)
issue: #31176

1. cannot get dbName correctly while describe alias #31978
2. return a valid json string even if the user doesn't have the whole
privileges to describe collection #31635
3. rename IndexParam.IndexConfig to IndexParam.Params
4. FieldSchema.ElementTypeParams, IndexParam.Params can not only accept
string

Signed-off-by: PowderLi <min.li@zilliz.com>
2024-04-13 21:55:29 +08:00
PowderLi ee56ae7299
fix: [CHERRY-PICK] [restful v2] add hybrid_search (#31395)
search issue: #30688

2.4 pr: #31344

Signed-off-by: PowderLi <min.li@zilliz.com>
2024-03-21 09:59:08 +08:00
PowderLi ed0526a1a4
fix: [restful v2] create a collection with vector field name (#31317)
fix: [restful v2] create a collection with vector field name

1. quickly create collection, create an index on the vector field #31149
2. valid consistencyLevel is needed, while create a collection
3. show collection's properties and aliases #31180
4. list aliase for one collection, default is the whole database

Signed-off-by: PowderLi <min.li@zilliz.com>
2024-03-17 20:03:07 +08:00
PowderLi afa4193fc2
fix: [restful v2] create collection & describe collection (#31269)
1. complete the output of describe collection #31180
2. add a new parameter `params` for create collection #31181 #31149
3. create index according to indexConfig while create collection #31199
4. autoID: false while create a collection quickly, take upsert into
consideration
5. polish error information while search without annsField #31010


fix: [restful v1] insert / upsert with default partitionName JIRA-4952

---------

Signed-off-by: PowderLi <min.li@zilliz.com>
2024-03-14 23:27:03 +08:00
yihao.dai b5c67948b7
enhance: Enhance and modify the return content of ImportV2 (#31192)
1. The Import APIs now provide detailed progress information for each
imported file, including details such as file name, file size, progress,
and more.
2. The APIs now return the collection name and the completion time.
3. Other modifications include changing jobID to jobId and other similar
adjustments.

issue: https://github.com/milvus-io/milvus/issues/28521

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-03-13 19:51:03 +08:00
PowderLi 58d7b9f902
fix: [restful v2] search & advanced_search API (#31113)
issue: #30688
former pr:  #30946

1. param `vector` is required #31012
2. param `annsField` is optional, for multiply vector fields #31010
3. support BinaryVector, Float16Vector, BFloat16Vector #31013
4. replace vector with data, to align with pymilvus milvus_client #31093
5. create collection quickly, to align with pymilvus milvus_client
#31149

---------

Signed-off-by: PowderLi <min.li@zilliz.com>
2024-03-11 15:57:01 +08:00
yihao.dai 3b66c17279
feat: Revise the RESTful bulk insert API (#29698)
Revise the RESTful bulk insert API from version 1 to version 2.

issue: https://github.com/milvus-io/milvus/issues/28521

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-03-05 15:03:00 +08:00
PowderLi 4869aaeb94
fix: [restful v2] align search api with milvus client (#30946)
issue: #30688

fix: [restful v2] align search api with milvus client #30688
fix: [restful v2] DescribeIndex return Response{ Status: IndexNotFound
}, nil #30722

Signed-off-by: PowderLi <min.li@zilliz.com>
2024-03-01 11:01:01 +08:00
PowderLi 19911a411f
fix: [restful v2] bug list (#30740)
bug list
1. element data type is needed while create a collection with an array
field #30638
2. spelling mistake about metricsType #30643
3. new collection enable dynamic field / auto id as user defined #30665
4. convert rowCount from string to int #30661
5. try it's best to create a new collection #30652
6. int64 percious #20415
7. insert into collection which has multi vector fields #30674
8. cannot rename a collection to other database #30700
9. update the request body of "indexes/create" #30769
10. got [] while list indexes of a collection which has no index #30722
11. restful need encode password before call
CreateCredential/UpdateCredential #30730
12. some parameters missed the required label #30737
13.define the field to be or not to be a partition key while create a
collection #30797

enhance: support opentelemetry
enhance: support dataType: Float16Vector & BFloat16Vector #22837
enhance: describe collection will show the field is partition key or not
#30789

Signed-off-by: PowderLi <min.li@zilliz.com>
2024-02-27 16:48:56 +08:00
PowderLi 1b87bcd60e
feat: restful support new features (#30485)
feat: restful support new features

1. search with groupingField #25324
2. hybrid search #25639

Signed-off-by: PowderLi <min.li@zilliz.com>
2024-02-07 17:16:47 +08:00
PowderLi 5cf9bb236e
enhance: restful support import jobs (#30343)
issue: #28521 #29732

include
1. list collection's import jobs
2. create a new import job
3. get the progress of an import job

fix:
1. mix the order of dbName & collectionName #29728
2. trace log keep the same as v1
3. support traceID
4. azure precheck, blob name cannot end with / #29703

---------

Signed-off-by: PowderLi <min.li@zilliz.com>
2024-01-31 17:57:04 +08:00
PowderLi 6abbab12fa
feat: restful phase two (#29728)
issue: #29732

Signed-off-by: PowderLi <min.li@zilliz.com>
2024-01-28 16:03:01 +08:00