Commit Graph

1352 Commits (243e311515e06d50e7e37536cc9df061e9361f56)

Author SHA1 Message Date
yihao.dai c4ace0f9d2
fix: Return specific error code when encountering incomplete requery results (#31343)
During requery, segments may change (e.g., due to compaction), so we
need to return specific error codes when encountering incomplete requery
results. Clients can then retry to avoid this issue.

issue: https://github.com/milvus-io/milvus/issues/29656

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-03-18 14:15:04 +08:00
zhagnlu e396ad9580
fix: fix wrong guaranteeTs for hybrid search (#31302)
#31281

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-03-17 20:21:04 +08:00
wei liu 6f39e35721
fix: Unstable ut TestGetClientFailed (#31296)
issue: #31295

This PR fix unstable ut TestGetClientFailed

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-03-15 17:41:04 +08:00
wei liu ca8eee2c47
fix: Set node unreachable when get shard client failed (#31277)
issue: #30531

cause get client from `shardClientMgr`, doesn't means query node is
unavailable. because of the ref counter policy in `shardClientMgr`,
which will clean the client, if no collection use qn as shard leader.

This PR fix that set node unreachable when get shard client failed.

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-03-15 10:23:03 +08:00
smellthemoon 056e9f0cf5
fix: unmarshal nil when check search params (#31139)
#31020

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-03-15 09:15:04 +08:00
Buqian Zheng 3c80083f51
feat: [Sparse Float Vector] add sparse vector support to milvus components (#30630)
add sparse float vector support to different milvus components,
including proxy, data node to receive and write sparse float vectors to
binlog, query node to handle search requests, index node to build index
for sparse float column, etc.

https://github.com/milvus-io/milvus/issues/29419

---------

Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com>
2024-03-13 14:32:54 -07:00
yihao.dai b5c67948b7
enhance: Enhance and modify the return content of ImportV2 (#31192)
1. The Import APIs now provide detailed progress information for each
imported file, including details such as file name, file size, progress,
and more.
2. The APIs now return the collection name and the completion time.
3. Other modifications include changing jobID to jobId and other similar
adjustments.

issue: https://github.com/milvus-io/milvus/issues/28521

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-03-13 19:51:03 +08:00
yihao.dai 87b3c25b15
fix: Fix binlog import (#31205)
1. File type validation is omitted during binlog import.
2. System fields are appended to the schema during binlog import.

issue: https://github.com/milvus-io/milvus/issues/28521

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-03-13 10:35:04 +08:00
cai.zhang de2c95d00c
enhance: Constraint dynamic field as key-value format (#31183)
issue: #31051

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-03-12 12:45:03 +08:00
cai.zhang 6a83f16871
feat: Support for multiple forms of JSON (#31052)
issue: #31051

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-03-11 19:55:02 +08:00
yihao.dai 8cb06acfed
feat: Replacing the current import API with the v2 implementation (#31046)
Replacing the current import API v1 implementation with the v2
implementation.

issue: https://github.com/milvus-io/milvus/issues/28521

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-03-10 12:23:02 +08:00
Chun Han 3574bdf858
enhance: ban range-search iteration for search-group-by (#30824)
related: #30033

Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
2024-03-08 14:17:00 +08:00
Bingyi Sun eb0cfe3e84
fix: set CollectionID field for AlterCollection request (#30814)
issue: https://github.com/milvus-io/milvus/issues/30800

---------

Signed-off-by: sunby <sunbingyi1992@gmail.com>
2024-03-07 14:49:02 +08:00
congqixia 007fab183c
enhance: Skip partition key name check if feature not enabled (#31067)
Skip partition key naming & hash value pre process if collection schema
does not have partition key
The PR removes mislead warning when collection has no partition key

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-03-06 19:33:00 +08:00
MrPresent-Han d0eeea4b44
fix: reduce incorrectly for group-by with offset(#30828) (#30882)
related: #30828

Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
2024-03-06 16:47:00 +08:00
Jiquan Long a88c896733
enhance: purge client infos periodically (#31037)
https://github.com/milvus-io/milvus/issues/31007

---------

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2024-03-06 12:50:59 +08:00
Xiaofan 4bda6c33ad
fix: binary vector should not limit dimension to 32768 (#30676)
all the vector dimension check should happen on collection creation but
not index build
fix #30285

Signed-off-by: xiaofanluan <xiaofan.luan@zilliz.com>
2024-03-05 14:21:00 +08:00
aoiasd 0dacf0172e
enhance: access log support print output fields of Query or Search (#30768)
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-03-05 11:54:58 +08:00
congqixia 3b5ce73ded
enhance: Change proxy connection manager to concurrent safe (#31008)
See also #31007

This PR:
- Add param item for connection manager behavior: TTL & check interval
- Change clientInfo map to concurrent map

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-03-05 10:39:00 +08:00
congqixia 9b3005f1be
enhance: Avoid create schema helper for each read task (#30981)
See also #30806

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-03-04 19:39:00 +08:00
yihao.dai a434d33e75
feat: Add import scheduler and manager (#29367)
This PR introduces novel managerial roles for importv2:
1. ImportMeta: To manage all the import tasks;
2. ImportScheduler: To process tasks and modify their states;
3. ImportChecker: To ascertain the completion of all tasks and instigate
relevant operations.

issue: https://github.com/milvus-io/milvus/issues/28521

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-03-01 18:31:02 +08:00
Bingyi Sun 7783098ddd
feat: support lazy load on querycoord (#30372)
https://github.com/milvus-io/milvus/issues/30361

Signed-off-by: sunby <sunbingyi1992@gmail.com>
2024-03-01 18:15:29 +08:00
zhagnlu 8b25ecffff
fix: add pk sort for same distance of hybridsearch (#30901)
#29923

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-03-01 18:09:02 +08:00
aoiasd 6f184d354f
enhance: Refine describe collection error message when collection not found (#30766)
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-03-01 12:53:00 +08:00
aoiasd f3d1c75499
enhance: optimize access log formatter and add bench test (#30829)
relate: https://github.com/milvus-io/milvus/issues/28948
https://github.com/milvus-io/milvus/issues/30806

---------

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-03-01 10:41:00 +08:00
chyezh 0c7474d7e8
enhance: add graceful stop timeout to avoid node stop hang under extreme cases (#30317)
1. add coordinator graceful stop timeout to 5s
2. change the order of datacoord component while stop
3. change querynode grace stop timeout to 900s, and we should
potentially change this to 600s when graceful stop is smooth

issue: #30310
also see pr: #30306

---------

Signed-off-by: chyezh <chyezh@outlook.com>
2024-02-29 17:01:50 +08:00
chyezh e59bc1e526
enhance: use latest milvus proto, add unimplemented server (#30889)
issue: #30647

Signed-off-by: chyezh <chyezh@outlook.com>
2024-02-29 10:13:00 +08:00
aoiasd 81a2d9ced6
enhance: access log support get sdk type by user agent (#30760)
Support get sdk type by user agent when we can't get sdk version by
connection in access log.

---------

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-02-28 18:47:00 +08:00
smellthemoon a4f3e01a3a
fix: add the range search params check in proxy (#30423)
if check in Segcore, will not do the it when not insert data.
so, check "radius" and "range_filter" in proxy.
related with #30365

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-02-28 11:24:58 +08:00
chyezh 77477d6340
fix: wrong context passing into NewClient, error handling lost in session_util (#30817)
issue: #30799

Signed-off-by: chyezh <chyezh@outlook.com>
2024-02-28 10:40:09 +08:00
zhagnlu a0531b72aa
fix: reorder hybridsearch result for L2 metric (#30739)
#30694

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-02-26 14:18:55 +08:00
Xiaofan d970a9fc6c
enhance: refine error log of json parse error (#30671)
add more error info when json format does not match.
relate to https://github.com/milvus-io/milvus/issues/29837

Signed-off-by: xiaofanluan <xiaofan.luan@zilliz.com>
2024-02-24 12:32:14 +08:00
zhagnlu c5363c70db
fix: fix upsert using wrong field to compute partition key (#30772)
#30607

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-02-23 09:54:53 +08:00
aoiasd e5353ad049
enhance: access log should get get client info by get method (#30502)
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-02-22 14:58:52 +08:00
MrPresent-Han 77eb6defb1
feat: support groupby on growing and non-indexed sealed egment(#30307) (#30644)
related: #30308

Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
2024-02-21 14:02:53 +08:00
smellthemoon 81d6cb1a0c
enhance: The error information is not clear (#30658)
issue:#30516

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-02-21 10:16:53 +08:00
cai.zhang 40ca98f57f
enhance: Skip timestamp allocation when search/query consistency level is eventually (#29773)
issue: #29772 
1. Skip timestamp allocation when search/query consistency level is
eventually.

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-02-21 09:52:59 +08:00
aoiasd c863b82476
enhance: Return parse expression failed error with reason (#30548)
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-02-20 16:02:52 +08:00
aoiasd bbff9193d9
enhance: support clean paramtable config event in test (#30534)
relate: https://github.com/milvus-io/milvus/issues/30441

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-02-20 14:16:51 +08:00
chyezh 941dc755df
feat: add collection level flush rate control (#29567)
flush rate control at collection level to avoid generate too much
segment.
0.1 qps by default.

issue: #29477

Signed-off-by: chyezh <ye.zhen@zilliz.com>
2024-02-18 15:32:50 +08:00
zhenshan.cao bb93b22c84
fix: should return collectionName in response of ListAliases (#30532)
issue : https://github.com/milvus-io/milvus/issues/30369

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2024-02-12 08:30:55 +08:00
aoiasd 23aa136444
enhance: access log method support get status when method only return error instead of status (#30409)
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-02-04 16:35:05 +08:00
XuanYang-cn 6959630652
fix: donot set l0 segment as growing when savebinlogs (#29194)
This PR fixes negative growing L0 segments in Metrics

See also: #29204, #30441

---------

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-02-04 10:21:06 +08:00
xige-16 05ebe7b6ae
fix: fix hybrid search panic when has multi shard (#30464)
/kind bug
issue:#30457
Signed-off-by: xige-16 <xi.ge@zilliz.com>

Signed-off-by: xige-16 <xi.ge@zilliz.com>
2024-02-02 17:25:12 +08:00
SimFG 02de84239d
fix: add more requests to the database interceptor (#30452)
issue: #30368

Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-02-02 16:49:06 +08:00
xige-16 0a78b38bb8
fix: fix requery without partitionIDs in hybrid search (#30444)
issue: #30412 
Signed-off-by: xige-16 <xi.ge@zilliz.com>

Signed-off-by: xige-16 <xi.ge@zilliz.com>
2024-02-02 16:47:13 +08:00
Bingyi Sun dcdf85977c
enhance: [skip e2e] add alter collection task ut (#30447)
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2024-02-02 15:25:04 +08:00
congqixia 1c1dd48b71
fix: Deprecated shard leader cache while deprecating collection (#30446)
Resolves: #30445

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-02-02 09:11:05 +08:00
Bingyi Sun e22e8b30d4
enhance: check load state before altering collection (#30399)
/kind improvement

Signed-off-by: sunby <sunbingyi1992@gmail.com>
2024-02-01 18:03:04 +08:00
xige-16 060c8603a3
fix: Support mvcc with hybrid serach (#30114)
issue: https://github.com/milvus-io/milvus/issues/29656
/kind bug

Signed-off-by: xige-16 <xi.ge@zilliz.com>

---------

Signed-off-by: xige-16 <xi.ge@zilliz.com>
2024-02-01 16:03:03 +08:00