a2c4cd59ce
fix: drop partition can not be successful if load failed[2.5] ( #38874 )
...
fix https://github.com/milvus-io/milvus/issues/38649
pr: #38793
when partition load failed, the partition drop will also fail due to the
wrong error message
Signed-off-by: xiaofanluan <xiaofan.luan@zilliz.com>
2025-01-02 09:56:53 +08:00
11bedf5e76
fix: Revert "Expose metrics of stanby coordinators ( #27698 )" ( #38621 )
...
issue: #38608
pr: #38620
Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-12-20 18:04:47 +08:00
306e5e6898
enhance: clean compaction task in compactionHandler ( #38170 )
...
issue: #35711
---------
Signed-off-by: wayblink <anyang.wang@zilliz.com>
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
Co-authored-by: wayblink <anyang.wang@zilliz.com>
2024-12-19 12:36:47 +08:00
78438ef41e
fix: revert optimize CPU usage for CheckHealth requests ( #35589 ) ( #38555 )
...
issue: #35563
Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-12-19 00:38:45 +08:00
28fdbc4e30
enhance: optimize CPU usage for CheckHealth requests ( #35589 )
...
issue: #35563
1. Use an internal health checker to monitor the cluster's health state,
storing the latest state on the coordinator node. The CheckHealth
request retrieves the cluster's health from this latest state on the
proxy sides, which enhances cluster stability.
2. Each health check will assess all collections and channels, with
detailed failure messages temporarily saved in the latest state.
3. Use CheckHealth request instead of the heavy GetMetrics request on
the querynode and datanode
Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-12-17 11:02:45 +08:00
2afe2eaf3e
feat: support to replicate collection when the services contains the system tt msg ( #37559 )
...
- issue: #37105
---------
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-12-17 09:08:46 +08:00
a65d395ecd
fix: grant v2 collection name and privilege error typo ( #38195 )
...
issue: https://github.com/milvus-io/milvus/issues/38042
Signed-off-by: shaoting-huang <shaoting.huang@zilliz.com>
2024-12-04 16:14:38 +08:00
23dc313c44
fix: fix grant/revoke v2 meta and unclear error messages ( #38110 )
...
related issue: https://github.com/milvus-io/milvus/issues/37031
fixed issues:
#37974 : better error messages for grant v2 interface
#37903 : fix meta built-in privilege group object name
#37843 : better error messages for custom privilege group interface
#38002 : fix built-in privilege group meta to pass proxy interceptor
check
#38008 : fix revoke v2 to support revoking v1 granted privileges
Signed-off-by: shaoting-huang <shaoting.huang@zilliz.com>
2024-12-02 11:36:39 +08:00
b6612e02b4
enhance: Reduce GetIndexInfos calls ( #37695 )
...
Batch `GetIndexInfos` calls for segments to reduce RPC calls.
issue: https://github.com/milvus-io/milvus/issues/37634
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-11-19 14:24:31 +08:00
9a0e1c82bc
fix: repeated error code in milvus and segcore ( #37359 )
...
issue: #37357
Signed-off-by: chyezh <chyezh@outlook.com>
2024-11-05 16:28:23 +08:00
da227ff9a1
feat: Support create collection with functions ( #35973 )
...
relate: https://github.com/milvus-io/milvus/issues/35853
Support create collection with functions. Prepare for support bm25
function.
---------
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-09-12 10:43:06 +08:00
2c9bb4dfa3
feat: Support stats task to sort segment by PK ( #35054 )
...
issue: #33744
This PR includes the following changes:
1. Added a new task type to the task scheduler in datacoord: stats task,
which sorts segments by primary key.
2. Implemented segment sorting in indexnode.
3. Added a new field `FieldStatsLog` to SegmentInfo to store token index
information.
---------
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-09-02 14:19:03 +08:00
031ee6f155
enhance: support httpv1/v2 throttle and add it for httpV2( #35350 ) ( #35470 )
...
related: #35350
Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
2024-08-20 16:16:55 +08:00
c4a240acb1
enhance: memory value are displayed in MB ( #35292 )
...
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-08-09 14:14:25 +08:00
c79d1af390
enhance: Add compaction task slot usage logic ( #34581 )
...
#34544
Signed-off-by: wayblink <anyang.wang@zilliz.com>
2024-07-18 10:27:41 +08:00
60be454db0
enhance: add disk quota and max collections into db properties ( #34368 )
...
issue: #34385
Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-07-05 18:22:17 +08:00
186757e622
enhance: support mark error as user error ( #33498 )
...
relate: https://github.com/milvus-io/milvus/issues/33492
---------
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-07-01 14:56:12 +08:00
0426390f06
enhance: improve check health ( #33800 )
...
issue: #34264
Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-07-01 10:16:06 +08:00
73ffc1b424
Refine clustering_compaction_task retry mechanism ( #34194 )
...
#32939
Signed-off-by: wayblink <anyang.wang@zilliz.com>
2024-06-30 20:22:09 +08:00
f9a0f7bb25
Add an option to enable/disable vector field clustering key ( #34097 )
...
#30633
Signed-off-by: wayblink <anyang.wang@zilliz.com>
2024-06-25 18:52:04 +08:00
a1232fafda
feat: Major compaction ( #33620 )
...
#30633
Signed-off-by: wayblink <anyang.wang@zilliz.com>
Co-authored-by: MrPresent-Han <chun.han@zilliz.com>
2024-06-10 21:34:08 +08:00
f53ab54c5d
enhance: async cgo utility ( #33133 )
...
issue: #30926 , #33132
- implement future-based cgo utility.
---------
Signed-off-by: chyezh <chyezh@outlook.com>
2024-06-09 22:55:53 +08:00
ac4f3997ce
enhance: Reconstructing Compaction to possess persistence capability ( #33265 )
...
issue #33586
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2024-06-05 10:17:50 +08:00
fecd9c21ba
feat: LRU cache implementation ( #32567 )
...
issue: https://github.com/milvus-io/milvus/issues/32783
This pr is the implementation of lru cache on branch lru-dev.
Signed-off-by: sunby <sunbingyi1992@gmail.com>
Co-authored-by: chyezh <chyezh@outlook.com>
Co-authored-by: MrPresent-Han <chun.han@zilliz.com>
Co-authored-by: Ted Xu <ted.xu@zilliz.com>
Co-authored-by: jaime <yun.zhang@zilliz.com>
Co-authored-by: wayblink <anyang.wang@zilliz.com>
2024-05-06 20:29:30 +08:00
b287fbaa2e
fix: return collection on recovering but not collection not loaded when target is not recovered ( #32447 )
...
issue: #32398
Signed-off-by: chyezh <chyezh@outlook.com>
2024-04-25 11:21:26 +08:00
8594b55ad5
enhance: add `max insert request size` and `must use partition key` configs ( #32433 )
...
issue: https://github.com/milvus-io/milvus/issues/30577
/kind improvement
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-04-19 10:31:20 +08:00
48fe977a9d
enhance: declarative resource group api ( #31930 )
...
issue: #30647
- Add declarative resource group api
- Add config for resource group management
- Resource group recovery enhancement
---------
Signed-off-by: chyezh <chyezh@outlook.com>
2024-04-15 08:13:19 +08:00
1b767669a4
enhance: Throw error instead of crash when index cannot be built ( #31844 )
...
issue: #27589
---------
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-04-09 11:51:18 +08:00
92971707de
enhance: Add restful api for devops to execute rolling upgrade ( #29998 )
...
issue: #29261
This PR Add restful api for devops to execute rolling upgrade, including
suspend/resume balance and manual transfer segments/channels.
---------
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-03-27 16:15:19 +08:00
c4ace0f9d2
fix: Return specific error code when encountering incomplete requery results ( #31343 )
...
During requery, segments may change (e.g., due to compaction), so we
need to return specific error codes when encountering incomplete requery
results. Clients can then retry to avoid this issue.
issue: https://github.com/milvus-io/milvus/issues/29656
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-03-18 14:15:04 +08:00
9cfe183253
enhance: remove duplicated target node id check ( #31087 )
...
issue: #31109
This PR remove duplicate target node id check, due to server id has
already been checked in rpc's interceptor
---------
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-03-11 15:31:02 +08:00
229fc4f755
enhance: retry to read when the s3 get the unexpect eof error ( #30861 )
...
/kind improvement
issue: #30877
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-02-28 16:28:53 +08:00
c6665c2a4c
test: support multiple data/querynodes in integration test ( #30618 )
...
issue: https://github.com/milvus-io/milvus/issues/29507
Signed-off-by: yiwangdr <yiwangdr@gmail.com>
2024-02-21 11:54:53 +08:00
8f083e45cb
enhance: enable converting segcore error to merr ( #29914 )
...
this converts the segcore error to merr if possible
Signed-off-by: yah01 <yang.cen@zilliz.com>
2024-01-17 11:28:53 +08:00
2a0eb1d2e6
feat: support general capacity restrict for cloud-side resoure contro… ( #29845 )
...
related: #29844
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
2024-01-16 16:32:53 +08:00
1df3f90696
feat: Implement DescribeAlias and ListAliases interfaces ( #29641 )
...
#22882
/kind feature
Signed-off-by: wayblink <anyang.wang@zilliz.com>
2024-01-11 19:12:51 +08:00
635a7f777c
feat: add clustering key in create/describe collection ( #29506 )
...
#28410
/kind feature
Signed-off-by: wayblink <anyang.wang@zilliz.com>
2024-01-07 19:56:48 +08:00
a0e1a1eb31
feat: support enable/disable mmap for index ( #29005 )
...
support enable/disable mmap for index, the user could alter the index's
mode by `AlterIndex` method
related: https://github.com/milvus-io/milvus/issues/21866
---------
Signed-off-by: yah01 <yah2er0ne@outlook.com>
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-12-21 18:07:24 +08:00
f457b9f7c9
fix: Return time tick delay error and refine quota error messages ( #29289 )
...
This pr:
1. Handles the time tick delay error when converting old error codes to
milvus errors.
2. Enhances quota error messages by eliminating "force deny" and
substituting it with "quota exceeded."
issue: https://github.com/milvus-io/milvus/issues/29288
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-12-20 22:52:44 +08:00
1b7a503f89
enhance: Revert import support csv format ( #28760 )
...
Revert import support csv format.
issue: #28778
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2023-11-28 14:32:27 +08:00
c29b60e18e
enhance: Support Array DataType for bulk_insert ( #28341 )
...
issue: #28272
Support array DataType for bulk_insert with json, binlog files.
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2023-11-27 13:50:27 +08:00
bfccfcd0ca
enhance: refine error messages ( #28424 )
...
- Split the simple reason and full detail
- Refine existing error messages
related: #28422
---------
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-11-21 17:02:24 +08:00
bf2f62c1e7
Add `WriteBuffer` to provide abstraction for delta policy ( #27874 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-11-04 12:10:17 +08:00
9658367a3c
Refine chunk manager errors ( #27590 )
...
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-10-31 12:18:15 +08:00
9b0ecbdca7
Support to replicate the mq message ( #27240 )
...
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-10-20 14:26:09 +08:00
020ad9a6bc
Rectify wrong exception messages associated with Array datatype ( #27769 )
...
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2023-10-19 17:24:07 +08:00
49b3a12804
Return newly defined merr instead of grpc unimplemented err ( #27751 )
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-10-18 15:32:11 +08:00
e3f2122618
Expose metrics of stanby coordinators ( #27698 )
...
Signed-off-by: wayblink <anyang.wang@zilliz.com>
2023-10-16 15:04:09 +08:00
be980fbc38
Refine state check ( #27541 )
...
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-10-11 21:01:35 +08:00
2df9908c6a
Map old error code to new error ( #27616 )
...
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-10-11 12:43:33 +08:00
Xiaofan
jaime
cai.zhang
SimFG
sthuang
yihao.dai
Zhen Ye
aoiasd
Chun Han
smellthemoon
wayblink
zhenshan.cao
Bingyi Sun
wei liu
yiwangdr
yah01
yah01
congqixia