e389909547
enhance: Bump milvus version to 2.3.10 ( #30776 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-02-23 13:48:53 +08:00
3a1338436a
enhance: Bump milvus version to v2.3.9 ( #30635 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-02-18 17:48:49 +08:00
be1bd9615a
enhance: add configurable memory index load predict memory usage factor ( #30563 )
...
pr: #30561
related pr: #30475
Signed-off-by: chyezh <chyezh@outlook.com>
2024-02-06 22:00:49 +08:00
7e7722ed43
enhance: [skip e2e] set logrus log level to reduce output error logs ( #30478 )
...
issue: #30295
Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-02-04 17:23:06 +08:00
cea5396c33
enhance: Bump milvus & milvus-proto version to v2.3.8 ( #30492 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-02-04 12:57:05 +08:00
3036c19867
fix: can't not get search_cache_budget_gb in create index ( #30353 )
...
issue:https://github.com/milvus-io/milvus/issues/30375
pr: https://github.com/milvus-io/milvus/pull/30119
Signed-off-by: cqy123456 <qianya.cheng@zilliz.com>
2024-01-31 15:49:03 +08:00
21c944beaa
enhance: add basic information of milvus into metrics ( #29666 )
...
add basic build information and runtime component dependency into
metrics.
issue: #29664
pr: #29665
Signed-off-by: chyezh <ye.zhen@zilliz.com>
2024-01-29 15:49:04 +08:00
77e123762f
enhance: add graceful stop timeout to avoid node stop hang under extreme cases ( #30320 )
...
1. add coordinator and proxy graceful stop timeout to 5s.
3. add other work node graceful stop timeout to 900s, and we should
potentially change this to 600s when graceful stop is smooth
4. change the order of datacoord component while stop.
5. `LivenessCheck` do not perform graceful shutdown now.
issue: https://github.com/milvus-io/milvus/issues/30310
pr: #30317
also see: https://github.com/milvus-io/milvus/pull/30306
---------
Signed-off-by: chyezh <chyezh@outlook.com>
2024-01-27 08:45:02 +08:00
650dcc512e
fix: dead lock while getting configs ( #30319 )
...
issue: https://github.com/milvus-io/milvus/issues/30295
pr: #30318
Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-01-26 20:15:01 +08:00
26df754514
enhance: Bump milvus version to 2.3.7 ( #30297 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-01-26 11:07:02 +08:00
e0f987ee9b
enhance: Allows proactive warming up of chunk cache ( #30182 ) ( #30289 )
...
Allows proactive warming up of chunk cache. Original vector data will be
asynchronously loaded into the chunk cache during the load process. It
has the potential to significantly reduce query/search latency for a
certain duration after the load, albeit with a concurrent increase in
disk usage.
issue: https://github.com/milvus-io/milvus/issues/30181
pr: https://github.com/milvus-io/milvus/pull/30182
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-01-26 09:57:01 +08:00
2c4d0605ef
enhance: add a weight for growing row count when balancing segments ( #30293 )
...
Cherry-pick from master
pr: #30271
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2024-01-26 09:17:03 +08:00
565d37ced8
enhance: Bump version to 2.3.6 ( #30184 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-01-22 19:04:55 +08:00
0e71923408
enhance: enable converting segcore error to merr ( #29914 ) ( #30178 )
...
this converts the segcore error to merr if possible
pr: #29914
Signed-off-by: yah01 <yang.cen@zilliz.com>
2024-01-22 16:56:55 +08:00
1cc5a613d5
enhance: adjust the GPU pool size ( #29937 ) ( #30177 )
...
according to benchmark, the GPU pool size with 6 performs best
pr: #29937
Signed-off-by: yah01 <yang.cen@zilliz.com>
2024-01-22 16:55:04 +08:00
6aaccdd5f4
feat: support general capacity restrict for cloud-side resoure contro… ( #30017 )
...
related: #29844
pr: #https://github.com/milvus-io/milvus/pull/29845
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
2024-01-22 16:18:56 +08:00
2465d86138
enhance: [2.3] support related privilege for grant api ( #30154 )
...
/kind improvement
pr: #30153
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-01-22 14:42:55 +08:00
b95f0cc0a1
enhance: Add a counter monitoring for the rate-limit requests ( #30109 ) ( #30132 )
...
Add a counter monitoring metric for the ratelimited rpc requests with
labels: proxy nodeID, rpc request type, and state.
issue: https://github.com/milvus-io/milvus/issues/30052
pr: https://github.com/milvus-io/milvus/pull/30109
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-01-21 14:44:59 +08:00
0700434c58
fix: patching search cache param when index meta does not hold one ( #30116 )
...
patch search cache param from index configs when index meta could not
get the search cache size key
issue: #30113
pr: #30119
Signed-off-by: xianliang <xianliang.li@zilliz.com>
2024-01-19 11:50:56 +08:00
be1470a654
enhance: [2.3] Add load/release partitions to replicate msg stream ( #30001 )
...
/kind improvement
pr: #28399
---------
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-01-18 22:50:55 +08:00
7f32576f36
enhance: [cherry-pick] replace magic number with ParamItem for dist handler ( #30020 ) ( #30070 )
...
Cherry-pick from master
pr: #30020
See also #28817
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-01-18 15:58:54 +08:00
0447ef5df3
fix: Unexpected rpc msg size limit ( #29682 ) ( #29983 )
...
pr: #29682
due to `clientMaxSendSize` and `serverMaxRecvSize` will limit the rpc
request size limit, they should use same config value, and
`serverMaxSendSize` and `clientMaxRecvSize` will limit the rpc response
size limit, they should use same config value too.
This PR fix unexpected rpc msg limit which caused by the wrong usage of
misunderstanding rpc config items
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-01-16 11:18:52 +08:00
962b3ea5fa
fix: [cherry-pick]Remove logging data when logging skip msg ( #29708 )
...
See also: #29696
pr: #29707
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-01-15 17:46:53 +08:00
a689ea3228
feat: Add RBAC functionality to alias ( #29885 ) ( #29947 )
...
issue: https://github.com/milvus-io/milvus/issues/29781
issue: https://github.com/milvus-io/milvus-proto/issues/237
pr : https://github.com/milvus-io/milvus/pull/29885
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2024-01-12 23:36:52 +08:00
fb956536b9
fix: remove checking if running inside container ( #29941 )
...
issue: https://github.com/milvus-io/milvus/issues/29846
pr: #29940
Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-01-12 21:12:52 +08:00
6d8146a09a
enhance: bump milvus & proto version to 2.3.5 ( #29946 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-01-12 20:54:51 +08:00
e1446da83c
feat: [Cherry-pick] Implement DescribeAlias and ListAliases interfaces ( #29896 )
...
#22882
pr: #29641
Signed-off-by: wayblink <anyang.wang@zilliz.com>
2024-01-12 16:30:51 +08:00
5520bfbb05
enhance: Change some frequency log to rated level ( #29720 ) ( #29903 )
...
pr: #29720
This PR change some frequency log to rated level
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-01-12 11:46:52 +08:00
c0b711e9fb
enhance: Support read hardware metrics for cgroupv2 ( #29847 )
...
issue: #29846
pr: #29850
Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-01-11 19:20:57 +08:00
cd93954214
enhance: [Cherry-pick] pre-allocate result FieldData space to reduce growslice ( #29726 ) ( #29866 )
...
Cherry-pick from master
pr: #29726
See also: #29113
Add a new utitliy function in `pkg/util/typetuil` to pre-allocate field
data slice capacity acoording to search limit. This shall avoid copying
the data during `AppendFieldData` when previous slice is out of space.
And shall also save CPU time during high paylog.
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-01-11 17:59:01 +08:00
322e9f39a3
fix: Remove Unnecessary lock in config manager ( #29855 )
...
issue: #29709 #291712
pr: #29836
to avoid concurrent recursive RLock and Lock cause deadlock, This PR
remove the unnecessary lock in config manager
---------
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-01-11 15:01:01 +08:00
38c61594c0
enhance: use GPU pool for gpu tasks ( #29678 ) ( #29706 )
...
- this much improve the performance for GPU index
- this also reduce 1x copy while parsing index meta
pr: #29678
Signed-off-by: yah01 <yang.cen@zilliz.com>
2024-01-09 14:04:48 +08:00
fcd23e2ec2
enhance:Add upsert vector metrics( #29226 ) ( #29674 )
...
pr: https://github.com/milvus-io/milvus/pull/29226
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-01-05 15:44:47 +08:00
a2365e4b2a
enhance: [2.3] Add concurrency for datacoord segment GC ( #29557 )
...
issue: #29553
pr: https://github.com/milvus-io/milvus/pull/29561
/kind improvement
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-01-03 13:16:57 +08:00
757834602a
enhance: add param for bloomfilter( #29388 ) ( #29614 )
...
related: https://github.com/milvus-io/milvus/issues/29388
pr: https://github.com/milvus-io/milvus/pull/29490
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
2024-01-02 18:58:47 +08:00
7a192da870
enhance: Bump version 2.3.4 & milvus-proto ( #29598 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-12-29 18:32:45 +08:00
151a5c3ca8
fix: iterator lose data for duplicted result( #29406 ) ( #29446 )
...
related: #29406
pr: #29451
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
2023-12-27 23:22:46 +08:00
26b1853c54
fix: Auto balance param can't be updated by dynamic( #29501 ) ( #29502 )
...
pr: #29501
This PR fixed that auto balance param can't be updated by dynamic
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-12-27 14:30:53 +08:00
f5bba79963
enhance: [Cherry-Pick] add default local path for access log ( #29461 )
...
relate: https://github.com/milvus-io/milvus/issues/29459
pr: https://github.com/milvus-io/milvus/pull/29460
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2023-12-26 17:52:47 +08:00
6eeb4b7f9a
enhance: [Cherry-Pick] Refine delete by expression for forbid proxy dml task scheduler hang ( #29359 )
...
relate: https://github.com/milvus-io/milvus/issues/29146
pr: https://github.com/milvus-io/milvus/pull/29340
---------
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2023-12-26 17:50:48 +08:00
95d1056e32
refine the range of chunk size config value( #29388 ) ( #29472 )
...
related: #https://github.com/milvus-io/milvus/issues/29388
pr: https://github.com/milvus-io/milvus/pull/29389
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
2023-12-26 14:34:46 +08:00
514da535e5
enhance: add metrics for stopping querynode balance progress ( #29201 ) ( #29390 )
...
pr: #29201
This PR add three metrics to track the stopping balance progress.
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-12-26 10:02:46 +08:00
477def9368
fix: [Cherry-Pick]Use uber atomic instead sync/atomic which only supported after go v1.20 ( #29379 )
...
relate: https://github.com/milvus-io/milvus/issues/29376
pr: https://github.com/milvus-io/milvus/pull/29377
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2023-12-26 10:00:47 +08:00
0c3de8d886
fix: Partition related privileges should belong to Collection ( #29429 )
...
issue: #29427
master pr: #29428
it will try to find privilege_name from the object_type, while grant a
privilege to a role
Signed-off-by: PowderLi <min.li@zilliz.com>
2023-12-25 14:18:43 +08:00
5fa63da4e3
fix: Rename invalid parameterutil package path ( #29334 ) ( #29391 )
...
pr: #29334
This PR renames the invalid parameterutil package path
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-12-22 10:06:41 +08:00
74e72ce27e
enhance: [2.3] Support to get the param value in the runtime ( #29298 )
...
pr: #29297
/kind improvement
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-12-21 20:36:43 +08:00
d8800f5ca5
fix: Return time tick delay error and refine quota error messages ( #29289 ) ( #29363 )
...
This pr:
Handles the time tick delay error when converting old error codes to
milvus errors.
Enhances quota error messages by eliminating "force deny" and
substituting it with "quota exceeded."
pr: https://github.com/milvus-io/milvus/pull/29289
issue: https://github.com/milvus-io/milvus/issues/29288
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-12-21 14:58:49 +08:00
9acf32a0b7
enhance: [cherry-pick] change cp metric to absolute unix ts ( #29328 ) ( #29337 )
...
Cherry pick from master
pr: #29328
See also #29327
Change channel checkpoint metrics to unix seconds instead of checkpoint
timestamp lag value
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-12-20 15:04:42 +08:00
8ef0c571b3
enhance: make consistency level used in delete configurable ( #29280 ) ( #29284 )
...
pr: #29280
issue: #29279
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2023-12-20 14:14:34 +08:00
af54ce9e20
fix: [Cherry-pick] msgstream memory leak caused by config event don't deregister ( #29268 )
...
pr: https://github.com/milvus-io/milvus/pull/29266
relate: https://github.com/milvus-io/milvus/issues/28620
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2023-12-19 12:34:39 +08:00
congqixia
chyezh
jaime
cqy123456
yihao.dai
Bingyi Sun
yah01
MrPresent-Han
SimFG
foxspy
wei liu
XuanYang-cn
zhenshan.cao
wayblink
smellthemoon
aoiasd
PowderLi
Jiquan Long