b7635ed989
enhance: [Cherry-pick] Change proxy connection manager to concurrent safe ( #31009 )
...
Cherry-pick from master
pr: #31008
See also #31007
This PR:
- Add param item for connection manager behavior: TTL & check interval
- Change clientInfo map to concurrent map
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-03-05 14:13:00 +08:00
5b695d7e86
fix: Clean kafka default configuration ( #30925 )
...
issue: #30917
pr: #30924
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2024-03-01 18:15:29 +08:00
483a32bced
feat: add collection level flush rate control ( #29568 )
...
flush rate control at collection level to avoid generate too much
segment.
0.1 qps by default.
issue: #29477
pr: #29567
Signed-off-by: chyezh <ye.zhen@zilliz.com>
2024-03-01 10:23:01 +08:00
2009c3c783
fix: Support TLS for kafka connection ( #30466 )
...
issue: https://github.com/milvus-io/milvus/discussions/27977
pr: #30468
Add extra configurations in milvus.yaml to pass certificates for kafka.
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2024-02-28 18:43:07 +08:00
e0f987ee9b
enhance: Allows proactive warming up of chunk cache ( #30182 ) ( #30289 )
...
Allows proactive warming up of chunk cache. Original vector data will be
asynchronously loaded into the chunk cache during the load process. It
has the potential to significantly reduce query/search latency for a
certain duration after the load, albeit with a concurrent increase in
disk usage.
issue: https://github.com/milvus-io/milvus/issues/30181
pr: https://github.com/milvus-io/milvus/pull/30182
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-01-26 09:57:01 +08:00
2c4d0605ef
enhance: add a weight for growing row count when balancing segments ( #30293 )
...
Cherry-pick from master
pr: #30271
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2024-01-26 09:17:03 +08:00
6aaccdd5f4
feat: support general capacity restrict for cloud-side resoure contro… ( #30017 )
...
related: #29844
pr: #https://github.com/milvus-io/milvus/pull/29845
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
2024-01-22 16:18:56 +08:00
0447ef5df3
fix: Unexpected rpc msg size limit ( #29682 ) ( #29983 )
...
pr: #29682
due to `clientMaxSendSize` and `serverMaxRecvSize` will limit the rpc
request size limit, they should use same config value, and
`serverMaxSendSize` and `clientMaxRecvSize` will limit the rpc response
size limit, they should use same config value too.
This PR fix unexpected rpc msg limit which caused by the wrong usage of
misunderstanding rpc config items
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-01-16 11:18:52 +08:00
757834602a
enhance: add param for bloomfilter( #29388 ) ( #29614 )
...
related: https://github.com/milvus-io/milvus/issues/29388
pr: https://github.com/milvus-io/milvus/pull/29490
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
2024-01-02 18:58:47 +08:00
26b1853c54
fix: Auto balance param can't be updated by dynamic( #29501 ) ( #29502 )
...
pr: #29501
This PR fixed that auto balance param can't be updated by dynamic
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-12-27 14:30:53 +08:00
95d1056e32
refine the range of chunk size config value( #29388 ) ( #29472 )
...
related: #https://github.com/milvus-io/milvus/issues/29388
pr: https://github.com/milvus-io/milvus/pull/29389
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
2023-12-26 14:34:46 +08:00
8ef0c571b3
enhance: make consistency level used in delete configurable ( #29280 ) ( #29284 )
...
pr: #29280
issue: #29279
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2023-12-20 14:14:34 +08:00
1e2a0dd127
enhance: Support otlp with insecure ( #29131 )
...
pr: https://github.com/milvus-io/milvus/pull/29115
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-12-12 16:44:37 +08:00
e49860cb80
feat: Introduce channelCheckpointUpdater to reduce goroutine in ttNode ( #29107 )
...
pr: #28570
Signed-off-by: wayblink <anyang.wang@zilliz.com>
2023-12-12 13:48:42 +08:00
ac0f000c29
enhance: Add proxy.ginLogSkipPaths ( #29008 )
...
pr: #28945
/cc @czs007
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
2023-12-08 17:55:30 +08:00
ecbb6160aa
enhance: storeageType default value change to remote ( #29009 )
...
pr: #28792
/cc @czs007
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
2023-12-08 17:54:33 +08:00
bba0e159ba
feat: [pick] Add bypassing logic for ttMsg in flowgraph of DataNode( #28756 ) ( #29036 )
...
In order to minimize the CPU usage of the coroutine and avoid frequent
execution of time-consuming operations in the flowgraph when the message
stream consists solely of "ttMsg," it is recommended to implement a
mechanism for quickly bypassing the subsequent flowgraph node processing
logic.
If "ttMsg" is continuously received for a certain period of time
(coldTime), the flowgraph enters skipMode. Once in skipMode, every
skipNum "ttMsg" messages are merged into one for processing. If a
non-"ttMsg" message is received while in skipMode, the flowgraph exits
skipMode.
pr: #28756
Signed-off-by: wayblink <anyang.wang@zilliz.com>
Co-authored-by: wayblink <anyang.wang@zilliz.com>
2023-12-08 12:10:39 +08:00
b639fd66b4
fix: [Cherry-pick] accesslog can not print search expression ( #28931 )
...
relate: https://github.com/milvus-io/milvus/issues/28893
pr: https://github.com/milvus-io/milvus/pull/28899
---------
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2023-12-08 02:00:36 +08:00
cb7b1d1d51
enhance: [Cherry-pick] pack proxy connection code and support accesslog print SDK_Version ( #28844 )
...
relate: https://github.com/milvus-io/milvus/issues/28086
pr: https://github.com/milvus-io/milvus/pull/28835
---------
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2023-12-07 12:06:34 +08:00
8fd38c8eea
enhance:[cherry-pick] Use binlog index for better search performance ( #29012 )
...
this pr is cherry-pick from master:
pr: https://github.com/milvus-io/milvus/pull/28528
pr: https://github.com/milvus-io/milvus/pull/27673
related issue:
issue: https://github.com/milvus-io/milvus/issues/27678
Signed-off-by: cqy123456 <qianya.cheng@zilliz.com>
2023-12-07 09:52:34 +08:00
224f2755c5
enhance: [Cherry-pick]Refine accesslog to support format by yaml and support more metric ( #28740 )
...
relate: https://github.com/milvus-io/milvus/issues/28086
pr: https://github.com/milvus-io/milvus/issues/28086
---------
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2023-11-28 15:34:28 +08:00
918333817e
Disable auto balance when old node exists ( #28191 ) ( #28224 )
...
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-11-08 07:10:17 +08:00
bbcaf7a703
Fix coordinator fast restart ( #28205 )
...
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2023-11-07 10:04:16 +08:00
218a988941
Increase the ChunkManager request timeout ( #28015 ) ( #28154 )
...
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-11-04 10:26:17 +08:00
53246b1b38
Set accesslog default to close and use stdout ( #27891 )
...
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2023-10-25 10:30:10 +08:00
40723a292e
reduce compact parallel task num ( #27899 )
...
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-10-25 09:40:12 +08:00
4640928280
Fix initialization and backward compatibility issue for GRPC compression ( #27894 )
...
Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-10-24 22:14:14 +08:00
351c64b606
fix some typos ( #27851 )
...
1. fix some typos in md,yaml #22893
Signed-off-by: Sheldon <chuanfeng.liu@zilliz.com>
2023-10-24 09:30:10 +08:00
2ea7579dbb
Reduce rpc size for GetRecoveryInfoV2 ( #27483 )
...
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2023-10-23 21:44:09 +08:00
6060dd7ea8
Add chunk manager request timeout ( #27692 )
...
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2023-10-23 20:08:08 +08:00
9b0ecbdca7
Support to replicate the mq message ( #27240 )
...
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-10-20 14:26:09 +08:00
4b0ec156b3
Set channel work pool size in datanode ( #27728 )
...
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2023-10-19 08:28:08 +08:00
106c17f304
Make read ahead policy in ChunkCache configurable ( #27291 )
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-09-28 15:47:27 +08:00
5db4a0489e
dynamic index version control ( #27335 )
...
Co-authored-by: longjiquan <jiquan.long@zilliz.com>
2023-09-25 21:39:27 +08:00
370b6fde58
milvus support multi index engine ( #27178 )
...
Co-authored-by: longjiquan <jiquan.long@zilliz.com>
2023-09-22 09:59:26 +08:00
fe01d54eca
Set kafka read timeout to 10s and make it configurable ( #27238 )
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-09-20 22:01:27 +08:00
d1f6825b86
Specify componet ip ( #27161 )
...
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2023-09-20 11:49:25 +08:00
791e6ef6c6
[Improvement] add pulsar metrics and fix timeout ( #26907 )
...
Signed-off-by: chyezh <ye.zhen@zilliz.com>
2023-09-13 12:03:19 +08:00
337edc321b
tikv integration ( #26246 )
...
Signed-off-by: yiwangdr <yiwangdr@gmail.com>
2023-09-07 07:25:14 +08:00
28681276e2
Improve the retry of the rpc client ( #26795 )
...
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-09-06 17:43:14 +08:00
1097776477
stop heartbeat if reach heartbeat limit ( #26728 )
...
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-09-04 17:51:48 +08:00
b2e7cbdf4b
Remove TimeTravel in compactor ( #26785 )
...
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2023-09-04 17:41:48 +08:00
7056e9c0f7
Increase minio log level to avoid unnecessary log ( #26776 )
...
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2023-09-04 17:25:48 +08:00
0530fd80c9
[Fixup] remove nats from default ( #26791 )
...
Signed-off-by: chyezh <ye.zhen@zilliz.com>
2023-09-04 10:01:04 +08:00
7d5a4b2994
add more event for segcore search( #26277 ) ( #26688 )
...
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
2023-08-30 14:15:01 +08:00
c603f1c244
Remove mysql metastore ( #26633 )
...
Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-08-29 14:36:26 +08:00
ee7aef9272
Make pulsar request timeout configurable ( #26525 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-08-23 20:46:23 +08:00
d13ca54414
Change the default gracefulStopTimeout and only warn when fail to refresh policy cache ( #26450 )
...
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-08-22 09:04:21 +08:00
c5a1b41f95
Update session ttl to 60s ( #26346 )
...
Signed-off-by: wayblink <anyang.wang@zilliz.com>
2023-08-18 22:52:20 +08:00
79ccf06cf6
refine proxy to querynode heartbeat interval ( #26426 )
...
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-08-18 14:26:19 +08:00
congqixia
groot
chyezh
yihao.dai
Bingyi Sun
MrPresent-Han
wei liu
Jiquan Long
Enwei Jiao
wayblink
shaoyue
zhenshan.cao
aoiasd
cqy123456
Xiaofan
yah01
jaime
Sheldon
zhagnlu
SimFG
smellthemoon
foxspy
chyezh
yiwangdr
XuanYang-cn