milvus/pkg/mocks/streaming
Zhen Ye 446f06eb02
enhance: Implement rate limiting in WAL append operations (#47179)
issue: #47178

This commit introduces a rate limiting mechanism for Write-Ahead Logging
(WAL) operations to prevent overload during high traffic. Key changes
include:

- Added `RateLimitObserver` to monitor and control the rate of DML
operations.
- Add Adaptive RateLimitController to apply the strategy of rate limit.
- WAL will slow down if the recovery-storage works on catchup mode or
node memory is high.
- Updated `WAL` and related components to handle rate limit states,
including rejection and slowdown.
- Introduced new error codes for rate limit rejection in the streaming
error handling.
- Enhanced tests to cover the new rate limiting functionality.

These changes aim to improve the stability and performance of the
streaming service under load.

---------

Signed-off-by: chyezh <chyezh@outlook.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-18 16:19:26 +08:00
..
mock_walimpls
util enhance: Implement rate limiting in WAL append operations (#47179) 2026-03-18 16:19:26 +08:00