chore(influxdb3): Data durability: replace diagram, cleanup brand terms.

pull/5881/head
Jason Stirnaman 2025-03-20 13:33:27 -05:00
parent 4dbdfba22c
commit 072cada97e
2 changed files with 9 additions and 10 deletions

View File

@ -2,13 +2,12 @@
When data is written to {{% product-name %}}, it progresses through multiple stages to ensure durability, optimize performance, and enable efficient querying. Configuration options at each stage affect system behavior, balancing reliability and resource usage.
## Data flow
## Data flow for writes
As data moves through {{% product-name %}}, it follows a structured path to ensure durability, efficient querying, and optimized storage.
As written data moves through {{% product-name %}}, it follows a structured path to ensure durability, efficient querying, and optimized storage.
The figure below shows how written data flows through the database.
{{< img-hd src="/img/influxdb/influxdb-3-write-path.png" alt="Write Path for InfluxDB 3 Core & Enterprise" />}}
{{< img-hd src="/img/influxdb3/influxdb3-core-enterprise-ingest-path-flow.png" alt="Ingest path and data flow for InfluxDB 3 Core & Enterprise" />}}
<span class="caption">Figure: Write request, response, and ingest flow for InfluxDB 3 Core and Enterprise</span>
1. [Write validation and memory buffer](#1-write-validation-and-memory-buffer)
2. [Write-ahead log (WAL) persistence](#2-write-ahead-log-wal-persistence)
@ -20,14 +19,14 @@ The figure below shows how written data flows through the database.
- **Process**: InfluxDB validates incoming data before accepting it into the system.
- **Impact**: Prevents malformed or unsupported data from entering the database.
- **Details**: The system validates incoming data and stores it in the write buffer (in memory). If [`no_sync=true`](#no-sync-write-option), the server sends a response to acknowledge the write.
- **Details**: The database validates incoming data and stores it in the write buffer (in memory). If [`no_sync=true`](#no-sync-write-option), the server sends a response to acknowledge the write.
### Write-ahead log (WAL) persistence
- **Process**: The system flushes the write buffer to the WAL every second (default).
- **Process**: The database flushes the write buffer to the WAL every second (default).
- **Impact**: Ensures durability by persisting data to object storage.
- **Tradeoff**: More frequent flushing improves durability but increases I/O overhead.
- **Details**: Every second (default), the system flushes the write buffer to the Write-Ahead Log (WAL) for persistence in the Object store. If [`no_sync=false`](#no-sync-write-option) (default), the server sends a response to acknowledge the write.
- **Details**: Every second (default), the database flushes the write buffer to the Write-Ahead Log (WAL) for persistence in the Object store. If [`no_sync=false`](#no-sync-write-option) (default), the server sends a response to acknowledge the write.
### Query availability
@ -41,10 +40,10 @@ The figure below shows how written data flows through the database.
- **Process**: Every ten minutes (default), data is persisted to Parquet files in object storage.
- **Impact**: Provides durable, long-term storage.
- **Tradeoff**: More frequent persistence reduces reliance on the WAL but increases I/O costs.
- **Details**: Every ten minutes (default), the system persists the oldest data from the queryable buffer to the Object store in Parquet format. InfluxDB keeps the remaining data (the most recent 5 minutes) in memory.
- **Details**: Every ten minutes (default), the {{% product-name %}} persists the oldest data from the queryable buffer to the Object store in Parquet format, and keeps the remaining data (the most recent 5 minutes) in memory.
### In-memory cache
- **Process**: Recently persisted Parquet files are cached in memory.
- **Impact**: Reduces query latency by minimizing object storage access.
- **Details**: InfluxDB puts Parquet files into an in-memory cache so that queries against the most recently persisted data don't have to go to object storage.
- **Details**: {{% product-name %}} puts Parquet files into an in-memory cache so that queries against the most recently persisted data don't have to go to object storage.

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.2 MiB