chore(influxdb3): Data durability: replace diagram, cleanup brand terms.

pull/5881/head
Jason Stirnaman 2025-03-20 13:33:27 -05:00
parent 4dbdfba22c
commit 072cada97e
2 changed files with 9 additions and 10 deletions

View File

@ -2,13 +2,12 @@
When data is written to {{% product-name %}}, it progresses through multiple stages to ensure durability, optimize performance, and enable efficient querying. Configuration options at each stage affect system behavior, balancing reliability and resource usage. When data is written to {{% product-name %}}, it progresses through multiple stages to ensure durability, optimize performance, and enable efficient querying. Configuration options at each stage affect system behavior, balancing reliability and resource usage.
## Data flow ## Data flow for writes
As data moves through {{% product-name %}}, it follows a structured path to ensure durability, efficient querying, and optimized storage. As written data moves through {{% product-name %}}, it follows a structured path to ensure durability, efficient querying, and optimized storage.
The figure below shows how written data flows through the database. {{< img-hd src="/img/influxdb3/influxdb3-core-enterprise-ingest-path-flow.png" alt="Ingest path and data flow for InfluxDB 3 Core & Enterprise" />}}
<span class="caption">Figure: Write request, response, and ingest flow for InfluxDB 3 Core and Enterprise</span>
{{< img-hd src="/img/influxdb/influxdb-3-write-path.png" alt="Write Path for InfluxDB 3 Core & Enterprise" />}}
1. [Write validation and memory buffer](#1-write-validation-and-memory-buffer) 1. [Write validation and memory buffer](#1-write-validation-and-memory-buffer)
2. [Write-ahead log (WAL) persistence](#2-write-ahead-log-wal-persistence) 2. [Write-ahead log (WAL) persistence](#2-write-ahead-log-wal-persistence)
@ -20,14 +19,14 @@ The figure below shows how written data flows through the database.
- **Process**: InfluxDB validates incoming data before accepting it into the system. - **Process**: InfluxDB validates incoming data before accepting it into the system.
- **Impact**: Prevents malformed or unsupported data from entering the database. - **Impact**: Prevents malformed or unsupported data from entering the database.
- **Details**: The system validates incoming data and stores it in the write buffer (in memory). If [`no_sync=true`](#no-sync-write-option), the server sends a response to acknowledge the write. - **Details**: The database validates incoming data and stores it in the write buffer (in memory). If [`no_sync=true`](#no-sync-write-option), the server sends a response to acknowledge the write.
### Write-ahead log (WAL) persistence ### Write-ahead log (WAL) persistence
- **Process**: The system flushes the write buffer to the WAL every second (default). - **Process**: The database flushes the write buffer to the WAL every second (default).
- **Impact**: Ensures durability by persisting data to object storage. - **Impact**: Ensures durability by persisting data to object storage.
- **Tradeoff**: More frequent flushing improves durability but increases I/O overhead. - **Tradeoff**: More frequent flushing improves durability but increases I/O overhead.
- **Details**: Every second (default), the system flushes the write buffer to the Write-Ahead Log (WAL) for persistence in the Object store. If [`no_sync=false`](#no-sync-write-option) (default), the server sends a response to acknowledge the write. - **Details**: Every second (default), the database flushes the write buffer to the Write-Ahead Log (WAL) for persistence in the Object store. If [`no_sync=false`](#no-sync-write-option) (default), the server sends a response to acknowledge the write.
### Query availability ### Query availability
@ -41,10 +40,10 @@ The figure below shows how written data flows through the database.
- **Process**: Every ten minutes (default), data is persisted to Parquet files in object storage. - **Process**: Every ten minutes (default), data is persisted to Parquet files in object storage.
- **Impact**: Provides durable, long-term storage. - **Impact**: Provides durable, long-term storage.
- **Tradeoff**: More frequent persistence reduces reliance on the WAL but increases I/O costs. - **Tradeoff**: More frequent persistence reduces reliance on the WAL but increases I/O costs.
- **Details**: Every ten minutes (default), the system persists the oldest data from the queryable buffer to the Object store in Parquet format. InfluxDB keeps the remaining data (the most recent 5 minutes) in memory. - **Details**: Every ten minutes (default), the {{% product-name %}} persists the oldest data from the queryable buffer to the Object store in Parquet format, and keeps the remaining data (the most recent 5 minutes) in memory.
### In-memory cache ### In-memory cache
- **Process**: Recently persisted Parquet files are cached in memory. - **Process**: Recently persisted Parquet files are cached in memory.
- **Impact**: Reduces query latency by minimizing object storage access. - **Impact**: Reduces query latency by minimizing object storage access.
- **Details**: InfluxDB puts Parquet files into an in-memory cache so that queries against the most recently persisted data don't have to go to object storage. - **Details**: {{% product-name %}} puts Parquet files into an in-memory cache so that queries against the most recently persisted data don't have to go to object storage.

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.2 MiB