From 765f669d06923d0f4dc05a18466d27ce824cc015 Mon Sep 17 00:00:00 2001 From: Jason Stirnaman Date: Fri, 6 Mar 2026 08:11:11 -0600 Subject: [PATCH] feat(influxdb3): document --checkpoint-interval serve option (#6896) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * feat(influxdb3): document --checkpoint-interval serve option Add documentation for the previously undocumented --checkpoint-interval flag, which aggregates WAL snapshot files into monthly checkpoint files to significantly reduce server startup time. Changes: - config-options.md: add checkpoint-interval entry in the WAL section, including description, default (disabled), env var, and usage example - performance-tuning.md: add "Startup optimization" section explaining the startup-time tradeoff and recommended intervals by scenario Closes #6884 https://claude.ai/code/session_01BArJYJTXBsUjNcdTJVyg6v * docs(influxdb3): add version metadata to checkpoint-interval heading (#6897) * Initial plan * feat(influxdb3): add version metadata to checkpoint-interval heading Co-authored-by: jstirnaman <212227+jstirnaman@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: jstirnaman <212227+jstirnaman@users.noreply.github.com> * Apply suggestions from code review * fix(workflows): trigger pr-preview on ready_for_review event (#6898) * Initial plan * fix(workflows): trigger pr-preview on ready_for_review event Co-authored-by: jstirnaman <212227+jstirnaman@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: jstirnaman <212227+jstirnaman@users.noreply.github.com> * Update content/shared/influxdb3-cli/config-options.md * fix(influxdb3): remove WAL snapshot conflation and add release note - Replace "WAL snapshot files" with "snapshots" shorthand, linking to backup-restore file structure on first use - Simplify "snapshot files"→"snapshots" and "checkpoint files"→"checkpoints" throughout checkpoint-interval and startup optimization sections - Add checkpoint-interval feature to v3.8.2 release notes Addresses review feedback from #6896. * Update content/shared/influxdb3-cli/config-options.md --------- Co-authored-by: Claude Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com> --- .github/workflows/pr-preview.yml | 2 +- .../influxdb3-admin/performance-tuning.md | 48 +++++++++++++++++++ .../shared/influxdb3-cli/config-options.md | 41 ++++++++++++++++ .../_index.md | 1 + 4 files changed, 91 insertions(+), 1 deletion(-) diff --git a/.github/workflows/pr-preview.yml b/.github/workflows/pr-preview.yml index 0eb5d6b42..18c8ca378 100644 --- a/.github/workflows/pr-preview.yml +++ b/.github/workflows/pr-preview.yml @@ -2,7 +2,7 @@ name: PR Preview on: pull_request: - types: [opened, reopened, synchronize, closed] + types: [opened, reopened, synchronize, closed, ready_for_review] paths: - 'content/**' - 'layouts/**' diff --git a/content/shared/influxdb3-admin/performance-tuning.md b/content/shared/influxdb3-admin/performance-tuning.md index 3ed16c6e6..45201eb65 100644 --- a/content/shared/influxdb3-admin/performance-tuning.md +++ b/content/shared/influxdb3-admin/performance-tuning.md @@ -12,6 +12,7 @@ based on your workload characteristics. {{% /show-in %}} - [Memory tuning](#memory-tuning) - [Advanced tuning options](#advanced-tuning-options) +- [Startup optimization](#startup-optimization) - [Monitoring and validation](#monitoring-and-validation) - [Common performance issues](#common-performance-issues-1) @@ -577,6 +578,53 @@ For all available configuration options, see: - [CLI serve command reference](/influxdb3/version/reference/cli/influxdb3/serve/) - [Configuration options](/influxdb3/version/reference/config-options/) +## Startup optimization + +Server startup time scales with the number of +[snapshots](/influxdb3/version/admin/backup-restore/#file-structure) +stored in the object store. +Snapshots accumulate over time and are not automatically deleted. + +Without checkpointing, the server loads individual snapshots on startup. +The number of snapshots is determined by the lookback window +([`gen1-lookback-duration`](/influxdb3/version/reference/config-options/#gen1-lookback-duration), +default 1 month) divided by +[`gen1-duration`](/influxdb3/version/reference/config-options/#gen1-duration) +(default 10 minutes), with a minimum of 100. +With default settings, a long-running server can accumulate up to ~4,320 +snapshots, causing slow restarts. + +Two configuration options reduce startup time: + +- [`--checkpoint-interval`](/influxdb3/version/reference/config-options/#checkpoint-interval)-- + periodically consolidates snapshot metadata into monthly checkpoints. + On startup, the server loads one to two checkpoints per calendar month, + then loads only snapshots created since the last checkpoint. +- [`--gen1-lookback-duration`](/influxdb3/version/reference/config-options/#gen1-lookback-duration)-- + limits how far back the server loads gen1 file index metadata on startup. + Files outside this window still exist in object storage but are not indexed. + +> [!Note] +> Enabling checkpointing does not delete old snapshots. +> They remain in object storage but are no longer needed for startup. + +### Recommended checkpoint intervals + +| Scenario | Recommended interval | +| :------- | :------------------- | +| Production servers | `1h` | +| Development / testing | `10m` | + +### Enable checkpoint creation + + +```bash +influxdb3 serve --checkpoint-interval 1h +``` + +For all checkpoint configuration options, see +[checkpoint-interval](/influxdb3/version/reference/config-options/#checkpoint-interval). + ## Monitoring and validation ### Monitor thread utilization diff --git a/content/shared/influxdb3-cli/config-options.md b/content/shared/influxdb3-cli/config-options.md index 8cf4ebca4..d82eafaba 100644 --- a/content/shared/influxdb3-cli/config-options.md +++ b/content/shared/influxdb3-cli/config-options.md @@ -1227,6 +1227,7 @@ percentage (portion of available memory) or absolute value in MB--for example: ` ### Write-Ahead Log (WAL) +- [checkpoint-interval](#checkpoint-interval) - [wal-flush-interval](#wal-flush-interval) - [wal-snapshot-size](#wal-snapshot-size) - [wal-max-write-buffer-size](#wal-max-write-buffer-size) @@ -1234,6 +1235,46 @@ percentage (portion of available memory) or absolute value in MB--for example: ` - [wal-replay-fail-on-error](#wal-replay-fail-on-error) - [wal-replay-concurrency-limit](#wal-replay-concurrency-limit) +#### checkpoint-interval {#checkpoint-interval metadata="v3.8.2+"} + +Sets the interval for consolidating +[snapshots](/influxdb3/version/admin/backup-restore/#file-structure) into +monthly checkpoints for faster server startup. +Snapshots accumulate in object storage over time and are not automatically deleted. + +Without checkpointing, the server loads individual snapshots on startup. +The number of snapshots is determined by the lookback window +([`gen1-lookback-duration`](#gen1-lookback-duration), default 1 month) +divided by [`gen1-duration`](#gen1-duration) (default 10 minutes), +with a minimum of 100. +With default settings, that can be up to ~4,320 snapshots. + +With checkpointing enabled, the server periodically consolidates snapshot +metadata into checkpoints in object storage. +On startup, the server loads one to two checkpoints per calendar month, +then loads only snapshots created since the last checkpoint. +Enabling checkpointing does not delete old snapshots. + +Up to 10 checkpoints load concurrently during startup. +The server retains two checkpoints per calendar month and handles month rollovers automatically. + +Accepts a [duration](/influxdb3/version/reference/glossary/#duration) value--for example: `1h`, `30m`, `10m`. + +**Default:** _Not set (disabled)_ + +| influxdb3 serve option | Environment variable | +| :---------------------- | :------------------------------ | +| `--checkpoint-interval` | `INFLUXDB3_CHECKPOINT_INTERVAL` | + +##### Example + + +```bash +influxdb3 serve --checkpoint-interval 1h +``` + +*** + #### wal-flush-interval Specifies the interval to flush buffered data to a WAL file. Writes that wait diff --git a/content/shared/v3-core-enterprise-release-notes/_index.md b/content/shared/v3-core-enterprise-release-notes/_index.md index c8e747da3..07503f63c 100644 --- a/content/shared/v3-core-enterprise-release-notes/_index.md +++ b/content/shared/v3-core-enterprise-release-notes/_index.md @@ -40,6 +40,7 @@ - **`_internal` database default retention**: The `_internal` system database now defaults to a 7-day retention period (previously infinite). Only admin tokens can modify retention on the `_internal` database. +- **Snapshot checkpointing for faster startup**: Use the new [`--checkpoint-interval`](/influxdb3/version/reference/config-options/#checkpoint-interval) serve option to periodically consolidate snapshots into monthly checkpoints. On startup, the server loads one to two checkpoints per calendar month instead of thousands of individual snapshots, reducing startup time for long-running servers. #### Bug fixes