From 500a31cdd928ce4bd85a749c128a205282bc6cee Mon Sep 17 00:00:00 2001 From: Jason Stirnaman Date: Thu, 15 May 2025 15:12:20 -0500 Subject: [PATCH 1/6] feat(oss2): Release notes for OSS 2.7.12, fix broken link, list item indentation, and failing codeblock tests --- .../v2/reference/release-notes/influxdb.md | 75 +++++++++++-------- 1 file changed, 44 insertions(+), 31 deletions(-) diff --git a/content/influxdb/v2/reference/release-notes/influxdb.md b/content/influxdb/v2/reference/release-notes/influxdb.md index 18319bddb..18a3f75cf 100644 --- a/content/influxdb/v2/reference/release-notes/influxdb.md +++ b/content/influxdb/v2/reference/release-notes/influxdb.md @@ -8,6 +8,19 @@ menu: weight: 101 --- +## v2.7.12 {date="2025-05-15"} + +### Features + +- Add a `--pid-file` option to write a PID file to the specified location on startup. InfluxDB removes the PID file on shutdown. +- Add a `--storage-wal-flush-on-shutdown` option to flush the WAL on database shutdown to ensure all data is written to disk. +- Improve logging for dropped points, providing more detailed error messages for partial writes. + +### Maintenance + +- Improved startup logging with an "are we there yet" counter. + + ## v2.7.11 {date="2024-12-02"} ### Features @@ -606,7 +619,7 @@ to migrate InfluxDB key-value metadata schemas to earlier 2.x versions when nece #### Flux - Update to [Flux v0.139.0](/flux/v0/release-notes/#v01390). -- Enable writing to remote hosts using the Flux [`to()`](/flux/v0/stdlib/influxdata/influxdb/to/) and [`experimental.to()`](/flux/v0/v0.x/stdlib/experimental/to/) functions. +- Enable writing to remote hosts using the Flux [`to()`](/flux/v0/stdlib/influxdata/influxdb/to/) and [`experimental.to()`](/flux/v0/stdlib/experimental/to/) functions. - Flux now supports locations that dynamically modify time offsets based on your specified timezone. You can also specify fixed time offsets relative to UTC. - Perform [bitwise operations](/flux/v0/stdlib/experimental/bitwise/) on integers and unsigned integers. @@ -673,24 +686,24 @@ New features include: - Add a new route `/api/v2/resources` that returns a list of known resources to the platform, including the following resource types. Makes it easier to update All Access tokens with current resources: - - `AuthorizationsResourceType` - - `BucketsResourceType` - - `ChecksResourceType` - - `DashboardsResourceType` - - `DBRPResourceType` - - `DocumentsResourceType` - - `LabelsResourceType` - - `NotificationEndpointResourceType` - - `NotificationRuleResourceType` - - `OrgsResourceType` - - `ScraperResourceType` - - `SecretsResourceType` - - `SourcesResourceType` - - `TasksResourceType` - - `TelegrafsResourceType` - - `UsersResourceType` - - `VariablesResourceType` - - `ViewsResourceType` + - `AuthorizationsResourceType` + - `BucketsResourceType` + - `ChecksResourceType` + - `DashboardsResourceType` + - `DBRPResourceType` + - `DocumentsResourceType` + - `LabelsResourceType` + - `NotificationEndpointResourceType` + - `NotificationRuleResourceType` + - `OrgsResourceType` + - `ScraperResourceType` + - `SecretsResourceType` + - `SourcesResourceType` + - `TasksResourceType` + - `TelegrafsResourceType` + - `UsersResourceType` + - `VariablesResourceType` + - `ViewsResourceType` #### Flux updates @@ -992,10 +1005,10 @@ The startup process automatically generates replacement `tsi1` indexes for shard - Fix timeout setup for `influxd` graceful shutdown. - Require user to set password during initial user onboarding. - Error message improvements: - - Remove duplication from task error messages. - - Improve error message shown when influx CLI can't find an `org` by name. - - Improve error message when opening BoltDB with unsupported file system options. - - Improve messages in DBRP API validation errors. + - Remove duplication from task error messages. + - Improve error message shown when influx CLI can't find an `org` by name. + - Improve error message when opening BoltDB with unsupported file system options. + - Improve messages in DBRP API validation errors. - `influxd upgrade` improvements: - Add confirmation step with file sizes before copying data files. - Prevent panic in `influxd upgrade` when v1 users exist but v1 config is missing. @@ -1072,8 +1085,8 @@ Previously, the database retention policy (DBRP) mapping API did not match the s ### Features - Improvements to upgrade from 1.x to 2.x: - - Warning appears if auth is not enabled in 1.x (`auth-enabled = false`), which is not an option in 2.x. For details, see [Upgrade from InfluxDB 1.x to InfluxDB 2.0](/influxdb/v2/upgrade/v1-to-v2/). - - `upgrade` command now checks to see if continuous queries are running and automatically exports them to a local file. + - Warning appears if auth is not enabled in 1.x (`auth-enabled = false`), which is not an option in 2.x. For details, see [Upgrade from InfluxDB 1.x to InfluxDB 2.0](/influxdb/v2/upgrade/v1-to-v2/). + - `upgrade` command now checks to see if continuous queries are running and automatically exports them to a local file. - Upgrade to [Flux v0.95.0](/flux/v0/release-notes/#v0-95-0). - Upgrade `flux-lsp-browser` to v.0.5.23. - Manage database retention policy (DBRP) mappings via CLI. See [`influx v1 dbrp`](/influxdb/v2/reference/cli/influx/v1/dbrp/). @@ -1117,8 +1130,8 @@ When there are multiple [DBRP mappings](/influxdb/v2/reference/api/influxdb-1x/d Highlights include: - Support for **upgrading to InfluxDB 2.0**: - - To upgrade **from InfluxDB 1.x**, see [Upgrade from InfluxDB 1.x to InfluxDB 2.0](/influxdb/v2/upgrade/v1-to-v2). - - To upgrade **from InfluxDB 2.0 beta 16 or earlier**, see [Upgrade from InfluxDB 2.0 beta to InfluxDB 2.0](/influxdb/v2/upgrade/v2-beta-to-v2). + - To upgrade **from InfluxDB 1.x**, see [Upgrade from InfluxDB 1.x to InfluxDB 2.0](/influxdb/v2/upgrade/v1-to-v2). + - To upgrade **from InfluxDB 2.0 beta 16 or earlier**, see [Upgrade from InfluxDB 2.0 beta to InfluxDB 2.0](/influxdb/v2/install/upgrade/v2-beta-to-v2/). - **Flux**, our powerful new functional data scripting language designed for querying, analyzing, and acting on data. This release includes [Flux v0.94.0](/flux/v0/release-notes/#v0940). If you're new to Flux, [check out how to get started with Flux](/influxdb/v2/query-data/get-started/). Next, delve deeper into the [Flux standard library](/flux/v0/stdlib//) reference docs and see how to [query with Flux](/influxdb/v2/query-data/flux/). - Support for [InfluxDB 1.x API compatibility](/influxdb/v2/reference/api/influxdb-1x/). - **Templates** and **stacks**. Discover how to [use community templates](/influxdb/v2/tools/influxdb-templates/use/) and how to [manage templates with stacks](/influxdb/v2/tools/influxdb-templates/stacks/). @@ -1241,14 +1254,14 @@ If you're new to InfluxDB 2.0, we recommend checking out [how to get started](/i {{% warn %}} #### Manual upgrade required -To simplify the migration for existing users of InfluxDB 1.x, this release includes significant breaking changes that require a manual upgrade from all alpha and beta versions. For more information, see [Upgrade to InfluxDB OSS 2.0rc](/influxdb/v2/upgrade/v2-beta-to-v2/), +To simplify the migration for existing users of InfluxDB 1.x, this release includes significant breaking changes that require a manual upgrade from all alpha and beta versions. For more information, see [Upgrade to InfluxDB OSS 2.0rc](/influxdb/v2/install/upgrade/v2-beta-to-v2/), {{% /warn %}} ### Breaking changes #### Manual upgrade -- To continue using data from InfluxDB 2.0 beta 16 or earlier, you must move all existing data out of the `~/.influxdbv2` (or equivalent) path, including `influxd.bolt`. All existing dashboards, tasks, integrations, alerts, users, and tokens must be recreated. For information on how to migrate your data, see [Upgrade to InfluxDB OSS 2.0rc](/influxdb/v2/upgrade/v2-beta-to-v2/). +- To continue using data from InfluxDB 2.0 beta 16 or earlier, you must move all existing data out of the `~/.influxdbv2` (or equivalent) path, including `influxd.bolt`. All existing dashboards, tasks, integrations, alerts, users, and tokens must be recreated. For information on how to migrate your data, see [Upgrade to InfluxDB OSS 2.0rc](/influxdb/v2/install/upgrade/v2-beta-to-v2/). #### Port update to 8086 @@ -2045,7 +2058,7 @@ _**This will remove all data from your InfluxDB v2.0 instance including time ser ###### Linux and macOS ```sh -rm ~/.influxdbv2/influxd.bolt +rm -f ~/.influxdbv2/influxd.bolt ``` Once completed, `v2.0.0-alpha.6` can be started. @@ -2079,7 +2092,7 @@ run the following command. ###### Linux and macOS ```sh -rm -r ~/.influxdbv2/engine +rm -rf ~/.influxdbv2/engine ``` Once completed, InfluxDB v2.0.0-alpha.5 can be started. From 0a1179932fc988f265880eb4a8f0335242258c3b Mon Sep 17 00:00:00 2001 From: Jason Stirnaman Date: Thu, 15 May 2025 15:44:26 -0500 Subject: [PATCH 2/6] feat(oss2): option Part of influxdata/DAR/issues/500 --- .../influxdb/v2/reference/config-options.md | 55 +++++++++++++++++++ 1 file changed, 55 insertions(+) diff --git a/content/influxdb/v2/reference/config-options.md b/content/influxdb/v2/reference/config-options.md index 18f0eadcb..1698e37c6 100644 --- a/content/influxdb/v2/reference/config-options.md +++ b/content/influxdb/v2/reference/config-options.md @@ -2744,6 +2744,61 @@ storage-validate-keys = true --- +### storage-wal-flush-on-shutdown +Flush the WAL on shutdown. + + +**Default:** `false` + +| influxd flag | Environment variable | Configuration key | +| :------------------------------ | :------------------------------------ | :---------------------------- | +| `--storage-wal-flush-on-shutdown` | `INFLUXD_STORAGE_WAL_FLUSH_ON_SHUTDOWN` | `storage-wal-flush-on-shutdown` | + +If set, `influxd` flushes or snapshots all WALs prior to completing shutdown--`influxd` performs cache snapshots on shutdown, which +results in the WAL files being written to TSM files and then deleted. + +This is useful in upgrade and downgrade scenarios to prevent WAL format +compatibility issues. + +###### influxd flag + + +```sh +influxd --storage-wal-flush-on-shutdown +``` + +###### Environment variable +```sh +export INFLUXD_STORAGE_WAL_FLUSH_ON_SHUTDOWN=true +``` + +###### Configuration file +{{< code-tabs-wrapper >}} +{{% code-tabs %}} +[YAML](#) +[TOML](#) +[JSON](#) +{{% /code-tabs %}} +{{% code-tab-content %}} +```yml +storage-wal-flush-on-shutdown: true +``` +{{% /code-tab-content %}} +{{% code-tab-content %}} +```toml +storage-wal-flush-on-shutdown = true +``` +{{% /code-tab-content %}} +{{% code-tab-content %}} +```json +{ + "storage-wal-flush-on-shutdown": true +} +``` +{{% /code-tab-content %}} +{{< /code-tabs-wrapper >}} +--- + ### storage-wal-fsync-delay Duration a write will wait before fsyncing. A duration greater than `0` batches multiple fsync calls. From 79de24bef22b38c9ab67ea6a487e5c7730654b5d Mon Sep 17 00:00:00 2001 From: Jason Stirnaman Date: Thu, 15 May 2025 16:05:43 -0500 Subject: [PATCH 3/6] feat(oss2): Add influxd_flags entry for option Part of influxdata/DAR/issues/500 --- content/influxdb/v2/reference/config-options.md | 2 +- data/influxd_flags.yml | 3 +++ 2 files changed, 4 insertions(+), 1 deletion(-) diff --git a/content/influxdb/v2/reference/config-options.md b/content/influxdb/v2/reference/config-options.md index 1698e37c6..4f639b6ab 100644 --- a/content/influxdb/v2/reference/config-options.md +++ b/content/influxdb/v2/reference/config-options.md @@ -2747,7 +2747,6 @@ storage-validate-keys = true ### storage-wal-flush-on-shutdown Flush the WAL on shutdown. - **Default:** `false` | influxd flag | Environment variable | Configuration key | @@ -2797,6 +2796,7 @@ storage-wal-flush-on-shutdown = true ``` {{% /code-tab-content %}} {{< /code-tabs-wrapper >}} + --- ### storage-wal-fsync-delay diff --git a/data/influxd_flags.yml b/data/influxd_flags.yml index e3e1998de..29f57a2e7 100644 --- a/data/influxd_flags.yml +++ b/data/influxd_flags.yml @@ -161,6 +161,9 @@ - flag: "--storage-validate-keys" added: 2.0 +- flag: "--storage-wal-flush-on-shutdown" + added: 2.7 + - flag: "--storage-wal-fsync-delay" added: 2.0 From a0e627468ed83dfc8fb994c310b1192bda657c82 Mon Sep 17 00:00:00 2001 From: Jason Stirnaman Date: Thu, 15 May 2025 16:43:11 -0500 Subject: [PATCH 4/6] feat(oss2): Maintenance items for OSS 2.7.12 --- content/influxdb/v2/reference/release-notes/influxdb.md | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/content/influxdb/v2/reference/release-notes/influxdb.md b/content/influxdb/v2/reference/release-notes/influxdb.md index 18a3f75cf..69f2830c3 100644 --- a/content/influxdb/v2/reference/release-notes/influxdb.md +++ b/content/influxdb/v2/reference/release-notes/influxdb.md @@ -18,7 +18,10 @@ weight: 101 ### Maintenance -- Improved startup logging with an "are we there yet" counter. +- Improved startup logging with an "are we there yet" counter for the number and percentage of shards opened. +- Update Go to 1.23.9. +- Update Flux to v0.196.1. +- Refresh dependencies to address security vulnerabilities and improve stability. ## v2.7.11 {date="2024-12-02"} From cffe9402cf44803b9c5dc554a58589857f327727 Mon Sep 17 00:00:00 2001 From: Jason Stirnaman Date: Thu, 15 May 2025 17:12:12 -0500 Subject: [PATCH 5/6] feat(oss2): Revise and fix partial write enhancement for OSS 2.7.12 --- content/influxdb/v2/reference/release-notes/influxdb.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/influxdb/v2/reference/release-notes/influxdb.md b/content/influxdb/v2/reference/release-notes/influxdb.md index 69f2830c3..8ad4a0178 100644 --- a/content/influxdb/v2/reference/release-notes/influxdb.md +++ b/content/influxdb/v2/reference/release-notes/influxdb.md @@ -14,7 +14,7 @@ weight: 101 - Add a `--pid-file` option to write a PID file to the specified location on startup. InfluxDB removes the PID file on shutdown. - Add a `--storage-wal-flush-on-shutdown` option to flush the WAL on database shutdown to ensure all data is written to disk. -- Improve logging for dropped points, providing more detailed error messages for partial writes. +- Improve response error messages for dropped points, adding details including database, retention policy, and which bound was violated for partial writes. ### Maintenance From 50a9d7c796cb8b9cf8913b66595547403f448cc7 Mon Sep 17 00:00:00 2001 From: Jason Stirnaman Date: Tue, 20 May 2025 16:23:40 -0500 Subject: [PATCH 6/6] feat(oss2): 2.7.12 release notes: include bug fix, bump version --- content/influxdb/v2/reference/release-notes/influxdb.md | 7 ++++++- data/products.yml | 2 +- 2 files changed, 7 insertions(+), 2 deletions(-) diff --git a/content/influxdb/v2/reference/release-notes/influxdb.md b/content/influxdb/v2/reference/release-notes/influxdb.md index 8ad4a0178..9633439d6 100644 --- a/content/influxdb/v2/reference/release-notes/influxdb.md +++ b/content/influxdb/v2/reference/release-notes/influxdb.md @@ -8,7 +8,7 @@ menu: weight: 101 --- -## v2.7.12 {date="2025-05-15"} +## v2.7.12 {date="2025-05-20"} ### Features @@ -16,6 +16,10 @@ weight: 101 - Add a `--storage-wal-flush-on-shutdown` option to flush the WAL on database shutdown to ensure all data is written to disk. - Improve response error messages for dropped points, adding details including database, retention policy, and which bound was violated for partial writes. +### Bug Fixes + +- Fix a locking issue in `TagValueIterator` that could cause reads and writes in buckets to block. [PR #26414](https://github.com/influxdata/influxdb/pull/26414) + ### Maintenance - Improved startup logging with an "are we there yet" counter for the number and percentage of shards opened. @@ -23,6 +27,7 @@ weight: 101 - Update Flux to v0.196.1. - Refresh dependencies to address security vulnerabilities and improve stability. +--- ## v2.7.11 {date="2024-12-02"} diff --git a/data/products.yml b/data/products.yml index 47c2d258e..d427e7ffc 100644 --- a/data/products.yml +++ b/data/products.yml @@ -85,7 +85,7 @@ influxdb: - v1 latest: v2.7 latest_patches: - v2: 2.7.11 + v2: 2.7.12 v1: 1.11.8 latest_cli: v2: 2.7.5