From 500a31cdd928ce4bd85a749c128a205282bc6cee Mon Sep 17 00:00:00 2001 From: Jason Stirnaman Date: Thu, 15 May 2025 15:12:20 -0500 Subject: [PATCH 01/13] feat(oss2): Release notes for OSS 2.7.12, fix broken link, list item indentation, and failing codeblock tests --- .../v2/reference/release-notes/influxdb.md | 75 +++++++++++-------- 1 file changed, 44 insertions(+), 31 deletions(-) diff --git a/content/influxdb/v2/reference/release-notes/influxdb.md b/content/influxdb/v2/reference/release-notes/influxdb.md index 18319bddb..18a3f75cf 100644 --- a/content/influxdb/v2/reference/release-notes/influxdb.md +++ b/content/influxdb/v2/reference/release-notes/influxdb.md @@ -8,6 +8,19 @@ menu: weight: 101 --- +## v2.7.12 {date="2025-05-15"} + +### Features + +- Add a `--pid-file` option to write a PID file to the specified location on startup. InfluxDB removes the PID file on shutdown. +- Add a `--storage-wal-flush-on-shutdown` option to flush the WAL on database shutdown to ensure all data is written to disk. +- Improve logging for dropped points, providing more detailed error messages for partial writes. + +### Maintenance + +- Improved startup logging with an "are we there yet" counter. + + ## v2.7.11 {date="2024-12-02"} ### Features @@ -606,7 +619,7 @@ to migrate InfluxDB key-value metadata schemas to earlier 2.x versions when nece #### Flux - Update to [Flux v0.139.0](/flux/v0/release-notes/#v01390). -- Enable writing to remote hosts using the Flux [`to()`](/flux/v0/stdlib/influxdata/influxdb/to/) and [`experimental.to()`](/flux/v0/v0.x/stdlib/experimental/to/) functions. +- Enable writing to remote hosts using the Flux [`to()`](/flux/v0/stdlib/influxdata/influxdb/to/) and [`experimental.to()`](/flux/v0/stdlib/experimental/to/) functions. - Flux now supports locations that dynamically modify time offsets based on your specified timezone. You can also specify fixed time offsets relative to UTC. - Perform [bitwise operations](/flux/v0/stdlib/experimental/bitwise/) on integers and unsigned integers. @@ -673,24 +686,24 @@ New features include: - Add a new route `/api/v2/resources` that returns a list of known resources to the platform, including the following resource types. Makes it easier to update All Access tokens with current resources: - - `AuthorizationsResourceType` - - `BucketsResourceType` - - `ChecksResourceType` - - `DashboardsResourceType` - - `DBRPResourceType` - - `DocumentsResourceType` - - `LabelsResourceType` - - `NotificationEndpointResourceType` - - `NotificationRuleResourceType` - - `OrgsResourceType` - - `ScraperResourceType` - - `SecretsResourceType` - - `SourcesResourceType` - - `TasksResourceType` - - `TelegrafsResourceType` - - `UsersResourceType` - - `VariablesResourceType` - - `ViewsResourceType` + - `AuthorizationsResourceType` + - `BucketsResourceType` + - `ChecksResourceType` + - `DashboardsResourceType` + - `DBRPResourceType` + - `DocumentsResourceType` + - `LabelsResourceType` + - `NotificationEndpointResourceType` + - `NotificationRuleResourceType` + - `OrgsResourceType` + - `ScraperResourceType` + - `SecretsResourceType` + - `SourcesResourceType` + - `TasksResourceType` + - `TelegrafsResourceType` + - `UsersResourceType` + - `VariablesResourceType` + - `ViewsResourceType` #### Flux updates @@ -992,10 +1005,10 @@ The startup process automatically generates replacement `tsi1` indexes for shard - Fix timeout setup for `influxd` graceful shutdown. - Require user to set password during initial user onboarding. - Error message improvements: - - Remove duplication from task error messages. - - Improve error message shown when influx CLI can't find an `org` by name. - - Improve error message when opening BoltDB with unsupported file system options. - - Improve messages in DBRP API validation errors. + - Remove duplication from task error messages. + - Improve error message shown when influx CLI can't find an `org` by name. + - Improve error message when opening BoltDB with unsupported file system options. + - Improve messages in DBRP API validation errors. - `influxd upgrade` improvements: - Add confirmation step with file sizes before copying data files. - Prevent panic in `influxd upgrade` when v1 users exist but v1 config is missing. @@ -1072,8 +1085,8 @@ Previously, the database retention policy (DBRP) mapping API did not match the s ### Features - Improvements to upgrade from 1.x to 2.x: - - Warning appears if auth is not enabled in 1.x (`auth-enabled = false`), which is not an option in 2.x. For details, see [Upgrade from InfluxDB 1.x to InfluxDB 2.0](/influxdb/v2/upgrade/v1-to-v2/). - - `upgrade` command now checks to see if continuous queries are running and automatically exports them to a local file. + - Warning appears if auth is not enabled in 1.x (`auth-enabled = false`), which is not an option in 2.x. For details, see [Upgrade from InfluxDB 1.x to InfluxDB 2.0](/influxdb/v2/upgrade/v1-to-v2/). + - `upgrade` command now checks to see if continuous queries are running and automatically exports them to a local file. - Upgrade to [Flux v0.95.0](/flux/v0/release-notes/#v0-95-0). - Upgrade `flux-lsp-browser` to v.0.5.23. - Manage database retention policy (DBRP) mappings via CLI. See [`influx v1 dbrp`](/influxdb/v2/reference/cli/influx/v1/dbrp/). @@ -1117,8 +1130,8 @@ When there are multiple [DBRP mappings](/influxdb/v2/reference/api/influxdb-1x/d Highlights include: - Support for **upgrading to InfluxDB 2.0**: - - To upgrade **from InfluxDB 1.x**, see [Upgrade from InfluxDB 1.x to InfluxDB 2.0](/influxdb/v2/upgrade/v1-to-v2). - - To upgrade **from InfluxDB 2.0 beta 16 or earlier**, see [Upgrade from InfluxDB 2.0 beta to InfluxDB 2.0](/influxdb/v2/upgrade/v2-beta-to-v2). + - To upgrade **from InfluxDB 1.x**, see [Upgrade from InfluxDB 1.x to InfluxDB 2.0](/influxdb/v2/upgrade/v1-to-v2). + - To upgrade **from InfluxDB 2.0 beta 16 or earlier**, see [Upgrade from InfluxDB 2.0 beta to InfluxDB 2.0](/influxdb/v2/install/upgrade/v2-beta-to-v2/). - **Flux**, our powerful new functional data scripting language designed for querying, analyzing, and acting on data. This release includes [Flux v0.94.0](/flux/v0/release-notes/#v0940). If you're new to Flux, [check out how to get started with Flux](/influxdb/v2/query-data/get-started/). Next, delve deeper into the [Flux standard library](/flux/v0/stdlib//) reference docs and see how to [query with Flux](/influxdb/v2/query-data/flux/). - Support for [InfluxDB 1.x API compatibility](/influxdb/v2/reference/api/influxdb-1x/). - **Templates** and **stacks**. Discover how to [use community templates](/influxdb/v2/tools/influxdb-templates/use/) and how to [manage templates with stacks](/influxdb/v2/tools/influxdb-templates/stacks/). @@ -1241,14 +1254,14 @@ If you're new to InfluxDB 2.0, we recommend checking out [how to get started](/i {{% warn %}} #### Manual upgrade required -To simplify the migration for existing users of InfluxDB 1.x, this release includes significant breaking changes that require a manual upgrade from all alpha and beta versions. For more information, see [Upgrade to InfluxDB OSS 2.0rc](/influxdb/v2/upgrade/v2-beta-to-v2/), +To simplify the migration for existing users of InfluxDB 1.x, this release includes significant breaking changes that require a manual upgrade from all alpha and beta versions. For more information, see [Upgrade to InfluxDB OSS 2.0rc](/influxdb/v2/install/upgrade/v2-beta-to-v2/), {{% /warn %}} ### Breaking changes #### Manual upgrade -- To continue using data from InfluxDB 2.0 beta 16 or earlier, you must move all existing data out of the `~/.influxdbv2` (or equivalent) path, including `influxd.bolt`. All existing dashboards, tasks, integrations, alerts, users, and tokens must be recreated. For information on how to migrate your data, see [Upgrade to InfluxDB OSS 2.0rc](/influxdb/v2/upgrade/v2-beta-to-v2/). +- To continue using data from InfluxDB 2.0 beta 16 or earlier, you must move all existing data out of the `~/.influxdbv2` (or equivalent) path, including `influxd.bolt`. All existing dashboards, tasks, integrations, alerts, users, and tokens must be recreated. For information on how to migrate your data, see [Upgrade to InfluxDB OSS 2.0rc](/influxdb/v2/install/upgrade/v2-beta-to-v2/). #### Port update to 8086 @@ -2045,7 +2058,7 @@ _**This will remove all data from your InfluxDB v2.0 instance including time ser ###### Linux and macOS ```sh -rm ~/.influxdbv2/influxd.bolt +rm -f ~/.influxdbv2/influxd.bolt ``` Once completed, `v2.0.0-alpha.6` can be started. @@ -2079,7 +2092,7 @@ run the following command. ###### Linux and macOS ```sh -rm -r ~/.influxdbv2/engine +rm -rf ~/.influxdbv2/engine ``` Once completed, InfluxDB v2.0.0-alpha.5 can be started. From 0a1179932fc988f265880eb4a8f0335242258c3b Mon Sep 17 00:00:00 2001 From: Jason Stirnaman Date: Thu, 15 May 2025 15:44:26 -0500 Subject: [PATCH 02/13] feat(oss2): option Part of influxdata/DAR/issues/500 --- .../influxdb/v2/reference/config-options.md | 55 +++++++++++++++++++ 1 file changed, 55 insertions(+) diff --git a/content/influxdb/v2/reference/config-options.md b/content/influxdb/v2/reference/config-options.md index 18f0eadcb..1698e37c6 100644 --- a/content/influxdb/v2/reference/config-options.md +++ b/content/influxdb/v2/reference/config-options.md @@ -2744,6 +2744,61 @@ storage-validate-keys = true --- +### storage-wal-flush-on-shutdown +Flush the WAL on shutdown. + + +**Default:** `false` + +| influxd flag | Environment variable | Configuration key | +| :------------------------------ | :------------------------------------ | :---------------------------- | +| `--storage-wal-flush-on-shutdown` | `INFLUXD_STORAGE_WAL_FLUSH_ON_SHUTDOWN` | `storage-wal-flush-on-shutdown` | + +If set, `influxd` flushes or snapshots all WALs prior to completing shutdown--`influxd` performs cache snapshots on shutdown, which +results in the WAL files being written to TSM files and then deleted. + +This is useful in upgrade and downgrade scenarios to prevent WAL format +compatibility issues. + +###### influxd flag + + +```sh +influxd --storage-wal-flush-on-shutdown +``` + +###### Environment variable +```sh +export INFLUXD_STORAGE_WAL_FLUSH_ON_SHUTDOWN=true +``` + +###### Configuration file +{{< code-tabs-wrapper >}} +{{% code-tabs %}} +[YAML](#) +[TOML](#) +[JSON](#) +{{% /code-tabs %}} +{{% code-tab-content %}} +```yml +storage-wal-flush-on-shutdown: true +``` +{{% /code-tab-content %}} +{{% code-tab-content %}} +```toml +storage-wal-flush-on-shutdown = true +``` +{{% /code-tab-content %}} +{{% code-tab-content %}} +```json +{ + "storage-wal-flush-on-shutdown": true +} +``` +{{% /code-tab-content %}} +{{< /code-tabs-wrapper >}} +--- + ### storage-wal-fsync-delay Duration a write will wait before fsyncing. A duration greater than `0` batches multiple fsync calls. From 79de24bef22b38c9ab67ea6a487e5c7730654b5d Mon Sep 17 00:00:00 2001 From: Jason Stirnaman Date: Thu, 15 May 2025 16:05:43 -0500 Subject: [PATCH 03/13] feat(oss2): Add influxd_flags entry for option Part of influxdata/DAR/issues/500 --- content/influxdb/v2/reference/config-options.md | 2 +- data/influxd_flags.yml | 3 +++ 2 files changed, 4 insertions(+), 1 deletion(-) diff --git a/content/influxdb/v2/reference/config-options.md b/content/influxdb/v2/reference/config-options.md index 1698e37c6..4f639b6ab 100644 --- a/content/influxdb/v2/reference/config-options.md +++ b/content/influxdb/v2/reference/config-options.md @@ -2747,7 +2747,6 @@ storage-validate-keys = true ### storage-wal-flush-on-shutdown Flush the WAL on shutdown. - **Default:** `false` | influxd flag | Environment variable | Configuration key | @@ -2797,6 +2796,7 @@ storage-wal-flush-on-shutdown = true ``` {{% /code-tab-content %}} {{< /code-tabs-wrapper >}} + --- ### storage-wal-fsync-delay diff --git a/data/influxd_flags.yml b/data/influxd_flags.yml index e3e1998de..29f57a2e7 100644 --- a/data/influxd_flags.yml +++ b/data/influxd_flags.yml @@ -161,6 +161,9 @@ - flag: "--storage-validate-keys" added: 2.0 +- flag: "--storage-wal-flush-on-shutdown" + added: 2.7 + - flag: "--storage-wal-fsync-delay" added: 2.0 From a0e627468ed83dfc8fb994c310b1192bda657c82 Mon Sep 17 00:00:00 2001 From: Jason Stirnaman Date: Thu, 15 May 2025 16:43:11 -0500 Subject: [PATCH 04/13] feat(oss2): Maintenance items for OSS 2.7.12 --- content/influxdb/v2/reference/release-notes/influxdb.md | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/content/influxdb/v2/reference/release-notes/influxdb.md b/content/influxdb/v2/reference/release-notes/influxdb.md index 18a3f75cf..69f2830c3 100644 --- a/content/influxdb/v2/reference/release-notes/influxdb.md +++ b/content/influxdb/v2/reference/release-notes/influxdb.md @@ -18,7 +18,10 @@ weight: 101 ### Maintenance -- Improved startup logging with an "are we there yet" counter. +- Improved startup logging with an "are we there yet" counter for the number and percentage of shards opened. +- Update Go to 1.23.9. +- Update Flux to v0.196.1. +- Refresh dependencies to address security vulnerabilities and improve stability. ## v2.7.11 {date="2024-12-02"} From cffe9402cf44803b9c5dc554a58589857f327727 Mon Sep 17 00:00:00 2001 From: Jason Stirnaman Date: Thu, 15 May 2025 17:12:12 -0500 Subject: [PATCH 05/13] feat(oss2): Revise and fix partial write enhancement for OSS 2.7.12 --- content/influxdb/v2/reference/release-notes/influxdb.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/influxdb/v2/reference/release-notes/influxdb.md b/content/influxdb/v2/reference/release-notes/influxdb.md index 69f2830c3..8ad4a0178 100644 --- a/content/influxdb/v2/reference/release-notes/influxdb.md +++ b/content/influxdb/v2/reference/release-notes/influxdb.md @@ -14,7 +14,7 @@ weight: 101 - Add a `--pid-file` option to write a PID file to the specified location on startup. InfluxDB removes the PID file on shutdown. - Add a `--storage-wal-flush-on-shutdown` option to flush the WAL on database shutdown to ensure all data is written to disk. -- Improve logging for dropped points, providing more detailed error messages for partial writes. +- Improve response error messages for dropped points, adding details including database, retention policy, and which bound was violated for partial writes. ### Maintenance From 30f7495a72db19c0e8d56e66436139e1a966e803 Mon Sep 17 00:00:00 2001 From: Jason Stirnaman Date: Tue, 13 May 2025 16:05:42 -0500 Subject: [PATCH 06/13] chore(hugo): Update shortcodes and templates to avoid calling isset on --- layouts/_default/page-list.json | 16 ++++++------ layouts/partials/article/flux-contrib.html | 2 +- .../partials/article/flux-experimental.html | 2 +- layouts/partials/article/tags.html | 4 +-- layouts/partials/header/title.html | 6 ++--- layouts/partials/sidebar/nested-menu.html | 14 +++++----- layouts/shortcodes/children.html | 26 +++++++++---------- layouts/shortcodes/icon.html | 6 ++++- layouts/shortcodes/nav-icon.html | 5 +++- 9 files changed, 44 insertions(+), 37 deletions(-) diff --git a/layouts/_default/page-list.json b/layouts/_default/page-list.json index 507009897..e40823ffa 100644 --- a/layouts/_default/page-list.json +++ b/layouts/_default/page-list.json @@ -8,49 +8,49 @@ {{- if $index -}},{{- end }} { "name": {{ .Name | jsonify }}, - "url": {{ absURL (cond (isset .Params "url") .Params.url .URL) | jsonify }}, + "url": {{ absURL (default .URL .Params.url) | jsonify }}, "children": [ {{- range $index, $entry := .Children -}} {{- if $index -}},{{- end }} { "name": {{ .Name | jsonify }}, - "url": {{ absURL (cond (isset .Params "url") .Params.url .URL) | jsonify }}, + "url": {{ absURL (default .URL .Params.url) | jsonify }}, "children": [ {{- range $index, $entry := .Children -}} {{- if $index -}},{{- end }} { "name": {{ .Name | jsonify }}, - "url": {{ absURL (cond (isset .Params "url") .Params.url .URL) | jsonify }}, + "url": {{ absURL (default .URL .Params.url) | jsonify }}, "children": [ {{- range $index, $entry := .Children -}} {{- if $index -}},{{- end }} { "name": {{ .Name | jsonify }}, - "url": {{ absURL (cond (isset .Params "url") .Params.url .URL) | jsonify }}, + "url": {{ absURL (default .URL .Params.url) | jsonify }}, "children": [ {{- range $index, $entry := .Children -}} {{- if $index -}},{{- end }} { "name": {{ .Name | jsonify }}, - "url": {{ absURL (cond (isset .Params "url") .Params.url .URL) | jsonify }}, + "url": {{ absURL (default .URL .Params.url) | jsonify }}, "children": [ {{- range $index, $entry := .Children -}} {{- if $index -}},{{- end }} { "name": {{ .Name | jsonify }}, - "url": {{ absURL (cond (isset .Params "url") .Params.url .URL) | jsonify }}, + "url": {{ absURL (default .URL .Params.url) | jsonify }}, "children": [ {{- range $index, $entry := .Children -}} {{- if $index -}},{{- end }} { "name": {{ .Name | jsonify }}, - "url": {{ absURL (cond (isset .Params "url") .Params.url .URL) | jsonify }}, + "url": {{ absURL (default .URL .Params.url) | jsonify }}, "children": [ {{- range $index, $entry := .Children -}} {{- if $index -}},{{- end }} { "name": {{ .Name | jsonify }}, - "url": {{ absURL (cond (isset .Params "url") .Params.url .URL) | jsonify }}, + "url": {{ absURL (default .URL .Params.url) | jsonify }}, "children": [ {{ range .Children }} {{ end }} diff --git a/layouts/partials/article/flux-contrib.html b/layouts/partials/article/flux-contrib.html index dbd317e1d..6a0d1c749 100644 --- a/layouts/partials/article/flux-contrib.html +++ b/layouts/partials/article/flux-contrib.html @@ -13,7 +13,7 @@ {{ else if eq .Kind "section" }}
- {{ $packageTitle := cond (isset .Params "list_title") $.Params.list_title .Title }} + {{ $packageTitle := default .Title .Params.list_title }} {{ $packageName := replaceRE `^(.*)( package)` "$1$2" $packageTitle }}

The {{ $packageName | safeHTML }} is a user-contributed package diff --git a/layouts/partials/article/flux-experimental.html b/layouts/partials/article/flux-experimental.html index 3f951abff..9c3b75c50 100644 --- a/layouts/partials/article/flux-experimental.html +++ b/layouts/partials/article/flux-experimental.html @@ -13,7 +13,7 @@

{{ else if eq .Kind "section" }}
- {{ $packageTitle := cond (isset .Params "list_title") $.Params.list_title .Title }} + {{ $packageTitle := default .Title .Params.list_title }} {{ $packageName := replaceRE `^(.*)( package)` "$1$2" $packageTitle }}

The {{ $packageName | safeHTML }} is experimental and subject to change at any time. diff --git a/layouts/partials/article/tags.html b/layouts/partials/article/tags.html index 60b2f48ca..4c297c861 100644 --- a/layouts/partials/article/tags.html +++ b/layouts/partials/article/tags.html @@ -2,9 +2,9 @@ {{ $product := index $productPathData 0 }} {{ $version := index $productPathData 1 }} {{ $tagSet := print $product "/" $version "/tags" }} -{{ if isset .Params $tagSet }} +{{ with .Param $tagSet }}

- {{ range .Param $tagSet }} + {{ range . }} {{ $name := . }} {{ with $.Site.GetPage (printf "/%s/%s" $tagSet ($name | urlize)) }} {{ $name }} diff --git a/layouts/partials/header/title.html b/layouts/partials/header/title.html index e6b1c8910..0ef890d79 100644 --- a/layouts/partials/header/title.html +++ b/layouts/partials/header/title.html @@ -35,10 +35,10 @@ {{ else if eq $currentVersion nil}} {{ $scratch.Set "siteTitle" (print (index .Site.Data.products $product).name " Documentation") }} {{ else }} - {{ if (isset (index .Site.Data.products $product) "altname" ) }} - {{ $scratch.Set "siteTitle" (print (index .Site.Data.products $product).altname " Documentation") }} + {{ with (index .Site.Data.products $product).altname }} + {{ $scratch.Set "siteTitle" (print . " Documentation") }} {{ else }} - {{ $scratch.Set "siteTitle" (print (index .Site.Data.products $product).name " Documentation") }} + {{ $scratch.Set "siteTitle" (print (index $.Site.Data.products $product).name " Documentation") }} {{ end }} {{ end }} diff --git a/layouts/partials/sidebar/nested-menu.html b/layouts/partials/sidebar/nested-menu.html index feceb9ea8..0083b2a3e 100644 --- a/layouts/partials/sidebar/nested-menu.html +++ b/layouts/partials/sidebar/nested-menu.html @@ -4,37 +4,37 @@ {{ range $menu }}