From 09d1414e22c7c03dca196cfeb61d0fb363e303ff Mon Sep 17 00:00:00 2001 From: Scott Anderson Date: Mon, 28 Jul 2025 09:47:31 -0600 Subject: [PATCH] feat: InfluxDB OSS and Enterprise 1.12.1 (#6250) * InfluxDB OSS and Enterprise 1.12.1 * add message to enterprise 1.12 release notes * Update content/influxdb/v1/query_language/manage-database.md Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Apply suggestions from code review Co-authored-by: Jason Stirnaman * Apply suggestions from code review Co-authored-by: Jason Stirnaman * Apply suggestions from code review Co-authored-by: Jason Stirnaman * fix: update to address PR feedback --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: Jason Stirnaman --- .../v1/about-the-project/release-notes.md | 178 +- .../configure/config-data-nodes.md | 37 + .../v1/query_language/manage-database.md | 77 +- .../v1/query_language/spec.md | 43 +- .../v1/tools/influx_inspect.md | 221 ++- .../v1/tools/influxd-ctl/show-shards.md | 4 + .../frequently-asked-questions.md | 32 +- .../v1/about_the_project/release-notes.md | 498 +++--- .../v1/administration/backup_and_restore.md | 4 +- content/influxdb/v1/administration/config.md | 1572 ++++++++++------- content/influxdb/v1/administration/ports.md | 14 +- .../administration/subscription-management.md | 2 +- .../v1/concepts/file-system-layout.md | 6 +- .../influxdb/v1/concepts/storage_engine.md | 6 +- .../v1/guides/downsample_and_retain.md | 2 +- .../v1/query_language/manage-database.md | 76 +- content/influxdb/v1/query_language/spec.md | 112 +- .../v1/supported_protocols/prometheus.md | 20 +- content/influxdb/v1/tools/api.md | 4 +- content/influxdb/v1/tools/influx_inspect.md | 77 +- content/influxdb/v1/troubleshooting/errors.md | 23 +- .../frequently-asked-questions.md | 100 +- .../tools/measurements-internal.md | 3 + data/products.yml | 6 +- 24 files changed, 1895 insertions(+), 1222 deletions(-) diff --git a/content/enterprise_influxdb/v1/about-the-project/release-notes.md b/content/enterprise_influxdb/v1/about-the-project/release-notes.md index 6edaa9e4d..aae7eecd2 100644 --- a/content/enterprise_influxdb/v1/about-the-project/release-notes.md +++ b/content/enterprise_influxdb/v1/about-the-project/release-notes.md @@ -9,18 +9,81 @@ menu: parent: About the project --- -{{% note %}} -#### InfluxDB Enterprise and FIPS-compliance +## v1.12.1 {date="2025-06-26"} -**InfluxDB Enterprise 1.11+** introduces builds that are compliant with -[Federal Information Processing Standards (FIPS)](https://www.nist.gov/standardsgov/compliance-faqs-federal-information-processing-standards-fips) -and adhere to a strict set of security standards. Both standard and FIPS-compliant -InfluxDB Enterprise builds are available. For more information, see -[FIPS-compliant InfluxDB Enterprise builds](/enterprise_influxdb/v1/introduction/installation/fips-compliant/). -{{% /note %}} +> [!Important] +> #### Upgrade meta nodes first +> +> When upgrading to InfluxDB Enterprise 1.12.1+, upgrade meta nodes before +> upgrading data nodes. + +## Features + +- Add additional log output when using + [`influx_inspect buildtsi`](/enterprise_influxdb/v1/tools/influx_inspect/#buildtsi) to + rebuild the TSI index. +- Use [`influx_inspect export`](/enterprise_influxdb/v1/tools/influx_inspect/#export) with + [`-tsmfile` option](/enterprise_influxdb/v1/tools/influx_inspect/#--tsmfile-tsm_file-) to + export a single TSM file. +- Add `-m` flag to the [`influxd-ctl show-shards` command](/enterprise_influxdb/v1/tools/influxd-ctl/show-shards/) + to output inconsistent shards. +- Allow the specification of a write window for retention policies. +- Add `fluxQueryRespBytes` metric to the `/debug/vars` metrics endpoint. +- Log whenever meta gossip times exceed expiration. +- Add [`query-log-path` configuration option](/enterprise_influxdb/v1/administration/configure/config-data-nodes/#query-log-path) + to data nodes. +- Add [`aggressive-points-per-block` configuration option](/influxdb/v1/administration/config/#aggressive-points-per-block) + to prevent TSM files from not getting fully compacted. +- Log TLS configuration settings on startup. +- Check for TLS certificate and private key permissions. +- Add a warning if the TLS certificate is expired. +- Add authentication to the Raft portal and add the following related _data_ + node configuration options: + - [`[meta].raft-portal-auth-required`](/enterprise_influxdb/v1/administration/configure/config-data-nodes/#raft-portal-auth-required) + - [`[meta].raft-dialer-auth-required`](/enterprise_influxdb/v1/administration/configure/config-data-nodes/#raft-dialer-auth-required) +- Improve error handling. +- InfluxQL updates: + - Delete series by retention policy. + - Allow retention policies to discard writes that fall within their range, but + outside of [`FUTURE LIMIT`](/enterprise_influxdb/v1/query_language/manage-database/#future-limit) + and [`PAST LIMIT`](/enterprise_influxdb/v1/query_language/manage-database/#past-limit). + +## Bug fixes + +- Log rejected writes to subscriptions. +- Update `xxhash` and avoid `stringtoslicebyte` in the cache. +- Prevent a panic when a shard group has no shards. +- Fix file handle leaks in `Compactor.write`. +- Ensure fields in memory match the fields on disk. +- Ensure temporary files are removed after failed compactions. +- Do not panic on invalid multiple subqueries. +- Update the `/shard-status` API to return the correct result and use a + consistent "idleness" definition for shards. + +## Other + +- Update Go to 1.23.5. +- Upgrade Flux to v0.196.1. +- Upgrade InfluxQL to v1.4.1. +- Various other dependency updates. + +--- + +> [!Note] +> #### InfluxDB Enterprise and FIPS-compliance +> +> **InfluxDB Enterprise 1.11+** introduces builds that are compliant with +> [Federal Information Processing Standards (FIPS)](https://www.nist.gov/standardsgov/compliance-faqs-federal-information-processing-standards-fips) +> and adhere to a strict set of security standards. Both standard and FIPS-compliant +> InfluxDB Enterprise builds are available. For more information, see +> [FIPS-compliant InfluxDB Enterprise builds](/enterprise_influxdb/v1/introduction/installation/fips-compliant/). ## v1.11.8 {date="2024-11-15"} +### Features + +- Add a startup logger to InfluxDB Enterprise data nodes. + ### Bug Fixes - Strip double quotes from measurement names in the [`/api/v2/delete` compatibility @@ -28,6 +91,8 @@ InfluxDB Enterprise builds are available. For more information, see string comparisons (e.g. to allow special characters in measurement names). - Enable SHA256 for FIPS RPMs. +--- + ## v1.11.7 {date="2024-09-19"} ### Bug Fixes @@ -79,14 +144,13 @@ InfluxDB Enterprise builds are available. For more information, see ## v1.11.5 {date="2024-02-14"} -{{% note %}} -#### Upgrading from InfluxDB Enterprise v1.11.3 - -If upgrading from InfluxDB Enterprise v1.11.3+ to {{< latest-patch >}}, you can -now configure whether or not InfluxDB compacts series files on startup using the -[`compact-series-file` configuration option](/enterprise_influxdb/v1/administration/configure/config-data-nodes/#compact-series-file) -in your [InfluxDB Enterprise data node configuration file](/enterprise_influxdb/v1/administration/configure/config-data-nodes/). -{{% /note %}} +> [!Note] +> #### Upgrading from InfluxDB Enterprise v1.11.3 +> +> If upgrading from InfluxDB Enterprise v1.11.3+ to {{< latest-patch >}}, you can +> now configure whether or not InfluxDB compacts series files on startup using the +> [`compact-series-file` configuration option](/enterprise_influxdb/v1/administration/configure/config-data-nodes/#compact-series-file) +> in your [InfluxDB Enterprise data node configuration file](/enterprise_influxdb/v1/administration/configure/config-data-nodes/). ### Bug Fixes @@ -101,29 +165,28 @@ in your [InfluxDB Enterprise data node configuration file](/enterprise_influxdb/ ## v1.11.4 {date="2023-12-14"} -{{% note %}} -#### Series file compaction - -With InfluxDB Enterprise v1.11.4+, InfluxDB can be configured to optionally -[compact series files](/enterprise_influxdb/v1/tools/influx_inspect/#--compact-series-file-) -before data nodes are started. -Series files are stored in `_series` directories inside the -[InfluxDB data directory](/enterprise_influxdb/v1/concepts/file-system-layout/#data-node-file-system-layout). -Default: `/var/lib/data//_series`. - -To compact series files on startup, set the [`compact-series-file` configuration option](/enterprise_influxdb/v1/administration/configure/config-data-nodes/#compact-series-file) -to `true` in your [InfluxDB Enterprise data node configuration file](/enterprise_influxdb/v1/administration/configure/config-data-nodes/). - -- If any series files are corrupt, the `influx_inspect` or `influxd` processes on - the data node may fail to start. In both cases, delete the series file - directories before restarting the database. InfluxDB automatically - regenerates the necessary series directories and files when restarting. -- To check if series files are corrupt before starting the database, run the - [`influx_inspect verify-seriesfile` command](/enterprise_influxdb/v1/tools/influx_inspect/#verify-seriesfile) - while the database is off-line. -- If series files are large (20+ gigabytes), it may be faster to delete the - series file directories before starting the database. -{{% /note %}} +> [!Note] +> #### Series file compaction +> +> With InfluxDB Enterprise v1.11.4+, InfluxDB can be configured to optionally +> [compact series files](/enterprise_influxdb/v1/tools/influx_inspect/#--compact-series-file-) +> before data nodes are started. +> Series files are stored in `_series` directories inside the +> [InfluxDB data directory](/enterprise_influxdb/v1/concepts/file-system-layout/#data-node-file-system-layout). +> Default: `/var/lib/data//_series`. +> +> To compact series files on startup, set the [`compact-series-file` configuration option](/enterprise_influxdb/v1/administration/configure/config-data-nodes/#compact-series-file) +> to `true` in your [InfluxDB Enterprise data node configuration file](/enterprise_influxdb/v1/administration/configure/config-data-nodes/). +> +> - If any series files are corrupt, the `influx_inspect` or `influxd` processes on +> the data node may fail to start. In both cases, delete the series file +> directories before restarting the database. InfluxDB automatically +> regenerates the necessary series directories and files when restarting. +> - To check if series files are corrupt before starting the database, run the +> [`influx_inspect verify-seriesfile` command](/enterprise_influxdb/v1/tools/influx_inspect/#verify-seriesfile) +> while the database is off-line. +> - If series files are large (20+ gigabytes), it may be faster to delete the +> series file directories before starting the database. ### Bug Fixes @@ -448,8 +511,10 @@ An edge case regression was introduced into this version that may cause a consta ## v1.9.6 {date="2022-02-16"} -{{% note %}} InfluxDB Enterprise offerings are no longer available on AWS, Azure, and GCP marketplaces. Please [contact Sales](https://www.influxdata.com/contact-sales/) to request an license key to [install InfluxDB Enterprise in your own environment](/enterprise_influxdb/v1/introduction/installation/). -{{% /note %}} +> [!Note] +> InfluxDB Enterprise offerings are no longer available on AWS, Azure, and GCP +> marketplaces. Please [contact Sales](https://www.influxdata.com/contact-sales/) +> to request an license key to [install InfluxDB Enterprise in your own environment](/enterprise_influxdb/v1/introduction/installation/). ### Features @@ -495,10 +560,9 @@ An edge case regression was introduced into this version that may cause a consta ## v1.9.5 {date="2021-10-11"} -{{% note %}} -InfluxDB Enterprise 1.9.4 was not released. -Changes below are included in InfluxDB Enterprise 1.9.5. -{{% /note %}} +> [!Note] +> InfluxDB Enterprise 1.9.4 was not released. +> Changes below are included in InfluxDB Enterprise 1.9.5. ### Features @@ -581,7 +645,7 @@ in that there is no corresponding InfluxDB OSS release. ### Features - Upgrade to Go 1.15.10. -- Support user-defined *node labels*. +- Support user-defined _node labels_. Node labels let you assign arbitrary key-value pairs to meta and data nodes in a cluster. For instance, an operator might want to label nodes with the availability zone in which they're located. - Improve performance of `SHOW SERIES CARDINALITY` and `SHOW SERIES CARDINALITY from ` InfluxQL queries. @@ -646,10 +710,9 @@ in that there is no corresponding InfluxDB OSS release. Instead, use [`inch`](https://github.com/influxdata/inch) or [`influx-stress`](https://github.com/influxdata/influx-stress) (not to be confused with `influx_stress`). -{{% note %}} -**Note:** InfluxDB Enterprise 1.9.0 and 1.9.1 were not released. -Bug fixes intended for 1.9.0 and 1.9.1 were rolled into InfluxDB Enterprise 1.9.2. -{{% /note %}} +> [!Note] +> InfluxDB Enterprise 1.9.0 and 1.9.1 were not released. +> Bug fixes intended for 1.9.0 and 1.9.1 were rolled into InfluxDB Enterprise 1.9.2. --- @@ -756,11 +819,15 @@ For details on changes incorporated from the InfluxDB OSS release, see ### Features -#### **Back up meta data only** +#### Back up meta data only -- Add option to back up **meta data only** (users, roles, databases, continuous queries, and retention policies) using the new `-strategy` flag and `only meta` option: `influx ctl backup -strategy only meta `. +- Add option to back up **meta data only** (users, roles, databases, continuous + queries, and retention policies) using the new `-strategy` flag and `only meta` + option: `influx ctl backup -strategy only meta `. - > **Note:** To restore a meta data backup, use the `restore -full` command and specify your backup manifest: `influxd-ctl restore -full `. + > [!Note] + > To restore a meta data backup, use the `restore -full` command and specify + > your backup manifest: `influxd-ctl restore -full `. For more information, see [Perform a metastore only backup](/enterprise_influxdb/v1/administration/backup-and-restore/#perform-a-metastore-only-backup). @@ -1007,7 +1074,10 @@ The following summarizes the expected settings for proper configuration of JWT a `""`. - A long pass phrase is recommended for better security. ->**Note:** To provide encrypted internode communication, you must enable HTTPS. Although the JWT signature is encrypted, the the payload of a JWT token is encoded, but is not encrypted. +> [!Note] +> To provide encrypted internode communication, you must enable HTTPS. Although +> the JWT signature is encrypted, the the payload of a JWT token is encoded, but +> is not encrypted. ### Bug fixes diff --git a/content/enterprise_influxdb/v1/administration/configure/config-data-nodes.md b/content/enterprise_influxdb/v1/administration/configure/config-data-nodes.md index c45d343f7..ecddbd49c 100644 --- a/content/enterprise_influxdb/v1/administration/configure/config-data-nodes.md +++ b/content/enterprise_influxdb/v1/administration/configure/config-data-nodes.md @@ -259,6 +259,29 @@ For detailed configuration information, see [`meta.ensure-fips`](/enterprise_inf Environment variable: `INFLUXDB_META_ENSURE_FIPS` +#### raft-portal-auth-required {metadata="v1.12.0+"} + +Default is `false`. + +Require Raft clients to authenticate with server using the +[`meta-internal-shared-secret`](#meta-internal-shared-secret). +This requires that all meta nodes are running InfluxDB Enterprise v1.12.0+ and +are configured with the correct `meta-internal-shared-secret`. + +Environment variable: `INFLUXDB_META_RAFT_PORTAL_AUTH_REQUIRED` + +#### raft-dialer-auth-required {metadata="v1.12.0+"} + +Default is `false`. + +Require Raft servers to authenticate Raft clients using the +[`meta-internal-shared-secret`](#meta-internal-shared-secret). +This requires that all meta nodes are running InfluxDB Enterprise v1.12.0+, have +`raft-portal-auth-required=true`, and are configured with the correct +`meta-internal-shared-secret`. + +Environment variable: `INFLUXDB_META_RAFT_DIALER_AUTH_REQUIRED` + ----- ## Data settings @@ -305,6 +328,8 @@ Environment variable: `INFLUXDB_DATA_QUERY_LOG_ENABLED` #### query-log-path +Default is `""`. + An absolute path to the query log file. The default is `""` (queries aren't logged to a file). @@ -326,6 +351,8 @@ The following is an example of a `logrotate` configuration: } ``` +Environment variable: `INFLUXDB_DATA_QUERY_LOG_PATH` + #### wal-fsync-delay Default is `"0s"`. @@ -422,6 +449,16 @@ The duration at which to compact all TSM and TSI files in a shard if it has not Environment variable: `INFLUXDB_DATA_COMPACT_FULL_WRITE_COLD_DURATION` +#### aggressive-points-per-block {metadata="v1.12.0+"} + +Default is `10000`. + +The number of points per block to use during aggressive compaction. There are +certain cases where TSM files do not get fully compacted. This adjusts an +internal parameter to help ensure these files do get fully compacted. + +Environment variable: `INFLUXDB_DATA_AGGRESSIVE_POINTS_PER_BLOCK` + #### index-version Default is `"inmem"`. diff --git a/content/enterprise_influxdb/v1/query_language/manage-database.md b/content/enterprise_influxdb/v1/query_language/manage-database.md index 73d5ce7fc..c70c1cb52 100644 --- a/content/enterprise_influxdb/v1/query_language/manage-database.md +++ b/content/enterprise_influxdb/v1/query_language/manage-database.md @@ -62,17 +62,22 @@ Creates a new database. #### Syntax ```sql -CREATE DATABASE [WITH [DURATION ] [REPLICATION ] [SHARD DURATION ] [NAME ]] +CREATE DATABASE [WITH [DURATION ] [REPLICATION ] [SHARD DURATION ] [PAST LIMIT ] [FUTURE LIMIT ] [NAME ]] ``` #### Description of syntax `CREATE DATABASE` requires a database [name](/enterprise_influxdb/v1/troubleshooting/frequently-asked-questions/#what-words-and-characters-should-i-avoid-when-writing-data-to-influxdb). -The `WITH`, `DURATION`, `REPLICATION`, `SHARD DURATION`, and `NAME` clauses are optional and create a single [retention policy](/enterprise_influxdb/v1/concepts/glossary/#retention-policy-rp) associated with the created database. -If you do not specify one of the clauses after `WITH`, the relevant behavior defaults to the `autogen` retention policy settings. +The `WITH`, `DURATION`, `REPLICATION`, `SHARD DURATION`, `PAST LIMIT`, +`FUTURE LIMIT, and `NAME` clauses are optional and create a single +[retention policy](/enterprise_influxdb/v1/concepts/glossary/#retention-policy-rp) +associated with the created database. +If you do not specify one of the clauses after `WITH`, the relevant behavior +defaults to the `autogen` retention policy settings. The created retention policy automatically serves as the database's default retention policy. -For more information about those clauses, see [Retention Policy Management](/enterprise_influxdb/v1/query_language/manage-database/#retention-policy-management). +For more information about those clauses, see +[Retention Policy Management](/enterprise_influxdb/v1/query_language/manage-database/#retention-policy-management). A successful `CREATE DATABASE` query returns an empty result. If you attempt to create a database that already exists, InfluxDB does nothing and does not return an error. @@ -122,21 +127,25 @@ The `DROP SERIES` query deletes all points from a [series](/enterprise_influxdb/ and it drops the series from the index. The query takes the following form, where you must specify either the `FROM` clause or the `WHERE` clause: + ```sql DROP SERIES FROM WHERE ='' ``` Drop all series from a single measurement: + ```sql > DROP SERIES FROM "h2o_feet" ``` Drop series with a specific tag pair from a single measurement: + ```sql > DROP SERIES FROM "h2o_feet" WHERE "location" = 'santa_monica' ``` Drop all points in the series that have a specific tag pair from all measurements in the database: + ```sql > DROP SERIES WHERE "location" = 'santa_monica' ``` @@ -152,35 +161,48 @@ Unlike You must include either the `FROM` clause, the `WHERE` clause, or both: -``` +```sql DELETE FROM WHERE [=''] | [