From 735c843ff0fd41f13031849f9ebec88a86449a3a Mon Sep 17 00:00:00 2001 From: Scott Anderson Date: Thu, 17 Apr 2025 08:49:02 -0600 Subject: [PATCH 1/5] InfluxDB OSS and InfluxDB Enterprise 1.12.0 (#5992) * rework v1 configuration docs and links * fixing md lint errors * fix more md lint errors * updates release notes and new functionality * fixed md lint errors * added enterprise 1.12.0 release notes and config updates * fixed release notes for md lint * add influxql updates for oss and enterprise 1.12.0 * Apply suggestions from code review Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * ported new oss v1 config to enterprise v1 --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --- .../v1/about-the-project/release-notes.md | 83 +- .../configure/config-data-nodes.md | 37 + .../v1/query_language/manage-database.md | 76 +- .../v1/query_language/spec.md | 43 +- .../v1/tools/influx_inspect.md | 187 +- .../v1/tools/influxd-ctl/show-shards.md | 6 +- .../v1/about_the_project/release-notes.md | 85 +- .../v1/administration/backup_and_restore.md | 4 +- content/influxdb/v1/administration/config.md | 1572 ++++++++++------- content/influxdb/v1/administration/ports.md | 12 +- .../administration/subscription-management.md | 2 +- .../v1/concepts/file-system-layout.md | 6 +- .../influxdb/v1/concepts/storage_engine.md | 6 +- .../v1/guides/downsample_and_retain.md | 2 +- .../v1/query_language/manage-database.md | 76 +- content/influxdb/v1/query_language/spec.md | 48 +- .../v1/supported_protocols/prometheus.md | 2 +- content/influxdb/v1/tools/api.md | 4 +- content/influxdb/v1/tools/influx_inspect.md | 71 +- content/influxdb/v1/troubleshooting/errors.md | 18 +- .../frequently-asked-questions.md | 96 +- .../tools/measurements-internal.md | 3 + data/products.yml | 6 +- 23 files changed, 1532 insertions(+), 913 deletions(-) diff --git a/content/enterprise_influxdb/v1/about-the-project/release-notes.md b/content/enterprise_influxdb/v1/about-the-project/release-notes.md index 6edaa9e4d..fe2f32ba1 100644 --- a/content/enterprise_influxdb/v1/about-the-project/release-notes.md +++ b/content/enterprise_influxdb/v1/about-the-project/release-notes.md @@ -9,6 +9,60 @@ menu: parent: About the project --- +## v1.12.0 {date="2025-04-15"} + +## Features + +- Add additional log output when using + [`influx_inspect buildtsi`](/enterprise_influxdb/v1/tools/influx_inspect/#buildtsi) to + rebuild the TSI index. +- Use [`influx_inspect export`](/enterprise_influxdb/v1/tools/influx_inspect/#export) with + [`-tsmfile` option](/enterprise_influxdb/v1/tools/influx_inspect/#--tsmfile-tsm_file-) to + export a single TSM file. +- Add `-m` flag to the [`influxd-ctl show-shards` command](/enterprise_influxdb/v1/tools/influxd-ctl/show-shards/) + to output inconsistent shards. +- Allow the specification of a write window for retention policies. +- Add `fluxQueryRespBytes` metric to the `/debug/vars` metrics endpoint. +- Log whenever meta gossip times exceed expiration. +- Add [`query-log-path` configuration option](/enterprise_influxdb/v1/administration/configure/config-data-nodes/#query-log-path) + to data nodes. +- Add [`aggressive-points-per-block` configuration option](/influxdb/v1/administration/config/#aggressive-points-per-block) + to prevent TSM files from not getting fully compacted. +- Log TLS configuration settings on startup. +- Check for TLS certificate and private key permissions. +- Add a warning if the TLS certificate is expired. +- Add authentication to the Raft portal and add the following related _data_ + node configuration options: + - [`[meta].raft-portal-auth-required`](/enterprise_influxdb/v1/administration/configure/config-data-nodes/#raft-portal-auth-required) + - [`[meta].raft-dialer-auth-required`](/enterprise_influxdb/v1/administration/configure/config-data-nodes/#raft-dialer-auth-required) +- Improve error handling. +- InfluxQL updates: + - Delete series by retention policy. + - Allow retention policies to discard writes that fall within their range, but + outside of [`FUTURE LIMIT`](/enterprise_influxdb/v1/query_language/manage-database/#future-limit) + and [`PAST LIMIT`](/enterprise_influxdb/v1/query_language/manage-database/#past-limit). + +## Bug fixes + +- Log rejected writes to subscriptions. +- Update `xxhash` and avoid `stringtoslicebyte` in the cache. +- Prevent a panic when a shard group has no shards. +- Fix file handle leaks in `Compactor.write`. +- Ensure fields in memory match the fields on disk. +- Ensure temporary files are removed after failed compactions. +- Do not panic on invalid multiple subqueries. +- Update the `/shard-status` API to return the correct result and use a + consistent "idleness" definition for shards. + +## Other + +- Update Go to 1.23.5. +- Upgrade Flux to v0.196.1. +- Upgrade InfluxQL to v1.4.1. +- Various other dependency updates. + +--- + {{% note %}} #### InfluxDB Enterprise and FIPS-compliance @@ -21,6 +75,10 @@ InfluxDB Enterprise builds are available. For more information, see ## v1.11.8 {date="2024-11-15"} +### Features + +- Add a startup logger to InfluxDB Enterprise data nodes. + ### Bug Fixes - Strip double quotes from measurement names in the [`/api/v2/delete` compatibility @@ -28,6 +86,8 @@ InfluxDB Enterprise builds are available. For more information, see string comparisons (e.g. to allow special characters in measurement names). - Enable SHA256 for FIPS RPMs. +--- + ## v1.11.7 {date="2024-09-19"} ### Bug Fixes @@ -581,7 +641,7 @@ in that there is no corresponding InfluxDB OSS release. ### Features - Upgrade to Go 1.15.10. -- Support user-defined *node labels*. +- Support user-defined _node labels_. Node labels let you assign arbitrary key-value pairs to meta and data nodes in a cluster. For instance, an operator might want to label nodes with the availability zone in which they're located. - Improve performance of `SHOW SERIES CARDINALITY` and `SHOW SERIES CARDINALITY from ` InfluxQL queries. @@ -756,11 +816,15 @@ For details on changes incorporated from the InfluxDB OSS release, see ### Features -#### **Back up meta data only** +#### Back up meta data only -- Add option to back up **meta data only** (users, roles, databases, continuous queries, and retention policies) using the new `-strategy` flag and `only meta` option: `influx ctl backup -strategy only meta `. +- Add option to back up **meta data only** (users, roles, databases, continuous + queries, and retention policies) using the new `-strategy` flag and `only meta` + option: `influx ctl backup -strategy only meta `. - > **Note:** To restore a meta data backup, use the `restore -full` command and specify your backup manifest: `influxd-ctl restore -full `. + > [!Note] + > To restore a meta data backup, use the `restore -full` command and specify + > your backup manifest: `influxd-ctl restore -full `. For more information, see [Perform a metastore only backup](/enterprise_influxdb/v1/administration/backup-and-restore/#perform-a-metastore-only-backup). @@ -1007,7 +1071,10 @@ The following summarizes the expected settings for proper configuration of JWT a `""`. - A long pass phrase is recommended for better security. ->**Note:** To provide encrypted internode communication, you must enable HTTPS. Although the JWT signature is encrypted, the the payload of a JWT token is encoded, but is not encrypted. +> [!Note] +> To provide encrypted internode communication, you must enable HTTPS. Although +> the JWT signature is encrypted, the the payload of a JWT token is encoded, but +> is not encrypted. ### Bug fixes @@ -1082,8 +1149,10 @@ Please see the [InfluxDB OSS release notes](/influxdb/v1/about_the_project/relea ## v1.5.0 {date="2018-03-06"} -> ***Note:*** This release builds off of the 1.5 release of InfluxDB OSS. Please see the [InfluxDB OSS release -> notes](/influxdb/v1/about_the_project/release-notes/) for more information about the InfluxDB OSS release. +> [!Note] +> This release builds off of the 1.5 release of InfluxDB OSS. +> Please see the [InfluxDB OSS release notes](/influxdb/v1/about_the_project/release-notes/) +> for more information about the InfluxDB OSS release. For highlights of the InfluxDB 1.5 release, see [What's new in InfluxDB 1.5](/influxdb/v1/about_the_project/whats_new/). diff --git a/content/enterprise_influxdb/v1/administration/configure/config-data-nodes.md b/content/enterprise_influxdb/v1/administration/configure/config-data-nodes.md index c45d343f7..ecddbd49c 100644 --- a/content/enterprise_influxdb/v1/administration/configure/config-data-nodes.md +++ b/content/enterprise_influxdb/v1/administration/configure/config-data-nodes.md @@ -259,6 +259,29 @@ For detailed configuration information, see [`meta.ensure-fips`](/enterprise_inf Environment variable: `INFLUXDB_META_ENSURE_FIPS` +#### raft-portal-auth-required {metadata="v1.12.0+"} + +Default is `false`. + +Require Raft clients to authenticate with server using the +[`meta-internal-shared-secret`](#meta-internal-shared-secret). +This requires that all meta nodes are running InfluxDB Enterprise v1.12.0+ and +are configured with the correct `meta-internal-shared-secret`. + +Environment variable: `INFLUXDB_META_RAFT_PORTAL_AUTH_REQUIRED` + +#### raft-dialer-auth-required {metadata="v1.12.0+"} + +Default is `false`. + +Require Raft servers to authenticate Raft clients using the +[`meta-internal-shared-secret`](#meta-internal-shared-secret). +This requires that all meta nodes are running InfluxDB Enterprise v1.12.0+, have +`raft-portal-auth-required=true`, and are configured with the correct +`meta-internal-shared-secret`. + +Environment variable: `INFLUXDB_META_RAFT_DIALER_AUTH_REQUIRED` + ----- ## Data settings @@ -305,6 +328,8 @@ Environment variable: `INFLUXDB_DATA_QUERY_LOG_ENABLED` #### query-log-path +Default is `""`. + An absolute path to the query log file. The default is `""` (queries aren't logged to a file). @@ -326,6 +351,8 @@ The following is an example of a `logrotate` configuration: } ``` +Environment variable: `INFLUXDB_DATA_QUERY_LOG_PATH` + #### wal-fsync-delay Default is `"0s"`. @@ -422,6 +449,16 @@ The duration at which to compact all TSM and TSI files in a shard if it has not Environment variable: `INFLUXDB_DATA_COMPACT_FULL_WRITE_COLD_DURATION` +#### aggressive-points-per-block {metadata="v1.12.0+"} + +Default is `10000`. + +The number of points per block to use during aggressive compaction. There are +certain cases where TSM files do not get fully compacted. This adjusts an +internal parameter to help ensure these files do get fully compacted. + +Environment variable: `INFLUXDB_DATA_AGGRESSIVE_POINTS_PER_BLOCK` + #### index-version Default is `"inmem"`. diff --git a/content/enterprise_influxdb/v1/query_language/manage-database.md b/content/enterprise_influxdb/v1/query_language/manage-database.md index 73d5ce7fc..f6161716b 100644 --- a/content/enterprise_influxdb/v1/query_language/manage-database.md +++ b/content/enterprise_influxdb/v1/query_language/manage-database.md @@ -62,17 +62,22 @@ Creates a new database. #### Syntax ```sql -CREATE DATABASE [WITH [DURATION ] [REPLICATION ] [SHARD DURATION ] [NAME ]] +CREATE DATABASE [WITH [DURATION ] [REPLICATION ] [SHARD DURATION ] [PAST LIMIT ] [FUTURE LIMIT ] [NAME ]] ``` #### Description of syntax `CREATE DATABASE` requires a database [name](/enterprise_influxdb/v1/troubleshooting/frequently-asked-questions/#what-words-and-characters-should-i-avoid-when-writing-data-to-influxdb). -The `WITH`, `DURATION`, `REPLICATION`, `SHARD DURATION`, and `NAME` clauses are optional and create a single [retention policy](/enterprise_influxdb/v1/concepts/glossary/#retention-policy-rp) associated with the created database. -If you do not specify one of the clauses after `WITH`, the relevant behavior defaults to the `autogen` retention policy settings. +The `WITH`, `DURATION`, `REPLICATION`, `SHARD DURATION`, `PAST LIMIT`, +`FUTURE LIMIT, and `NAME` clauses are optional and create a single +[retention policy](/enterprise_influxdb/v1/concepts/glossary/#retention-policy-rp) +associated with the created database. +If you do not specify one of the clauses after `WITH`, the relevant behavior +defaults to the `autogen` retention policy settings. The created retention policy automatically serves as the database's default retention policy. -For more information about those clauses, see [Retention Policy Management](/enterprise_influxdb/v1/query_language/manage-database/#retention-policy-management). +For more information about those clauses, see +[Retention Policy Management](/enterprise_influxdb/v1/query_language/manage-database/#retention-policy-management). A successful `CREATE DATABASE` query returns an empty result. If you attempt to create a database that already exists, InfluxDB does nothing and does not return an error. @@ -122,21 +127,25 @@ The `DROP SERIES` query deletes all points from a [series](/enterprise_influxdb/ and it drops the series from the index. The query takes the following form, where you must specify either the `FROM` clause or the `WHERE` clause: + ```sql DROP SERIES FROM WHERE ='' ``` Drop all series from a single measurement: + ```sql > DROP SERIES FROM "h2o_feet" ``` Drop series with a specific tag pair from a single measurement: + ```sql > DROP SERIES FROM "h2o_feet" WHERE "location" = 'santa_monica' ``` Drop all points in the series that have a specific tag pair from all measurements in the database: + ```sql > DROP SERIES WHERE "location" = 'santa_monica' ``` @@ -152,35 +161,49 @@ Unlike You must include either the `FROM` clause, the `WHERE` clause, or both: -``` +```sql DELETE FROM WHERE [=''] | [