From 500a31cdd928ce4bd85a749c128a205282bc6cee Mon Sep 17 00:00:00 2001 From: Jason Stirnaman Date: Thu, 15 May 2025 15:12:20 -0500 Subject: [PATCH 01/10] feat(oss2): Release notes for OSS 2.7.12, fix broken link, list item indentation, and failing codeblock tests --- .../v2/reference/release-notes/influxdb.md | 75 +++++++++++-------- 1 file changed, 44 insertions(+), 31 deletions(-) diff --git a/content/influxdb/v2/reference/release-notes/influxdb.md b/content/influxdb/v2/reference/release-notes/influxdb.md index 18319bddb..18a3f75cf 100644 --- a/content/influxdb/v2/reference/release-notes/influxdb.md +++ b/content/influxdb/v2/reference/release-notes/influxdb.md @@ -8,6 +8,19 @@ menu: weight: 101 --- +## v2.7.12 {date="2025-05-15"} + +### Features + +- Add a `--pid-file` option to write a PID file to the specified location on startup. InfluxDB removes the PID file on shutdown. +- Add a `--storage-wal-flush-on-shutdown` option to flush the WAL on database shutdown to ensure all data is written to disk. +- Improve logging for dropped points, providing more detailed error messages for partial writes. + +### Maintenance + +- Improved startup logging with an "are we there yet" counter. + + ## v2.7.11 {date="2024-12-02"} ### Features @@ -606,7 +619,7 @@ to migrate InfluxDB key-value metadata schemas to earlier 2.x versions when nece #### Flux - Update to [Flux v0.139.0](/flux/v0/release-notes/#v01390). -- Enable writing to remote hosts using the Flux [`to()`](/flux/v0/stdlib/influxdata/influxdb/to/) and [`experimental.to()`](/flux/v0/v0.x/stdlib/experimental/to/) functions. +- Enable writing to remote hosts using the Flux [`to()`](/flux/v0/stdlib/influxdata/influxdb/to/) and [`experimental.to()`](/flux/v0/stdlib/experimental/to/) functions. - Flux now supports locations that dynamically modify time offsets based on your specified timezone. You can also specify fixed time offsets relative to UTC. - Perform [bitwise operations](/flux/v0/stdlib/experimental/bitwise/) on integers and unsigned integers. @@ -673,24 +686,24 @@ New features include: - Add a new route `/api/v2/resources` that returns a list of known resources to the platform, including the following resource types. Makes it easier to update All Access tokens with current resources: - - `AuthorizationsResourceType` - - `BucketsResourceType` - - `ChecksResourceType` - - `DashboardsResourceType` - - `DBRPResourceType` - - `DocumentsResourceType` - - `LabelsResourceType` - - `NotificationEndpointResourceType` - - `NotificationRuleResourceType` - - `OrgsResourceType` - - `ScraperResourceType` - - `SecretsResourceType` - - `SourcesResourceType` - - `TasksResourceType` - - `TelegrafsResourceType` - - `UsersResourceType` - - `VariablesResourceType` - - `ViewsResourceType` + - `AuthorizationsResourceType` + - `BucketsResourceType` + - `ChecksResourceType` + - `DashboardsResourceType` + - `DBRPResourceType` + - `DocumentsResourceType` + - `LabelsResourceType` + - `NotificationEndpointResourceType` + - `NotificationRuleResourceType` + - `OrgsResourceType` + - `ScraperResourceType` + - `SecretsResourceType` + - `SourcesResourceType` + - `TasksResourceType` + - `TelegrafsResourceType` + - `UsersResourceType` + - `VariablesResourceType` + - `ViewsResourceType` #### Flux updates @@ -992,10 +1005,10 @@ The startup process automatically generates replacement `tsi1` indexes for shard - Fix timeout setup for `influxd` graceful shutdown. - Require user to set password during initial user onboarding. - Error message improvements: - - Remove duplication from task error messages. - - Improve error message shown when influx CLI can't find an `org` by name. - - Improve error message when opening BoltDB with unsupported file system options. - - Improve messages in DBRP API validation errors. + - Remove duplication from task error messages. + - Improve error message shown when influx CLI can't find an `org` by name. + - Improve error message when opening BoltDB with unsupported file system options. + - Improve messages in DBRP API validation errors. - `influxd upgrade` improvements: - Add confirmation step with file sizes before copying data files. - Prevent panic in `influxd upgrade` when v1 users exist but v1 config is missing. @@ -1072,8 +1085,8 @@ Previously, the database retention policy (DBRP) mapping API did not match the s ### Features - Improvements to upgrade from 1.x to 2.x: - - Warning appears if auth is not enabled in 1.x (`auth-enabled = false`), which is not an option in 2.x. For details, see [Upgrade from InfluxDB 1.x to InfluxDB 2.0](/influxdb/v2/upgrade/v1-to-v2/). - - `upgrade` command now checks to see if continuous queries are running and automatically exports them to a local file. + - Warning appears if auth is not enabled in 1.x (`auth-enabled = false`), which is not an option in 2.x. For details, see [Upgrade from InfluxDB 1.x to InfluxDB 2.0](/influxdb/v2/upgrade/v1-to-v2/). + - `upgrade` command now checks to see if continuous queries are running and automatically exports them to a local file. - Upgrade to [Flux v0.95.0](/flux/v0/release-notes/#v0-95-0). - Upgrade `flux-lsp-browser` to v.0.5.23. - Manage database retention policy (DBRP) mappings via CLI. See [`influx v1 dbrp`](/influxdb/v2/reference/cli/influx/v1/dbrp/). @@ -1117,8 +1130,8 @@ When there are multiple [DBRP mappings](/influxdb/v2/reference/api/influxdb-1x/d Highlights include: - Support for **upgrading to InfluxDB 2.0**: - - To upgrade **from InfluxDB 1.x**, see [Upgrade from InfluxDB 1.x to InfluxDB 2.0](/influxdb/v2/upgrade/v1-to-v2). - - To upgrade **from InfluxDB 2.0 beta 16 or earlier**, see [Upgrade from InfluxDB 2.0 beta to InfluxDB 2.0](/influxdb/v2/upgrade/v2-beta-to-v2). + - To upgrade **from InfluxDB 1.x**, see [Upgrade from InfluxDB 1.x to InfluxDB 2.0](/influxdb/v2/upgrade/v1-to-v2). + - To upgrade **from InfluxDB 2.0 beta 16 or earlier**, see [Upgrade from InfluxDB 2.0 beta to InfluxDB 2.0](/influxdb/v2/install/upgrade/v2-beta-to-v2/). - **Flux**, our powerful new functional data scripting language designed for querying, analyzing, and acting on data. This release includes [Flux v0.94.0](/flux/v0/release-notes/#v0940). If you're new to Flux, [check out how to get started with Flux](/influxdb/v2/query-data/get-started/). Next, delve deeper into the [Flux standard library](/flux/v0/stdlib//) reference docs and see how to [query with Flux](/influxdb/v2/query-data/flux/). - Support for [InfluxDB 1.x API compatibility](/influxdb/v2/reference/api/influxdb-1x/). - **Templates** and **stacks**. Discover how to [use community templates](/influxdb/v2/tools/influxdb-templates/use/) and how to [manage templates with stacks](/influxdb/v2/tools/influxdb-templates/stacks/). @@ -1241,14 +1254,14 @@ If you're new to InfluxDB 2.0, we recommend checking out [how to get started](/i {{% warn %}} #### Manual upgrade required -To simplify the migration for existing users of InfluxDB 1.x, this release includes significant breaking changes that require a manual upgrade from all alpha and beta versions. For more information, see [Upgrade to InfluxDB OSS 2.0rc](/influxdb/v2/upgrade/v2-beta-to-v2/), +To simplify the migration for existing users of InfluxDB 1.x, this release includes significant breaking changes that require a manual upgrade from all alpha and beta versions. For more information, see [Upgrade to InfluxDB OSS 2.0rc](/influxdb/v2/install/upgrade/v2-beta-to-v2/), {{% /warn %}} ### Breaking changes #### Manual upgrade -- To continue using data from InfluxDB 2.0 beta 16 or earlier, you must move all existing data out of the `~/.influxdbv2` (or equivalent) path, including `influxd.bolt`. All existing dashboards, tasks, integrations, alerts, users, and tokens must be recreated. For information on how to migrate your data, see [Upgrade to InfluxDB OSS 2.0rc](/influxdb/v2/upgrade/v2-beta-to-v2/). +- To continue using data from InfluxDB 2.0 beta 16 or earlier, you must move all existing data out of the `~/.influxdbv2` (or equivalent) path, including `influxd.bolt`. All existing dashboards, tasks, integrations, alerts, users, and tokens must be recreated. For information on how to migrate your data, see [Upgrade to InfluxDB OSS 2.0rc](/influxdb/v2/install/upgrade/v2-beta-to-v2/). #### Port update to 8086 @@ -2045,7 +2058,7 @@ _**This will remove all data from your InfluxDB v2.0 instance including time ser ###### Linux and macOS ```sh -rm ~/.influxdbv2/influxd.bolt +rm -f ~/.influxdbv2/influxd.bolt ``` Once completed, `v2.0.0-alpha.6` can be started. @@ -2079,7 +2092,7 @@ run the following command. ###### Linux and macOS ```sh -rm -r ~/.influxdbv2/engine +rm -rf ~/.influxdbv2/engine ``` Once completed, InfluxDB v2.0.0-alpha.5 can be started. From 0a1179932fc988f265880eb4a8f0335242258c3b Mon Sep 17 00:00:00 2001 From: Jason Stirnaman Date: Thu, 15 May 2025 15:44:26 -0500 Subject: [PATCH 02/10] feat(oss2): option Part of influxdata/DAR/issues/500 --- .../influxdb/v2/reference/config-options.md | 55 +++++++++++++++++++ 1 file changed, 55 insertions(+) diff --git a/content/influxdb/v2/reference/config-options.md b/content/influxdb/v2/reference/config-options.md index 18f0eadcb..1698e37c6 100644 --- a/content/influxdb/v2/reference/config-options.md +++ b/content/influxdb/v2/reference/config-options.md @@ -2744,6 +2744,61 @@ storage-validate-keys = true --- +### storage-wal-flush-on-shutdown +Flush the WAL on shutdown. + + +**Default:** `false` + +| influxd flag | Environment variable | Configuration key | +| :------------------------------ | :------------------------------------ | :---------------------------- | +| `--storage-wal-flush-on-shutdown` | `INFLUXD_STORAGE_WAL_FLUSH_ON_SHUTDOWN` | `storage-wal-flush-on-shutdown` | + +If set, `influxd` flushes or snapshots all WALs prior to completing shutdown--`influxd` performs cache snapshots on shutdown, which +results in the WAL files being written to TSM files and then deleted. + +This is useful in upgrade and downgrade scenarios to prevent WAL format +compatibility issues. + +###### influxd flag + + +```sh +influxd --storage-wal-flush-on-shutdown +``` + +###### Environment variable +```sh +export INFLUXD_STORAGE_WAL_FLUSH_ON_SHUTDOWN=true +``` + +###### Configuration file +{{< code-tabs-wrapper >}} +{{% code-tabs %}} +[YAML](#) +[TOML](#) +[JSON](#) +{{% /code-tabs %}} +{{% code-tab-content %}} +```yml +storage-wal-flush-on-shutdown: true +``` +{{% /code-tab-content %}} +{{% code-tab-content %}} +```toml +storage-wal-flush-on-shutdown = true +``` +{{% /code-tab-content %}} +{{% code-tab-content %}} +```json +{ + "storage-wal-flush-on-shutdown": true +} +``` +{{% /code-tab-content %}} +{{< /code-tabs-wrapper >}} +--- + ### storage-wal-fsync-delay Duration a write will wait before fsyncing. A duration greater than `0` batches multiple fsync calls. From 79de24bef22b38c9ab67ea6a487e5c7730654b5d Mon Sep 17 00:00:00 2001 From: Jason Stirnaman Date: Thu, 15 May 2025 16:05:43 -0500 Subject: [PATCH 03/10] feat(oss2): Add influxd_flags entry for option Part of influxdata/DAR/issues/500 --- content/influxdb/v2/reference/config-options.md | 2 +- data/influxd_flags.yml | 3 +++ 2 files changed, 4 insertions(+), 1 deletion(-) diff --git a/content/influxdb/v2/reference/config-options.md b/content/influxdb/v2/reference/config-options.md index 1698e37c6..4f639b6ab 100644 --- a/content/influxdb/v2/reference/config-options.md +++ b/content/influxdb/v2/reference/config-options.md @@ -2747,7 +2747,6 @@ storage-validate-keys = true ### storage-wal-flush-on-shutdown Flush the WAL on shutdown. - **Default:** `false` | influxd flag | Environment variable | Configuration key | @@ -2797,6 +2796,7 @@ storage-wal-flush-on-shutdown = true ``` {{% /code-tab-content %}} {{< /code-tabs-wrapper >}} + --- ### storage-wal-fsync-delay diff --git a/data/influxd_flags.yml b/data/influxd_flags.yml index e3e1998de..29f57a2e7 100644 --- a/data/influxd_flags.yml +++ b/data/influxd_flags.yml @@ -161,6 +161,9 @@ - flag: "--storage-validate-keys" added: 2.0 +- flag: "--storage-wal-flush-on-shutdown" + added: 2.7 + - flag: "--storage-wal-fsync-delay" added: 2.0 From a0e627468ed83dfc8fb994c310b1192bda657c82 Mon Sep 17 00:00:00 2001 From: Jason Stirnaman Date: Thu, 15 May 2025 16:43:11 -0500 Subject: [PATCH 04/10] feat(oss2): Maintenance items for OSS 2.7.12 --- content/influxdb/v2/reference/release-notes/influxdb.md | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/content/influxdb/v2/reference/release-notes/influxdb.md b/content/influxdb/v2/reference/release-notes/influxdb.md index 18a3f75cf..69f2830c3 100644 --- a/content/influxdb/v2/reference/release-notes/influxdb.md +++ b/content/influxdb/v2/reference/release-notes/influxdb.md @@ -18,7 +18,10 @@ weight: 101 ### Maintenance -- Improved startup logging with an "are we there yet" counter. +- Improved startup logging with an "are we there yet" counter for the number and percentage of shards opened. +- Update Go to 1.23.9. +- Update Flux to v0.196.1. +- Refresh dependencies to address security vulnerabilities and improve stability. ## v2.7.11 {date="2024-12-02"} From cffe9402cf44803b9c5dc554a58589857f327727 Mon Sep 17 00:00:00 2001 From: Jason Stirnaman Date: Thu, 15 May 2025 17:12:12 -0500 Subject: [PATCH 05/10] feat(oss2): Revise and fix partial write enhancement for OSS 2.7.12 --- content/influxdb/v2/reference/release-notes/influxdb.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/influxdb/v2/reference/release-notes/influxdb.md b/content/influxdb/v2/reference/release-notes/influxdb.md index 69f2830c3..8ad4a0178 100644 --- a/content/influxdb/v2/reference/release-notes/influxdb.md +++ b/content/influxdb/v2/reference/release-notes/influxdb.md @@ -14,7 +14,7 @@ weight: 101 - Add a `--pid-file` option to write a PID file to the specified location on startup. InfluxDB removes the PID file on shutdown. - Add a `--storage-wal-flush-on-shutdown` option to flush the WAL on database shutdown to ensure all data is written to disk. -- Improve logging for dropped points, providing more detailed error messages for partial writes. +- Improve response error messages for dropped points, adding details including database, retention policy, and which bound was violated for partial writes. ### Maintenance From 18b90035a4aa54d8bfe19f65d0c5355de4bf3df6 Mon Sep 17 00:00:00 2001 From: Jason Stirnaman Date: Thu, 15 May 2025 13:46:17 -0500 Subject: [PATCH 06/10] fix: broken links and formatting, missing Management API ReDoc security section for auth types --- .../influxdb3/cloud-dedicated/management/openapi.yml | 1 + api-docs/influxdb3/core/v3/ref.yml | 2 +- api-docs/influxdb3/enterprise/v3/ref.yml | 4 +--- .../core/query-data/execute-queries/influxdb3-cli.md | 2 +- .../query-data/execute-queries/influxdb3-cli.md | 2 +- .../flight/csharp-flight.md | 2 +- .../flight/go-flight.md | 2 +- .../flight/java-flightsql.md | 2 +- .../flight/python-flight.md | 12 +++++------- .../flight/python-flightsql-dbapi.md | 12 ++++++------ .../shared/influxdb3-write-guides/troubleshoot.md | 2 +- 11 files changed, 20 insertions(+), 23 deletions(-) diff --git a/api-docs/influxdb3/cloud-dedicated/management/openapi.yml b/api-docs/influxdb3/cloud-dedicated/management/openapi.yml index ccf316d56..374a9ab72 100644 --- a/api-docs/influxdb3/cloud-dedicated/management/openapi.yml +++ b/api-docs/influxdb3/cloud-dedicated/management/openapi.yml @@ -40,6 +40,7 @@ tags: See how to [create a management token](/influxdb3/cloud-dedicated/admin/tokens/management/). By default, management tokens in InfluxDB 3 are short-lived tokens issued by an OAuth2 identity provider that grant a specific user administrative access to your InfluxDB cluster. However, for automation purposes, you can manually create management tokens that authenticate directly with your InfluxDB cluster and do not require human interaction with your identity provider. + - name: Database tokens description: Manage database read/write tokens for a cluster - name: Databases diff --git a/api-docs/influxdb3/core/v3/ref.yml b/api-docs/influxdb3/core/v3/ref.yml index 7d1ba6a38..0c320aada 100644 --- a/api-docs/influxdb3/core/v3/ref.yml +++ b/api-docs/influxdb3/core/v3/ref.yml @@ -52,7 +52,7 @@ tags: #### Related guides - [Manage tokens](/influxdb3/core/admin/tokens/) - - [Authentication and authorization](/influxdb3/core/reference/authentication/) + - [Authentication and authorization](/influxdb3/core/reference/internals/authentication/) x-traitTag: true - name: Cache data description: | diff --git a/api-docs/influxdb3/enterprise/v3/ref.yml b/api-docs/influxdb3/enterprise/v3/ref.yml index 8950419b0..a02ae5865 100644 --- a/api-docs/influxdb3/enterprise/v3/ref.yml +++ b/api-docs/influxdb3/enterprise/v3/ref.yml @@ -52,7 +52,7 @@ tags: #### Related guides - [Manage tokens](/influxdb3/enterprise/admin/tokens/) - - [Authentication and authorization](/influxdb3/enterprise/reference/authentication/) + - [Authentication and authorization](/influxdb3/enterprise/reference/internals/authentication/) x-traitTag: true - name: Cache data description: | @@ -1936,8 +1936,6 @@ components: scheme: bearer bearerFormat: JWT description: | - _During Alpha release, an API token is not required._ - A Bearer token for authentication. Provide the scheme and the API token in the `Authorization` header--for example: diff --git a/content/influxdb3/core/query-data/execute-queries/influxdb3-cli.md b/content/influxdb3/core/query-data/execute-queries/influxdb3-cli.md index 43222ff08..af911f486 100644 --- a/content/influxdb3/core/query-data/execute-queries/influxdb3-cli.md +++ b/content/influxdb3/core/query-data/execute-queries/influxdb3-cli.md @@ -13,7 +13,7 @@ related: - /influxdb3/core/reference/cli/influxdb3/query/ - /influxdb3/core/reference/sql/ - /influxdb3/core/reference/influxql/ - # - /influxdb3/core/get-started/query/#execute-an-sql-query, Get started querying data + # - /influxdb3/core/query-data/execute-queries/, Get started querying data list_code_example: | ```sh influxdb3 query \ diff --git a/content/influxdb3/enterprise/query-data/execute-queries/influxdb3-cli.md b/content/influxdb3/enterprise/query-data/execute-queries/influxdb3-cli.md index 8b19fdc9c..e3540b906 100644 --- a/content/influxdb3/enterprise/query-data/execute-queries/influxdb3-cli.md +++ b/content/influxdb3/enterprise/query-data/execute-queries/influxdb3-cli.md @@ -13,7 +13,7 @@ related: - /influxdb3/enterprise/reference/cli/influxdb3/query/ - /influxdb3/enterprise/reference/sql/ - /influxdb3/enterprise/reference/influxql/ - # - /influxdb3/enterprise/get-started/query/#execute-an-sql-query, Get started querying data + # - /influxdb3/enterprise/query-data/execute-queries/, Get started querying data list_code_example: | ```sh influxdb3 query \ diff --git a/content/shared/influxdb-client-libraries-reference/flight/csharp-flight.md b/content/shared/influxdb-client-libraries-reference/flight/csharp-flight.md index 1163b3a2e..ad0984cfd 100644 --- a/content/shared/influxdb-client-libraries-reference/flight/csharp-flight.md +++ b/content/shared/influxdb-client-libraries-reference/flight/csharp-flight.md @@ -8,5 +8,5 @@ For more information, see the [C# client example on GitHub](https://github.com/a > We recommend using the [`influxdb3-csharp` C# client library](/influxdb3/version/reference/client-libraries/v3/csharp/) for integrating InfluxDB 3 with your C# application code. > > [InfluxDB 3 client libraries](/influxdb3/version/reference/client-libraries/v3/) wrap Apache Arrow Flight clients -> and provide convenient methods for [writing](/influxdb3/version/get-started/write/#write-line-protocol-to-influxdb), [querying](/influxdb3/version/get-started/query/#execute-an-sql-query), and processing data stored in {{% product-name %}}. +> and provide convenient methods for [writing](/influxdb3/version/write-data/api-client-libraries/), [querying](/influxdb3/version/query-data/execute-queries/), and processing data stored in {{% product-name %}}. > Client libraries can query using SQL or InfluxQL. diff --git a/content/shared/influxdb-client-libraries-reference/flight/go-flight.md b/content/shared/influxdb-client-libraries-reference/flight/go-flight.md index cc1b97109..93e588f95 100644 --- a/content/shared/influxdb-client-libraries-reference/flight/go-flight.md +++ b/content/shared/influxdb-client-libraries-reference/flight/go-flight.md @@ -6,7 +6,7 @@ > We recommend using the [`influxdb3-go` Go client library](/influxdb3/version/reference/client-libraries/v3/go/) for integrating InfluxDB 3 with your Go application code. > > [InfluxDB 3 client libraries](/influxdb3/version/reference/client-libraries/v3/) wrap Apache Arrow Flight clients -> and provide convenient methods for [writing](/influxdb3/version/get-started/write/#write-line-protocol-to-influxdb), [querying](/influxdb3/version/get-started/query/#execute-an-sql-query), and processing data stored in {{% product-name %}}. +> and provide convenient methods for [writing](/influxdb3/version/write-data/api-client-libraries/), [querying](/influxdb3/version/query-data/execute-queries/), and processing data stored in {{% product-name %}}. > Client libraries can query using SQL or InfluxQL. ## Flight SQL client diff --git a/content/shared/influxdb-client-libraries-reference/flight/java-flightsql.md b/content/shared/influxdb-client-libraries-reference/flight/java-flightsql.md index b0a57d52e..e29b31da5 100644 --- a/content/shared/influxdb-client-libraries-reference/flight/java-flightsql.md +++ b/content/shared/influxdb-client-libraries-reference/flight/java-flightsql.md @@ -6,7 +6,7 @@ > We recommend using the [`influxdb3-java` Java client library](/influxdb3/version/reference/client-libraries/v3/java/) for integrating InfluxDB 3 with your Java application code. > > [InfluxDB 3 client libraries](/influxdb3/version/reference/client-libraries/v3/) wrap Apache Arrow Flight clients -> and provide convenient methods for [writing](/influxdb3/version/get-started/write/#write-line-protocol-to-influxdb), [querying](/influxdb3/version/get-started/query/#execute-an-sql-query), and processing data stored in {{% product-name %}}. +> and provide convenient methods for [writing](/influxdb3/version/write-data/api-client-libraries/), [querying](/influxdb3/version/query-data/execute-queries/), and processing data stored in {{% product-name %}}. > Client libraries can query using SQL or InfluxQL. diff --git a/content/shared/influxdb-client-libraries-reference/flight/python-flight.md b/content/shared/influxdb-client-libraries-reference/flight/python-flight.md index 5fa77472c..207f9bec7 100644 --- a/content/shared/influxdb-client-libraries-reference/flight/python-flight.md +++ b/content/shared/influxdb-client-libraries-reference/flight/python-flight.md @@ -6,21 +6,20 @@ > We recommend using the [`influxdb3-python` Python client library](/influxdb3/version/reference/client-libraries/v3/python/) for integrating InfluxDB 3 with your Python application code. > > [InfluxDB 3 client libraries](/influxdb3/version/reference/client-libraries/v3/) wrap Apache Arrow Flight clients -> and provide convenient methods for [writing](/influxdb3/version/get-started/write/#write-line-protocol-to-influxdb), [querying](/influxdb3/version/get-started/query/#execute-an-sql-query), and processing data stored in {{% product-name %}}. +> and provide convenient methods for [writing](/influxdb3/version/write-data/api-client-libraries/), [querying](/influxdb3/version/query-data/execute-queries/), and processing data stored in {{% product-name %}}. > Client libraries can query using SQL or InfluxQL. The following examples show how to use the `pyarrow.flight` and `pandas` Python modules to query and format data stored in an {{% product-name %}} database: -{{% code-tabs-wrapper %}} +{{< code-tabs-wrapper >}} {{% code-tabs %}} [SQL](#sql-python) [InfluxQL](#influxql-python) {{% /code-tabs %}} - {{% code-tab-content %}} {{% code-placeholders "DATABASE_NAME|DATABASE_TOKEN" %}} -```py +```python # Using pyarrow>=12.0.0 FlightClient from pyarrow.flight import FlightClient, Ticket, FlightCallOptions import json @@ -62,7 +61,7 @@ print(data_frame.to_markdown()) {{% code-tab-content %}} {{% code-placeholders "DATABASE_NAME|DATABASE_TOKEN" %}} -```py +```python # Using pyarrow>=12.0.0 FlightClient from pyarrow.flight import FlightClient, Ticket, FlightCallOptions import json @@ -97,6 +96,7 @@ print(data_frame.to_markdown()) {{% /code-placeholders %}} {{% /code-tab-content %}} +{{< /code-tabs-wrapper >}} Replace the following: @@ -104,5 +104,3 @@ Replace the following: - {{% code-placeholder-key %}}`DATABASE_TOKEN`{{% /code-placeholder-key %}}: a [database token](/influxdb3/version/admin/tokens/database/) with sufficient permissions to the specified database - -{{% /code-tabs-wrapper %}} diff --git a/content/shared/influxdb-client-libraries-reference/flight/python-flightsql-dbapi.md b/content/shared/influxdb-client-libraries-reference/flight/python-flightsql-dbapi.md index 2350510c5..61d75e8da 100644 --- a/content/shared/influxdb-client-libraries-reference/flight/python-flightsql-dbapi.md +++ b/content/shared/influxdb-client-libraries-reference/flight/python-flightsql-dbapi.md @@ -6,7 +6,7 @@ The [Python `flightsql-dbapi` Flight SQL DBAPI library](https://github.com/influ > We recommend using the [`influxdb3-python` Python client library](/influxdb3/version/reference/client-libraries/v3/python/) for integrating InfluxDB 3 with your Python application code. > > [InfluxDB 3 client libraries](/influxdb3/version/reference/client-libraries/v3/) wrap Apache Arrow Flight clients -> and provide convenient methods for [writing](/influxdb3/version/get-started/write/#write-line-protocol-to-influxdb), [querying](/influxdb3/version/get-started/query/#execute-an-sql-query), and processing data stored in {{% product-name %}}. +> and provide convenient methods for [writing](/influxdb3/version/write-data/api-client-libraries/), [querying](/influxdb3/version/query-data/execute-queries/), and processing data stored in {{% product-name %}}. > Client libraries can query using SQL or InfluxQL. ## Installation @@ -32,7 +32,7 @@ from flightsql import FlightSQLClient ``` - `flightsql.FlightSQLClient` class: an interface for [initializing -a client](#initialization) and interacting with a Flight SQL server. +a client](#initialize-a-client) and interacting with a Flight SQL server. ## API reference @@ -41,11 +41,11 @@ a client](#initialization) and interacting with a Flight SQL server. - [Initialize a client](#initialize-a-client) - [Instance methods](#instance-methods) - [FlightSQLClient.execute](#flightsqlclientexecute) - - [Syntax {#execute-query-syntax}](#syntax-execute-query-syntax) - - [Example {#execute-query-example}](#example-execute-query-example) + - [Syntax](#execute-query-syntax) + - [Example](#execute-query-example) - [FlightSQLClient.do_get](#flightsqlclientdo_get) - - [Syntax {#retrieve-data-syntax}](#syntax-retrieve-data-syntax) - - [Example {#retrieve-data-example}](#example-retrieve-data-example) + - [Syntax](#retrieve-data-syntax) + - [Example](#retrieve-data-example) ## Class FlightSQLClient diff --git a/content/shared/influxdb3-write-guides/troubleshoot.md b/content/shared/influxdb3-write-guides/troubleshoot.md index 1db4781c5..3d9691d2d 100644 --- a/content/shared/influxdb3-write-guides/troubleshoot.md +++ b/content/shared/influxdb3-write-guides/troubleshoot.md @@ -41,7 +41,7 @@ Write requests return the following status codes: | :-------------------------------| :--------------------------------------------------------------- | :------------- | | `204 "Success"` | | If InfluxDB ingested the data | | `400 "Bad request"` | error details about rejected points, up to 100 points: `line` contains the first rejected line, `message` describes rejections | If some or all request data isn't allowed (for example, if it is malformed or falls outside of the bucket's retention period)--the response body indicates whether a partial write has occurred or if all data has been rejected | -| `401 "Unauthorized"` | | If the `Authorization` header is missing or malformed or if the [token](/influxdb3/version/admin/tokens/) doesn't have [permission](/influxdb3/version/reference/cli/influxctl/token/create/#examples) to write to the database. See [examples using credentials](/influxdb3/version/get-started/write/#write-line-protocol-to-influxdb) in write requests. | +| `401 "Unauthorized"` | | If the `Authorization` header is missing or malformed or if the [token](/influxdb3/version/admin/tokens/) doesn't have [permission](/influxdb3/version/reference/cli/influxctl/token/create/#examples) to write to the database. See [examples using credentials](/influxdb3/version/write-data/api-client-libraries/) in write requests. | | `404 "Not found"` | requested **resource type** (for example, "organization" or "database"), and **resource name** | If a requested resource (for example, organization or database) wasn't found | | `500 "Internal server error"` | | Default status for an error | | `503` "Service unavailable" | | If the server is temporarily unavailable to accept writes. The `Retry-After` header describes when to try the write again. From 50a9d7c796cb8b9cf8913b66595547403f448cc7 Mon Sep 17 00:00:00 2001 From: Jason Stirnaman Date: Tue, 20 May 2025 16:23:40 -0500 Subject: [PATCH 07/10] feat(oss2): 2.7.12 release notes: include bug fix, bump version --- content/influxdb/v2/reference/release-notes/influxdb.md | 7 ++++++- data/products.yml | 2 +- 2 files changed, 7 insertions(+), 2 deletions(-) diff --git a/content/influxdb/v2/reference/release-notes/influxdb.md b/content/influxdb/v2/reference/release-notes/influxdb.md index 8ad4a0178..9633439d6 100644 --- a/content/influxdb/v2/reference/release-notes/influxdb.md +++ b/content/influxdb/v2/reference/release-notes/influxdb.md @@ -8,7 +8,7 @@ menu: weight: 101 --- -## v2.7.12 {date="2025-05-15"} +## v2.7.12 {date="2025-05-20"} ### Features @@ -16,6 +16,10 @@ weight: 101 - Add a `--storage-wal-flush-on-shutdown` option to flush the WAL on database shutdown to ensure all data is written to disk. - Improve response error messages for dropped points, adding details including database, retention policy, and which bound was violated for partial writes. +### Bug Fixes + +- Fix a locking issue in `TagValueIterator` that could cause reads and writes in buckets to block. [PR #26414](https://github.com/influxdata/influxdb/pull/26414) + ### Maintenance - Improved startup logging with an "are we there yet" counter for the number and percentage of shards opened. @@ -23,6 +27,7 @@ weight: 101 - Update Flux to v0.196.1. - Refresh dependencies to address security vulnerabilities and improve stability. +--- ## v2.7.11 {date="2024-12-02"} diff --git a/data/products.yml b/data/products.yml index 47c2d258e..d427e7ffc 100644 --- a/data/products.yml +++ b/data/products.yml @@ -85,7 +85,7 @@ influxdb: - v1 latest: v2.7 latest_patches: - v2: 2.7.11 + v2: 2.7.12 v1: 1.11.8 latest_cli: v2: 2.7.5 From e8e69d980471a6ef76244e473e42946490252448 Mon Sep 17 00:00:00 2001 From: peterbarnett03 Date: Wed, 21 May 2025 09:14:22 -0400 Subject: [PATCH 08/10] fix: clarify that a plugin filename is needed --- .../shared/influxdb3-cli/create/trigger.md | 33 ++++++++++++------- 1 file changed, 21 insertions(+), 12 deletions(-) diff --git a/content/shared/influxdb3-cli/create/trigger.md b/content/shared/influxdb3-cli/create/trigger.md index ea4f25da9..55134ff82 100644 --- a/content/shared/influxdb3-cli/create/trigger.md +++ b/content/shared/influxdb3-cli/create/trigger.md @@ -10,6 +10,7 @@ processing engine. influxdb3 create trigger [OPTIONS] \ --database \ --token \ + --plugin-filename \ --trigger-spec \ ``` @@ -20,16 +21,21 @@ influxdb3 create trigger [OPTIONS] \ ## Options -| Option | | Description | -| :----- | :--------------- | :--------------------------------------------------------------------------------------- | -| `-H` | `--host` | Host URL of the running {{< product-name >}} server (default is `http://127.0.0.1:8181`) | -| `-d` | `--database` | _({{< req >}})_ Name of the database to operate on | -| | `--token` | _({{< req >}})_ Authentication token | -| | `--trigger-spec` | Trigger specification--for example `table:` or `all_tables` | -| | `--disabled` | Create the trigger in disabled state | -| | `--tls-ca` | Path to a custom TLS certificate authority (for testing or self-signed certificates) | -| `-h` | `--help` | Print help information | -| | `--help-all` | Print detailed help information | +| Option | | Description | +| :----- | :------------------ | :------------------------------------------------------------------------------------------------------- | +| `-H` | `--host` | Host URL of the running {{< product-name >}} server (default is `http://127.0.0.1:8181`) | +| `-d` | `--database` | _({{< req >}})_ Name of the database to operate on | +| | `--token` | _({{< req >}})_ Authentication token | +| | `--plugin-filename` | _({{< req >}})_ Python file name of the file on the server's plugin-dir containing the plugin code. | +| | `--trigger-spec` | Trigger specification--for example `table:` or `all_tables` | +| | `--disabled` | Create the trigger in disabled state | +| | `--tls-ca` | Path to a custom TLS certificate authority (for testing or self-signed certificates) | +| `-h` | `--help` | Print help information | +| | `--help-all` | Print detailed help information | + +If you want to use a plugin from the [Plugin Library](https://github.com/influxdata/influxdb3_plugins) repo, use the url path with `gh:` specified as the prefix. +For example, to use the [System Metrics](https://github.com/influxdata/influxdb3_plugins/blob/main/examples/schedule/system_metrics/system_metrics.py) plugin, the plugin filename is `gh:examples/schedule/system_metrics/system_metrics.py`. + ### Option environment variables @@ -47,8 +53,8 @@ The following examples show how to use the `influxdb3 create trigger` command to - {{% code-placeholder-key %}}`DATABASE_NAME`{{% /code-placeholder-key %}}: Database name -- {{% code-placeholder-key %}}`AUTH_TOKEN`{{% /code-placeholder-key %}}: -Authentication token +- {{% code-placeholder-key %}}`AUTH_TOKEN`{{% /code-placeholder-key %}}: Authentication token +- {{% code-placeholder-key %}}`PLUGIN_FILENAME`{{% /code-placeholder-key %}}: Python plugin filename - {{% code-placeholder-key %}}`TRIGGER_NAME`{{% /code-placeholder-key %}}: Name of the trigger to create - {{% code-placeholder-key %}}`TABLE_NAME`{{% /code-placeholder-key %}}: @@ -66,6 +72,7 @@ Create a trigger that processes data from a specific table. influxdb3 create trigger \ --database DATABASE_NAME \ --token AUTH_TOKEN \ + --plugin-filename PLUGIN_FILENAME \ --trigger-spec table:TABLE_NAME \ TRIGGER_NAME ``` @@ -80,6 +87,7 @@ Create a trigger that applies to all tables in the specified database. influxdb3 create trigger \ --database DATABASE_NAME \ --token AUTH_TOKEN \ + --plugin-filename \ --trigger-spec all_tables \ TRIGGER_NAME ``` @@ -97,6 +105,7 @@ influxdb3 create trigger \ --disabled \ --database DATABASE_NAME \ --token AUTH_TOKEN \ + --plugin-filename \ --trigger-spec table:TABLE_NAME \ TRIGGER_NAME ``` From 630002bed1ed8fcda1420f26c20491e7a6714337 Mon Sep 17 00:00:00 2001 From: Jason Stirnaman Date: Wed, 21 May 2025 10:48:34 -0500 Subject: [PATCH 09/10] Update content/shared/influxdb3-cli/create/trigger.md --- content/shared/influxdb3-cli/create/trigger.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/shared/influxdb3-cli/create/trigger.md b/content/shared/influxdb3-cli/create/trigger.md index 55134ff82..dbb223128 100644 --- a/content/shared/influxdb3-cli/create/trigger.md +++ b/content/shared/influxdb3-cli/create/trigger.md @@ -26,7 +26,7 @@ influxdb3 create trigger [OPTIONS] \ | `-H` | `--host` | Host URL of the running {{< product-name >}} server (default is `http://127.0.0.1:8181`) | | `-d` | `--database` | _({{< req >}})_ Name of the database to operate on | | | `--token` | _({{< req >}})_ Authentication token | -| | `--plugin-filename` | _({{< req >}})_ Python file name of the file on the server's plugin-dir containing the plugin code. | +| | `--plugin-filename` | _({{< req >}})_ Name of the file, stored in the server's `plugin-dir`, that contains the Python plugin code to run | | | `--trigger-spec` | Trigger specification--for example `table:` or `all_tables` | | | `--disabled` | Create the trigger in disabled state | | | `--tls-ca` | Path to a custom TLS certificate authority (for testing or self-signed certificates) | From 950cb50a5be2a0e8f2432d946cc1730c68486399 Mon Sep 17 00:00:00 2001 From: Scott Anderson Date: Thu, 22 May 2025 12:57:50 -0500 Subject: [PATCH 10/10] hotfix: update v2 download links (#6085) --- assets/jsconfig.json | 3 ++- content/influxdb/v2/install/_index.md | 26 +++++++++++++------------- 2 files changed, 15 insertions(+), 14 deletions(-) diff --git a/assets/jsconfig.json b/assets/jsconfig.json index 377218ccb..4ad710c10 100644 --- a/assets/jsconfig.json +++ b/assets/jsconfig.json @@ -3,7 +3,8 @@ "baseUrl": ".", "paths": { "*": [ - "*" + "*", + "../node_modules/*" ] } } diff --git a/content/influxdb/v2/install/_index.md b/content/influxdb/v2/install/_index.md index c27d355f4..f7af4d919 100644 --- a/content/influxdb/v2/install/_index.md +++ b/content/influxdb/v2/install/_index.md @@ -164,13 +164,13 @@ gpg: key 7C3D57159FC2F927: public key "InfluxData Package Signing Key }}_darwin_amd64.tar.gz" \ + "https://download.influxdata.com/influxdb/releases/v{{< latest-patch >}}/influxdb2-{{< latest-patch >}}_darwin_amd64.tar.gz" \ ``` --> ```sh curl --silent --location \ - https://download.influxdata.com/influxdb/releases/influxdb2-{{< latest-patch >}}_darwin_amd64.tar.gz.asc \ + https://download.influxdata.com/influxdb/releases/v{{< latest-patch >}}/influxdb2-{{< latest-patch >}}_darwin_amd64.tar.gz.asc \ | gpg --verify - ~/Downloads/influxdb2-{{< latest-patch >}}_darwin_amd64.tar.gz \ 2>&1 | grep 'InfluxData Package Signing Key ' ``` @@ -239,12 +239,12 @@ brew install influxdb 1. In your browser or your terminal, download the InfluxDB package. - InfluxDB v2 (macOS) + InfluxDB v2 (macOS) ```sh # Download using cURL curl --location -O \ - "https://download.influxdata.com/influxdb/releases/influxdb2-{{< latest-patch >}}_darwin_amd64.tar.gz" + "https://download.influxdata.com/influxdb/releases/v{{< latest-patch >}}/influxdb2-{{< latest-patch >}}_darwin_amd64.tar.gz" ``` 2. {{< req text="Recommended:" color="magenta" >}}: Verify the integrity of the download--for example, enter the @@ -443,18 +443,18 @@ _If necessary, adjust the example file paths and utilities for your system._ 1. In your browser or your terminal, download the InfluxDB binary for your system architecture (AMD64 or ARM). - InfluxDB v2 (amd64) - InfluxDB v2 (arm) + InfluxDB v2 (amd64) + InfluxDB v2 (arm) @@ -463,7 +463,7 @@ _If necessary, adjust the example file paths and utilities for your system._ ```sh # Use curl to download the amd64 binary. curl --location -O \ - https://download.influxdata.com/influxdb/releases/influxdb2-{{< latest-patch >}}_linux_amd64.tar.gz + https://download.influxdata.com/influxdb/releases/v{{< latest-patch >}}/influxdb2-{{< latest-patch >}}_linux_amd64.tar.gz ``` @@ -471,7 +471,7 @@ _If necessary, adjust the example file paths and utilities for your system._ ```sh # Use curl to download the arm64 binary. curl --location -O \ - https://download.influxdata.com/influxdb/releases/influxdb2-{{< latest-patch >}}_linux_arm64.tar.gz + https://download.influxdata.com/influxdb/releases/v{{< latest-patch >}}/influxdb2-{{< latest-patch >}}_linux_arm64.tar.gz ``` 2. [Choose the InfluxData key-pair for your OS version](#choose-the-influxdata-key-pair-for-your-system). @@ -505,7 +505,7 @@ _If necessary, adjust the example file paths and utilities for your system._ | grep 'InfluxData Package Signing Key ' \ && # Download and verify the binary's signature file - curl --silent --location "https://download.influxdata.com/influxdb/releases/influxdb2-{{< latest-patch >}}_linux_amd64.tar.gz.asc" \ + curl --silent --location "https://download.influxdata.com/influxdb/releases/v{{< latest-patch >}}/influxdb2-{{< latest-patch >}}_linux_amd64.tar.gz.asc" \ | gpg --verify - influxdb2-{{< latest-patch >}}_linux_amd64.tar.gz \ 2>&1 | grep 'InfluxData Package Signing Key ' ``` @@ -519,7 +519,7 @@ _If necessary, adjust the example file paths and utilities for your system._ | grep 'InfluxData Package Signing Key ' \ && # Download and verify the binary's signature file - curl --silent --location "https://download.influxdata.com/influxdb/releases/influxdb2-{{< latest-patch >}}_linux_arm64.tar.gz.asc" \ + curl --silent --location "https://download.influxdata.com/influxdb/releases/v{{< latest-patch >}}/influxdb2-{{< latest-patch >}}_linux_arm64.tar.gz.asc" \ | gpg --verify - influxdb2-{{< latest-patch >}}_linux_arm64.tar.gz \ 2>&1 | grep 'InfluxData Package Signing Key ' ``` @@ -618,7 +618,7 @@ chmod 0750 ~/.influxdbv2 > > _You'll install the `influx CLI` in a [later step](#download-install-and-configure-the-influx-cli)._ -InfluxDB v2 (Windows) +InfluxDB v2 (Windows) Expand the downloaded archive into `C:\Program Files\InfluxData\` and rename the files if desired.