From 14276e516fa1cf42470737227024bfce0131203e Mon Sep 17 00:00:00 2001 From: peterbarnett03 Date: Tue, 30 Sep 2025 10:20:07 -0400 Subject: [PATCH] chore: add 3.5 release notes and config options (#6428) * chore: add 3.5 release notes and config options * docs(plugins): enhance --plugin-repo documentation with usage examples - Add comprehensive plugin-repo configuration documentation - Document custom repository setup with practical examples - Add Option 3 for custom plugin repositories in usage guide - Include use cases for private repos, air-gapped environments, and development - Cross-reference between config options and usage guide - Add note about custom repositories in setup section * Update content/shared/influxdb3-cli/config-options.md Co-authored-by: Jason Stirnaman --------- Co-authored-by: Jason Stirnaman --- .../shared/influxdb3-cli/config-options.md | 38 ++++++++++++++ content/shared/influxdb3-plugins/_index.md | 46 ++++++++++++++++- .../_index.md | 50 +++++++++++++++++-- 3 files changed, 129 insertions(+), 5 deletions(-) diff --git a/content/shared/influxdb3-cli/config-options.md b/content/shared/influxdb3-cli/config-options.md index 214a21a5e..8180b7f34 100644 --- a/content/shared/influxdb3-cli/config-options.md +++ b/content/shared/influxdb3-cli/config-options.md @@ -152,6 +152,7 @@ influxdb3 serve - [query-file-limit](#query-file-limit) - [Processing Engine](#processing-engine) - [plugin-dir](#plugin-dir) + - [plugin-repo](#plugin-repo) - [virtual-env-location](#virtual-env-location) - [package-manager](#package-manager) {{% show-in "enterprise" %}} @@ -1743,6 +1744,7 @@ the following side-effects: ### Processing Engine - [plugin-dir](#plugin-dir) +- [plugin-repo](#plugin-repo) - [virtual-env-location](#virtual-env-location) - [package-manager](#package-manager) @@ -1756,6 +1758,42 @@ Specifies the local directory that contains Python plugins and their test files. --- +#### plugin-repo + +Specifies the base URL of the remote repository used when referencing plugins with the `gh:` prefix. +When you create a trigger with a plugin filename starting with `gh:`, InfluxDB fetches +the plugin code from this repository URL. + +The URL construction automatically handles trailing slashes—both formats work identically: +- `https://example.com/plugins/` (with trailing slash) +- `https://example.com/plugins` (without trailing slash) + +**Default:** The official InfluxDB 3 plugins repository at `https://raw.githubusercontent.com/influxdata/influxdb3_plugins/main/` + +| influxdb3 serve option | Environment variable | +| :--------------------- | :---------------------- | +| `--plugin-repo` | `INFLUXDB3_PLUGIN_REPO` | + +##### Example usage + +```bash +# Use a custom organization repository +influxdb3 serve \ + --plugin-dir ~/.plugins \ + --plugin-repo "https://raw.githubusercontent.com/myorg/influxdb-plugins/main/" + +# Use an internal mirror +influxdb3 serve \ + --plugin-dir ~/.plugins \ + --plugin-repo "https://internal.company.com/influxdb-plugins/" + +# Set via environment variable +export INFLUXDB3_PLUGIN_REPO="https://custom-repo.example.com/plugins/" +influxdb3 serve --plugin-dir ~/.plugins +``` + +--- + #### virtual-env-location Specifies the location of the Python virtual environment that the processing diff --git a/content/shared/influxdb3-plugins/_index.md b/content/shared/influxdb3-plugins/_index.md index ea832675b..14dace282 100644 --- a/content/shared/influxdb3-plugins/_index.md +++ b/content/shared/influxdb3-plugins/_index.md @@ -32,7 +32,7 @@ Once you have all the prerequisites in place, follow these steps to implement th ## Set up the Processing Engine -To activate the Processing Engine, start your {{% product-name %}} server with the `--plugin-dir` flag. This flag tells InfluxDB where to load your plugin files. +To activate the Processing Engine, start your {{% product-name %}} server with the `--plugin-dir` flag. This flag tells InfluxDB where to load your plugin files. {{% code-placeholders "NODE_ID|OBJECT_STORE_TYPE|PLUGIN_DIR" %}} @@ -51,6 +51,14 @@ In the example above, replace the following: - {{% code-placeholder-key %}}`OBJECT_STORE_TYPE`{{% /code-placeholder-key %}}: Type of object store (for example, file or s3) - {{% code-placeholder-key %}}`PLUGIN_DIR`{{% /code-placeholder-key %}}: Absolute path to the directory where plugin files are stored. Store all plugin files in this directory or its subdirectories. +> [!Note] +> #### Use custom plugin repositories +> +> By default, plugins referenced with the `gh:` prefix are fetched from the official +> [influxdata/influxdb3_plugins](https://github.com/influxdata/influxdb3_plugins) repository. +> To use a custom repository, add the `--plugin-repo` flag when starting the server. +> See [Use a custom plugin repository](#option-3-use-a-custom-plugin-repository) for details. + ### Configure distributed environments When running {{% product-name %}} in a distributed setup, follow these steps to configure the Processing Engine: @@ -132,6 +140,42 @@ This approach: - Simplifies updates and maintenance - Reduces local storage requirements +##### Option 3: Use a custom plugin repository + +For organizations that maintain their own plugin repositories or need to use private/internal plugins, +configure a custom plugin repository URL: + +```bash +# Start the server with a custom plugin repository +influxdb3 serve \ + --node-id node0 \ + --object-store file \ + --data-dir ~/.influxdb3 \ + --plugin-dir ~/.plugins \ + --plugin-repo "https://internal.company.com/influxdb-plugins/" +``` + +Then reference plugins from your custom repository using the `gh:` prefix: + +```bash +# Fetches from: https://internal.company.com/influxdb-plugins/myorg/custom_plugin.py +influxdb3 create trigger \ + --trigger-spec "every:5m" \ + --plugin-filename "gh:myorg/custom_plugin.py" \ + --database my_database \ + custom_trigger +``` + +**Use cases for custom repositories:** + +- **Private plugins**: Host proprietary plugins not suitable for public repositories +- **Air-gapped environments**: Use internal mirrors when external internet access is restricted +- **Development and staging**: Test plugins from development branches before production deployment +- **Compliance requirements**: Meet data governance policies requiring internal hosting + +The `--plugin-repo` option accepts any HTTP/HTTPS URL that serves raw plugin files. +See the [plugin-repo configuration option](/influxdb3/version/reference/config-options/#plugin-repo) for more details. + Plugins have various functions such as: - Receive plugin-specific arguments (such as written data, call time, or an HTTP request) diff --git a/content/shared/v3-core-enterprise-release-notes/_index.md b/content/shared/v3-core-enterprise-release-notes/_index.md index f44f75c88..ab3d0eed4 100644 --- a/content/shared/v3-core-enterprise-release-notes/_index.md +++ b/content/shared/v3-core-enterprise-release-notes/_index.md @@ -5,13 +5,55 @@ > All updates to Core are automatically included in Enterprise. > The Enterprise sections below only list updates exclusive to Enterprise. -## v3.4.2 {date="2025-09-11"} +## v3.5.0 {date="2025-09-30"} ### Core #### Features -- No new features in this release +- **Custom Plugin Repository**: + - Use the `--plugin-repo` option with `influxdb3 serve` to specify custom plugin repositories. This enables loading plugins from personal repos or disabling remote repo access. + +#### Bug fixes + +- **Database reliability**: + - Table index updates now complete atomically before creating new indices, preventing race conditions that could corrupt database state ([#26838](https://github.com/influxdata/influxdb/pull/26838)) + - Delete operations are now idempotent, preventing errors during object store cleanup ([#26839](https://github.com/influxdata/influxdb/pull/26839)) +- **Write path**: + - Write operations to soft-deleted databases are now rejected, preventing data loss ([#26722](https://github.com/influxdata/influxdb/pull/26722)) +- **Runtime stability**: + - Fixed a compatibility issue that could cause deadlocks for concurrent operations ([#26804](https://github.com/influxdata/influxdb/pull/26804)) +- Other bug fixes and performance improvements + +#### Security & Misc + +- Sensitive environment variable values are now hidden in CLI output and log messages ([#26837](https://github.com/influxdata/influxdb/pull/26837)) + +### Enterprise + +All Core updates are included in Enterprise. Additional Enterprise-specific features and fixes: + +#### Features + +- **Cache optimization**: + - Last Value Cache (LVC) and Distinct Value Cache (DVC) now populate on creation and only on query nodes, reducing resource usage on ingest nodes. + +#### Bug fixes + +- **Object store reliability**: + - Object store operations now use retryable mechanisms with better error handling + +#### Operational improvements + +- **Compaction optimizations**: + - Compaction producer now waits 10 seconds before starting cycles, reducing resource contention during startup + - Enhanced scheduling algorithms distribute compaction work more efficiently across available resources +- **System tables**: + - System tables now provide consistent data across different node modes (ingest, query, compact), enabling better monitoring in multi-node deployments + +## v3.4.2 {date="2025-09-11"} + +### Core #### Bug fixes @@ -267,8 +309,8 @@ All Core updates are included in Enterprise. Additional Enterprise-specific feat - Tokens can now be granted `CREATE` permission for creating databases #### Additional Updates -- Last value caches populate on creation and reload on restart -- Distinct value caches populate on creation and reload on restart +- Last value caches reload on restart +- Distinct value caches reload on restart - Other performance improvements - Replaces remaining "INFLUXDB_IOX" Dockerfile environment variables with the following: - `ENV INFLUXDB3_OBJECT_STORE=file`