From a712d5bbebd0106e702e78a6039b535947a4c8c1 Mon Sep 17 00:00:00 2001 From: Jason Stirnaman Date: Thu, 26 Jun 2025 11:07:06 -0500 Subject: [PATCH 01/16] feat(monolith): Core and Ent v3.2.0 release notes and generation script: - Add v3.2.0 release notes for Core and Enterprise 3- Add a script to help analyze the diff between tags and generate release notes in a standard format following style guidelines --- .../_index.md | 50 +++++++ helper-scripts/generate-release-notes.sh | 135 ++++++++++++++++++ 2 files changed, 185 insertions(+) create mode 100755 helper-scripts/generate-release-notes.sh diff --git a/content/shared/v3-core-enterprise-release-notes/_index.md b/content/shared/v3-core-enterprise-release-notes/_index.md index ff46e3d03..f3f59bf05 100644 --- a/content/shared/v3-core-enterprise-release-notes/_index.md +++ b/content/shared/v3-core-enterprise-release-notes/_index.md @@ -5,6 +5,56 @@ > All updates to Core are automatically included in Enterprise. > The Enterprise sections below only list updates exclusive to Enterprise. +## v3.2.0 {date="2025-06-25"} +**Core**: revision 1ca3168bee + +**Enterprise**: revision 1ca3168bee + +### Core + +#### Features + +- **Hard delete for databases and tables**: Permanently delete databases and tables, enabling complete removal of data structures for compliance and storage management ([#26553](https://github.com/influxdata/influxdb/pull/26553)) +- **AWS credentials auto-reload**: Support dynamic reloading of ephemeral AWS credentials from files, improving security and reliability when using AWS services ([#26537](https://github.com/influxdata/influxdb/pull/26537)) +- **Database retention period support**: Add retention period support for databases via CLI commands (`create database` and `update database` commands) and HTTP APIs ([#26520](https://github.com/influxdata/influxdb/pull/26520)) +- **Configurable lookback duration**: Users can specify lookback duration for PersistedFiles buffer, providing better control over query performance ([#26528](https://github.com/influxdata/influxdb/pull/26528)) +- **WAL replay concurrency control**: Add concurrency limits for WAL (Write-Ahead Log) replay to improve startup performance and resource management ([#26483](https://github.com/influxdata/influxdb/pull/26483)) +- **Enhanced write path**: Separate write path executor with unbounded memory for improved write performance ([#26455](https://github.com/influxdata/influxdb/pull/26455)) + +#### Bug Fixes + +- **WAL corruption handling**: Handle corrupt WAL files during replay without panic, improving data recovery and system resilience ([#26556](https://github.com/influxdata/influxdb/pull/26556)) +- **Database naming validation**: Disallow underscores in database names when created via API to ensure consistency ([#26507](https://github.com/influxdata/influxdb/pull/26507)) +- **Object store cleanup**: Automatic intermediate directory cleanup for file object store, preventing storage bloat ([#26480](https://github.com/influxdata/influxdb/pull/26480)) + +#### Additional Updates + +- Track generation 1 duration in catalog for better performance monitoring ([#26508](https://github.com/influxdata/influxdb/pull/26508)) +- Add retention period support to the catalog ([#26479](https://github.com/influxdata/influxdb/pull/26479)) +- Update help text for improved user experience ([#26509](https://github.com/influxdata/influxdb/pull/26509)) + +### Enterprise + +All Core updates are included in Enterprise. Additional Enterprise-specific features and fixes: + +#### Features + +- **License management improvements**: + - New `influxdb3 show license` command to display current license information +- **Table-level retention period support**: Add retention period support for individual tables in addition to database-level retention, providing granular data lifecycle management + - New CLI commands: `create table --retention-period` and `update table --retention-period` + - Set or clear table-specific retention policies independent of database settings +- **Compaction improvements**: + - Address compactor restart issues for better reliability + - Track compacted generation durations in catalog for monitoring + - Disable parquet cache for ingest mode to optimize memory usage + +#### Bug Fixes + +- **Query optimization**: Correctly partition query chunks into generations for improved performance +- **Data integrity**: Don't delete generation 1 files as part of compaction process +- **License handling**: Trim whitespace from license file contents after reading to prevent validation issues + ## v3.1.0 {date="2025-05-29"} **Core**: revision 482dd8aac580c04f37e8713a8fffae89ae8bc264 diff --git a/helper-scripts/generate-release-notes.sh b/helper-scripts/generate-release-notes.sh new file mode 100755 index 000000000..58f34212e --- /dev/null +++ b/helper-scripts/generate-release-notes.sh @@ -0,0 +1,135 @@ +#!/bin/bash + +# Script to generate release notes for InfluxDB v3.x releases +# Usage: ./generate-release-notes.sh + +set -e + +# Default values +REPO_PATH="${3:-/Users/ja/Documents/github/influxdb}" +FROM_VERSION="${1:-v3.1.0}" +TO_VERSION="${2:-v3.2.0}" + +# Colors for output +RED='\033[0;31m' +GREEN='\033[0;32m' +YELLOW='\033[0;33m' +BLUE='\033[0;34m' +NC='\033[0m' # No Color + +echo -e "${BLUE}Generating release notes for ${TO_VERSION}${NC}" +echo -e "Repository: ${REPO_PATH}" +echo -e "From: ${FROM_VERSION} To: ${TO_VERSION}\n" + +# Function to extract PR number from commit message +extract_pr_number() { + echo "$1" | grep -oE '#[0-9]+' | head -1 | sed 's/#//' +} + +# Get the release date +RELEASE_DATE=$(git -C "$REPO_PATH" log -1 --format=%ai "$TO_VERSION" | cut -d' ' -f1) +echo -e "${GREEN}Release Date: ${RELEASE_DATE}${NC}\n" + +# Collect commits by category +echo -e "${YELLOW}Analyzing commits...${NC}" + +# Features +echo -e "\n${GREEN}Features:${NC}" +FEATURES=$(git -C "$REPO_PATH" log --format="%h %s" "${FROM_VERSION}..${TO_VERSION}" | grep -E "^[a-f0-9]+ feat:" | sed 's/^[a-f0-9]* feat: //') + +# Fixes +echo -e "\n${GREEN}Bug Fixes:${NC}" +FIXES=$(git -C "$REPO_PATH" log --format="%h %s" "${FROM_VERSION}..${TO_VERSION}" | grep -E "^[a-f0-9]+ fix:" | sed 's/^[a-f0-9]* fix: //') + +# Breaking changes +echo -e "\n${GREEN}Breaking Changes:${NC}" +BREAKING=$(git -C "$REPO_PATH" log --format="%h %s" "${FROM_VERSION}..${TO_VERSION}" | grep -iE "^[a-f0-9]+ .*(BREAKING|breaking change)" | sed 's/^[a-f0-9]* //') + +# Performance improvements +echo -e "\n${GREEN}Performance:${NC}" +PERF=$(git -C "$REPO_PATH" log --format="%h %s" "${FROM_VERSION}..${TO_VERSION}" | grep -E "^[a-f0-9]+ perf:" | sed 's/^[a-f0-9]* perf: //') + +# Generate markdown output +OUTPUT_FILE="release-notes-${TO_VERSION}.md" +cat > "$OUTPUT_FILE" << EOF +## ${TO_VERSION} {date="${RELEASE_DATE}"} + +### Features + +EOF + +# Add features +if [ -n "$FEATURES" ]; then + while IFS= read -r line; do + PR=$(extract_pr_number "$line") + # Clean up the commit message + CLEAN_LINE=$(echo "$line" | sed -E 's/ \(#[0-9]+\)$//') + if [ -n "$PR" ]; then + echo "- $CLEAN_LINE ([#$PR](https://github.com/influxdata/influxdb/pull/$PR))" >> "$OUTPUT_FILE" + else + echo "- $CLEAN_LINE" >> "$OUTPUT_FILE" + fi + done <<< "$FEATURES" +else + echo "- No new features in this release" >> "$OUTPUT_FILE" +fi + +# Add bug fixes +cat >> "$OUTPUT_FILE" << EOF + +### Bug Fixes + +EOF + +if [ -n "$FIXES" ]; then + while IFS= read -r line; do + PR=$(extract_pr_number "$line") + CLEAN_LINE=$(echo "$line" | sed -E 's/ \(#[0-9]+\)$//') + if [ -n "$PR" ]; then + echo "- $CLEAN_LINE ([#$PR](https://github.com/influxdata/influxdb/pull/$PR))" >> "$OUTPUT_FILE" + else + echo "- $CLEAN_LINE" >> "$OUTPUT_FILE" + fi + done <<< "$FIXES" +else + echo "- No bug fixes in this release" >> "$OUTPUT_FILE" +fi + +# Add breaking changes if any +if [ -n "$BREAKING" ]; then + cat >> "$OUTPUT_FILE" << EOF + +### Breaking Changes + +EOF + while IFS= read -r line; do + PR=$(extract_pr_number "$line") + CLEAN_LINE=$(echo "$line" | sed -E 's/ \(#[0-9]+\)$//') + if [ -n "$PR" ]; then + echo "- $CLEAN_LINE ([#$PR](https://github.com/influxdata/influxdb/pull/$PR))" >> "$OUTPUT_FILE" + else + echo "- $CLEAN_LINE" >> "$OUTPUT_FILE" + fi + done <<< "$BREAKING" +fi + +# Add performance improvements if any +if [ -n "$PERF" ]; then + cat >> "$OUTPUT_FILE" << EOF + +### Performance Improvements + +EOF + while IFS= read -r line; do + PR=$(extract_pr_number "$line") + CLEAN_LINE=$(echo "$line" | sed -E 's/ \(#[0-9]+\)$//') + if [ -n "$PR" ]; then + echo "- $CLEAN_LINE ([#$PR](https://github.com/influxdata/influxdb/pull/$PR))" >> "$OUTPUT_FILE" + else + echo "- $CLEAN_LINE" >> "$OUTPUT_FILE" + fi + done <<< "$PERF" +fi + +echo -e "\n${GREEN}Release notes generated in: ${OUTPUT_FILE}${NC}" +echo -e "${YELLOW}Please review and edit the generated notes before adding to documentation.${NC}" \ No newline at end of file From 5e0c6e7e20c42b5d32343af2937749aa3950be7a Mon Sep 17 00:00:00 2001 From: Jason Stirnaman Date: Thu, 26 Jun 2025 11:50:05 -0500 Subject: [PATCH 02/16] Apply suggestions from code review Co-authored-by: Scott Anderson --- content/shared/v3-core-enterprise-release-notes/_index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/shared/v3-core-enterprise-release-notes/_index.md b/content/shared/v3-core-enterprise-release-notes/_index.md index f3f59bf05..a526a14db 100644 --- a/content/shared/v3-core-enterprise-release-notes/_index.md +++ b/content/shared/v3-core-enterprise-release-notes/_index.md @@ -6,8 +6,8 @@ > The Enterprise sections below only list updates exclusive to Enterprise. ## v3.2.0 {date="2025-06-25"} -**Core**: revision 1ca3168bee +**Core**: revision 1ca3168bee **Enterprise**: revision 1ca3168bee ### Core From dbbbca8c58e9afe00e628375426c908ef67fc92a Mon Sep 17 00:00:00 2001 From: Jason Stirnaman Date: Thu, 26 Jun 2025 13:19:18 -0500 Subject: [PATCH 03/16] Apply suggestions from code review --- content/shared/v3-core-enterprise-release-notes/_index.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/content/shared/v3-core-enterprise-release-notes/_index.md b/content/shared/v3-core-enterprise-release-notes/_index.md index a526a14db..ae7d04ef7 100644 --- a/content/shared/v3-core-enterprise-release-notes/_index.md +++ b/content/shared/v3-core-enterprise-release-notes/_index.md @@ -42,8 +42,7 @@ All Core updates are included in Enterprise. Additional Enterprise-specific feat - **License management improvements**: - New `influxdb3 show license` command to display current license information - **Table-level retention period support**: Add retention period support for individual tables in addition to database-level retention, providing granular data lifecycle management - - New CLI commands: `create table --retention-period` and `update table --retention-period` - - Set or clear table-specific retention policies independent of database settings + - New CLI command: `influxdb3 create table --retention-period` - **Compaction improvements**: - Address compactor restart issues for better reliability - Track compacted generation durations in catalog for monitoring From 52e6eeca8a1aa1a9c9ec7a8210ec6155f2d1c9d2 Mon Sep 17 00:00:00 2001 From: Jason Stirnaman Date: Thu, 26 Jun 2025 16:36:26 -0500 Subject: [PATCH 04/16] feat: Update InfluxDB 3 CLI documentation for v3.2.0 features - Add influxdb3 show license command documentation for Enterprise - Update create database command with --retention-period option - Update create table command with Enterprise --retention-period option - Update delete database/table commands with --hard-delete options - Update CLI command index to include license command - Update planning documentation for CLI sync process - Closes #6160 --- .../reference/cli/influxdb3/show/license.md | 91 +++++++++++++++++++ .../shared/influxdb3-cli/create/database.md | 25 +++-- content/shared/influxdb3-cli/create/table.md | 35 ++++++- .../shared/influxdb3-cli/delete/database.md | 39 ++++++-- content/shared/influxdb3-cli/delete/table.md | 33 +++++-- content/shared/influxdb3-cli/show/_index.md | 1 + plans/cli-docs-sync/plan.md | 79 ++++++++++++++++ 7 files changed, 277 insertions(+), 26 deletions(-) create mode 100644 content/influxdb3/enterprise/reference/cli/influxdb3/show/license.md create mode 100644 plans/cli-docs-sync/plan.md diff --git a/content/influxdb3/enterprise/reference/cli/influxdb3/show/license.md b/content/influxdb3/enterprise/reference/cli/influxdb3/show/license.md new file mode 100644 index 000000000..ea7e73821 --- /dev/null +++ b/content/influxdb3/enterprise/reference/cli/influxdb3/show/license.md @@ -0,0 +1,91 @@ +--- +title: influxdb3 show license +description: > + The `influxdb3 show license` command displays license information for your + InfluxDB 3 Enterprise server. +menu: + influxdb3_enterprise: + parent: influxdb3 show + name: influxdb3 show license +weight: 300 +--- + +The `influxdb3 show license` command displays license information for your {{< product-name >}} instance. + +{{% show-in "enterprise" %}} +This command is only available in InfluxDB 3 Enterprise. +{{% /show-in %}} + +## Usage + + + +```bash +influxdb3 show license [OPTIONS] +``` + +## Options + +| Option | | Description | +| :----- | :----------- | :--------------------------------------------------------------------------------------- | +| `-H` | `--host` | Host URL of the running {{< product-name >}} server (default is `http://127.0.0.1:8181`) | +| | `--cluster-id` | _({{< req >}})_ Cluster identifier | +| | `--node-id` | _({{< req >}})_ Node identifier | +| | `--object-store` | _({{< req >}})_ Object store type (file, memory, s3, gcs, azure) | +| | `--token` | Authentication token | +| | `--tls-ca` | Path to a custom TLS certificate authority (for testing or self-signed certificates) | +| `-h` | `--help` | Print help information | +| | `--help-all` | Print detailed help information | + +> [!Note] +> **CLI help documentation bug in v3.2.0** +> +> The `influxdb3 show license --help` output in v3.2.0 does not display the required `--object-store`, `--cluster-id`, and `--node-id` options and related object store configuration options. +> This command requires object store configuration and cluster/node identification to function properly. + +### Additional object store options + +Depending on the `--object-store` type specified, additional configuration options may be required: + +- **S3**: AWS credentials and bucket configuration +- **GCS**: Google Cloud credentials and bucket configuration +- **Azure**: Azure credentials and container configuration +- **File**: Local file system path configuration + +### Option environment variables + +You can use the following environment variables to set command options: + +| Environment Variable | Option | +| :------------------------ | :----------- | +| `INFLUXDB3_HOST_URL` | `--host` | +| `INFLUXDB3_AUTH_TOKEN` | `--token` | + +## Examples + +### Display license information with file object store + +{{% code-placeholders "AUTH_TOKEN|CLUSTER_ID|NODE_ID" %}} + + + +```bash +influxdb3 show license \ + --cluster-id CLUSTER_ID \ + --node-id NODE_ID \ + --object-store file \ + --token AUTH_TOKEN +``` + +{{% /code-placeholders %}} + +In the example above, replace the following: + +- {{% code-placeholder-key %}}`AUTH_TOKEN`{{% /code-placeholder-key %}}: + Authentication token +- {{% code-placeholder-key %}}`CLUSTER_ID`{{% /code-placeholder-key %}}: + Your cluster identifier +- {{% code-placeholder-key %}}`NODE_ID`{{% /code-placeholder-key %}}: + Your node identifier + +The command displays information about your Enterprise license, including license type, expiration date, and usage limits. \ No newline at end of file diff --git a/content/shared/influxdb3-cli/create/database.md b/content/shared/influxdb3-cli/create/database.md index 630546c14..30c9c78eb 100644 --- a/content/shared/influxdb3-cli/create/database.md +++ b/content/shared/influxdb3-cli/create/database.md @@ -19,13 +19,14 @@ You can also set the database name using the `INFLUXDB3_DATABASE_NAME` environme ## Options -| Option | | Description | -| :----- | :----------- | :--------------------------------------------------------------------------------------- | -| `-H` | `--host` | Host URL of the running {{< product-name >}} server (default is `http://127.0.0.1:8181`) | -| | `--token` | Authentication token | -| | `--tls-ca` | Path to a custom TLS certificate authority (for testing or self-signed certificates) | -| `-h` | `--help` | Print help information | -| | `--help-all` | Print detailed help information | +| Option | | Description | +| :----- | :------------------- | :----------------------------------------------------------------------------------------------------------------------------------------------- | +| `-H` | `--host` | Host URL of the running {{< product-name >}} server (default is `http://127.0.0.1:8181`) | +| | `--retention-period` | Database [retention period](/influxdb3/version/reference/glossary/#retention-period) ([duration](/influxdb3/version/reference/glossary/#duration) value, for example: `30d`, `24h`, `1h`) | +| | `--token` | Authentication token | +| | `--tls-ca` | Path to a custom TLS certificate authority (for testing or self-signed certificates) | +| `-h` | `--help` | Print help information | +| | `--help-all` | Print detailed help information | ### Option environment variables @@ -69,4 +70,14 @@ Flags override their associated environment variables. influxdb3 create database --token AUTH_TOKEN DATABASE_NAME ``` +### Create a database with a retention period + +Creates a database with a specific retention period. + + + +```bash +influxdb3 create database --retention-period 30d DATABASE_NAME +``` + {{% /code-placeholders %}} diff --git a/content/shared/influxdb3-cli/create/table.md b/content/shared/influxdb3-cli/create/table.md index e3b858970..79c4e9ce9 100644 --- a/content/shared/influxdb3-cli/create/table.md +++ b/content/shared/influxdb3-cli/create/table.md @@ -24,6 +24,7 @@ influxdb3 create table [OPTIONS] \ ## Options +{{% hide-in "enterprise" %}} | Option | | Description | | :----- | :----------- | :--------------------------------------------------------------------------------------- | | `-H` | `--host` | Host URL of the running {{< product-name >}} server (default is `http://127.0.0.1:8181`) | @@ -34,6 +35,22 @@ influxdb3 create table [OPTIONS] \ | | `--tls-ca` | Path to a custom TLS certificate authority (for testing or self-signed certificates) | | `-h` | `--help` | Print help information | | | `--help-all` | Print detailed help information | +{{% /hide-in %}} + + +{{% show-in "enterprise" %}} +| Option | | Description | +| :----- | :------------------- | :----------------------------------------------------------------------------------------------------------------------------------------------- | +| `-H` | `--host` | Host URL of the running {{< product-name >}} server (default is `http://127.0.0.1:8181`) | +| `-d` | `--database` | _({{< req >}})_ Name of the database to operate on | +| | `--retention-period` | [Retention period](/influxdb3/version/reference/glossary/#retention-period) ([duration](/influxdb3/version/reference/glossary/#duration) value, for example: `30d`, `24h`, `1h`) for data in the table| +| | `--token` | _({{< req >}})_ Authentication token | +| | `--tags` | _({{< req >}})_ Comma-separated list of tag columns to include in the table | +| | `--fields` | Comma-separated list of field columns and their types to include in the table | +| | `--tls-ca` | Path to a custom TLS certificate authority (for testing or self-signed certificates) | +| `-h` | `--help` | Print help information | +| | `--help-all` | Print detailed help information | +{{% /show-in %}} > [!Important] > @@ -90,6 +107,22 @@ influxdb3 create table \ TABLE_NAME ``` +{{% show-in "enterprise" %}} +### Create a table with a retention period + + + +```bash +influxdb3 create table \ + --tags room,sensor_id \ + --fields temp:float64,hum:float64 \ + --retention-period 7d \ + --database DATABASE_NAME \ + --token AUTH_TOKEN \ + TABLE_NAME +``` +{{% /show-in %}} + ### Verification Use the `SHOW TABLES` query to verify that the table was created successfully: @@ -114,7 +147,7 @@ Example output: +---------------+--------------------+----------------------------+------------+ ``` ->[!Note] +> [!Note] > `SHOW TABLES` is an SQL query. It isn't supported in InfluxQL. {{% /code-placeholders %}} diff --git a/content/shared/influxdb3-cli/delete/database.md b/content/shared/influxdb3-cli/delete/database.md index 6d675b414..c6fbda211 100644 --- a/content/shared/influxdb3-cli/delete/database.md +++ b/content/shared/influxdb3-cli/delete/database.md @@ -17,13 +17,14 @@ influxdb3 delete database [OPTIONS] ## Options -| Option | | Description | -| :----- | :----------- | :--------------------------------------------------------------------------------------- | -| `-H` | `--host` | Host URL of the running {{< product-name >}} server (default is `http://127.0.0.1:8181`) | -| | `--token` | Authentication token | -| | `--tls-ca` | Path to a custom TLS certificate authority (for testing or self-signed certificates) | -| `-h` | `--help` | Print help information | -| | `--help-all` | Print detailed help information | +| Option | | Description | +| :----- | :------------ | :--------------------------------------------------------------------------------------- | +| `-H` | `--host` | Host URL of the running {{< product-name >}} server (default is `http://127.0.0.1:8181`) | +| | `--hard-delete` | When to hard delete data (never/now/default/timestamp). Default behavior is a soft delete that allows recovery | +| | `--token` | Authentication token | +| | `--tls-ca` | Path to a custom TLS certificate authority (for testing or self-signed certificates) | +| `-h` | `--help` | Print help information | +| | `--help-all` | Print detailed help information | ### Option environment variables @@ -36,8 +37,8 @@ You can use the following environment variables to set command options: ## Examples -- [Delete a database](#delete-a-new-database) -- [Delete a database while specifying the token inline](#delete-a-new-database-while-specifying-the-token-inline) +- [Delete a database](#delete-a-database) +- [Delete a database while specifying the token inline](#delete-a-database-while-specifying-the-token-inline) In the examples below, replace the following: @@ -64,4 +65,24 @@ influxdb3 delete database DATABASE_NAME influxdb3 delete database --token AUTH_TOKEN DATABASE_NAME ``` +### Hard delete a database immediately + +Permanently delete a database and all its data immediately without the ability to recover. + + + +```bash +influxdb3 delete database --hard-delete now DATABASE_NAME +``` + +### Hard delete a database at a specific time + +Schedule a database for permanent deletion at a specific timestamp. + + + +```bash +influxdb3 delete database --hard-delete "2024-01-01T00:00:00Z" DATABASE_NAME +``` + {{% /code-placeholders %}} diff --git a/content/shared/influxdb3-cli/delete/table.md b/content/shared/influxdb3-cli/delete/table.md index c15674071..b6c83602e 100644 --- a/content/shared/influxdb3-cli/delete/table.md +++ b/content/shared/influxdb3-cli/delete/table.md @@ -15,14 +15,15 @@ influxdb3 delete table [OPTIONS] --database ## Options -| Option | | Description | -| :----- | :----------- | :--------------------------------------------------------------------------------------- | -| `-H` | `--host` | Host URL of the running {{< product-name >}} server (default is `http://127.0.0.1:8181`) | -| `-d` | `--database` | _({{< req >}})_ Name of the database to operate on | -| | `--token` | _({{< req >}})_ Authentication token | -| | `--tls-ca` | Path to a custom TLS certificate authority (for testing or self-signed certificates) | -| `-h` | `--help` | Print help information | -| | `--help-all` | Print detailed help information | +| Option | | Description | +| :----- | :------------ | :--------------------------------------------------------------------------------------- | +| `-H` | `--host` | Host URL of the running {{< product-name >}} server (default is `http://127.0.0.1:8181`) | +| `-d` | `--database` | _({{< req >}})_ Name of the database to operate on | +| | `--hard-delete` | When to hard delete data (never/now/default/timestamp). Default behavior is a soft delete that allows recovery | +| | `--token` | _({{< req >}})_ Authentication token | +| | `--tls-ca` | Path to a custom TLS certificate authority (for testing or self-signed certificates) | +| `-h` | `--help` | Print help information | +| | `--help-all` | Print detailed help information | ### Option environment variables @@ -49,9 +50,23 @@ influxdb3 delete table \ TABLE_NAME ``` +### Hard delete a table immediately + +Permanently delete a table and all its data immediately without the ability to recover. + + + +```bash +influxdb3 delete table \ + --database DATABASE_NAME \ + --token AUTH_TOKEN \ + --hard-delete now \ + TABLE_NAME +``` + {{% /code-placeholders %}} -In the example above, replace the following: +In the examples above, replace the following: - {{% code-placeholder-key %}}`DATABASE_NAME`{{% /code-placeholder-key %}}: Database name diff --git a/content/shared/influxdb3-cli/show/_index.md b/content/shared/influxdb3-cli/show/_index.md index 4a5bbf935..eea02dcf0 100644 --- a/content/shared/influxdb3-cli/show/_index.md +++ b/content/shared/influxdb3-cli/show/_index.md @@ -14,6 +14,7 @@ influxdb3 show | Subcommand | Description | | :---------------------------------------------------------------------- | :--------------------------------------------- | | [databases](/influxdb3/version/reference/cli/influxdb3/show/databases/) | List database | +{{% show-in "enterprise" %}}| [license](/influxdb3/version/reference/cli/influxdb3/show/license/) | Display license information |{{% /show-in %}} | [system](/influxdb3/version/reference/cli/influxdb3/show/system/) | Display system table data | | [tokens](/influxdb3/version/reference/cli/influxdb3/show/tokens/) | List authentication tokens | | help | Print command help or the help of a subcommand | diff --git a/plans/cli-docs-sync/plan.md b/plans/cli-docs-sync/plan.md new file mode 100644 index 000000000..ca1d3afb9 --- /dev/null +++ b/plans/cli-docs-sync/plan.md @@ -0,0 +1,79 @@ +# Plan: Update InfluxDB 3 CLI Reference Documentation + +## Automation and Process Improvements + +### Immediate Improvements: +1. **Create CLI documentation sync script:** + ```bash + # Script: /Users/ja/Documents/github/docs-v2/scripts/sync-cli-docs.sh + # - Extract help text from influxdb3 CLI at /Users/ja/.influxdb//influxdb3 + # - Compare with existing docs + # - Generate report of differences + # - Auto-update basic command syntax + # - Real-time CLI verification capability established + ``` + +2. **Establish documentation standards:** + - Standardize frontmatter across CLI docs + - Create templates for command documentation + - Define Enterprise vs Core content patterns using Hugo shortcodes + +### Long-term Automation Strategy: +1. **CI/CD Integration:** + - Add GitHub Actions workflow to detect CLI changes + - Auto-generate CLI help extraction on new releases + - Create pull requests for documentation updates + +2. **Release Process Integration:** + - Include CLI documentation review in release checklist + - Link release notes to specific CLI documentation updates + - Automated cross-referencing between release notes and CLI docs + +3. **Content Management Improvements:** + - Use Hugo shortcodes for Enterprise-specific content + - Implement version-aware documentation + - Create shared content templates for common CLI patterns + +## Phase 4: Validation and Testing + +### Content accuracy verification: +- ✅ **CLI Access Available**: Direct verification via `influxdb3 --help` commands +- ✅ **Real-time Validation**: All commands and options verified against actual CLI output +- **Process**: Use `influxdb3 [command] --help` to validate documentation accuracy +- **Verification**: Cross-reference documented options with actual CLI behavior + +### Documentation completeness check: +- Ensure all v3.2.0 features are documented +- Verify examples and use cases +- Check internal links and cross-references + +## Suggested Recurring Process + +### Pre-release (during development): +- Monitor CLI changes in pull requests +- Update documentation as features are added +- Maintain CLI help extraction automation + +### At release (when tagging versions): +- Run automated CLI documentation sync +- Review and approve auto-generated updates +- Publish updated documentation + +### Post-release (after release): +- Validate documentation accuracy +- Gather user feedback on CLI documentation +- Plan improvements for next cycle + +## Related Documentation Paths + +### InfluxDB 3 Product Documentation (affects CLI usage examples): +- `/content/influxdb3/core/write-data/influxdb3-cli.md` +- `/content/influxdb3/enterprise/write-data/influxdb3-cli.md` +- `/content/shared/influxdb3-write-guides/influxdb3-cli.md` + +### Admin Documentation (affects retention and license features): +- `/content/influxdb3/core/admin/` +- `/content/influxdb3/enterprise/admin/` +- `/content/influxdb3/enterprise/admin/license.md` + +This plan ensures comprehensive documentation updates for v3.2.0 while establishing sustainable processes for future releases. \ No newline at end of file From 566f7dcb56fd987770464dbaefe73528de0ea891 Mon Sep 17 00:00:00 2001 From: Jason Stirnaman Date: Thu, 26 Jun 2025 17:04:32 -0500 Subject: [PATCH 05/16] Update content/shared/influxdb3-cli/delete/table.md --- content/shared/influxdb3-cli/delete/table.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/shared/influxdb3-cli/delete/table.md b/content/shared/influxdb3-cli/delete/table.md index b6c83602e..c96178345 100644 --- a/content/shared/influxdb3-cli/delete/table.md +++ b/content/shared/influxdb3-cli/delete/table.md @@ -66,7 +66,7 @@ influxdb3 delete table \ {{% /code-placeholders %}} -In the examples above, replace the following: +Replace the following: - {{% code-placeholder-key %}}`DATABASE_NAME`{{% /code-placeholder-key %}}: Database name From 6dac1421275bc2338b6c3d37fe867b3be5132742 Mon Sep 17 00:00:00 2001 From: Jason Stirnaman Date: Fri, 27 Jun 2025 12:46:56 -0500 Subject: [PATCH 06/16] feat(influxdb3): Core and Ent: CLI reference for update database, update table (Ent3 only)- Add CLI reference for influxdb3 update database command and retention period option in Core and Ent - Add CLI ref for influxdb3 update table and ret. period option in Ent- Change volume source to /test for core and ent3 in compose.yaml - Add alt_links instructions in CONTRIBUTING.md --- .../instructions/contributing.instructions.md | 30 ++++++- CONTRIBUTING.md | 26 ++++++ compose.yaml | 22 +++-- .../core/reference/cli/influxdb3/_index.md | 1 + .../reference/cli/influxdb3/update/_index.md | 15 ++++ .../cli/influxdb3/update/database.md | 15 ++++ .../reference/cli/influxdb3/_index.md | 1 + .../reference/cli/influxdb3/update/_index.md | 15 ++++ .../cli/influxdb3/update/database.md | 15 ++++ .../reference/cli/influxdb3/update/table.md | 17 ++++ content/shared/influxdb3-cli/update/_index.md | 33 ++++++++ .../influxdb3-cli/update/database/_index.md | 84 +++++++++++++++++++ .../influxdb3-cli/update/table/_index.md | 74 ++++++++++++++++ test/.gitignore | 1 + 14 files changed, 340 insertions(+), 9 deletions(-) create mode 100644 content/influxdb3/core/reference/cli/influxdb3/update/_index.md create mode 100644 content/influxdb3/core/reference/cli/influxdb3/update/database.md create mode 100644 content/influxdb3/enterprise/reference/cli/influxdb3/update/_index.md create mode 100644 content/influxdb3/enterprise/reference/cli/influxdb3/update/database.md create mode 100644 content/influxdb3/enterprise/reference/cli/influxdb3/update/table.md create mode 100644 content/shared/influxdb3-cli/update/_index.md create mode 100644 content/shared/influxdb3-cli/update/database/_index.md create mode 100644 content/shared/influxdb3-cli/update/table/_index.md diff --git a/.github/instructions/contributing.instructions.md b/.github/instructions/contributing.instructions.md index 2774f384f..3ddd6bb83 100644 --- a/.github/instructions/contributing.instructions.md +++ b/.github/instructions/contributing.instructions.md @@ -2,11 +2,11 @@ applyTo: "content/**/*.md, layouts/**/*.html" --- -# GitHub Copilot Instructions for InfluxData Documentation +# Contributing instructions for InfluxData Documentation ## Purpose and scope -GitHub Copilot should help document InfluxData products +Help document InfluxData products by creating clear, accurate technical content with proper code examples, frontmatter, shortcodes, and formatting. @@ -375,6 +375,9 @@ list_query_example:# Code examples included with article descriptions in childre # References to examples in data/query_examples canonical: # Path to canonical page, overrides auto-gen'd canonical URL v2: # Path to v2 equivalent page +alt_links: # Alternate pages in other products/versions for cross-product navigation + cloud-dedicated: /influxdb3/cloud-dedicated/path/to/page/ + core: /influxdb3/core/path/to/page/ prepend: # Prepend markdown content to an article (especially powerful with cascade) block: # (Optional) Wrap content in a block style (note, warn, cloud) content: # Content to prepend to article @@ -466,6 +469,29 @@ add the following frontmatter to the 1.x page: v2: /influxdb/v2.0/get-started/ ``` +### Alternative links for cross-product navigation + +Use the `alt_links` frontmatter to specify equivalent pages in other InfluxDB products, +for example, when a page exists at a different path in a different version or if +the feature doesn't exist in that product. +This enables the product switcher to navigate users to the corresponding page when they +switch between products. If a page doesn't exist in another product (for example, an +Enterprise-only feature), point to the nearest parent page if relevant. + +```yaml +alt_links: + cloud-dedicated: /influxdb3/cloud-dedicated/admin/tokens/create-token/ + cloud-serverless: /influxdb3/cloud-serverless/admin/tokens/create-token/ + core: /influxdb3/core/reference/cli/influxdb3/update/ # Points to parent if exact page doesn't exist +``` + +Supported product keys for InfluxDB 3: +- `core` +- `enterprise` +- `cloud-serverless` +- `cloud-dedicated` +- `clustered` + ### Prepend and append content to a page Use the `prepend` and `append` frontmatter to add content to the top or bottom of a page. diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index b2a6bdee3..e9f5fd25c 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -363,6 +363,9 @@ list_query_example:# Code examples included with article descriptions in childre # References to examples in data/query_examples canonical: # Path to canonical page, overrides auto-gen'd canonical URL v2: # Path to v2 equivalent page +alt_links: # Alternate pages in other products/versions for cross-product navigation + cloud-dedicated: /influxdb3/cloud-dedicated/path/to/page/ + core: /influxdb3/core/path/to/page/ prepend: # Prepend markdown content to an article (especially powerful with cascade) block: # (Optional) Wrap content in a block style (note, warn, cloud) content: # Content to prepend to article @@ -454,6 +457,29 @@ add the following frontmatter to the 1.x page: v2: /influxdb/v2.0/get-started/ ``` +### Alternative links for cross-product navigation + +Use the `alt_links` frontmatter to specify equivalent pages in other InfluxDB products, +for example, when a page exists at a different path in a different version or if +the feature doesn't exist in that product. +This enables the product switcher to navigate users to the corresponding page when they +switch between products. If a page doesn't exist in another product (for example, an +Enterprise-only feature), point to the nearest parent page if relevant. + +```yaml +alt_links: + cloud-dedicated: /influxdb3/cloud-dedicated/admin/tokens/create-token/ + cloud-serverless: /influxdb3/cloud-serverless/admin/tokens/create-token/ + core: /influxdb3/core/reference/cli/influxdb3/update/ # Points to parent if exact page doesn't exist +``` + +Supported product keys for InfluxDB 3: +- `core` +- `enterprise` +- `cloud-serverless` +- `cloud-dedicated` +- `clustered` + ### Prepend and append content to a page Use the `prepend` and `append` frontmatter to add content to the top or bottom of a page. diff --git a/compose.yaml b/compose.yaml index 203f853c9..0b160f3d2 100644 --- a/compose.yaml +++ b/compose.yaml @@ -304,13 +304,21 @@ services: image: influxdb:3-core ports: - 8181:8181 + volumes: + - type: bind + source: test/influxdb3/core/data + target: /var/lib/influxdb3/data + - type: bind + source: test/influxdb3/core/plugins + target: /var/lib/influxdb3-plugins command: - influxdb3 - serve - - --node-id=sensors_node0 + - --node-id=node0 - --log-filter=debug - --object-store=file - - --data-dir=/var/lib/influxdb3 + - --data-dir=/var/lib/influxdb3/data + - --plugin-dir=/var/lib/influxdb3/plugins influxdb3-enterprise: container_name: influxdb3-enterprise image: influxdb:3-enterprise @@ -326,16 +334,16 @@ services: - --cluster-id=cluster0 - --log-filter=debug - --object-store=file - - --data-dir=/var/lib/influxdb3 + - --data-dir=/var/lib/influxdb3/data - --plugin-dir=/var/lib/influxdb3/plugins - --license-email=${INFLUXDB3_LICENSE_EMAIL} volumes: - type: bind - source: docker/influxdb3/data - target: /var/lib/influxdb3 + source: test/influxdb3/enterprise/data + target: /var/lib/influxdb3/data - type: bind - source: docker/influxdb3/plugins - target: /var/lib/influxdb3-plugins + source: test/influxdb3/enterprise/plugins + target: /var/lib/influxdb3/plugins telegraf-pytest: container_name: telegraf-pytest image: influxdata/docs-pytest diff --git a/content/influxdb3/core/reference/cli/influxdb3/_index.md b/content/influxdb3/core/reference/cli/influxdb3/_index.md index 221e4e654..831597dc6 100644 --- a/content/influxdb3/core/reference/cli/influxdb3/_index.md +++ b/content/influxdb3/core/reference/cli/influxdb3/_index.md @@ -32,6 +32,7 @@ influxdb3 [GLOBAL-OPTIONS] [COMMAND] | [serve](/influxdb3/core/reference/cli/influxdb3/serve/) | Run the {{% product-name %}} server | | [show](/influxdb3/core/reference/cli/influxdb3/show/) | List resources | | [test](/influxdb3/core/reference/cli/influxdb3/test/) | Test plugins | +| [update](/influxdb3/core/reference/cli/influxdb3/update/) | Update resources | | [write](/influxdb3/core/reference/cli/influxdb3/write/) | Write to {{% product-name %}} | ## Global options diff --git a/content/influxdb3/core/reference/cli/influxdb3/update/_index.md b/content/influxdb3/core/reference/cli/influxdb3/update/_index.md new file mode 100644 index 000000000..0cc4e0846 --- /dev/null +++ b/content/influxdb3/core/reference/cli/influxdb3/update/_index.md @@ -0,0 +1,15 @@ +--- +title: influxdb3 update +description: > + The `influxdb3 update` command updates resources such as databases. +menu: + influxdb3_core: + parent: influxdb3 + name: influxdb3 update +weight: 300 +source: /shared/influxdb3-cli/update/_index.md +--- + + \ No newline at end of file diff --git a/content/influxdb3/core/reference/cli/influxdb3/update/database.md b/content/influxdb3/core/reference/cli/influxdb3/update/database.md new file mode 100644 index 000000000..754f70d38 --- /dev/null +++ b/content/influxdb3/core/reference/cli/influxdb3/update/database.md @@ -0,0 +1,15 @@ +--- +title: influxdb3 update database +description: > + The `influxdb3 update database` command updates an existing database. +menu: + influxdb3_core: + parent: influxdb3 update + name: influxdb3 update database +weight: 400 +source: /shared/influxdb3-cli/update/database/_index.md +--- + + \ No newline at end of file diff --git a/content/influxdb3/enterprise/reference/cli/influxdb3/_index.md b/content/influxdb3/enterprise/reference/cli/influxdb3/_index.md index db57936cb..863318111 100644 --- a/content/influxdb3/enterprise/reference/cli/influxdb3/_index.md +++ b/content/influxdb3/enterprise/reference/cli/influxdb3/_index.md @@ -32,6 +32,7 @@ influxdb3 [GLOBAL-OPTIONS] [COMMAND] | [serve](/influxdb3/enterprise/reference/cli/influxdb3/serve/) | Run the {{% product-name %}} server | | [show](/influxdb3/enterprise/reference/cli/influxdb3/show/) | List resources | | [test](/influxdb3/enterprise/reference/cli/influxdb3/test/) | Test plugins | +| [update](/influxdb3/enterprise/reference/cli/influxdb3/update/) | Update resources | | [write](/influxdb3/enterprise/reference/cli/influxdb3/write/) | Write to {{% product-name %}} | ## Global options diff --git a/content/influxdb3/enterprise/reference/cli/influxdb3/update/_index.md b/content/influxdb3/enterprise/reference/cli/influxdb3/update/_index.md new file mode 100644 index 000000000..8166b3bbb --- /dev/null +++ b/content/influxdb3/enterprise/reference/cli/influxdb3/update/_index.md @@ -0,0 +1,15 @@ +--- +title: influxdb3 update +description: > + The `influxdb3 update` command updates resources such as databases and tables. +menu: + influxdb3_enterprise: + parent: influxdb3 + name: influxdb3 update +weight: 300 +source: /shared/influxdb3-cli/update/_index.md +--- + + \ No newline at end of file diff --git a/content/influxdb3/enterprise/reference/cli/influxdb3/update/database.md b/content/influxdb3/enterprise/reference/cli/influxdb3/update/database.md new file mode 100644 index 000000000..0a8130a62 --- /dev/null +++ b/content/influxdb3/enterprise/reference/cli/influxdb3/update/database.md @@ -0,0 +1,15 @@ +--- +title: influxdb3 update database +description: > + The `influxdb3 update database` command updates an existing database. +menu: + influxdb3_enterprise: + parent: influxdb3 update + name: influxdb3 update database +weight: 400 +source: /shared/influxdb3-cli/update/database/_index.md +--- + + \ No newline at end of file diff --git a/content/influxdb3/enterprise/reference/cli/influxdb3/update/table.md b/content/influxdb3/enterprise/reference/cli/influxdb3/update/table.md new file mode 100644 index 000000000..d1a2b0bf2 --- /dev/null +++ b/content/influxdb3/enterprise/reference/cli/influxdb3/update/table.md @@ -0,0 +1,17 @@ +--- +title: influxdb3 update table +description: > + The `influxdb3 update table` command updates an existing table. +menu: + influxdb3_enterprise: + parent: influxdb3 update + name: influxdb3 update table +weight: 400 +source: /shared/influxdb3-cli/update/table/_index.md +alt_links: + core: /influxdb3/core/reference/cli/influxdb3/update/ +--- + + \ No newline at end of file diff --git a/content/shared/influxdb3-cli/update/_index.md b/content/shared/influxdb3-cli/update/_index.md new file mode 100644 index 000000000..afe2a22db --- /dev/null +++ b/content/shared/influxdb3-cli/update/_index.md @@ -0,0 +1,33 @@ +The `influxdb3 update` command updates resources such as databases and tables. + +## Usage + + + +```bash +influxdb3 update +``` + +## Subcommands + +{{% show-in "enterprise" %}} +| Subcommand | Description | +| :----------------------------------------------------------------- | :--------------------- | +| [database](/influxdb3/version/reference/cli/influxdb3/update/database/) | Update a database | +| [table](/influxdb3/version/reference/cli/influxdb3/update/table/) | Update a table | +| help | Print command help or the help of a subcommand | +{{% /show-in %}} + +{{% show-in "core" %}} +| Subcommand | Description | +| :----------------------------------------------------------------- | :--------------------- | +| [database](/influxdb3/version/reference/cli/influxdb3/update/database/) | Update a database | +| help | Print command help or the help of a subcommand | +{{% /show-in %}} + +## Options + +| Option | | Description | +| :----- | :----------- | :------------------------------ | +| `-h` | `--help` | Print help information | +| | `--help-all` | Print detailed help information | \ No newline at end of file diff --git a/content/shared/influxdb3-cli/update/database/_index.md b/content/shared/influxdb3-cli/update/database/_index.md new file mode 100644 index 000000000..c2be9879b --- /dev/null +++ b/content/shared/influxdb3-cli/update/database/_index.md @@ -0,0 +1,84 @@ +The `influxdb3 update database` command updates an existing database in your {{< product-name >}} instance. + +Use this command to update a database's retention period. + +## Usage + + + +```bash +influxdb3 update database [OPTIONS] --database +``` + +## Arguments + +- **`DATABASE_NAME`**: (Required) The name of the database to update. Valid database names are alphanumeric and start with a letter or number. Dashes (-) and underscores (_) are allowed. + +You can also set the database name using the `INFLUXDB3_DATABASE_NAME` environment variable. + +## Options + +| Option | | Description | +| :----- | :------------------- | :----------------------------------------------------------------------------------------------------------------------------------------------- | +| `-H` | `--host` | Host URL of the running {{< product-name >}} server (default is `http://127.0.0.1:8181`) | +| `-d` | `--database` | The name of the database to update | +| | `--token` | Authentication token | +| `-r` | `--retention-period` | The retention period as a [duration](/influxdb3/version/reference/glossary/#duration) value (for example: `30d`, `24h`) or `none` to clear | +| | `--tls-ca` | Path to a custom TLS certificate authority (for testing or self-signed certificates) | +| `-h` | `--help` | Print help information | +| | `--help-all` | Print detailed help information | + +### Option environment variables + +You can use the following environment variables instead of providing CLI options directly: + +| Environment Variable | Option | +| :------------------------ | :----------- | +| `INFLUXDB3_HOST_URL` | `--host` | +| `INFLUXDB3_DATABASE_NAME` | `--database` | +| `INFLUXDB3_AUTH_TOKEN` | `--token` | +| `INFLUXDB3_TLS_CA` | `--tls-ca` | + +## Examples + +The following examples show how to update a database. + +In your commands replace the following: +- {{% code-placeholder-key %}}`DATABASE_NAME`{{% /code-placeholder-key %}}: + Database name +- {{% code-placeholder-key %}}`AUTH_TOKEN`{{% /code-placeholder-key %}}: + Authentication token + +{{% code-placeholders "DATABASE_NAME|AUTH_TOKEN" %}} + +### Update a database retention period + +Updates a database retention period to 30 days. + + + +```bash +influxdb3 update database --retention-period 30d DATABASE_NAME +``` + +### Clear a database retention period + +Removes the retention period from a database by setting it to `none`. + + + +```bash +influxdb3 update database --retention-period none DATABASE_NAME +``` + +### Update a database with authentication + +Updates a database using an authentication token. + + + +```bash +influxdb3 update database --token AUTH_TOKEN --retention-period 7d DATABASE_NAME +``` + +{{% /code-placeholders %}} \ No newline at end of file diff --git a/content/shared/influxdb3-cli/update/table/_index.md b/content/shared/influxdb3-cli/update/table/_index.md new file mode 100644 index 000000000..928ca8573 --- /dev/null +++ b/content/shared/influxdb3-cli/update/table/_index.md @@ -0,0 +1,74 @@ +The `influxdb3 update table` command updates an existing table in a database in your {{< product-name >}} instance. + +Use this command to update a table's retention period. + +## Usage + + + +```bash +influxdb3 update table [OPTIONS] --database +``` + +## Arguments + +- **`TABLE_NAME`**: (Required) The name of the table to update + +## Options + +| Option | | Description | +| :----- | :------------------- | :----------------------------------------------------------------------------------------------------------------------------------------------- | +| `-H` | `--host` | Host URL of the running {{< product-name >}} server (default is `http://127.0.0.1:8181`) | +| `-d` | `--database` | The name of the database containing the table | +| | `--token` | Authentication token | +| `-r` | `--retention-period` | The retention period as a [duration](/influxdb3/version/reference/glossary/#duration) value (for example: `30d`, `24h`) or `none` to clear | +| | `--tls-ca` | Path to a custom TLS certificate authority (for testing or self-signed certificates) | +| `-h` | `--help` | Print help information | +| | `--help-all` | Print detailed help information | + +### Option environment variables + +You can use the following environment variables instead of providing CLI options directly: + +| Environment Variable | Option | +| :------------------------ | :----------- | +| `INFLUXDB3_HOST_URL` | `--host` | +| `INFLUXDB3_DATABASE_NAME` | `--database` | +| `INFLUXDB3_AUTH_TOKEN` | `--token` | +| `INFLUXDB3_TLS_CA` | `--tls-ca` | + +## Examples + +The following examples show how to update a table. + +In your commands replace the following: +- {{% code-placeholder-key %}}`DATABASE_NAME`{{% /code-placeholder-key %}}: + Database name +- {{% code-placeholder-key %}}`TABLE_NAME`{{% /code-placeholder-key %}}: + Table name +- {{% code-placeholder-key %}}`AUTH_TOKEN`{{% /code-placeholder-key %}}: + Authentication token + +{{% code-placeholders "DATABASE_NAME|TABLE_NAME|AUTH_TOKEN" %}} + +### Update a table retention period + +Updates a table retention period to 30 days. + + + +```bash +influxdb3 update table --database DATABASE_NAME --token AUTH_TOKEN --retention-period 30d TABLE_NAME +``` + +### Clear a table retention period + +Removes the retention period from a table by setting it to `none`. + + + +```bash +influxdb3 update table --database DATABASE_NAME --retention-period none TABLE_NAME +``` + +{{% /code-placeholders %}} \ No newline at end of file diff --git a/test/.gitignore b/test/.gitignore index 726e3ff57..bcdfd10db 100644 --- a/test/.gitignore +++ b/test/.gitignore @@ -2,6 +2,7 @@ /Cargo.lock config.toml content +influxdb3/**/data _*_pid node_modules shared From 429e3fc0792331eb3d278da2e252be078804822d Mon Sep 17 00:00:00 2001 From: Jason Stirnaman Date: Fri, 27 Jun 2025 14:00:02 -0500 Subject: [PATCH 07/16] Apply suggestions from code review Co-authored-by: Scott Anderson --- .../enterprise/reference/cli/influxdb3/show/license.md | 3 --- content/shared/influxdb3-cli/delete/database.md | 2 ++ content/shared/influxdb3-cli/update/database/_index.md | 2 +- 3 files changed, 3 insertions(+), 4 deletions(-) diff --git a/content/influxdb3/enterprise/reference/cli/influxdb3/show/license.md b/content/influxdb3/enterprise/reference/cli/influxdb3/show/license.md index ea7e73821..579cfb8e5 100644 --- a/content/influxdb3/enterprise/reference/cli/influxdb3/show/license.md +++ b/content/influxdb3/enterprise/reference/cli/influxdb3/show/license.md @@ -12,9 +12,6 @@ weight: 300 The `influxdb3 show license` command displays license information for your {{< product-name >}} instance. -{{% show-in "enterprise" %}} -This command is only available in InfluxDB 3 Enterprise. -{{% /show-in %}} ## Usage diff --git a/content/shared/influxdb3-cli/delete/database.md b/content/shared/influxdb3-cli/delete/database.md index c6fbda211..14e1d4817 100644 --- a/content/shared/influxdb3-cli/delete/database.md +++ b/content/shared/influxdb3-cli/delete/database.md @@ -39,6 +39,8 @@ You can use the following environment variables to set command options: - [Delete a database](#delete-a-database) - [Delete a database while specifying the token inline](#delete-a-database-while-specifying-the-token-inline) +- [Hard delete a database immediately](#hard-delete-a-database-immediately) +- [Hard delete a database at a specific time](#hard-delete-a-database-at-a-specific-time) In the examples below, replace the following: diff --git a/content/shared/influxdb3-cli/update/database/_index.md b/content/shared/influxdb3-cli/update/database/_index.md index c2be9879b..a292283a4 100644 --- a/content/shared/influxdb3-cli/update/database/_index.md +++ b/content/shared/influxdb3-cli/update/database/_index.md @@ -12,7 +12,7 @@ influxdb3 update database [OPTIONS] --database ## Arguments -- **`DATABASE_NAME`**: (Required) The name of the database to update. Valid database names are alphanumeric and start with a letter or number. Dashes (-) and underscores (_) are allowed. +- **`DATABASE_NAME`**: (Required) The name of the database to update. You can also set the database name using the `INFLUXDB3_DATABASE_NAME` environment variable. From 63d31907ff0719f548af1fcf8b123dd7dd19dd12 Mon Sep 17 00:00:00 2001 From: Jason Stirnaman Date: Thu, 26 Jun 2025 12:30:11 -0500 Subject: [PATCH 08/16] feat: add documentation planning structure and CLI sync plan MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Create structured approach for managing recurring documentation tasks: Benefits: - Claude can easily reference plans via @plans/[task]/plan-[version].md - Version-controlled plans alongside code for iteration tracking - Progress tracking with execution logs for lessons learned - Reusable templates for recurring documentation patterns - Organized structure for multiple concurrent documentation tasks Usage Examples: - "Claude, execute Phase 1 of @plans/cli-docs-sync/plan-v3.2.0.md" - "Claude, create plan-v3.3.0.md based on previous plan and new release notes" - Create similar structures for other recurring documentation tasks Includes comprehensive CLI documentation sync plan for v3.2.0 with: - Analysis of existing CLI docs and v3.2.0 changes - Specific file paths for updates (retention periods, license management) - Automation strategy for future releases - Phase-based execution approach 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude --- plans/README.md | 38 ++++++ plans/cli-docs-sync/execution-log.md | 47 ++++++++ plans/cli-docs-sync/plan-v3.2.0.md | 165 +++++++++++++++++++++++++++ 3 files changed, 250 insertions(+) create mode 100644 plans/README.md create mode 100644 plans/cli-docs-sync/execution-log.md create mode 100644 plans/cli-docs-sync/plan-v3.2.0.md diff --git a/plans/README.md b/plans/README.md new file mode 100644 index 000000000..098c81915 --- /dev/null +++ b/plans/README.md @@ -0,0 +1,38 @@ +# Documentation Plans + +This directory contains plans for recurring documentation tasks and automation. + +## Active Plans + +### CLI Documentation Synchronization +- **Location**: `cli-docs-sync/` +- **Purpose**: Keep InfluxDB 3 CLI reference documentation in sync with actual CLI commands +- **Current Version**: `plan-v3.2.0.md` +- **Status**: Ready for execution + +### Release Notes Automation +- **Location**: `release-notes-automation/` +- **Purpose**: Automate generation of release notes from git commits +- **Scripts**: Available in `/scripts/generate-release-notes.sh` + +## Plan Structure + +Each plan directory should contain: +- `plan-[version].md` - The detailed plan +- `execution-log.md` - Track progress and notes +- `templates/` - Reusable templates and patterns + +## Usage for Claude + +When working with Claude on these plans: +1. Reference the specific plan file: `@plans/[task]/plan-[version].md` +2. Update execution logs as you progress +3. Iterate on plans by creating new versions +4. Use templates for consistency + +## Best Practices + +- Keep plans versioned by release or iteration +- Document lessons learned in execution logs +- Create reusable templates for recurring patterns +- Link plans to related scripts and automation \ No newline at end of file diff --git a/plans/cli-docs-sync/execution-log.md b/plans/cli-docs-sync/execution-log.md new file mode 100644 index 000000000..ec2cf65af --- /dev/null +++ b/plans/cli-docs-sync/execution-log.md @@ -0,0 +1,47 @@ +# CLI Documentation Sync - Execution Log + +## v3.2.0 Execution + +### Status: Planning Complete ✅ +**Date**: 2025-06-26 +**Plan**: `plan-v3.2.0.md` + +### Phase 1: Analysis (Not Started) +- [ ] Audit existing CLI documentation structure +- [ ] Extract v3.2.0 changes from release notes +- [ ] Generate current CLI help output + +### Phase 2: Update Documentation (Not Started) +#### Files to Create: +- [ ] `/content/shared/influxdb3-cli/update/_index.md` +- [ ] `/content/shared/influxdb3-cli/update/database.md` +- [ ] `/content/shared/influxdb3-cli/update/table.md` (Enterprise) +- [ ] `/content/shared/influxdb3-cli/show/license.md` (Enterprise) + +#### Files to Update: +- [ ] `/content/shared/influxdb3-cli/create/database.md` (add `--retention-period`) +- [ ] `/content/shared/influxdb3-cli/create/table.md` (add Enterprise `--retention-period`) +- [ ] `/content/shared/influxdb3-cli/show/_index.md` (add license command) +- [ ] `/content/shared/influxdb3-cli/delete/database.md` (review hard delete) +- [ ] `/content/shared/influxdb3-cli/delete/table.md` (review hard delete) + +### Phase 3: Automation (Not Started) +- [ ] Create CLI documentation sync script +- [ ] Establish documentation standards +- [ ] Plan CI/CD integration + +### Phase 4: Validation (Not Started) +- [ ] Test documented commands +- [ ] Verify completeness +- [ ] Check cross-references + +### Notes and Lessons Learned +- Release notes analysis was crucial for identifying specific CLI changes +- Having automation scripts in place (`generate-release-notes.sh`) helped identify features +- Need to distinguish between Core and Enterprise features clearly + +### Next Actions +When ready to execute: +1. Reference plan: `@plans/cli-docs-sync/plan-v3.2.0.md` +2. Start with Phase 1 analysis +3. Update this log as progress is made \ No newline at end of file diff --git a/plans/cli-docs-sync/plan-v3.2.0.md b/plans/cli-docs-sync/plan-v3.2.0.md new file mode 100644 index 000000000..725c2275b --- /dev/null +++ b/plans/cli-docs-sync/plan-v3.2.0.md @@ -0,0 +1,165 @@ +# Plan: Update InfluxDB 3 CLI Reference Documentation for v3.2.0 + +## Phase 1: Analyze Current State and Changes + +### 1. Audit existing CLI documentation structure +**Existing files in `/content/shared/influxdb3-cli/`:** +- `/content/shared/influxdb3-cli/create/_index.md` +- `/content/shared/influxdb3-cli/create/database.md` ⚠️ **NEEDS UPDATE** (add `--retention-period`) +- `/content/shared/influxdb3-cli/create/distinct_cache.md` +- `/content/shared/influxdb3-cli/create/file_index.md` +- `/content/shared/influxdb3-cli/create/last_cache.md` +- `/content/shared/influxdb3-cli/create/table.md` ⚠️ **NEEDS UPDATE** (add Enterprise `--retention-period`) +- `/content/shared/influxdb3-cli/create/token/_index.md` +- `/content/shared/influxdb3-cli/create/token/admin.md` +- `/content/shared/influxdb3-cli/create/trigger.md` +- `/content/shared/influxdb3-cli/delete/_index.md` +- `/content/shared/influxdb3-cli/delete/database.md` ⚠️ **REVIEW** (hard delete features) +- `/content/shared/influxdb3-cli/delete/distinct_cache.md` +- `/content/shared/influxdb3-cli/delete/file_index.md` +- `/content/shared/influxdb3-cli/delete/last_cache.md` +- `/content/shared/influxdb3-cli/delete/table.md` ⚠️ **REVIEW** (hard delete features) +- `/content/shared/influxdb3-cli/delete/trigger.md` +- `/content/shared/influxdb3-cli/disable/_index.md` +- `/content/shared/influxdb3-cli/disable/trigger.md` +- `/content/shared/influxdb3-cli/enable/_index.md` +- `/content/shared/influxdb3-cli/enable/trigger.md` +- `/content/shared/influxdb3-cli/query.md` +- `/content/shared/influxdb3-cli/show/_index.md` ⚠️ **NEEDS UPDATE** (add license command) +- `/content/shared/influxdb3-cli/show/databases.md` +- `/content/shared/influxdb3-cli/show/system/_index.md` +- `/content/shared/influxdb3-cli/show/system/summary.md` +- `/content/shared/influxdb3-cli/show/system/table-list.md` +- `/content/shared/influxdb3-cli/show/system/table.md` +- `/content/shared/influxdb3-cli/show/tokens.md` +- `/content/shared/influxdb3-cli/test/_index.md` +- `/content/shared/influxdb3-cli/test/schedule_plugin.md` +- `/content/shared/influxdb3-cli/test/wal_plugin.md` +- `/content/shared/influxdb3-cli/write.md` + +### 2. Extract v3.2.0 changes from release notes +**From `/content/shared/v3-core-enterprise-release-notes/_index.md`:** + +**Core v3.2.0 Features:** +- Database retention period support: `create database --retention-period`, `update database --retention-period` +- Hard delete for databases and tables +- AWS credentials auto-reload +- WAL improvements + +**Enterprise v3.2.0 Features:** +- License management: `influxdb3 show license` +- Table retention period support: `create table --retention-period`, `update table --retention-period` +- All Core features plus Enterprise-specific enhancements + +### 3. Generate current CLI help output +- Run `influxdb3 --help` for both Core and Enterprise versions +- Extract new commands, options, and help text +- Compare with existing documentation + +## Phase 2: Update Documentation Files + +### Files to Create (NEW): +- `/content/shared/influxdb3-cli/update/_index.md` 🆕 +- `/content/shared/influxdb3-cli/update/database.md` 🆕 (retention period management) +- `/content/shared/influxdb3-cli/update/table.md` 🆕 (Enterprise-only, retention period management) +- `/content/shared/influxdb3-cli/show/license.md` 🆕 (Enterprise-only) + +### Files to Update (EXISTING): +- `/content/shared/influxdb3-cli/create/database.md` ⚠️ (add `--retention-period` option) +- `/content/shared/influxdb3-cli/create/table.md` ⚠️ (add Enterprise `--retention-period` option) +- `/content/shared/influxdb3-cli/show/_index.md` ⚠️ (include license command) +- `/content/shared/influxdb3-cli/delete/database.md` ⚠️ (review hard delete options) +- `/content/shared/influxdb3-cli/delete/table.md` ⚠️ (review hard delete options) + +### Content Changes by Category: + +**1. Retention Period Documentation:** +- Update `/content/shared/influxdb3-cli/create/database.md` with `--retention-period` option +- Create `/content/shared/influxdb3-cli/update/` directory structure +- Create `/content/shared/influxdb3-cli/update/database.md` for retention management +- Update `/content/shared/influxdb3-cli/create/table.md` with Enterprise `--retention-period` option +- Create `/content/shared/influxdb3-cli/update/table.md` for Enterprise table retention management + +**2. License Management Documentation:** +- Update `/content/shared/influxdb3-cli/show/_index.md` to include license command +- Create `/content/shared/influxdb3-cli/show/license.md` for Enterprise license display + +**3. Hard Delete Documentation:** +- Review and update `/content/shared/influxdb3-cli/delete/database.md` with hard delete options +- Review and update `/content/shared/influxdb3-cli/delete/table.md` with hard delete options + +## Phase 3: Automation and Process Improvements + +### Immediate Improvements: +1. **Create CLI documentation sync script:** + ```bash + # Script: /Users/ja/Documents/github/docs-v2/scripts/sync-cli-docs.sh + # - Extract help text from influxdb3 CLI + # - Compare with existing docs + # - Generate report of differences + # - Auto-update basic command syntax + ``` + +2. **Establish documentation standards:** + - Standardize frontmatter across CLI docs + - Create templates for command documentation + - Define Enterprise vs Core content patterns using Hugo shortcodes + +### Long-term Automation Strategy: +1. **CI/CD Integration:** + - Add GitHub Actions workflow to detect CLI changes + - Auto-generate CLI help extraction on new releases + - Create pull requests for documentation updates + +2. **Release Process Integration:** + - Include CLI documentation review in release checklist + - Link release notes to specific CLI documentation updates + - Automated cross-referencing between release notes and CLI docs + +3. **Content Management Improvements:** + - Use Hugo shortcodes for Enterprise-specific content + - Implement version-aware documentation + - Create shared content templates for common CLI patterns + +## Phase 4: Validation and Testing + +### Content accuracy verification: +- Test all documented commands and options against actual CLIs +- Verify Enterprise vs Core feature availability +- Cross-reference with actual CLI behavior + +### Documentation completeness check: +- Ensure all v3.2.0 features are documented +- Verify examples and use cases +- Check internal links and cross-references + +## Suggested Recurring Process + +### Pre-release (during development): +- Monitor CLI changes in pull requests +- Update documentation as features are added +- Maintain CLI help extraction automation + +### At release (when tagging versions): +- Run automated CLI documentation sync +- Review and approve auto-generated updates +- Publish updated documentation + +### Post-release (after release): +- Validate documentation accuracy +- Gather user feedback on CLI documentation +- Plan improvements for next cycle + +## Related Documentation Paths + +### InfluxDB 3 Product Documentation (affects CLI usage examples): +- `/content/influxdb3/core/write-data/influxdb3-cli.md` +- `/content/influxdb3/enterprise/write-data/influxdb3-cli.md` +- `/content/shared/influxdb3-write-guides/influxdb3-cli.md` + +### Admin Documentation (affects retention and license features): +- `/content/influxdb3/core/admin/` +- `/content/influxdb3/enterprise/admin/` +- `/content/influxdb3/enterprise/admin/license.md` + +This plan ensures comprehensive documentation updates for v3.2.0 while establishing sustainable processes for future releases. \ No newline at end of file From 18611368ccaaa1819ab02d2d4c1bebb0bea22099 Mon Sep 17 00:00:00 2001 From: Jason Stirnaman Date: Fri, 27 Jun 2025 14:15:12 -0500 Subject: [PATCH 09/16] fix(influxdb3): Add update database and update table commands --- content/shared/v3-core-enterprise-release-notes/_index.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/content/shared/v3-core-enterprise-release-notes/_index.md b/content/shared/v3-core-enterprise-release-notes/_index.md index ae7d04ef7..fa11b397f 100644 --- a/content/shared/v3-core-enterprise-release-notes/_index.md +++ b/content/shared/v3-core-enterprise-release-notes/_index.md @@ -16,7 +16,8 @@ - **Hard delete for databases and tables**: Permanently delete databases and tables, enabling complete removal of data structures for compliance and storage management ([#26553](https://github.com/influxdata/influxdb/pull/26553)) - **AWS credentials auto-reload**: Support dynamic reloading of ephemeral AWS credentials from files, improving security and reliability when using AWS services ([#26537](https://github.com/influxdata/influxdb/pull/26537)) -- **Database retention period support**: Add retention period support for databases via CLI commands (`create database` and `update database` commands) and HTTP APIs ([#26520](https://github.com/influxdata/influxdb/pull/26520)) +- **Database retention period support**: Add retention period support for databases via CLI commands (`create database` and `update database` commands) and HTTP APIs ([#26520](https://github.com/influxdata/influxdb/pull/26520)): + - New CLI command: `update database --retention-period` - **Configurable lookback duration**: Users can specify lookback duration for PersistedFiles buffer, providing better control over query performance ([#26528](https://github.com/influxdata/influxdb/pull/26528)) - **WAL replay concurrency control**: Add concurrency limits for WAL (Write-Ahead Log) replay to improve startup performance and resource management ([#26483](https://github.com/influxdata/influxdb/pull/26483)) - **Enhanced write path**: Separate write path executor with unbounded memory for improved write performance ([#26455](https://github.com/influxdata/influxdb/pull/26455)) @@ -42,7 +43,8 @@ All Core updates are included in Enterprise. Additional Enterprise-specific feat - **License management improvements**: - New `influxdb3 show license` command to display current license information - **Table-level retention period support**: Add retention period support for individual tables in addition to database-level retention, providing granular data lifecycle management - - New CLI command: `influxdb3 create table --retention-period` + - New CLI commands: `create table --retention-period` and `update table --retention-period` + - Set or clear table-specific retention policies independent of database settings - **Compaction improvements**: - Address compactor restart issues for better reliability - Track compacted generation durations in catalog for monitoring From 3d1ef2bfc3f8e2ce394a8b4fc488048e35396d04 Mon Sep 17 00:00:00 2001 From: Jason Stirnaman Date: Fri, 27 Jun 2025 15:26:28 -0500 Subject: [PATCH 10/16] feat(helper-scripts): Enhance release notes generation with multi-repo and API change detection MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - Add support for multiple repositories (primary + additional repos) - Automatically fetch latest commits from all repositories by default - Add --no-fetch and --pull command-line options for different workflows - Detect and categorize REST API changes across v1, v2, v3 endpoints - Include specific API endpoint analysis (/write, /query, /ping, /health, /metrics) - Add repository tagging to commit categorization for multi-repo visibility - Improve error handling with fallback options for git operations 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude --- helper-scripts/generate-release-notes.sh | 325 +++++++++++++++++++---- 1 file changed, 272 insertions(+), 53 deletions(-) diff --git a/helper-scripts/generate-release-notes.sh b/helper-scripts/generate-release-notes.sh index 58f34212e..d47ae5592 100755 --- a/helper-scripts/generate-release-notes.sh +++ b/helper-scripts/generate-release-notes.sh @@ -1,14 +1,55 @@ #!/bin/bash # Script to generate release notes for InfluxDB v3.x releases -# Usage: ./generate-release-notes.sh +# Usage: ./generate-release-notes.sh [--no-fetch] [--pull] [additional_repo_paths...] +# +# Options: +# --no-fetch Skip fetching latest commits from remote +# --pull Pull latest changes (implies fetch) - use with caution as it may change your working directory +# +# Example: ./generate-release-notes.sh v3.1.0 v3.2.0 /path/to/influxdb /path/to/influxdb_pro /path/to/influxdb_iox +# Example: ./generate-release-notes.sh --no-fetch v3.1.0 v3.2.0 /path/to/influxdb +# Example: ./generate-release-notes.sh --pull v3.1.0 v3.2.0 /path/to/influxdb /path/to/influxdb_pro set -e -# Default values -REPO_PATH="${3:-/Users/ja/Documents/github/influxdb}" +# Parse command line options +FETCH_COMMITS=true +PULL_COMMITS=false + +while [[ $# -gt 0 ]]; do + case $1 in + --no-fetch) + FETCH_COMMITS=false + shift + ;; + --pull) + PULL_COMMITS=true + FETCH_COMMITS=true + shift + ;; + -*) + echo "Unknown option $1" + exit 1 + ;; + *) + break + ;; + esac +done + +# Parse remaining arguments FROM_VERSION="${1:-v3.1.0}" TO_VERSION="${2:-v3.2.0}" +PRIMARY_REPO="${3:-/Users/ja/Documents/github/influxdb}" + +# Collect additional repositories (all arguments after the third) +ADDITIONAL_REPOS=() +shift 3 2>/dev/null || true +while [ $# -gt 0 ]; do + ADDITIONAL_REPOS+=("$1") + shift +done # Colors for output RED='\033[0;31m' @@ -18,7 +59,13 @@ BLUE='\033[0;34m' NC='\033[0m' # No Color echo -e "${BLUE}Generating release notes for ${TO_VERSION}${NC}" -echo -e "Repository: ${REPO_PATH}" +echo -e "Primary Repository: ${PRIMARY_REPO}" +if [ ${#ADDITIONAL_REPOS[@]} -gt 0 ]; then + echo -e "Additional Repositories:" + for repo in "${ADDITIONAL_REPOS[@]}"; do + echo -e " - ${repo}" + done +fi echo -e "From: ${FROM_VERSION} To: ${TO_VERSION}\n" # Function to extract PR number from commit message @@ -26,28 +73,156 @@ extract_pr_number() { echo "$1" | grep -oE '#[0-9]+' | head -1 | sed 's/#//' } +# Function to get commits from a repository +get_commits_from_repo() { + local repo_path="$1" + local pattern="$2" + local format="${3:-%h %s}" + + if [ -d "$repo_path" ]; then + git -C "$repo_path" log --format="$format" "${FROM_VERSION}..${TO_VERSION}" 2>/dev/null | grep -E "$pattern" || true + fi +} + +# Function to analyze API-related commits +analyze_api_changes() { + local repo_path="$1" + local repo_name="$2" + + if [ ! -d "$repo_path" ]; then + return + fi + + # Look for API-related file changes + local api_files=$(git -C "$repo_path" diff --name-only "${FROM_VERSION}..${TO_VERSION}" 2>/dev/null | grep -E "(api|handler|endpoint|route)" | head -10 || true) + + # Look for specific API endpoint patterns in commit messages and diffs + local api_commits=$(git -C "$repo_path" log --format="%h %s" "${FROM_VERSION}..${TO_VERSION}" 2>/dev/null | \ + grep -iE "(api|endpoint|/write|/query|/ping|/health|/metrics|v1|v2|v3)" || true) + + if [ -n "$api_files" ] || [ -n "$api_commits" ]; then + echo " Repository: $repo_name" + if [ -n "$api_files" ]; then + echo " Modified API files:" + echo "$api_files" | while read -r file; do + echo " - $file" + done + fi + if [ -n "$api_commits" ]; then + echo " API-related commits:" + echo "$api_commits" | while read -r commit; do + echo " - $commit" + done + fi + echo + fi +} + # Get the release date -RELEASE_DATE=$(git -C "$REPO_PATH" log -1 --format=%ai "$TO_VERSION" | cut -d' ' -f1) +RELEASE_DATE=$(git -C "$PRIMARY_REPO" log -1 --format=%ai "$TO_VERSION" | cut -d' ' -f1) echo -e "${GREEN}Release Date: ${RELEASE_DATE}${NC}\n" -# Collect commits by category -echo -e "${YELLOW}Analyzing commits...${NC}" +# Create array of all repositories +ALL_REPOS=("$PRIMARY_REPO") +for repo in "${ADDITIONAL_REPOS[@]}"; do + ALL_REPOS+=("$repo") +done -# Features -echo -e "\n${GREEN}Features:${NC}" -FEATURES=$(git -C "$REPO_PATH" log --format="%h %s" "${FROM_VERSION}..${TO_VERSION}" | grep -E "^[a-f0-9]+ feat:" | sed 's/^[a-f0-9]* feat: //') +# Fetch latest commits from all repositories (if enabled) +if [ "$FETCH_COMMITS" = true ]; then + if [ "$PULL_COMMITS" = true ]; then + echo -e "${YELLOW}Pulling latest changes from all repositories...${NC}" + echo -e "${RED}Warning: This will modify your working directories!${NC}" + else + echo -e "${YELLOW}Fetching latest commits from all repositories...${NC}" + fi + + for repo in "${ALL_REPOS[@]}"; do + if [ -d "$repo" ]; then + repo_name=$(basename "$repo") + + if [ "$PULL_COMMITS" = true ]; then + echo -e " Pulling changes in $repo_name..." + if git -C "$repo" pull origin 2>/dev/null; then + echo -e " ${GREEN}✓${NC} Successfully pulled changes in $repo_name" + else + echo -e " ${RED}✗${NC} Failed to pull changes in $repo_name (trying fetch only)" + if git -C "$repo" fetch origin 2>/dev/null; then + echo -e " ${GREEN}✓${NC} Successfully fetched from $repo_name" + else + echo -e " ${RED}✗${NC} Failed to fetch from $repo_name (continuing with local commits)" + fi + fi + else + echo -e " Fetching from $repo_name..." + if git -C "$repo" fetch origin 2>/dev/null; then + echo -e " ${GREEN}✓${NC} Successfully fetched from $repo_name" + else + echo -e " ${RED}✗${NC} Failed to fetch from $repo_name (continuing with local commits)" + fi + fi + else + echo -e " ${RED}✗${NC} Repository not found: $repo" + fi + done +else + echo -e "${YELLOW}Skipping fetch (using local commits only)${NC}" +fi -# Fixes -echo -e "\n${GREEN}Bug Fixes:${NC}" -FIXES=$(git -C "$REPO_PATH" log --format="%h %s" "${FROM_VERSION}..${TO_VERSION}" | grep -E "^[a-f0-9]+ fix:" | sed 's/^[a-f0-9]* fix: //') +# Collect commits by category from all repositories +echo -e "\n${YELLOW}Analyzing commits across all repositories...${NC}" -# Breaking changes -echo -e "\n${GREEN}Breaking Changes:${NC}" -BREAKING=$(git -C "$REPO_PATH" log --format="%h %s" "${FROM_VERSION}..${TO_VERSION}" | grep -iE "^[a-f0-9]+ .*(BREAKING|breaking change)" | sed 's/^[a-f0-9]* //') +# Initialize variables +FEATURES="" +FIXES="" +BREAKING="" +PERF="" +API_CHANGES="" -# Performance improvements -echo -e "\n${GREEN}Performance:${NC}" -PERF=$(git -C "$REPO_PATH" log --format="%h %s" "${FROM_VERSION}..${TO_VERSION}" | grep -E "^[a-f0-9]+ perf:" | sed 's/^[a-f0-9]* perf: //') +# Collect commits from all repositories +for repo in "${ALL_REPOS[@]}"; do + if [ -d "$repo" ]; then + repo_name=$(basename "$repo") + echo -e " Analyzing $repo_name..." + + # Features + repo_features=$(get_commits_from_repo "$repo" "^[a-f0-9]+ feat:" | sed "s/^[a-f0-9]* feat: /- [$repo_name] /") + if [ -n "$repo_features" ]; then + FEATURES="$FEATURES$repo_features"$'\n' + fi + + # Fixes + repo_fixes=$(get_commits_from_repo "$repo" "^[a-f0-9]+ fix:" | sed "s/^[a-f0-9]* fix: /- [$repo_name] /") + if [ -n "$repo_fixes" ]; then + FIXES="$FIXES$repo_fixes"$'\n' + fi + + # Breaking changes + repo_breaking=$(get_commits_from_repo "$repo" "^[a-f0-9]+ .*(BREAKING|breaking change)" | sed "s/^[a-f0-9]* /- [$repo_name] /") + if [ -n "$repo_breaking" ]; then + BREAKING="$BREAKING$repo_breaking"$'\n' + fi + + # Performance improvements + repo_perf=$(get_commits_from_repo "$repo" "^[a-f0-9]+ perf:" | sed "s/^[a-f0-9]* perf: /- [$repo_name] /") + if [ -n "$repo_perf" ]; then + PERF="$PERF$repo_perf"$'\n' + fi + + # API changes + repo_api=$(get_commits_from_repo "$repo" "(api|endpoint|/write|/query|/ping|/health|/metrics|v1|v2|v3)" | sed "s/^[a-f0-9]* /- [$repo_name] /") + if [ -n "$repo_api" ]; then + API_CHANGES="$API_CHANGES$repo_api"$'\n' + fi + fi +done + +# Analyze API changes in detail +echo -e "\n${YELLOW}Analyzing HTTP API changes...${NC}" +for repo in "${ALL_REPOS[@]}"; do + repo_name=$(basename "$repo") + analyze_api_changes "$repo" "$repo_name" +done # Generate markdown output OUTPUT_FILE="release-notes-${TO_VERSION}.md" @@ -60,16 +235,18 @@ EOF # Add features if [ -n "$FEATURES" ]; then - while IFS= read -r line; do - PR=$(extract_pr_number "$line") - # Clean up the commit message - CLEAN_LINE=$(echo "$line" | sed -E 's/ \(#[0-9]+\)$//') - if [ -n "$PR" ]; then - echo "- $CLEAN_LINE ([#$PR](https://github.com/influxdata/influxdb/pull/$PR))" >> "$OUTPUT_FILE" - else - echo "- $CLEAN_LINE" >> "$OUTPUT_FILE" + echo "$FEATURES" | while IFS= read -r line; do + if [ -n "$line" ]; then + PR=$(extract_pr_number "$line") + # Clean up the commit message + CLEAN_LINE=$(echo "$line" | sed -E 's/ \(#[0-9]+\)$//') + if [ -n "$PR" ]; then + echo "$CLEAN_LINE ([#$PR](https://github.com/influxdata/influxdb/pull/$PR))" >> "$OUTPUT_FILE" + else + echo "$CLEAN_LINE" >> "$OUTPUT_FILE" + fi fi - done <<< "$FEATURES" + done else echo "- No new features in this release" >> "$OUTPUT_FILE" fi @@ -82,15 +259,17 @@ cat >> "$OUTPUT_FILE" << EOF EOF if [ -n "$FIXES" ]; then - while IFS= read -r line; do - PR=$(extract_pr_number "$line") - CLEAN_LINE=$(echo "$line" | sed -E 's/ \(#[0-9]+\)$//') - if [ -n "$PR" ]; then - echo "- $CLEAN_LINE ([#$PR](https://github.com/influxdata/influxdb/pull/$PR))" >> "$OUTPUT_FILE" - else - echo "- $CLEAN_LINE" >> "$OUTPUT_FILE" + echo "$FIXES" | while IFS= read -r line; do + if [ -n "$line" ]; then + PR=$(extract_pr_number "$line") + CLEAN_LINE=$(echo "$line" | sed -E 's/ \(#[0-9]+\)$//') + if [ -n "$PR" ]; then + echo "$CLEAN_LINE ([#$PR](https://github.com/influxdata/influxdb/pull/$PR))" >> "$OUTPUT_FILE" + else + echo "$CLEAN_LINE" >> "$OUTPUT_FILE" + fi fi - done <<< "$FIXES" + done else echo "- No bug fixes in this release" >> "$OUTPUT_FILE" fi @@ -102,15 +281,17 @@ if [ -n "$BREAKING" ]; then ### Breaking Changes EOF - while IFS= read -r line; do - PR=$(extract_pr_number "$line") - CLEAN_LINE=$(echo "$line" | sed -E 's/ \(#[0-9]+\)$//') - if [ -n "$PR" ]; then - echo "- $CLEAN_LINE ([#$PR](https://github.com/influxdata/influxdb/pull/$PR))" >> "$OUTPUT_FILE" - else - echo "- $CLEAN_LINE" >> "$OUTPUT_FILE" + echo "$BREAKING" | while IFS= read -r line; do + if [ -n "$line" ]; then + PR=$(extract_pr_number "$line") + CLEAN_LINE=$(echo "$line" | sed -E 's/ \(#[0-9]+\)$//') + if [ -n "$PR" ]; then + echo "$CLEAN_LINE ([#$PR](https://github.com/influxdata/influxdb/pull/$PR))" >> "$OUTPUT_FILE" + else + echo "$CLEAN_LINE" >> "$OUTPUT_FILE" + fi fi - done <<< "$BREAKING" + done fi # Add performance improvements if any @@ -120,16 +301,54 @@ if [ -n "$PERF" ]; then ### Performance Improvements EOF - while IFS= read -r line; do - PR=$(extract_pr_number "$line") - CLEAN_LINE=$(echo "$line" | sed -E 's/ \(#[0-9]+\)$//') - if [ -n "$PR" ]; then - echo "- $CLEAN_LINE ([#$PR](https://github.com/influxdata/influxdb/pull/$PR))" >> "$OUTPUT_FILE" - else - echo "- $CLEAN_LINE" >> "$OUTPUT_FILE" + echo "$PERF" | while IFS= read -r line; do + if [ -n "$line" ]; then + PR=$(extract_pr_number "$line") + CLEAN_LINE=$(echo "$line" | sed -E 's/ \(#[0-9]+\)$//') + if [ -n "$PR" ]; then + echo "$CLEAN_LINE ([#$PR](https://github.com/influxdata/influxdb/pull/$PR))" >> "$OUTPUT_FILE" + else + echo "$CLEAN_LINE" >> "$OUTPUT_FILE" + fi fi - done <<< "$PERF" + done fi +# Add HTTP API changes if any +if [ -n "$API_CHANGES" ]; then + cat >> "$OUTPUT_FILE" << EOF + +### HTTP API Changes + +EOF + echo "$API_CHANGES" | while IFS= read -r line; do + if [ -n "$line" ]; then + PR=$(extract_pr_number "$line") + CLEAN_LINE=$(echo "$line" | sed -E 's/ \(#[0-9]+\)$//') + if [ -n "$PR" ]; then + echo "$CLEAN_LINE ([#$PR](https://github.com/influxdata/influxdb/pull/$PR))" >> "$OUTPUT_FILE" + else + echo "$CLEAN_LINE" >> "$OUTPUT_FILE" + fi + fi + done +fi + +# Add API analysis summary +cat >> "$OUTPUT_FILE" << EOF + +### API Analysis Summary + +The following endpoints may have been affected in this release: +- v1 API endpoints: \`/write\`, \`/query\`, \`/ping\` +- v2 API endpoints: \`/api/v2/write\`, \`/api/v2/query\` +- v3 API endpoints: \`/api/v3/*\` +- System endpoints: \`/health\`, \`/metrics\` + +Please review the commit details above and consult the API documentation for specific changes. + +EOF + echo -e "\n${GREEN}Release notes generated in: ${OUTPUT_FILE}${NC}" -echo -e "${YELLOW}Please review and edit the generated notes before adding to documentation.${NC}" \ No newline at end of file +echo -e "${YELLOW}Please review and edit the generated notes before adding to documentation.${NC}" +echo -e "${BLUE}API changes have been automatically detected and included.${NC}" \ No newline at end of file From 9588022cebdf701826b6b62a3a8b13bebc6d7b5f Mon Sep 17 00:00:00 2001 From: Jason Stirnaman Date: Mon, 30 Jun 2025 08:29:15 -0500 Subject: [PATCH 11/16] fix(ent3): Update Ent3 API reference request trigger_specification syntax to string request: --- api-docs/influxdb3/enterprise/v3/ref.yml | 208 +++++++++++++++++++++-- 1 file changed, 193 insertions(+), 15 deletions(-) diff --git a/api-docs/influxdb3/enterprise/v3/ref.yml b/api-docs/influxdb3/enterprise/v3/ref.yml index 9f1a7fca1..92a65905a 100644 --- a/api-docs/influxdb3/enterprise/v3/ref.yml +++ b/api-docs/influxdb3/enterprise/v3/ref.yml @@ -922,9 +922,25 @@ paths: summary: Delete a database description: | Soft deletes a database. - The database is scheduled for deletion and unavailable for querying. + The database is scheduled for deletion and unavailable for querying. + Use the `hard_delete_at` parameter to schedule a hard deletion. parameters: - $ref: '#/components/parameters/db' + - name: hard_delete_at + in: query + required: false + schema: + type: string + format: date-time + description: | + Schedule the database for hard deletion at the specified time. + If not provided, the database will be soft deleted. + Use ISO 8601 date-time format (for example, "2025-12-31T23:59:59Z"). + + #### Deleting a database cannot be undone + + Deleting a database is a destructive action. + Once a database is deleted, data stored in that database cannot be recovered. responses: '200': description: Success. Database deleted. @@ -961,7 +977,13 @@ paths: summary: Delete a table description: | Soft deletes a table. - The table is scheduled for deletion and unavailable for querying. + The table is scheduled for deletion and unavailable for querying. + Use the `hard_delete_at` parameter to schedule a hard deletion. + + #### Deleting a table cannot be undone + + Deleting a table is a destructive action. + Once a table is deleted, data stored in that table cannot be recovered. parameters: - $ref: '#/components/parameters/db' - name: table @@ -969,6 +991,16 @@ paths: required: true schema: type: string + - name: hard_delete_at + in: query + required: false + schema: + type: string + format: date-time + description: | + Schedule the table for hard deletion at the specified time. + If not provided, the table will be soft deleted. + Use ISO 8601 format (for example, "2025-12-31T23:59:59Z"). responses: '200': description: Success (no content). The table has been deleted. @@ -978,6 +1010,77 @@ paths: description: Table not found. tags: - Table + patch: + operationId: PatchConfigureTable + summary: Update a table + description: | + Updates table configuration, such as retention period. + requestBody: + required: true + content: + application/json: + schema: + $ref: '#/components/schemas/UpdateTableRequest' + responses: + '200': + description: Success. The table has been updated. + '400': + description: Bad request. + '401': + $ref: '#/components/responses/Unauthorized' + '404': + description: Table not found. + tags: + - Table + /api/v3/configure/database/{db}: + patch: + operationId: PatchConfigureDatabase + summary: Update a database + description: | + Updates database configuration, such as retention period. + parameters: + - name: db + in: path + required: true + schema: + type: string + description: The name of the database to update. + requestBody: + required: true + content: + application/json: + schema: + $ref: '#/components/schemas/UpdateDatabaseRequest' + responses: + '200': + description: Success. The database has been updated. + '400': + description: Bad request. + '401': + $ref: '#/components/responses/Unauthorized' + '404': + description: Database not found. + tags: + - Database + /api/v3/show/license: + get: + operationId: GetShowLicense + summary: Show license information + description: | + Retrieves information about the current InfluxDB 3 Enterprise license. + responses: + '200': + description: Success. The response body contains license information. + content: + application/json: + schema: + $ref: '#/components/schemas/LicenseResponse' + '401': + $ref: '#/components/responses/Unauthorized' + '403': + description: Access denied. + tags: + - Server information /api/v3/configure/distinct_cache: post: operationId: PostConfigureDistinctCache @@ -1136,7 +1239,7 @@ paths: db: mydb plugin_filename: request.py trigger_name: hello_world_trigger - trigger_specification: path:hello-world + trigger_specification: "request:hello-world" cron_friday_afternoon: summary: Cron trigger for Friday afternoons description: | @@ -1365,16 +1468,16 @@ paths: description: Plugin not enabled. tags: - Processing engine - /api/v3/engine/{plugin_path}: + /api/v3/engine/{request_path}: parameters: - - name: plugin_path + - name: request_path description: | - The path configured in the request trigger specification "path:"` for the plugin. + The path configured in the request trigger specification "request:"` for the plugin. For example, if you define a trigger with the following: ```json - trigger-spec: "path:hello-world" + trigger_specification: "request:hello-world" ``` then, the HTTP API exposes the following plugin endpoint: @@ -1390,7 +1493,7 @@ paths: operationId: GetProcessingEnginePluginRequest summary: On Request processing engine plugin request description: | - Executes the On Request processing engine plugin specified in ``. + Executes the On Request processing engine plugin specified in the trigger's `plugin_filename`. The request can include request headers, query string parameters, and a request body, which InfluxDB passes to the plugin. An On Request plugin implements the following signature: @@ -1417,7 +1520,7 @@ paths: operationId: PostProcessingEnginePluginRequest summary: On Request processing engine plugin request description: | - Executes the On Request processing engine plugin specified in ``. + Executes the On Request processing engine plugin specified in the trigger's `plugin_filename`. The request can include request headers, query string parameters, and a request body, which InfluxDB passes to the plugin. An On Request plugin implements the following signature: @@ -1812,6 +1915,16 @@ components: properties: db: type: string + pattern: '^[a-zA-Z0-9][a-zA-Z0-9-]*[a-zA-Z0-9]$|^[a-zA-Z0-9]$' + description: |- + The database name. Database names cannot contain underscores (_). + Names must start and end with alphanumeric characters and can contain hyphens (-) in the middle. + retention_period: + type: string + description: |- + The retention period for the database. Specifies how long data should be retained. + Use duration format (for example, "1d", "1h", "30m", "7d"). + example: "7d" required: - db CreateTableRequest: @@ -1843,6 +1956,12 @@ components: required: - name - type + retention_period: + type: string + description: |- + The retention period for the table. Specifies how long data in this table should be retained. + Use duration format (for example, "1d", "1h", "30m", "7d"). + example: "30d" required: - db - table @@ -1929,7 +2048,7 @@ components: `schedule.py` or `endpoints/report.py`. The path can be absolute or relative to the `--plugins-dir` directory configured when starting InfluxDB 3. - The plugin file must implement the trigger interface associated with the trigger's specification (`trigger_spec`). + The plugin file must implement the trigger interface associated with the trigger's specification. trigger_name: type: string trigger_specification: @@ -1972,12 +2091,12 @@ components: - `table:TABLE_NAME` - Triggers on write events to a specific table ### On-demand triggers - Format: `path:ENDPOINT_NAME` + Format: `request:REQUEST_PATH` - Creates an HTTP endpoint `/api/v3/engine/ENDPOINT_NAME` for manual invocation: - - `path:hello-world` - Creates endpoint `/api/v3/engine/hello-world` - - `path:data-export` - Creates endpoint `/api/v3/engine/data-export` - pattern: ^(cron:[0-9 *,/-]+|every:[0-9]+[smhd]|all_tables|table:[a-zA-Z_][a-zA-Z0-9_]*|path:[a-zA-Z0-9_-]+)$ + Creates an HTTP endpoint `/api/v3/engine/REQUEST_PATH` for manual invocation: + - `request:hello-world` - Creates endpoint `/api/v3/engine/hello-world` + - `request:data-export` - Creates endpoint `/api/v3/engine/data-export` + pattern: ^(cron:[0-9 *,/-]+|every:[0-9]+[smhd]|all_tables|table:[a-zA-Z_][a-zA-Z0-9_]*|request:[a-zA-Z0-9_-]+)$ example: cron:0 0 6 * * 1-5 trigger_arguments: type: object @@ -2074,6 +2193,65 @@ components: - m - h type: string + UpdateDatabaseRequest: + type: object + properties: + retention_period: + type: string + description: | + The retention period for the database. Specifies how long data should be retained. + Use duration format (for example, "1d", "1h", "30m", "7d"). + example: "7d" + description: Request schema for updating database configuration. + UpdateTableRequest: + type: object + properties: + db: + type: string + description: The name of the database containing the table. + table: + type: string + description: The name of the table to update. + retention_period: + type: string + description: | + The retention period for the table. Specifies how long data in this table should be retained. + Use duration format (for example, "1d", "1h", "30m", "7d"). + example: "30d" + required: + - db + - table + description: Request schema for updating table configuration. + LicenseResponse: + type: object + properties: + license_type: + type: string + description: The type of license (for example, "enterprise", "trial"). + example: "enterprise" + expires_at: + type: string + format: date-time + description: The expiration date of the license in ISO 8601 format. + example: "2025-12-31T23:59:59Z" + features: + type: array + items: + type: string + description: List of features enabled by the license. + example: + - "clustering" + - "processing_engine" + - "advanced_auth" + status: + type: string + enum: + - "active" + - "expired" + - "invalid" + description: The current status of the license. + example: "active" + description: Response schema for license information. responses: Unauthorized: description: Unauthorized access. From 5a44f6c10eb2fd9348ba25a4a869952eadfe4ae3 Mon Sep 17 00:00:00 2001 From: Jason Stirnaman Date: Mon, 30 Jun 2025 08:59:47 -0500 Subject: [PATCH 12/16] fix(influxdb3): Update request trigger specification format for Enterprise MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Due to a bug in InfluxDB 3 Enterprise, the request trigger specification format diverges from InfluxDB 3 Core. Enterprise requires a JSON object format: {"request_path": {"path": "path-name"}} instead of the simpler "request:path-name" format used in Core. This commit updates: - API examples to use the correct JSON object format - Documentation to clearly show the required format - Schema validation to accept both string and object formats - Added clarifying note about the format difference Related to influxdata/influxdb#6171 which will address this in the 3.2.1 release. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude --- api-docs/influxdb3/enterprise/v3/ref.yml | 33 ++++++++++++++++++------ 1 file changed, 25 insertions(+), 8 deletions(-) diff --git a/api-docs/influxdb3/enterprise/v3/ref.yml b/api-docs/influxdb3/enterprise/v3/ref.yml index 92a65905a..78ae2aad3 100644 --- a/api-docs/influxdb3/enterprise/v3/ref.yml +++ b/api-docs/influxdb3/enterprise/v3/ref.yml @@ -1239,7 +1239,8 @@ paths: db: mydb plugin_filename: request.py trigger_name: hello_world_trigger - trigger_specification: "request:hello-world" + # trigger_specification: "request:hello-world" - For 3.2.1 (issue#6171) + trigger_specification: '{"request_path": {"path": "hello-world"}}' cron_friday_afternoon: summary: Cron trigger for Friday afternoons description: | @@ -1472,19 +1473,22 @@ paths: parameters: - name: request_path description: | - The path configured in the request trigger specification "request:"` for the plugin. + The path configured in the request trigger specification for the plugin. For example, if you define a trigger with the following: ```json - trigger_specification: "request:hello-world" + trigger_specification: {"request_path": {"path": "hello-world"}} ``` - + then, the HTTP API exposes the following plugin endpoint: ``` /api/v3/engine/hello-world ``` + + ***Note:*** Currently, due to a bug in InfluxDB 3 Enterprise, the request trigger specification is different from Core. + in: path required: true schema: @@ -2091,12 +2095,25 @@ components: - `table:TABLE_NAME` - Triggers on write events to a specific table ### On-demand triggers - Format: `request:REQUEST_PATH` + Format: `{"request_path": {"path": "REQUEST_PATH"}}` Creates an HTTP endpoint `/api/v3/engine/REQUEST_PATH` for manual invocation: - - `request:hello-world` - Creates endpoint `/api/v3/engine/hello-world` - - `request:data-export` - Creates endpoint `/api/v3/engine/data-export` - pattern: ^(cron:[0-9 *,/-]+|every:[0-9]+[smhd]|all_tables|table:[a-zA-Z_][a-zA-Z0-9_]*|request:[a-zA-Z0-9_-]+)$ + - `{"request_path": {"path": "hello-world"}}` - Creates endpoint `/api/v3/engine/hello-world` + - `{"request_path": {"path": "data-export"}}` - Creates endpoint `/api/v3/engine/data-export` + + ***Note:*** Currently, due to a bug in InfluxDB 3 Enterprise, the request trigger specification is different from Core. Use the JSON object format shown above. + + oneOf: + - type: string + pattern: ^(cron:[0-9 *,/-]+|every:[0-9]+[smhd]|all_tables|table:[a-zA-Z_][a-zA-Z0-9_]*)$ + - type: object + properties: + request_path: + type: object + properties: + path: + type: string + pattern: ^[a-zA-Z0-9_-]+$ example: cron:0 0 6 * * 1-5 trigger_arguments: type: object From 1db855046229ad828f756fc943919955a43c1606 Mon Sep 17 00:00:00 2001 From: Jason Stirnaman Date: Mon, 30 Jun 2025 13:33:52 -0500 Subject: [PATCH 13/16] fix(influxdb3): Update API reference and fix processing engine documentation for 3.2 release MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Resolves issue #6161 with InfluxDB 3.2 API reference updates. ## API Reference Updates - Add `hard_delete_at` parameter to database and table deletion endpoints for scheduled hard deletion - Add update database and update table endpoints with retention period configuration - Add license endpoint response schema - Fix request trigger specification format from `path:` to `request:` in Core API - Fix OpenAPI schema validation error in Enterprise API by removing conflicting type declaration ## Processing Engine Documentation Fixes - Standardize terminology across documentation: - "Data write" (not "WAL flush" or "On WAL flush") - "Scheduled" (not "On Schedule") - "HTTP request" (not "On Request") - Fix placeholder inconsistencies: use REQUEST_PATH instead of ENDPOINT_PATH - Add Enterprise-specific warnings about request trigger format differences - Update trigger specification format in Enterprise API to use JSON object format - Add proper table of contents with consistent structure - Improve distributed cluster configuration documentation ## Enterprise Request Trigger Bug Documentation Due to a bug in InfluxDB 3 Enterprise, the request trigger specification format differs between CLI and API: - CLI: `request:` (same as Core CLI and API) - Enterprise API: `{"request_path": {"path": ""}}` This difference is now properly documented with warnings in all relevant sections. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude --- api-docs/influxdb3/core/v3/ref.yml | 123 +++++++++++++++--- api-docs/influxdb3/enterprise/v3/ref.yml | 3 +- .../shared/influxdb3-cli/create/trigger.md | 14 +- .../processing-engine.md | 39 ++++-- content/shared/v3-core-plugins/_index.md | 98 +++++++++----- test/influxdb3/engine.test.sh | 20 +++ 6 files changed, 235 insertions(+), 62 deletions(-) create mode 100644 test/influxdb3/engine.test.sh diff --git a/api-docs/influxdb3/core/v3/ref.yml b/api-docs/influxdb3/core/v3/ref.yml index 3f4a738e8..e15c42f9f 100644 --- a/api-docs/influxdb3/core/v3/ref.yml +++ b/api-docs/influxdb3/core/v3/ref.yml @@ -922,9 +922,25 @@ paths: summary: Delete a database description: | Soft deletes a database. - The database is scheduled for deletion and unavailable for querying. + The database is scheduled for deletion and unavailable for querying. + Use the `hard_delete_at` parameter to schedule a hard deletion. parameters: - $ref: '#/components/parameters/db' + - name: hard_delete_at + in: query + required: false + schema: + type: string + format: date-time + description: | + Schedule the database for hard deletion at the specified time. + If not provided, the database will be soft deleted. + Use ISO 8601 date-time format (for example, "2025-12-31T23:59:59Z"). + + #### Deleting a database cannot be undone + + Deleting a database is a destructive action. + Once a database is deleted, data stored in that database cannot be recovered. responses: '200': description: Success. Database deleted. @@ -961,7 +977,13 @@ paths: summary: Delete a table description: | Soft deletes a table. - The table is scheduled for deletion and unavailable for querying. + The table is scheduled for deletion and unavailable for querying. + Use the `hard_delete_at` parameter to schedule a hard deletion. + + #### Deleting a table cannot be undone + + Deleting a table is a destructive action. + Once a table is deleted, data stored in that table cannot be recovered. parameters: - $ref: '#/components/parameters/db' - name: table @@ -969,6 +991,16 @@ paths: required: true schema: type: string + - name: hard_delete_at + in: query + required: false + schema: + type: string + format: date-time + description: | + Schedule the table for hard deletion at the specified time. + If not provided, the table will be soft deleted. + Use ISO 8601 format (for example, "2025-12-31T23:59:59Z"). responses: '200': description: Success (no content). The table has been deleted. @@ -1078,7 +1110,7 @@ paths: In `"cron:CRON_EXPRESSION"`, `CRON_EXPRESSION` uses extended 6-field cron format. The cron expression `0 0 6 * * 1-5` means the trigger will run at 6:00 AM every weekday (Monday to Friday). value: - db: DATABASE_NAME + db: mydb plugin_filename: schedule.py trigger_name: schedule_cron_trigger trigger_specification: cron:0 0 6 * * 1-5 @@ -1136,7 +1168,7 @@ paths: db: mydb plugin_filename: request.py trigger_name: hello_world_trigger - trigger_specification: path:hello-world + trigger_specification: request:hello-world cron_friday_afternoon: summary: Cron trigger for Friday afternoons description: | @@ -1365,16 +1397,16 @@ paths: description: Plugin not enabled. tags: - Processing engine - /api/v3/engine/{plugin_path}: + /api/v3/engine/{request_path}: parameters: - - name: plugin_path + - name: request_path description: | - The path configured in the request trigger specification "path:"` for the plugin. + The path configured in the request trigger specification for the plugin. For example, if you define a trigger with the following: ```json - trigger-spec: "path:hello-world" + trigger_specification: "request:hello-world" ``` then, the HTTP API exposes the following plugin endpoint: @@ -1390,7 +1422,7 @@ paths: operationId: GetProcessingEnginePluginRequest summary: On Request processing engine plugin request description: | - Executes the On Request processing engine plugin specified in ``. + Executes the On Request processing engine plugin specified in the trigger's `plugin_filename`. The request can include request headers, query string parameters, and a request body, which InfluxDB passes to the plugin. An On Request plugin implements the following signature: @@ -1417,7 +1449,7 @@ paths: operationId: PostProcessingEnginePluginRequest summary: On Request processing engine plugin request description: | - Executes the On Request processing engine plugin specified in ``. + Executes the On Request processing engine plugin specified in the trigger's `plugin_filename`. The request can include request headers, query string parameters, and a request body, which InfluxDB passes to the plugin. An On Request plugin implements the following signature: @@ -1868,7 +1900,7 @@ components: `schedule.py` or `endpoints/report.py`. The path can be absolute or relative to the `--plugins-dir` directory configured when starting InfluxDB 3. - The plugin file must implement the trigger interface associated with the trigger's specification (`trigger_spec`). + The plugin file must implement the trigger interface associated with the trigger's specification. trigger_name: type: string trigger_specification: @@ -1911,12 +1943,12 @@ components: - `table:TABLE_NAME` - Triggers on write events to a specific table ### On-demand triggers - Format: `path:ENDPOINT_NAME` + Format: `request:REQUEST_PATH` - Creates an HTTP endpoint `/api/v3/engine/ENDPOINT_NAME` for manual invocation: - - `path:hello-world` - Creates endpoint `/api/v3/engine/hello-world` - - `path:data-export` - Creates endpoint `/api/v3/engine/data-export` - pattern: ^(cron:[0-9 *,/-]+|every:[0-9]+[smhd]|all_tables|table:[a-zA-Z_][a-zA-Z0-9_]*|path:[a-zA-Z0-9_-]+)$ + Creates an HTTP endpoint `/api/v3/engine/REQUEST_PATH` for manual invocation: + - `request:hello-world` - Creates endpoint `/api/v3/engine/hello-world` + - `request:data-export` - Creates endpoint `/api/v3/engine/data-export` + pattern: ^(cron:[0-9 *,/-]+|every:[0-9]+[smhd]|all_tables|table:[a-zA-Z_][a-zA-Z0-9_]*|request:[a-zA-Z0-9_-]+)$ example: cron:0 0 6 * * 1-5 trigger_arguments: type: object @@ -2013,6 +2045,65 @@ components: - m - h type: string + UpdateDatabaseRequest: + type: object + properties: + retention_period: + type: string + description: | + The retention period for the database. Specifies how long data should be retained. + Use duration format (for example, "1d", "1h", "30m", "7d"). + example: "7d" + description: Request schema for updating database configuration. + UpdateTableRequest: + type: object + properties: + db: + type: string + description: The name of the database containing the table. + table: + type: string + description: The name of the table to update. + retention_period: + type: string + description: | + The retention period for the table. Specifies how long data in this table should be retained. + Use duration format (for example, "1d", "1h", "30m", "7d"). + example: "30d" + required: + - db + - table + description: Request schema for updating table configuration. + LicenseResponse: + type: object + properties: + license_type: + type: string + description: The type of license (for example, "enterprise", "trial"). + example: "enterprise" + expires_at: + type: string + format: date-time + description: The expiration date of the license in ISO 8601 format. + example: "2025-12-31T23:59:59Z" + features: + type: array + items: + type: string + description: List of features enabled by the license. + example: + - "clustering" + - "processing_engine" + - "advanced_auth" + status: + type: string + enum: + - "active" + - "expired" + - "invalid" + description: The current status of the license. + example: "active" + description: Response schema for license information. responses: Unauthorized: description: Unauthorized access. diff --git a/api-docs/influxdb3/enterprise/v3/ref.yml b/api-docs/influxdb3/enterprise/v3/ref.yml index 78ae2aad3..b1cb43d7c 100644 --- a/api-docs/influxdb3/enterprise/v3/ref.yml +++ b/api-docs/influxdb3/enterprise/v3/ref.yml @@ -1240,7 +1240,7 @@ paths: plugin_filename: request.py trigger_name: hello_world_trigger # trigger_specification: "request:hello-world" - For 3.2.1 (issue#6171) - trigger_specification: '{"request_path": {"path": "hello-world"}}' + trigger_specification: {"request_path": {"path": "hello-world"}} cron_friday_afternoon: summary: Cron trigger for Friday afternoons description: | @@ -2056,7 +2056,6 @@ components: trigger_name: type: string trigger_specification: - type: string description: | Specifies when and how the processing engine trigger should be invoked. diff --git a/content/shared/influxdb3-cli/create/trigger.md b/content/shared/influxdb3-cli/create/trigger.md index dbb223128..baaadd91c 100644 --- a/content/shared/influxdb3-cli/create/trigger.md +++ b/content/shared/influxdb3-cli/create/trigger.md @@ -27,7 +27,7 @@ influxdb3 create trigger [OPTIONS] \ | `-d` | `--database` | _({{< req >}})_ Name of the database to operate on | | | `--token` | _({{< req >}})_ Authentication token | | | `--plugin-filename` | _({{< req >}})_ Name of the file, stored in the server's `plugin-dir`, that contains the Python plugin code to run | -| | `--trigger-spec` | Trigger specification--for example `table:` or `all_tables` | +| | `--trigger-spec` | Trigger specification: `table:`, `all_tables`, `every:`, `cron:`, or `request:` | | | `--disabled` | Create the trigger in disabled state | | | `--tls-ca` | Path to a custom TLS certificate authority (for testing or self-signed certificates) | | `-h` | `--help` | Print help information | @@ -113,3 +113,15 @@ influxdb3 create trigger \ Creating a trigger in a disabled state prevents it from running immediately. You can enable it later when you're ready to activate it. {{% /code-placeholders %}} + +{{% show-in "enterprise" %}} +> [!Warning] +> #### Request trigger specification format differs between CLI and API +> +> Due to a bug in InfluxDB 3 Enterprise, the request trigger specification format differs: +> +> - **CLI**: `request:` (same as Core CLI and API) +> - **Enterprise API**: `{"request_path": {"path": ""}}` +> +> See the [API reference](/influxdb3/enterprise/api/#operation/PostConfigureProcessingEngineTrigger) for examples. Use `influxdb3 show summary` to verify the actual trigger specification. +{{% /show-in %}} diff --git a/content/shared/influxdb3-get-started/processing-engine.md b/content/shared/influxdb3-get-started/processing-engine.md index 4365b35de..1e1121762 100644 --- a/content/shared/influxdb3-get-started/processing-engine.md +++ b/content/shared/influxdb3-get-started/processing-engine.md @@ -23,24 +23,36 @@ engine [trigger](#trigger). ### Trigger When you create a trigger, you specify a [plugin](#plugin), a database, optional -arguments, and a _trigger-spec_, which defines when the plugin is executed and +arguments, and a trigger specification, which defines when the plugin is executed and what data it receives. #### Trigger types InfluxDB 3 provides the following types of triggers, each with specific -trigger-specs: +specifications: -- **On WAL flush**: Sends a batch of written data (for a specific table or all - tables) to a plugin (by default, every second). -- **On Schedule**: Executes a plugin on a user-configured schedule (using a +- **Data write** (`table:` or `all_tables`): Sends a batch of written data (for a specific table or all + tables) to a plugin when the database flushes data to the Write-Ahead Log (by default, every second). +- **Scheduled** (`every:` or `cron:`): Executes a plugin on a user-configured schedule (using a crontab or a duration). This trigger type is useful for data collection and deadman monitoring. -- **On Request**: Binds a plugin to a custom HTTP API endpoint at - `/api/v3/engine/`. +- **HTTP request** (`request:`): Binds a plugin to a custom HTTP API endpoint at + `/api/v3/engine/`. The plugin receives the HTTP request headers and content, and can parse, process, and send the data into the database or to third-party services. +{{% show-in "enterprise" %}} +> [!Warning] +> #### Request trigger specification format differs between CLI and API +> +> Due to a bug in InfluxDB 3 Enterprise, the request trigger specification format differs: +> +> - **CLI**: `request:` (same as Core CLI and API) +> - **Enterprise API**: `{"request_path": {"path": ""}}` +> +> See the [API reference](/influxdb3/enterprise/api/#operation/PostConfigureProcessingEngineTrigger) for examples. Use `influxdb3 show summary` to verify the actual trigger specification. +{{% /show-in %}} + ## Activate the processing engine To activate the processing engine, include the `--plugin-dir ` option @@ -64,10 +76,10 @@ to the current working directory of the `influxdb3` server. ## Create a plugin To create a plugin, write and store a Python file in your configured `PLUGIN_DIR`. -The following example is a WAL flush plugin that processes data before it gets +The following example is a data write plugin that processes data before it gets persisted to the object store. -##### Example Python plugin for WAL rows +##### Example Python plugin for data writes ```python # This is the basic structure for Python plugin code that runs in the @@ -77,9 +89,9 @@ persisted to the object store. # allowing you to write generic code that uses variables such as monitoring # thresholds, environment variables, and host names. # -# Use the following exact signature to define a function for the WAL flush +# Use the following exact signature to define a function for the data write # trigger. -# When you create a trigger for a WAL flush plugin, you specify the database +# When you create a trigger for a data write plugin, you specify the database # and tables that the plugin receives written data from on every WAL flush # (default is once per second). def process_writes(influxdb3_local, table_batches, args=None): @@ -98,9 +110,8 @@ def process_writes(influxdb3_local, table_batches, args=None): # value. influxdb3_local.info("query result: " + str(query_result)) - # this is the data that is sent when the WAL is flushed of writes the server - # received for the DB or table of interest. One batch for each table (will - # only be one if triggered on a single table) + # this is the data that is sent when data is written to the database and flushed to the WAL. + # One batch for each table (will only be one if triggered on a single table) for table_batch in table_batches: # here you can see that the table_name is available. influxdb3_local.info("table: " + table_batch["table_name"]) diff --git a/content/shared/v3-core-plugins/_index.md b/content/shared/v3-core-plugins/_index.md index 4e4c1a762..32cb2c37b 100644 --- a/content/shared/v3-core-plugins/_index.md +++ b/content/shared/v3-core-plugins/_index.md @@ -22,20 +22,26 @@ Ensure you have: Once you have all the prerequisites in place, follow these steps to implement the Processing Engine for your data automation needs. -1. [Set up the Processing Engine](#set-up-the-processing-engine) -2. [Add a Processing Engine plugin](#add-a-processing-engine-plugin) - - [Use example plugins](#use-example-plugins) - - [Create a custom plugin](#create-a-custom-plugin) -3. [Set up a trigger](#set-up-a-trigger) - - [Understand trigger types](#understand-trigger-types) - - [Use the create trigger command](#use-the-create-trigger-command) - - [Trigger specification examples](#trigger-specification-examples) -4. [Advanced trigger configuration](#advanced-trigger-configuration) - - [Access community plugins from GitHub](#access-community-plugins-from-github) - - [Pass arguments to plugins](#pass-arguments-to-plugins) - - [Control trigger execution](#control-trigger-execution) - - [Configure error handling for a trigger](#configure-error-handling-for-a-trigger) - - [Install Python dependencies](#install-python-dependencies) +- [Set up the Processing Engine](#set-up-the-processing-engine) + - [Configure distributed environments](#configure-distributed-environments) +- [Add a Processing Engine plugin](#add-a-processing-engine-plugin) + - [Choose a plugin strategy](#choose-a-plugin-strategy) + - [Use example plugins](#use-example-plugins) + - [Create a custom plugin](#create-a-custom-plugin) +- [Set up a trigger](#set-up-a-trigger) + - [Understand trigger types](#understand-trigger-types) + - [Use the create trigger command](#use-the-create-trigger-command) + - [Trigger specification examples](#trigger-specification-examples) + - [Pass arguments to plugins](#pass-arguments-to-plugins) + - [Control trigger execution](#control-trigger-execution) + - [Configure error handling for a trigger](#configure-error-handling-for-a-trigger) +- [Advanced trigger configuration](#advanced-trigger-configuration) + - [Access community plugins from GitHub](#access-community-plugins-from-github) + - [Configure your triggers](#configure-your-triggers) + - [Install Python dependencies](#install-python-dependencies) +- [Distributed cluster considerations](#distributed-cluster-considerations) + - [Match plugin types to the correct node](#match-plugin-types-to-the-correct-node) + - [Route third-party clients to querier nodes](#route-third-party-clients-to-querier-nodes) ## Set up the Processing Engine @@ -75,6 +81,8 @@ When running {{% product-name %}} in a distributed setup, follow these steps to > > Configure your plugin directory on the same system as the nodes that run the triggers and plugins. +For more information about configuring distributed environments, see the [Distributed cluster considerations](#distributed-cluster-considerations) section. + ## Add a Processing Engine plugin A plugin is a Python script that defines a specific function signature for a trigger (_trigger spec_). When the specified event occurs, InfluxDB runs the plugin. @@ -168,11 +176,11 @@ Before you begin, make sure: Choose a plugin type based on your automation goals: -| Plugin Type | Best For | Trigger Type | -| ---------------- | ------------------------------------------- | ------------------------ | -| **Data write** | Processing data as it arrives | `table:` or `all_tables` | -| **Scheduled** | Running code at specific intervals or times | `every:` or `cron:` | -| **HTTP request** | Running code on demand via API endpoints | `path:` | +| Plugin Type | Best For | +| ---------------- | ------------------------------------------- | +| **Data write** | Processing data as it arrives | +| **Scheduled** | Running code at specific intervals or times | +| **HTTP request** | Running code on demand via API endpoints | #### Create your plugin file @@ -184,7 +192,7 @@ After writing your plugin, [create a trigger](#use-the-create-trigger-command) t #### Create a data write plugin -Use a data write plugin to process data as it's written to the database. Ideal use cases include: +Use a data write plugin to process data as it's written to the database. These plugins use [`table:` or `all_tables:`](#trigger-on-data-writes) trigger specifications. Ideal use cases include: - Data transformation and enrichment - Alerting on incoming values @@ -209,7 +217,7 @@ def process_writes(influxdb3_local, table_batches, args=None): #### Create a scheduled plugin -Scheduled plugins run at defined intervals. Use them for: +Scheduled plugins run at defined intervals using [`every:` or `cron:`](#trigger-on-a-schedule) trigger specifications. Use them for: - Periodic data aggregation - Report generation @@ -231,7 +239,7 @@ def process_scheduled_call(influxdb3_local, call_time, args=None): #### Create an HTTP request plugin -HTTP request plugins respond to API calls. Use them for: +HTTP request plugins respond to API calls using [`request:`](#trigger-on-http-requests) trigger specifications{{% show-in "enterprise" %}} (CLI) or `{"request_path": {"path": "..."}}` (API){{% /show-in %}}. Use them for: - Creating custom API endpoints - Webhooks for external integrations @@ -270,7 +278,7 @@ After writing your plugin: |------------|----------------------|-----------------| | Data write | `table:` or `all_tables` | When data is written to tables | | Scheduled | `every:` or `cron:` | At specified time intervals | -| HTTP request | `path:` | When HTTP requests are received | +| HTTP request | `request:`{{% show-in "enterprise" %}} (CLI) or `{"request_path": {"path": ""}}`{{% /show-in %}} | When HTTP requests are received | ### Use the create trigger command @@ -302,7 +310,7 @@ In the example above, replace the following: ### Trigger specification examples -#### Data write example +#### Trigger on data writes ```bash # Trigger on writes to a specific table @@ -325,13 +333,13 @@ The trigger runs when the database flushes ingested data for the specified table The plugin receives the written data and table information. -#### Scheduled events example +#### Trigger on a schedule ```bash # Run every 5 minutes influxdb3 create trigger \ --trigger-spec "every:5m" \ - --plugin-filename "hourly_check.py" \ + --plugin-filename "periodic_check.py" \ --database my_database \ regular_check @@ -346,7 +354,7 @@ influxdb3 create trigger \ The plugin receives the scheduled call time. -#### HTTP requests example +#### Trigger on HTTP requests ```bash # Create an endpoint at /api/v3/engine/webhook @@ -357,7 +365,9 @@ influxdb3 create trigger \ webhook_processor ``` -Access your endpoint available at `/api/v3/engine/`. +Access your endpoint at `/api/v3/engine/{REQUEST_PATH}` (in this example, `/api/v3/engine/webhook`). +The trigger is enabled by default and runs when an HTTP request is received at the specified path. + To run the plugin, send a `GET` or `POST` request to the endpoint--for example: ```bash @@ -366,6 +376,24 @@ curl http://{{% influxdb/host %}}/api/v3/engine/webhook The plugin receives the HTTP request object with methods, headers, and body. +To view triggers associated with a database, use the `influxdb3 show summary` command: + +```bash +influxdb3 show summary --database my_database --token AUTH_TOKEN +``` + +{{% show-in "enterprise" %}} +> [!Warning] +> #### Request trigger specification format differs between CLI and API +> +> Due to a bug in InfluxDB 3 Enterprise, the request trigger specification format differs: +> +> - **CLI**: `request:` (same as Core CLI and API) +> - **Enterprise API**: `{"request_path": {"path": ""}}` +> +> See the [API reference](/influxdb3/enterprise/api/#operation/PostConfigureProcessingEngineTrigger) for examples. Use `influxdb3 show summary` to verify the actual trigger specification. +{{% /show-in %}} + ### Pass arguments to plugins Use trigger arguments to pass configuration from a trigger to the plugin it runs. You can use this for: @@ -587,7 +615,19 @@ Each plugin must run on a node that supports its trigger type: |--------------------|--------------------------|-----------------------------| | Data write | `table:` or `all_tables` | Ingester nodes | | Scheduled | `every:` or `cron:` | Any node with scheduler | -| HTTP request | `path:` | Nodes that serve API traffic| +| HTTP request | `request:`{{% show-in "enterprise" %}} (CLI) or `{"request_path": {"path": "..."}}`{{% /show-in %}} | Nodes that serve API traffic| + +{{% show-in "enterprise" %}} +> [!Note] +> #### Request trigger specification format differs between CLI and API +> +> Due to a bug in InfluxDB 3 Enterprise, the request trigger specification format differs: +> +> - **CLI**: `request:` (same as Core CLI and API) +> - **Enterprise API**: `{"request_path": {"path": ""}}` +> +> See the [API reference](/influxdb3/enterprise/api/#operation/PostConfigureProcessingEngineTrigger) for examples. +{{% /show-in %}} For example: - Run write-ahead log (WAL) plugins on ingester nodes. diff --git a/test/influxdb3/engine.test.sh b/test/influxdb3/engine.test.sh new file mode 100644 index 000000000..8732ac1c3 --- /dev/null +++ b/test/influxdb3/engine.test.sh @@ -0,0 +1,20 @@ +# Create a processing engine request trigger +# // SECTION - influxdb3-core +curl -v -X POST "http://localhost:8181/api/v3/configure/processingengine/trigger" \ + --header "Authorization: Bearer ${INFLUXDB3_ENTERPRISE_ADMIN_TOKEN}" \ + --json '{ + "db": "sensors", + "plugin_filename": "request.py", + "trigger_name": "Process request trigger", + "trigger_specification": "request:process-request" +}' + +# // SECTION - influxdb3-enterprise +curl -v -X POST "http://localhost:8181/api/v3/configure/processingengine/trigger" \ + --header "Authorization: Bearer ${INFLUXDB3_ENTERPRISE_ADMIN_TOKEN}" \ + --json '{ + "db": "sensors", + "plugin_filename": "request.py", + "trigger_name": "Process request trigger", + "trigger_specification": {"request_path": {"path": "process-request"}} +}' \ No newline at end of file From feea4b8df2cf8ac1b37f8b42677ac0890d35e548 Mon Sep 17 00:00:00 2001 From: Jason Stirnaman Date: Mon, 30 Jun 2025 13:44:19 -0500 Subject: [PATCH 14/16] fix(plugins): Missing descriptor. Cleanup. --- .../processing-engine.md | 2 +- content/shared/v3-core-plugins/_index.md | 21 +++---------------- 2 files changed, 4 insertions(+), 19 deletions(-) diff --git a/content/shared/influxdb3-get-started/processing-engine.md b/content/shared/influxdb3-get-started/processing-engine.md index 1e1121762..61f3399f1 100644 --- a/content/shared/influxdb3-get-started/processing-engine.md +++ b/content/shared/influxdb3-get-started/processing-engine.md @@ -162,7 +162,7 @@ affecting actual data. During a plugin test: To test a plugin: -1. Save the [example plugin code](#example-python-plugin-for-wal-rows) to a +1. Save the [example plugin code](#example-python-plugin-for-data-writes) to a plugin file inside of the plugin directory. If you haven't yet written data to the table in the example, comment out the lines where it queries. 2. To run the test, enter the following command with the following options: diff --git a/content/shared/v3-core-plugins/_index.md b/content/shared/v3-core-plugins/_index.md index 32cb2c37b..f225fdb27 100644 --- a/content/shared/v3-core-plugins/_index.md +++ b/content/shared/v3-core-plugins/_index.md @@ -23,25 +23,10 @@ Ensure you have: Once you have all the prerequisites in place, follow these steps to implement the Processing Engine for your data automation needs. - [Set up the Processing Engine](#set-up-the-processing-engine) - - [Configure distributed environments](#configure-distributed-environments) - [Add a Processing Engine plugin](#add-a-processing-engine-plugin) - - [Choose a plugin strategy](#choose-a-plugin-strategy) - - [Use example plugins](#use-example-plugins) - - [Create a custom plugin](#create-a-custom-plugin) - [Set up a trigger](#set-up-a-trigger) - - [Understand trigger types](#understand-trigger-types) - - [Use the create trigger command](#use-the-create-trigger-command) - - [Trigger specification examples](#trigger-specification-examples) - - [Pass arguments to plugins](#pass-arguments-to-plugins) - - [Control trigger execution](#control-trigger-execution) - - [Configure error handling for a trigger](#configure-error-handling-for-a-trigger) - [Advanced trigger configuration](#advanced-trigger-configuration) - - [Access community plugins from GitHub](#access-community-plugins-from-github) - - [Configure your triggers](#configure-your-triggers) - - [Install Python dependencies](#install-python-dependencies) - [Distributed cluster considerations](#distributed-cluster-considerations) - - [Match plugin types to the correct node](#match-plugin-types-to-the-correct-node) - - [Route third-party clients to querier nodes](#route-third-party-clients-to-querier-nodes) ## Set up the Processing Engine @@ -239,7 +224,7 @@ def process_scheduled_call(influxdb3_local, call_time, args=None): #### Create an HTTP request plugin -HTTP request plugins respond to API calls using [`request:`](#trigger-on-http-requests) trigger specifications{{% show-in "enterprise" %}} (CLI) or `{"request_path": {"path": "..."}}` (API){{% /show-in %}}. Use them for: +HTTP request plugins respond to API calls using [`request:`](#trigger-on-http-requests) trigger specifications{{% show-in "enterprise" %}} (CLI) or `{"request_path": {"path": "..."}}` (HTTP API){{% /show-in %}}. Use them for: - Creating custom API endpoints - Webhooks for external integrations @@ -278,7 +263,7 @@ After writing your plugin: |------------|----------------------|-----------------| | Data write | `table:` or `all_tables` | When data is written to tables | | Scheduled | `every:` or `cron:` | At specified time intervals | -| HTTP request | `request:`{{% show-in "enterprise" %}} (CLI) or `{"request_path": {"path": ""}}`{{% /show-in %}} | When HTTP requests are received | +| HTTP request | `request:`{{% show-in "enterprise" %}} (CLI) or `{"request_path": {"path": ""}}` (HTTP API){{% /show-in %}} | When HTTP requests are received | ### Use the create trigger command @@ -615,7 +600,7 @@ Each plugin must run on a node that supports its trigger type: |--------------------|--------------------------|-----------------------------| | Data write | `table:` or `all_tables` | Ingester nodes | | Scheduled | `every:` or `cron:` | Any node with scheduler | -| HTTP request | `request:`{{% show-in "enterprise" %}} (CLI) or `{"request_path": {"path": "..."}}`{{% /show-in %}} | Nodes that serve API traffic| +| HTTP request | `request:`{{% show-in "enterprise" %}} (CLI) or `{"request_path": {"path": "..."}}` (HTTP API){{% /show-in %}} | Nodes that serve API traffic| {{% show-in "enterprise" %}} > [!Note] From 7cae3660e2ede627eb388ee5aac2e5a7f4e529ce Mon Sep 17 00:00:00 2001 From: Jason Stirnaman Date: Mon, 30 Jun 2025 14:15:45 -0500 Subject: [PATCH 15/16] fix(explorer): broken link --- content/influxdb3/explorer/manage-plugins/_index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/influxdb3/explorer/manage-plugins/_index.md b/content/influxdb3/explorer/manage-plugins/_index.md index afffa12e6..701761a83 100644 --- a/content/influxdb3/explorer/manage-plugins/_index.md +++ b/content/influxdb3/explorer/manage-plugins/_index.md @@ -44,7 +44,7 @@ Use the InfluxDB 3 Explorer UI to enable, disable, or configure triggers for eac - [View installed plugins](#view-installed-plugins) - [Filter installed plugins](#filter-installed-plugins) - [Enable or disable a plugin](#enable-or-disable-a-plugin) -- [ View Plugin Logs](#-view-plugin-logs) +- [ View Plugin Logs](#view-plugin-logs) - [Delete a plugin](#delete-a-plugin) - [Use the Plugin Library](#use-the-plugin-library) From 3405e93ea61e51fdde6dcb70eae8b01b482fe232 Mon Sep 17 00:00:00 2001 From: Jason Stirnaman Date: Mon, 30 Jun 2025 14:57:33 -0500 Subject: [PATCH 16/16] Update test/influxdb3/engine.test.sh Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --- test/influxdb3/engine.test.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/test/influxdb3/engine.test.sh b/test/influxdb3/engine.test.sh index 8732ac1c3..7c884129a 100644 --- a/test/influxdb3/engine.test.sh +++ b/test/influxdb3/engine.test.sh @@ -1,7 +1,7 @@ # Create a processing engine request trigger # // SECTION - influxdb3-core curl -v -X POST "http://localhost:8181/api/v3/configure/processingengine/trigger" \ - --header "Authorization: Bearer ${INFLUXDB3_ENTERPRISE_ADMIN_TOKEN}" \ + --header "Authorization: Bearer ${INFLUXDB3_CORE_ADMIN_TOKEN}" \ --json '{ "db": "sensors", "plugin_filename": "request.py",