Merge remote-tracking branch 'origin/jts-3-2-release-notes' into core-ent-3.2

2025-06-30 14:10:45 -05:00 · 2025-06-30 14:10:45 -05:00 · ae85ae45d1
parent ba1e7f8d2b 18611368cc
commit ae85ae45d1
5 changed files with 436 additions and 0 deletions
--- a/content/shared/v3-core-enterprise-release-notes/_index.md
+++ b/content/shared/v3-core-enterprise-release-notes/_index.md
@ -5,6 +5,57 @@
 > All updates to Core are automatically included in Enterprise.
 > The Enterprise sections below only list updates exclusive to Enterprise.

+## v3.2.0 {date="2025-06-25"}
+
+**Core**: revision 1ca3168bee  
+**Enterprise**: revision 1ca3168bee
+
+### Core
+
+#### Features
+
+- **Hard delete for databases and tables**: Permanently delete databases and tables, enabling complete removal of data structures for compliance and storage management ([#26553](https://github.com/influxdata/influxdb/pull/26553))
+- **AWS credentials auto-reload**: Support dynamic reloading of ephemeral AWS credentials from files, improving security and reliability when using AWS services ([#26537](https://github.com/influxdata/influxdb/pull/26537))
+- **Database retention period support**: Add retention period support for databases via CLI commands (`create database` and `update database` commands) and HTTP APIs ([#26520](https://github.com/influxdata/influxdb/pull/26520)):
+  - New CLI command: `update database --retention-period`
+- **Configurable lookback duration**: Users can specify lookback duration for PersistedFiles buffer, providing better control over query performance ([#26528](https://github.com/influxdata/influxdb/pull/26528))
+- **WAL replay concurrency control**: Add concurrency limits for WAL (Write-Ahead Log) replay to improve startup performance and resource management ([#26483](https://github.com/influxdata/influxdb/pull/26483))
+- **Enhanced write path**: Separate write path executor with unbounded memory for improved write performance ([#26455](https://github.com/influxdata/influxdb/pull/26455))
+
+#### Bug Fixes
+
+- **WAL corruption handling**: Handle corrupt WAL files during replay without panic, improving data recovery and system resilience ([#26556](https://github.com/influxdata/influxdb/pull/26556))
+- **Database naming validation**: Disallow underscores in database names when created via API to ensure consistency ([#26507](https://github.com/influxdata/influxdb/pull/26507))
+- **Object store cleanup**: Automatic intermediate directory cleanup for file object store, preventing storage bloat ([#26480](https://github.com/influxdata/influxdb/pull/26480))
+
+#### Additional Updates
+
+- Track generation 1 duration in catalog for better performance monitoring ([#26508](https://github.com/influxdata/influxdb/pull/26508))
+- Add retention period support to the catalog ([#26479](https://github.com/influxdata/influxdb/pull/26479))
+- Update help text for improved user experience ([#26509](https://github.com/influxdata/influxdb/pull/26509))
+
+### Enterprise
+
+All Core updates are included in Enterprise. Additional Enterprise-specific features and fixes:
+
+#### Features
+
+- **License management improvements**: 
+  - New `influxdb3 show license` command to display current license information
+- **Table-level retention period support**: Add retention period support for individual tables in addition to database-level retention, providing granular data lifecycle management
+   - New CLI commands: `create table --retention-period` and `update table --retention-period`
+   - Set or clear table-specific retention policies independent of database settings
+- **Compaction improvements**:
+  - Address compactor restart issues for better reliability
+  - Track compacted generation durations in catalog for monitoring
+  - Disable parquet cache for ingest mode to optimize memory usage
+
+#### Bug Fixes
+
+- **Query optimization**: Correctly partition query chunks into generations for improved performance
+- **Data integrity**: Don't delete generation 1 files as part of compaction process
+- **License handling**: Trim whitespace from license file contents after reading to prevent validation issues
+
 ## v3.1.0 {date="2025-05-29"}
 **Core**: revision 482dd8aac580c04f37e8713a8fffae89ae8bc264

--- a/helper-scripts/generate-release-notes.sh
+++ b/helper-scripts/generate-release-notes.sh
@ -0,0 +1,135 @@
+#!/bin/bash
+
+# Script to generate release notes for InfluxDB v3.x releases
+# Usage: ./generate-release-notes.sh <from_version> <to_version> <repo_path>
+
+set -e
+
+# Default values
+REPO_PATH="${3:-/Users/ja/Documents/github/influxdb}"
+FROM_VERSION="${1:-v3.1.0}"
+TO_VERSION="${2:-v3.2.0}"
+
+# Colors for output
+RED='\033[0;31m'
+GREEN='\033[0;32m'
+YELLOW='\033[0;33m'
+BLUE='\033[0;34m'
+NC='\033[0m' # No Color
+
+echo -e "${BLUE}Generating release notes for ${TO_VERSION}${NC}"
+echo -e "Repository: ${REPO_PATH}"
+echo -e "From: ${FROM_VERSION} To: ${TO_VERSION}\n"
+
+# Function to extract PR number from commit message
+extract_pr_number() {
+    echo "$1" | grep -oE '#[0-9]+' | head -1 | sed 's/#//'
+}
+
+# Get the release date
+RELEASE_DATE=$(git -C "$REPO_PATH" log -1 --format=%ai "$TO_VERSION" | cut -d' ' -f1)
+echo -e "${GREEN}Release Date: ${RELEASE_DATE}${NC}\n"
+
+# Collect commits by category
+echo -e "${YELLOW}Analyzing commits...${NC}"
+
+# Features
+echo -e "\n${GREEN}Features:${NC}"
+FEATURES=$(git -C "$REPO_PATH" log --format="%h %s" "${FROM_VERSION}..${TO_VERSION}" | grep -E "^[a-f0-9]+ feat:" | sed 's/^[a-f0-9]* feat: //')
+
+# Fixes
+echo -e "\n${GREEN}Bug Fixes:${NC}"
+FIXES=$(git -C "$REPO_PATH" log --format="%h %s" "${FROM_VERSION}..${TO_VERSION}" | grep -E "^[a-f0-9]+ fix:" | sed 's/^[a-f0-9]* fix: //')
+
+# Breaking changes
+echo -e "\n${GREEN}Breaking Changes:${NC}"
+BREAKING=$(git -C "$REPO_PATH" log --format="%h %s" "${FROM_VERSION}..${TO_VERSION}" | grep -iE "^[a-f0-9]+ .*(BREAKING|breaking change)" | sed 's/^[a-f0-9]* //')
+
+# Performance improvements
+echo -e "\n${GREEN}Performance:${NC}"
+PERF=$(git -C "$REPO_PATH" log --format="%h %s" "${FROM_VERSION}..${TO_VERSION}" | grep -E "^[a-f0-9]+ perf:" | sed 's/^[a-f0-9]* perf: //')
+
+# Generate markdown output
+OUTPUT_FILE="release-notes-${TO_VERSION}.md"
+cat > "$OUTPUT_FILE" << EOF
+## ${TO_VERSION} {date="${RELEASE_DATE}"}
+
+### Features
+
+EOF
+
+# Add features
+if [ -n "$FEATURES" ]; then
+    while IFS= read -r line; do
+        PR=$(extract_pr_number "$line")
+        # Clean up the commit message
+        CLEAN_LINE=$(echo "$line" | sed -E 's/ \(#[0-9]+\)$//')
+        if [ -n "$PR" ]; then
+            echo "- $CLEAN_LINE ([#$PR](https://github.com/influxdata/influxdb/pull/$PR))" >> "$OUTPUT_FILE"
+        else
+            echo "- $CLEAN_LINE" >> "$OUTPUT_FILE"
+        fi
+    done <<< "$FEATURES"
+else
+    echo "- No new features in this release" >> "$OUTPUT_FILE"
+fi
+
+# Add bug fixes
+cat >> "$OUTPUT_FILE" << EOF
+
+### Bug Fixes
+
+EOF
+
+if [ -n "$FIXES" ]; then
+    while IFS= read -r line; do
+        PR=$(extract_pr_number "$line")
+        CLEAN_LINE=$(echo "$line" | sed -E 's/ \(#[0-9]+\)$//')
+        if [ -n "$PR" ]; then
+            echo "- $CLEAN_LINE ([#$PR](https://github.com/influxdata/influxdb/pull/$PR))" >> "$OUTPUT_FILE"
+        else
+            echo "- $CLEAN_LINE" >> "$OUTPUT_FILE"
+        fi
+    done <<< "$FIXES"
+else
+    echo "- No bug fixes in this release" >> "$OUTPUT_FILE"
+fi
+
+# Add breaking changes if any
+if [ -n "$BREAKING" ]; then
+    cat >> "$OUTPUT_FILE" << EOF
+
+### Breaking Changes
+
+EOF
+    while IFS= read -r line; do
+        PR=$(extract_pr_number "$line")
+        CLEAN_LINE=$(echo "$line" | sed -E 's/ \(#[0-9]+\)$//')
+        if [ -n "$PR" ]; then
+            echo "- $CLEAN_LINE ([#$PR](https://github.com/influxdata/influxdb/pull/$PR))" >> "$OUTPUT_FILE"
+        else
+            echo "- $CLEAN_LINE" >> "$OUTPUT_FILE"
+        fi
+    done <<< "$BREAKING"
+fi
+
+# Add performance improvements if any
+if [ -n "$PERF" ]; then
+    cat >> "$OUTPUT_FILE" << EOF
+
+### Performance Improvements
+
+EOF
+    while IFS= read -r line; do
+        PR=$(extract_pr_number "$line")
+        CLEAN_LINE=$(echo "$line" | sed -E 's/ \(#[0-9]+\)$//')
+        if [ -n "$PR" ]; then
+            echo "- $CLEAN_LINE ([#$PR](https://github.com/influxdata/influxdb/pull/$PR))" >> "$OUTPUT_FILE"
+        else
+            echo "- $CLEAN_LINE" >> "$OUTPUT_FILE"
+        fi
+    done <<< "$PERF"
+fi
+
+echo -e "\n${GREEN}Release notes generated in: ${OUTPUT_FILE}${NC}"
+echo -e "${YELLOW}Please review and edit the generated notes before adding to documentation.${NC}"
--- a/plans/README.md
+++ b/plans/README.md
@ -0,0 +1,38 @@
+# Documentation Plans
+
+This directory contains plans for recurring documentation tasks and automation.
+
+## Active Plans
+
+### CLI Documentation Synchronization
+- **Location**: `cli-docs-sync/`
+- **Purpose**: Keep InfluxDB 3 CLI reference documentation in sync with actual CLI commands
+- **Current Version**: `plan-v3.2.0.md`
+- **Status**: Ready for execution
+
+### Release Notes Automation
+- **Location**: `release-notes-automation/`
+- **Purpose**: Automate generation of release notes from git commits
+- **Scripts**: Available in `/scripts/generate-release-notes.sh`
+
+## Plan Structure
+
+Each plan directory should contain:
+- `plan-[version].md` - The detailed plan
+- `execution-log.md` - Track progress and notes
+- `templates/` - Reusable templates and patterns
+
+## Usage for Claude
+
+When working with Claude on these plans:
+1. Reference the specific plan file: `@plans/[task]/plan-[version].md`
+2. Update execution logs as you progress
+3. Iterate on plans by creating new versions
+4. Use templates for consistency
+
+## Best Practices
+
+- Keep plans versioned by release or iteration
+- Document lessons learned in execution logs
+- Create reusable templates for recurring patterns
+- Link plans to related scripts and automation
--- a/plans/cli-docs-sync/execution-log.md
+++ b/plans/cli-docs-sync/execution-log.md
@ -0,0 +1,47 @@
+# CLI Documentation Sync - Execution Log
+
+## v3.2.0 Execution
+
+### Status: Planning Complete ✅
+**Date**: 2025-06-26
+**Plan**: `plan-v3.2.0.md`
+
+### Phase 1: Analysis (Not Started)
+- [ ] Audit existing CLI documentation structure
+- [ ] Extract v3.2.0 changes from release notes
+- [ ] Generate current CLI help output
+
+### Phase 2: Update Documentation (Not Started)
+#### Files to Create:
+- [ ] `/content/shared/influxdb3-cli/update/_index.md`
+- [ ] `/content/shared/influxdb3-cli/update/database.md`
+- [ ] `/content/shared/influxdb3-cli/update/table.md` (Enterprise)
+- [ ] `/content/shared/influxdb3-cli/show/license.md` (Enterprise)
+
+#### Files to Update:
+- [ ] `/content/shared/influxdb3-cli/create/database.md` (add `--retention-period`)
+- [ ] `/content/shared/influxdb3-cli/create/table.md` (add Enterprise `--retention-period`)
+- [ ] `/content/shared/influxdb3-cli/show/_index.md` (add license command)
+- [ ] `/content/shared/influxdb3-cli/delete/database.md` (review hard delete)
+- [ ] `/content/shared/influxdb3-cli/delete/table.md` (review hard delete)
+
+### Phase 3: Automation (Not Started)
+- [ ] Create CLI documentation sync script
+- [ ] Establish documentation standards
+- [ ] Plan CI/CD integration
+
+### Phase 4: Validation (Not Started)
+- [ ] Test documented commands
+- [ ] Verify completeness
+- [ ] Check cross-references
+
+### Notes and Lessons Learned
+- Release notes analysis was crucial for identifying specific CLI changes
+- Having automation scripts in place (`generate-release-notes.sh`) helped identify features
+- Need to distinguish between Core and Enterprise features clearly
+
+### Next Actions
+When ready to execute:
+1. Reference plan: `@plans/cli-docs-sync/plan-v3.2.0.md`
+2. Start with Phase 1 analysis
+3. Update this log as progress is made
--- a/plans/cli-docs-sync/plan-v3.2.0.md
+++ b/plans/cli-docs-sync/plan-v3.2.0.md
@ -0,0 +1,165 @@
+# Plan: Update InfluxDB 3 CLI Reference Documentation for v3.2.0
+
+## Phase 1: Analyze Current State and Changes
+
+### 1. Audit existing CLI documentation structure
+**Existing files in `/content/shared/influxdb3-cli/`:**
+- `/content/shared/influxdb3-cli/create/_index.md`
+- `/content/shared/influxdb3-cli/create/database.md` ⚠️ **NEEDS UPDATE** (add `--retention-period`)
+- `/content/shared/influxdb3-cli/create/distinct_cache.md`
+- `/content/shared/influxdb3-cli/create/file_index.md`
+- `/content/shared/influxdb3-cli/create/last_cache.md`
+- `/content/shared/influxdb3-cli/create/table.md` ⚠️ **NEEDS UPDATE** (add Enterprise `--retention-period`)
+- `/content/shared/influxdb3-cli/create/token/_index.md`
+- `/content/shared/influxdb3-cli/create/token/admin.md`
+- `/content/shared/influxdb3-cli/create/trigger.md`
+- `/content/shared/influxdb3-cli/delete/_index.md`
+- `/content/shared/influxdb3-cli/delete/database.md` ⚠️ **REVIEW** (hard delete features)
+- `/content/shared/influxdb3-cli/delete/distinct_cache.md`
+- `/content/shared/influxdb3-cli/delete/file_index.md`
+- `/content/shared/influxdb3-cli/delete/last_cache.md`
+- `/content/shared/influxdb3-cli/delete/table.md` ⚠️ **REVIEW** (hard delete features)
+- `/content/shared/influxdb3-cli/delete/trigger.md`
+- `/content/shared/influxdb3-cli/disable/_index.md`
+- `/content/shared/influxdb3-cli/disable/trigger.md`
+- `/content/shared/influxdb3-cli/enable/_index.md`
+- `/content/shared/influxdb3-cli/enable/trigger.md`
+- `/content/shared/influxdb3-cli/query.md`
+- `/content/shared/influxdb3-cli/show/_index.md` ⚠️ **NEEDS UPDATE** (add license command)
+- `/content/shared/influxdb3-cli/show/databases.md`
+- `/content/shared/influxdb3-cli/show/system/_index.md`
+- `/content/shared/influxdb3-cli/show/system/summary.md`
+- `/content/shared/influxdb3-cli/show/system/table-list.md`
+- `/content/shared/influxdb3-cli/show/system/table.md`
+- `/content/shared/influxdb3-cli/show/tokens.md`
+- `/content/shared/influxdb3-cli/test/_index.md`
+- `/content/shared/influxdb3-cli/test/schedule_plugin.md`
+- `/content/shared/influxdb3-cli/test/wal_plugin.md`
+- `/content/shared/influxdb3-cli/write.md`
+
+### 2. Extract v3.2.0 changes from release notes
+**From `/content/shared/v3-core-enterprise-release-notes/_index.md`:**
+
+**Core v3.2.0 Features:**
+- Database retention period support: `create database --retention-period`, `update database --retention-period`
+- Hard delete for databases and tables
+- AWS credentials auto-reload
+- WAL improvements
+
+**Enterprise v3.2.0 Features:**
+- License management: `influxdb3 show license`
+- Table retention period support: `create table --retention-period`, `update table --retention-period`
+- All Core features plus Enterprise-specific enhancements
+
+### 3. Generate current CLI help output
+- Run `influxdb3 --help` for both Core and Enterprise versions
+- Extract new commands, options, and help text
+- Compare with existing documentation
+
+## Phase 2: Update Documentation Files
+
+### Files to Create (NEW):
+- `/content/shared/influxdb3-cli/update/_index.md` 🆕
+- `/content/shared/influxdb3-cli/update/database.md` 🆕 (retention period management)
+- `/content/shared/influxdb3-cli/update/table.md` 🆕 (Enterprise-only, retention period management)
+- `/content/shared/influxdb3-cli/show/license.md` 🆕 (Enterprise-only)
+
+### Files to Update (EXISTING):
+- `/content/shared/influxdb3-cli/create/database.md` ⚠️ (add `--retention-period` option)
+- `/content/shared/influxdb3-cli/create/table.md` ⚠️ (add Enterprise `--retention-period` option)
+- `/content/shared/influxdb3-cli/show/_index.md` ⚠️ (include license command)
+- `/content/shared/influxdb3-cli/delete/database.md` ⚠️ (review hard delete options)
+- `/content/shared/influxdb3-cli/delete/table.md` ⚠️ (review hard delete options)
+
+### Content Changes by Category:
+
+**1. Retention Period Documentation:**
+- Update `/content/shared/influxdb3-cli/create/database.md` with `--retention-period` option
+- Create `/content/shared/influxdb3-cli/update/` directory structure
+- Create `/content/shared/influxdb3-cli/update/database.md` for retention management
+- Update `/content/shared/influxdb3-cli/create/table.md` with Enterprise `--retention-period` option
+- Create `/content/shared/influxdb3-cli/update/table.md` for Enterprise table retention management
+
+**2. License Management Documentation:**
+- Update `/content/shared/influxdb3-cli/show/_index.md` to include license command
+- Create `/content/shared/influxdb3-cli/show/license.md` for Enterprise license display
+
+**3. Hard Delete Documentation:**
+- Review and update `/content/shared/influxdb3-cli/delete/database.md` with hard delete options
+- Review and update `/content/shared/influxdb3-cli/delete/table.md` with hard delete options
+
+## Phase 3: Automation and Process Improvements
+
+### Immediate Improvements:
+1. **Create CLI documentation sync script:**
+   ```bash
+   # Script: /Users/ja/Documents/github/docs-v2/scripts/sync-cli-docs.sh
+   # - Extract help text from influxdb3 CLI
+   # - Compare with existing docs
+   # - Generate report of differences
+   # - Auto-update basic command syntax
+   ```
+
+2. **Establish documentation standards:**
+   - Standardize frontmatter across CLI docs
+   - Create templates for command documentation
+   - Define Enterprise vs Core content patterns using Hugo shortcodes
+
+### Long-term Automation Strategy:
+1. **CI/CD Integration:**
+   - Add GitHub Actions workflow to detect CLI changes
+   - Auto-generate CLI help extraction on new releases
+   - Create pull requests for documentation updates
+
+2. **Release Process Integration:**
+   - Include CLI documentation review in release checklist
+   - Link release notes to specific CLI documentation updates
+   - Automated cross-referencing between release notes and CLI docs
+
+3. **Content Management Improvements:**
+   - Use Hugo shortcodes for Enterprise-specific content
+   - Implement version-aware documentation
+   - Create shared content templates for common CLI patterns
+
+## Phase 4: Validation and Testing
+
+### Content accuracy verification:
+- Test all documented commands and options against actual CLIs
+- Verify Enterprise vs Core feature availability
+- Cross-reference with actual CLI behavior
+
+### Documentation completeness check:
+- Ensure all v3.2.0 features are documented
+- Verify examples and use cases
+- Check internal links and cross-references
+
+## Suggested Recurring Process
+
+### Pre-release (during development):
+- Monitor CLI changes in pull requests
+- Update documentation as features are added
+- Maintain CLI help extraction automation
+
+### At release (when tagging versions):
+- Run automated CLI documentation sync
+- Review and approve auto-generated updates
+- Publish updated documentation
+
+### Post-release (after release):
+- Validate documentation accuracy
+- Gather user feedback on CLI documentation
+- Plan improvements for next cycle
+
+## Related Documentation Paths
+
+### InfluxDB 3 Product Documentation (affects CLI usage examples):
+- `/content/influxdb3/core/write-data/influxdb3-cli.md`
+- `/content/influxdb3/enterprise/write-data/influxdb3-cli.md`
+- `/content/shared/influxdb3-write-guides/influxdb3-cli.md`
+
+### Admin Documentation (affects retention and license features):
+- `/content/influxdb3/core/admin/`
+- `/content/influxdb3/enterprise/admin/`
+- `/content/influxdb3/enterprise/admin/license.md`
+
+This plan ensures comprehensive documentation updates for v3.2.0 while establishing sustainable processes for future releases.