Commit Graph

29 Commits (42ae0fbf6b3db05ee07fed2dc701e0b1ca09336a)

Author SHA1 Message Date
Jason Stirnaman b8418d9788
fix(shortcodes): latest-patch CLI version for cloud-serverless (#6655)
* fix(shortcodes): latest-patch CLI version for cloud-serverless

Add cloud-serverless to the condition that handles cloud products,
and use influxdb.latest directly for CLI version lookups.

Root cause: The shortcode only checked for "cloud" version when
deciding to use the influxdb CLI version. Cloud Serverless pages
have $product="influxdb3" which doesn't exist in products.yml,
causing $latestVersion to be empty.

Fix: Calculate $influxdbLatest directly from products.influxdb.latest
within the CLI block, ensuring cloud products always use the correct
lookup key (v2) for the influx CLI version.

Includes Cypress test covering v2, Cloud, and Cloud Serverless.

Closes #6646

* Restore latest-patch shortcode for cloud-serverless. Revert "hotfix: influxdb (v2) CLI version (#6644)"

This reverts commit 5f792bd47a.

* chore(link-checker): update configs for v1.3.0 severity classification (#6688)

Remove exclusions for sites that return 403/429 (bot protection) and
5xx (server errors) - these are now handled by severity classification:
- 403/401/429 → info (shown but don't fail CI)
- 5xx/timeout → warning (shown but don't fail CI)
- 404/410/DNS → error (fail CI)

Removed exclusions:
- GitHub, Slack, Reddit, StackOverflow
- Docker Hub, Grafana, Microsoft Learn
- Claude.ai, Dremio, Scarf, InfluxData support

Kept exclusions:
- Localhost/local network URLs
- Example/placeholder URLs
- CI-specific workarounds (canonical URLs, file fragments)

Added [severity] configuration section with default thresholds.
2026-01-02 12:47:12 -06:00
Jason Stirnaman 0fbf9e5e9b
fix(ci): complete PR preview path offset and fork PR handling (#6667)
* fix(ci): add path offset for PR preview subdirectory baseURL

When PR preview builds use a subdirectory baseURL like
/docs-v2/pr-preview/pr-XXXX/, shortcodes that parse .RelPermalink
to detect product context fail because the path has extra segments.

This fix:
- Adds config/pr-preview/params.yml with prPreviewPathOffset: 3
- Updates workflow to use -e pr-preview environment
- Updates api-endpoint, influxdb/host, and children shortcodes to
  use the offset when indexing path segments
- Adds nil-safety with default fallback for placeholder_host

Normal builds are unaffected (offset defaults to 0).

* fix(ci): add path offset to product-name and sidebar for PR previews

Apply the same prPreviewPathOffset fix to product-name.html and
sidebar.html that was applied in the initial PR #6665.

These templates parse RelPermalink to detect product context, but when
baseURL includes a subdirectory path (e.g., /docs-v2/pr-preview/pr-XXXX/),
the path indices shift. This fix uses the configurable offset to skip
extra path segments in PR preview builds.

* fix(ci): skip PR preview for fork PRs and add notice comment

Fork PRs cannot deploy to gh-pages because GITHUB_TOKEN has read-only
access to the base repository. This is a GitHub security feature.

Changes:
- Add condition to skip preview job for fork PRs
- Add fork-notice job to post helpful comment explaining limitation
- Include local preview instructions for contributors
2025-12-22 15:30:26 -06:00
Jason Stirnaman 74c33e9483
fix(ci): add path offset for PR preview subdirectory baseURL (#6662)
When PR preview builds use a subdirectory baseURL like
/docs-v2/pr-preview/pr-XXXX/, shortcodes that parse .RelPermalink
to detect product context fail because the path has extra segments.

This fix:
- Adds config/pr-preview/params.yml with prPreviewPathOffset: 3
- Updates workflow to use -e pr-preview environment
- Updates api-endpoint, influxdb/host, and children shortcodes to
  use the offset when indexing path segments
- Adds nil-safety with default fallback for placeholder_host

Normal builds are unaffected (offset defaults to 0).
2025-12-22 10:03:27 -06:00
Jason Stirnaman 61a88db39b
fix(ci): set baseURL for PR preview subdirectory deployment (#6658)
* refactor(ci): improve PR preview with products.yml and index page

- Load product prefixes from data/products.yml (single source of truth)
- Build regex pattern dynamically from loaded namespaces
- Fix CSS copying: Hugo outputs fingerprinted CSS at root level
- Generate index.html with clickable links to preview pages
- Add test for influxdb3_explorer namespace

This keeps the URL parser in sync as new products are added and
provides a better preview experience with navigable links.

* fix(ci): set baseURL for preview subdirectory deployment

Hugo generates absolute paths for fingerprinted assets based on baseURL.
When deploying to GitHub Pages subdirectory (pr-preview/pr-XXXX/), assets
were incorrectly resolving to the root instead of the preview path.

This sets --baseURL to the correct preview URL so all asset paths work.
2025-12-19 18:31:37 -06:00
Jason Stirnaman e203482b62
feat(ci): add PR preview system for GitHub Pages (#6636)
* docs(ci): add PR preview system design

Document the design for GitHub Pages PR previews including:
- Selective deployment of changed pages only
- Reuse of existing content-utils.js change detection
- URL parsing from PR descriptions for layout/asset changes
- Automatic cleanup on PR close
- Storage budget management

* docs(ci): add PR preview implementation plan

Detailed task-by-task implementation plan including:
- 8 tasks with complete code and exact file paths
- Reuses existing content-utils.js for change detection
- Scripts for URL parsing, change detection, file staging
- Main workflow and cleanup workflow
- Setup documentation and testing steps

* feat(ci): add PR URL parser for preview page detection

* fix(ci): harden PR URL parser against malicious input

- Add path validation to reject path traversal attacks (..)
- Add validation to reject HTML/script injection attempts
- Add validation to reject URL-encoded characters
- Update regex to capture markdown link paths: [text](/path/)
- Add comprehensive test suite with 27 security and functionality tests

Security improvements:
- Reject paths containing '..' to prevent directory traversal
- Reject paths with suspicious characters: <, >, |, {, }, `, etc.
- Reject URL-encoded characters to prevent encoding attacks
- Validate all paths against known product prefixes

Functionality improvements:
- Support markdown link syntax: [text](/influxdb3/core/)
- Support reference-style markdown links
- Maintain existing support for production URLs, localhost URLs, and relative paths

* feat(ci): add change detection script for PR previews

* fix(ci): remove unused import from detect-preview-pages.js

* feat(ci): add preview file staging script for selective deployment

* fix(ci): add input validation to preview file staging script

- Add path traversal validation to urlToHtmlPath() to reject paths containing '..'
- Add array validation at start of preparePreviewFiles() with clear error message
- Fix copiedCount to only increment when safeCopy() actually succeeds
- Add return value to copyPage() to track success/failure

Addresses security vulnerabilities and improves accuracy of preview stats.

* feat(ci): add preview comment manager for sticky PR comments

* fix(ci): sanitize user input in preview comments to prevent XSS

* feat(ci): add PR preview workflow for GitHub Pages deployment

* fix(ci): correct environment variables and checkout in PR preview workflow

* feat(ci): add weekly stale preview cleanup workflow

* docs(ci): add PR preview setup guide

* Delete docs/plans/2025-12-16-pr-preview-design.md

* Delete docs/plans/2025-12-16-pr-preview-implementation.md

* fix(ci): add input validation for PR preview security

- Add single quote to rejected characters in URL path validation
  to prevent JavaScript injection in workflow string interpolation
- Add BASE_REF validation regex to prevent command injection,
  allowing slashes for feature branches (e.g., feature/new-auth)
- Add PR_NUM numeric validation in cleanup workflow
- Add tests for single quote handling and BASE_REF validation
2025-12-18 13:44:14 -06:00
Jason Stirnaman ebbb0d4721
chore(ci): update link-checker to v1.2.5 (#6602)
Update link-checker to v1.2.5 which adds --root-dir support for lychee v0.22.0+.

This fixes CI failures caused by lychee v0.22.0 (released Dec 5, 2025)
requiring --root-dir to resolve root-relative links in local files.
The link-checker now automatically sets --root-dir to the Hugo public/
directory when checking local HTML files.

Fixes failures for PRs touching /telegraf/v1 and /influxdb3/explorer paths.
2025-12-10 09:52:51 -06:00
meelahme 3c30f19436 Disable screenshot generation for MVP 2025-11-17 13:30:01 -08:00
meelahme e0d56ad41e fix: added npm install playwrite 2025-11-14 13:25:54 -08:00
meelahme 930bb7091b fix: correct relative path to influxdb3_plugins dir 2025-11-14 10:53:57 -08:00
meelahme e18e2a1daf debug: add file structure inspection to identify path issue 2025-11-14 10:28:02 -08:00
meelahme df8d5f8378 fix: combine PAT and path fixes for plugin sync 2025-11-10 10:00:39 -08:00
meelahme d645e75215 fix: use PAT for reading from influxdb3_plugins 2025-11-10 09:42:50 -08:00
Jason Stirnaman 20562d5bb6
Automate plugin documentation sync workflow between influxdb3_plugins and docs-v2 (#6329)
* feat(plugins): add automated plugin documentation sync workflow

- Add GitHub Actions workflow for syncing plugin docs from influxdb3_plugins
- Create issue template for triggering sync requests
- Add Node.js transformation script (port_to_docs.js) with ES modules
- Add mapping configuration (docs_mapping.yaml) for all official plugins
- Add npm scripts for plugin sync operations
- Include comprehensive documentation in helper-scripts/influxdb3-plugins/README.md

The workflow provides:
- Issue-triggered automation with no cross-repo secrets required
- Validation of source READMEs against template requirements
- Content transformation with Hugo shortcodes and GitHub URLs
- Screenshot generation for visual validation
- Automatic PR creation with detailed change summaries

* Updated source paths in docs_mapping.yaml to use ./influxdb3_plugins/ instead of ../influxdb3_plugins/ to match what the
  GitHub Actions workflow expects when it clones the
  repository:

  1. GitHub Actions workflow clones to ./influxdb3_plugins/
  2. docs_mapping.yaml references
  ./influxdb3_plugins/influxdata/[plugin]/README.md
  3. Local development can manually clone the repo to the same
   location for testing

  Tupdated all the source paths in
  docs_mapping.yaml to use ./influxdb3_plugins/ instead of
  ../influxdb3_plugins/. This now matches exactly what the
  GitHub Actions workflow expects when it clones the
  repository.

  The paths are now consistent:

  1. GitHub Actions workflow clones to ./influxdb3_plugins/
  2. docs_mapping.yaml references
  ./influxdb3_plugins/influxdata/[plugin]/README.md
  3. Local development can manually clone the repo to the same
   location for testing

  This resolves the inconsistency and makes the automation
  more reliable. For local development, developers would
  just need to run:

  git clone
  https://github.com/influxdata/influxdb3_plugins.git

  From the docs-v2 root directory, and then they can use the
  same paths that the automation uses.

* Updated source paths in docs_mapping.yaml to use ./influxdb3_plugins/ instead of ../influxdb3_plugins/ to match what the
  GitHub Actions workflow expects when it clones the
  repository:

  1. GitHub Actions workflow clones to ./influxdb3_plugins/
  2. docs_mapping.yaml references
  ./influxdb3_plugins/influxdata/[plugin]/README.md
  3. Local development can manually clone the repo to the same
   location for testing

  Tupdated all the source paths in
  docs_mapping.yaml to use ./influxdb3_plugins/ instead of
  ../influxdb3_plugins/. This now matches exactly what the
  GitHub Actions workflow expects when it clones the
  repository.

  The paths are now consistent:

  1. GitHub Actions workflow clones to ./influxdb3_plugins/
  2. docs_mapping.yaml references
  ./influxdb3_plugins/influxdata/[plugin]/README.md
  3. Local development can manually clone the repo to the same
   location for testing

  This resolves the inconsistency and makes the automation
  more reliable. For local development, developers would
  just need to run:

  git clone
  https://github.com/influxdata/influxdb3_plugins.git

  From the docs-v2 root directory, and then they can use the
  same paths that the automation uses.

* Jmercer/plugin sync test (#6468)

* fix: update workflow branch references from master to main

* fix: update issue template branch references from master to main

- Update sync-plugin-docs.yml description (line 40)
- Update placeholder from 'master' to 'main' (line 41)
- Update default value from 'master' to 'main' (line 42)
- Add influxdb3_plugins/ to .gitignore
- Ensures template matches influxdb3_plugins default branch

* Update .gitignore

---------

Co-authored-by: Jason Stirnaman <stirnamanj@gmail.com>

* Update docs_mapping.yaml

* Apply suggestion from @Copilot

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Apply suggestion from @jstirnaman

* Apply suggestion from @jstirnaman

* Apply suggestion from @jstirnaman

* Apply suggestion from @jstirnaman

* Use <docs-root>/.ext/influxdb3_plugins for the source path

---------

Co-authored-by: Jameelah Mercer <36314199+MeelahMe@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-10-20 17:18:34 -05:00
Jason Stirnaman 33549bed68 chore(ci): update link-checker to v1.2.4
- Update GitHub Actions workflow to use link-checker v1.2.4
- Ensures CI uses latest link validation tooling with improvements and bug fixes
2025-09-12 16:46:48 -05:00
Jason Stirnaman 194bf0446d fix(ci): update link-checker to v1.2.3 to fix false positives for new content
- Use link-checker-v1.2.3 which includes fix for base URL detection
- Prevents false positives when checking local files for new pages
- Resolves broken link errors for pages that exist locally but not in production yet
2025-08-22 17:30:36 -05:00
Jason Stirnaman cba3b21f1c docs(testing): document link-checker binary release process
Add comprehensive documentation for maintainers on how to:
- Create releases in docs-tooling (automated)
- Manually distribute binaries to docs-v2 (required for private repo)
- Update workflow references when needed

This addresses the missing process documentation for link-checker
binary distribution between the two repositories.

feat(ci): update link-checker to v1.2.2 and add manual sync workflow

- Update pr-link-check.yml to use link-checker-v1.2.2 with latest fixes
- Add sync-link-checker-binary.yml for manual binary distribution
- Improvements in v1.2.2: base URL detection, anchor validation, JSON parsing

The v1.2.2 release fixes the Hugo base URL detection issue and
improves anchor link validation that was tested in this PR.
2025-08-18 18:02:44 -05:00
Jason Stirnaman e10340b6ec chore(ci): Add config for new link validation tool (docs-tooling/link-checker)
chore(ci): Replaced PR link validation workflow with new workflow from docs-tooling/link-checker/.github-workflows-link-check.yml

chore: organize .gitignore

test: add content to trigger link-checker workflow

This small change tests the pr-link-check.yml workflow

feat: update link-checker workflow and documentation

- Add production config with corrected User-Agent placement
- Remove old link validation actions (replaced by link-checker)

fix: update link-checker workflow configuration

- Update Node.js version to 20 for dependency compatibility

feat: use pre-built link-checker binary from docs-tooling releases

- Replace building from source with downloading from releases
- Use GitHub API to get latest release and binary
- Maintain same artifact structure for downstream job

fix: improve change detection in pr-link-check workflow

- Use GitHub API for reliable PR file detection
- Add debug output to show all changed files
- Fix conditional logic for when jobs should run

docs: update TESTING.md with binary distribution and automated GitHub Actions integration

- Document pre-built binary download as recommended installation method
- Explain automated PR link checking workflow for docs-v2
- Replace manual GitHub Actions example with automated integration details
- Remove exaggerated language and specify actual exclusion types

fix(ci): download link-checker binary from docs-v2 releases

- Change binary source from private docs-tooling to public docs-v2 releases
- Fixes GitHub Actions permission issues accessing private repos
- Binary is now stored as a release asset on docs-v2 itself

test: add test file with valid links to verify workflow passes

test: remove temporary test file for link checker workflow

The test file was only needed to verify the workflow functionality
and should not be part of the documentation.

docs: update TESTING.md to document docs-v2 binary distribution

- Change primary installation method to download from docs-v2 releases
- Explain that binary distribution enables reliable GitHub Actions access
- Update automated workflow description to reflect docs-v2 release usage
- Maintain build-from-source as alternative option

refactor(ci): combine workflow into single job for cleaner PR display

- Merge detect-changes, build-site, and download-link-checker into single job
- All setup steps now run conditionally within one job
- Cleaner PR display shows only 'Check links in affected files'
- Maintains all functionality with improved UX

fix(ci): exclude problematic URLs from link checking

- Add reddit.com exclusions (blocks bots)
- Add support.influxdata.com exclusion (SSL certificate issues in CI)
- Prevents false positive failures in automated link checking
2025-08-16 22:14:27 -05:00
Jason Stirnaman 89739cd792 chore(ci): temporarily disable link validation workflow
- Add if: false conditions to all jobs to prevent execution
- Add disabled-check job to indicate the workflow is disabled
- Preserves original conditions in comments for easy re-enabling

To re-enable: Remove the 'if: false' conditions and disabled-check job
2025-07-29 23:09:00 -05:00
Jason Stirnaman eac1acfdf8 ci: fix timeout command, set timeout options, enhance logging:
1. Timeout Handling: Proper timeout command syntax
  with specific timeout error detection
  2. Process Management: Environment variable for Hugo
   shutdown timeout
  3. Debugging: Enhanced log collection and artifact
  uploads
  4. Error Context: Better error messages that help
  identify root causes (EPIPE, timeouts, etc.)
  5. Resource Constraints: Memory limits and
  CI-specific optimizations
2025-07-28 21:03:31 -05:00
Jason Stirnaman d762e7800e fix: apply code review suggestions: High Priority Issues (Fixed):
1. Script execution detection in matrix-generator.js -
  Added fileURLToPath import and updated comparison
  2. Script execution detection in incremental-validator.js -
   Added fileURLToPath import and updated comparison
  3. Script execution detection in link-extractor.js - Added
  fileURLToPath import and updated comparison
  4. Script execution detection in comment-generator.js -
  Added fileURLToPath import and updated comparison

  Medium Priority Issues (Fixed):

  5. Extracted duplicated URL transformation logic - Created
  shared utility module and updated both files to use it
  6. Fixed cache key strategy - Updated GitHub workflow to
  use content-based hashing instead of base SHA

  Changes Made:

  - 4 JavaScript files: Updated with robust script execution
  detection using fileURLToPath
  - 1 utility module: Created
  /.github/scripts/utils/url-transformer.js for shared logic
  - 2 files: Updated to use the shared URL transformation
  utility
  - 1 workflow file: Improved cache key strategy for better
  cache hit rates
2025-07-28 16:24:24 -05:00
Jason Stirnaman 6a4e8827eb feat(testing): add link validation automation and improvements
- Add GitHub Actions for automated link validation on PRs
- Implement incremental validation with caching (30-day TTL, configurable)
- Add matrix generator for parallel validation strategy
- Create comprehensive TESTING.md documentation
- Add cache manager with configurable TTL via env var or CLI
- Implement smart link extraction and validation
- Add PR comment generator for broken link reports
- Update Cypress tests to use incremental validation
- Consolidate testing docs from CONTRIBUTING.md to TESTING.md

Key improvements:
- Cache-aware validation only checks changed content
- Parallel execution for large changesets
- Detailed PR comments with broken link reports
- Support for LINK_CACHE_TTL_DAYS env var
- Local testing with yarn test:links
- Reduced false positives through intelligent caching
2025-07-28 16:24:24 -05:00
Jason Stirnaman 105938f3aa Updated all README files in the
helper-scripts directory to:

  1. Main helper-scripts/README.md:
  Updated to describe
  generate-release-notes.js instead of
  the old bash script, including new
  configuration options and examples.
  2. helper-scripts/common/README.md:
  Updated to describe the JavaScript
  version with all its new features like
  configuration files,
  integrated/separated formats, and PR
  link options.
  3. helper-scripts/influxdb3-monolith/RE
  ADME.md: Completely updated to reflect
  the actual JavaScript scripts present
  (audit-cli-documentation.js and
  apply-cli-patches.js) instead of the
  non-existent bash scripts, updated
  prerequisites, examples, and workflow
  documentation.
2025-07-07 12:45:48 -05:00
Jason Stirnaman a1d7a6399c chore(ci): Migrate generate-release-notes.sh to generate-release-notes.js. Update and cleanup workflows to use the new script and the Markdown-formatted output. 2025-07-07 12:13:37 -05:00
Jason Stirnaman 6506c5ff5d chore(ci): Cleanup JS 2025-07-07 12:13:36 -05:00
Jason Stirnaman 1cb33bfb13 chore(ci): Validate git tags for workflows: - Reusable utility for validating git tags across all
scripts
  - Supports special case for local development mode
  - Provides helpful error messages with available tags
  - Can be used as CLI tool or imported module
2025-07-07 12:13:36 -05:00
Jason Stirnaman fa069a77ea fix(ci): GitHub Actions:\Usage Examples
Now you can run specific audits directly:

  # Run specific audits
  gh act workflow_dispatch -j cli-3-enterprise
  gh act workflow_dispatch -j cli-3-core
  gh act workflow_dispatch -j api-3-cloud-dedicated

  # Run with custom version
  gh act workflow_dispatch -j cli-3-enterprise --input
  version=3.1.0

  # Run all audits (scheduled behavior)
  gh act workflow_dispatch
2025-07-07 12:13:36 -05:00
Jason Stirnaman 786f44cbe2 fix(ci): replace the problematic multi-line template
literal with an array of strings that gets joined with
  newlines, which avoids the YAML parsing issues with backticks
  and special characters.
2025-07-07 12:13:36 -05:00
Jason Stirnaman c74060210b chore(ci): Audit documentation for changes to influxdb3 CLI. Configures a GitHub release workflow to generate release notes and run the audit documentation script. 2025-07-07 12:13:36 -05:00
Jason Stirnaman 9a4721aa40 chore(ci): Automation scripts to compare influxdb3 CLI help to reference documentation and generate an audit report, runs influxdb3 core and enterprise using Docker, improves compose.yaml, restructures helper-scripts for different versions 2025-07-07 12:13:36 -05:00