Commit Graph

15 Commits (70481539c0c670062732f33dd57e316be6fee67d)

Author SHA1 Message Date
Jason Stirnaman 2afcfafb1f fix(ci): resolve merge conflict markers in detect-preview-pages.js
Unresolved conflict markers caused SyntaxError on Node.js parse,
which made the preview workflow skip page detection entirely.
2026-03-13 23:27:12 -05:00
Jason Stirnaman a809dc5313 fix(ci): replace HTTP polling with commit status signaling for preview coordination
pr-preview.yml now emits a `preview/deploy` commit status (pending/success/failure)
so doc-review.yml can detect failures instantly instead of polling an HTTP URL for
10 minutes. Also centralizes the api-docs TypeScript build command in package.json
and adds a lefthook pre-commit hook for api-docs script changes.
2026-03-13 23:13:05 -05:00
Jason Stirnaman 064e0248f4
Merge branch 'master' into api-docs-uplift 2026-03-13 22:20:39 -05:00
Copilot 0a0bebc42c
Fix PR preview and Copilot visual review skipping docs home page when layouts/index.html changes (#6922)
* Initial plan

* fix: include docs home page in PR preview and Copilot visual review when layouts/index.html changes

Co-authored-by: jstirnaman <212227+jstirnaman@users.noreply.github.com>

* fix: improve detect-preview-pages to avoid false needs-author-input when home page auto-detected

Co-authored-by: jstirnaman <212227+jstirnaman@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: jstirnaman <212227+jstirnaman@users.noreply.github.com>
Co-authored-by: Jason Stirnaman <jstirnaman@influxdata.com>
2026-03-13 08:47:38 -05:00
Jason Stirnaman a8b879fa72
Merge branch 'master' into copilot/fix-pr-preview-api-pages 2026-03-10 15:35:06 -05:00
Jason Stirnaman b951d3c27a
feat(ci): add doc review pipeline with auto-labeling and Copilot review (#6890)
* feat: add doc review pipeline implementation plan

Detailed plan for two interconnected systems:
1. Label system overhaul (22 automation-driven labels replacing 30+ inconsistent ones)
2. Doc review workflow (Claude visual review + Copilot structural review with screenshots)

This is a plan document only — no implementation changes.

https://claude.ai/code/session_01D5rLaHdQv9iBL55UEsdQFt

* fix: split product:v3 into v3-monolith and v3-distributed labels

- product:v3-monolith: Core, Enterprise (single-node / clusterable)
- product:v3-distributed: Cloud Serverless, Cloud Dedicated, Clustered
- Updated auto-label path mappings to match content directory structure
- Updated migration mapping and label count (22 → 23)

https://claude.ai/code/session_01D5rLaHdQv9iBL55UEsdQFt

* feat: define agent instruction file architecture in Phase 3

- One CLAUDE.md (pointer) → role-specific files in .claude/agents/
- doc-triage-agent.md: label taxonomy, path mapping, priority rules
- doc-review-agent.md: review scope, severity classification, output format
- Prompt file (.github/prompts/) references agent file, stays workflow-specific
- Updated file summary and implementation order

https://claude.ai/code/session_01D5rLaHdQv9iBL55UEsdQFt

* feat: move visual/screenshot review from Claude to Copilot

Claude now handles diff-only Markdown review (frontmatter, shortcodes,
style, terminology). Copilot handles visual review by analyzing
screenshots posted as images in PR comments.

Key changes:
- Job 3 (Claude) runs in parallel with Jobs 1→2→4 (diff-only, no screenshots)
- Job 4 (Copilot) analyzes screenshots via @copilot PR comment mentions
- Two prompt files: doc-review.md (Claude), copilot-visual-review.md (Copilot)
- doc-review-agent.md scoped to diff-only (no screenshot analysis)
- Q1 resolved: screenshots delivered to Copilot via PR comment images
- Reduced Claude API cost (no image processing)
- Added Copilot failure handling (fallback to human review of artifacts)

https://claude.ai/code/session_01D5rLaHdQv9iBL55UEsdQFt

* chore: resolve all open questions in doc review pipeline plan

Convert Q2–Q5 from open recommendations to resolved decisions:
- Q2: Advisory only (no required status checks) until false-positive rate confirmed
- Q3: Playwright for CI screenshots, Puppeteer for local debugging
- Q4: Poll preview URL with 15s interval and 10-min timeout
- Q5: Cost acceptable with existing mitigations (path filters, skip-review, concurrency)

Rename section from "Open Questions" to "Decisions (Resolved)".

https://claude.ai/code/session_01D5rLaHdQv9iBL55UEsdQFt

* Apply suggestions from code review

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Fix label naming inconsistency and document workflow migration requirements (#6893)

* Initial plan

* fix: resolve label naming inconsistency and document workflow updates

- Rename review:approved to approval:codeowner to avoid confusion with review/* labels
- Add note explaining the distinct prefix to prevent implementor confusion
- Document required workflow updates for sync-plugin-docs label migration
- Specify exact files and line numbers that need updating

Co-authored-by: jstirnaman <212227+jstirnaman@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: jstirnaman <212227+jstirnaman@users.noreply.github.com>

* feat(ci): add doc review pipeline and deduplicate instruction files

Add Phase 2-3 pipeline components: doc-review workflow (3-job
architecture), Claude/Copilot review prompts, URL resolver script,
triage and review agents, and label guide.

Deduplicate AGENTS.md (254→96 lines) by removing content available
in referenced docs. Remove duplicate sections from
copilot-instructions.md (264→221 lines). AGENTS.md now contains
only high-signal guidelines loaded every session: commands,
constraints, style rules, product paths, and reference pointers.

* task: updated PR pipeline plan

* task: remove old cli plan

* task: products file now contains content path mappings and (GitHub) label groups for each product. Product group labels will be used to assign reviewers and help with content checks.

* feat(ci): add auto-label workflow for PR product detection

Add auto-label workflow that applies product and source labels to
PRs based on changed file paths, using data/products.yml as the
source of truth. Add workflow-utils.js shared helper for product
path matching.

* refactor(ci): extract shared label and review format definitions

Consolidate duplicated label definitions and review comment format
into shared source-of-truth files to prevent context drift.

New files:
- data/labels.yml: source, waiting, workflow, review, and
  product:shared label definitions (names, colors, descriptions)
- .github/templates/review-comment.md: severity levels, comment
  structure, result rules, and result-to-label mapping

Updated consumers to reference shared files instead of inline copies:
- .claude/agents/doc-review-agent.md
- .claude/agents/doc-triage-agent.md
- .github/prompts/copilot-visual-review.md
- .github/LABEL_GUIDE.md

Workflow fixes:
- Pin all GitHub Actions to SHA hashes
- Fix fromJson() for url-count comparison in doc-review.yml
- Fix fallback handler to remove all review:* labels before re-adding

* refactor(ci): replace Claude review with Copilot native code review

- Remove claude-code-action job, use `copilot-reviews` reviewer instead
- Extract review criteria to .github/instructions/content-review.instructions.md
- Simplify copilot-instructions.md by removing duplicated content
- Harden auto-label workflow (scoped permissions, pagination, concurrency)
- Upsert visual review comments instead of creating duplicates
- Delete unused .github/prompts/doc-review.md

* Update .github/workflows/doc-review.yml

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update .github/DOC-REVIEW-PIPELINE-PLAN.md

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* task: update review pipeline plan

* task: add label-migration scripts. These are one-use scripts that we'll remove after the label migration

* Update .github/workflows/doc-review.yml

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update .github/workflows/doc-review.yml

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update .github/workflows/auto-label.yml

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* style: remove unnecessary escape in regex character class

* feat(ci): add workflow_dispatch to auto-label and doc-review workflows

- Add workflow_dispatch with pr_number input to both workflows for
  manual testing and on-demand re-runs
- Migrate sync-plugin-docs label references to source:sync
- Add area:agents, area:ci, area:links, release:*, good-first-issue,
  source:feedback, waiting:pr to labels.yml
- Update products.yml: influxdb_cloud label_group v2 -> v2-cloud
- Track label renames and deletions in LABEL_GUIDE.md

* fix(ci): replace npm ci with targeted js-yaml install in auto-label

npm ci fails in sparse checkout because package-lock.json is not
included. The workflow only needs js-yaml for YAML parsing.

* fix(ci): add --legacy-peer-deps to auto-label npm install

* task: updated labels for migration

* docs: update pipeline plan with test results and completion status

* test: reapply reverted serve.md changes for e2e pipeline test

Reverse the revert from 2f8efd6 to provide content changes that
exercise the auto-label and doc-review workflows end-to-end.

* fix(ci): fix preview URL polling in doc-review visual review

curl --head outputs response headers before the status code from -w,
so STATUS contained "HTTP/2 200 ...200" instead of just "200".
Drop --head and add -o /dev/null to capture only the status code.

* fix: correct broken fragment links in InfluxDB 3 serve.md files (#6910)

* Initial plan

* fix: correct broken links in serve.md files for enterprise config-options

Co-authored-by: jstirnaman <212227+jstirnaman@users.noreply.github.com>

* Update content/influxdb3/enterprise/reference/cli/influxdb3/serve.md

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: jstirnaman <212227+jstirnaman@users.noreply.github.com>
Co-authored-by: Jason Stirnaman <jstirnaman@influxdata.com>

---------

Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
2026-03-09 20:22:31 -05:00
copilot-swe-agent[bot] de07f79a45 fix: run API docs build in PR preview workflow for /api pages
- Add detectApiPages() to detect-preview-pages.js to auto-map changed
  api-docs/ files to their content URL paths via .config.yml
- Add has-api-doc-changes output to detect-preview-pages.js
- Fix needs-author-input logic to not request input when API pages
  are already auto-detected
- Add Build API docs step to pr-preview.yml that runs yarn run
  build:api-docs before the Hugo build when api-doc changes detected

Co-authored-by: jstirnaman <212227+jstirnaman@users.noreply.github.com>
2026-03-08 22:24:19 +00:00
Jason Stirnaman 42ae0fbf6b
dar issue 563 - Cloud 1 documentation, support, and migration (#6729)
* docs(v2): Specify versions for v2 Cloud and OSS

* docs(v2): Use specific version for OSS v2, add ToC

* docs(v1): Use specific versions and names. Replace Enterprise v1 with 3
Ent.

* docs(v1): More detailed description for OSS v1 release notes. Repetition
fixes.

* docs(v1): Update Download instructions. Add version specificity

* docs(v1): OSS v1 specificity, fix config commands, cleanup lists

* docs(cloud1): Shared note shortcode to guide Cloud 1 users to Enterprise
documentation, Cloud 1 support, and v3 migration

* Provide Cloud 1.x in version detector and product selector menu,
simplify InfluxDB 1.x section

- Simplify InfluxDB 1.x section on platform page to product links and
  migration guidance
- Remove detailed TICK stack and Enterprise feature descriptions
- Update all Cloud 1 links to point to /platform/#influxdb-cloud-1

- Keep Cloud 1 infrastructure intact:
  - products.yml configuration
  - Product selector entry (links to platform page section)
  - Version detector for *.influxcloud.net service URLs
  - Ask AI integration

Files changed:
- content/platform/_index.md (simplified, updated links)
- content/shared/identify-version.md (updated links)
- layouts/shortcodes/influxdb-cloud1-note.html (updated links)
- assets/js/utils/product-mappings.ts
- assets/js/influxdb-version-detector.ts

Closes /influxdata/dar/issues/563

* Delete content/influxcloud/v1/_index.md

* Update content/enterprise_influxdb/v1/_index.md

* Update content/influxdb/v1/introduction/install.md

* Fix PR Preview skipping when layout changes include wildcard URL patterns (#6725)

* Initial plan

* Fix: Strip wildcards from URL paths in PR Preview detection

- Update normalizeUrlPath() to remove asterisk wildcards
- Collapse multiple consecutive slashes after wildcard removal
- Add backtick as valid URL delimiter for code-wrapped URLs
- Add comprehensive test cases for wildcard handling
- Update backtick test to reflect safer truncation behavior

Fixes issue where PR descriptions with wildcard patterns like
`/influxdb3/enterprise/*` were not properly extracted, causing
PR Preview to skip even when URLs were provided.

Co-authored-by: jstirnaman <212227+jstirnaman@users.noreply.github.com>

* docs: Clarify backtick handling in URL validation

Add comment explaining that backticks act as delimiters in regex
extraction, preventing them from appearing in extracted paths even
though they're in the rejection pattern.

Co-authored-by: jstirnaman <212227+jstirnaman@users.noreply.github.com>

* docs: Improve comments explaining normalization and regex logic

- Clarify why wildcards are removed before slash collapsing
- Document the defense-in-depth backtick handling
- Add examples of the normalization process

Co-authored-by: jstirnaman <212227+jstirnaman@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: jstirnaman <212227+jstirnaman@users.noreply.github.com>

* Update content/platform/_index.md

Co-authored-by: Scott Anderson <sanderson@users.noreply.github.com>

* refactor: organize cloud1-note shortcode into influxdb directory (#6734)

Move the Cloud 1 note shortcode from influxdb-cloud1-note.html
to influxdb/cloud1-note.html to follow the existing organizational
pattern in layouts/shortcodes/influxdb/.

This change also applies the Cloud 1 content updates from PR #6729
using the new organized shortcode path: {{< influxdb/cloud1-note >}}

closes influxdata/docs-v2#6729

Co-authored-by: Claude <noreply@anthropic.com>

---------

Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
Co-authored-by: Scott Anderson <sanderson@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
2026-01-15 14:55:06 -06:00
Jason Stirnaman e03e7874fc
refactor(ci): improve PR preview with products.yml and index page (#6656)
- Load product prefixes from data/products.yml (single source of truth)
- Build regex pattern dynamically from loaded namespaces
- Fix CSS copying: Hugo outputs fingerprinted CSS at root level
- Generate index.html with clickable links to preview pages
- Add test for influxdb3_explorer namespace

This keeps the URL parser in sync as new products are added and
provides a better preview experience with navigable links.
2025-12-19 12:12:20 -06:00
Jason Stirnaman e203482b62
feat(ci): add PR preview system for GitHub Pages (#6636)
* docs(ci): add PR preview system design

Document the design for GitHub Pages PR previews including:
- Selective deployment of changed pages only
- Reuse of existing content-utils.js change detection
- URL parsing from PR descriptions for layout/asset changes
- Automatic cleanup on PR close
- Storage budget management

* docs(ci): add PR preview implementation plan

Detailed task-by-task implementation plan including:
- 8 tasks with complete code and exact file paths
- Reuses existing content-utils.js for change detection
- Scripts for URL parsing, change detection, file staging
- Main workflow and cleanup workflow
- Setup documentation and testing steps

* feat(ci): add PR URL parser for preview page detection

* fix(ci): harden PR URL parser against malicious input

- Add path validation to reject path traversal attacks (..)
- Add validation to reject HTML/script injection attempts
- Add validation to reject URL-encoded characters
- Update regex to capture markdown link paths: [text](/path/)
- Add comprehensive test suite with 27 security and functionality tests

Security improvements:
- Reject paths containing '..' to prevent directory traversal
- Reject paths with suspicious characters: <, >, |, {, }, `, etc.
- Reject URL-encoded characters to prevent encoding attacks
- Validate all paths against known product prefixes

Functionality improvements:
- Support markdown link syntax: [text](/influxdb3/core/)
- Support reference-style markdown links
- Maintain existing support for production URLs, localhost URLs, and relative paths

* feat(ci): add change detection script for PR previews

* fix(ci): remove unused import from detect-preview-pages.js

* feat(ci): add preview file staging script for selective deployment

* fix(ci): add input validation to preview file staging script

- Add path traversal validation to urlToHtmlPath() to reject paths containing '..'
- Add array validation at start of preparePreviewFiles() with clear error message
- Fix copiedCount to only increment when safeCopy() actually succeeds
- Add return value to copyPage() to track success/failure

Addresses security vulnerabilities and improves accuracy of preview stats.

* feat(ci): add preview comment manager for sticky PR comments

* fix(ci): sanitize user input in preview comments to prevent XSS

* feat(ci): add PR preview workflow for GitHub Pages deployment

* fix(ci): correct environment variables and checkout in PR preview workflow

* feat(ci): add weekly stale preview cleanup workflow

* docs(ci): add PR preview setup guide

* Delete docs/plans/2025-12-16-pr-preview-design.md

* Delete docs/plans/2025-12-16-pr-preview-implementation.md

* fix(ci): add input validation for PR preview security

- Add single quote to rejected characters in URL path validation
  to prevent JavaScript injection in workflow string interpolation
- Add BASE_REF validation regex to prevent command injection,
  allowing slashes for feature branches (e.g., feature/new-auth)
- Add PR_NUM numeric validation in cleanup workflow
- Add tests for single quote handling and BASE_REF validation
2025-12-18 13:44:14 -06:00
Jason Stirnaman 660170435f ci: convert cypress scripts to CommonJS:
Verification Results

  - Direct module loading:  Works perfectly
  - Incremental validation:  Processes files correctly

  - Subprocess calls:  No EPIPE errors
  - Cache functionality:  Operating normally

  🔧 Technical Details

  - All modules now use CommonJS require() statements
  - Proper module.exports for compatibility
  - File extensions changed to .cjs to work with type:
   module in package.json
  - Maintained all existing functionality and error
  handling
2025-07-28 21:58:09 -05:00
Jason Stirnaman 891553ac33 Apply suggestions from code review
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-07-28 16:24:24 -05:00
Jason Stirnaman 27c358037a fix: resolve code review findings:1. Fixed Cypress Race Condition: Moved logging
statements inside the async callback to ensure
  subjects array is populated before accessing its
   length
  2. Proper Async Handling: Used
  cy.wrap(Promise.all()) to handle multiple async
  Cypress tasks correctly
  3. Maintained Functionality: All existing GitHub
   Actions workflows and Cypress tests will
  continue to work

  Both naming conventions are appropriate for
  their use cases:
  - filePathToUrl: Transforms content file paths
  to URL paths
  - fileURLToPath: Converts ES module file URLs to
   file system paths
2025-07-28 16:24:24 -05:00
Jason Stirnaman d762e7800e fix: apply code review suggestions: High Priority Issues (Fixed):
1. Script execution detection in matrix-generator.js -
  Added fileURLToPath import and updated comparison
  2. Script execution detection in incremental-validator.js -
   Added fileURLToPath import and updated comparison
  3. Script execution detection in link-extractor.js - Added
  fileURLToPath import and updated comparison
  4. Script execution detection in comment-generator.js -
  Added fileURLToPath import and updated comparison

  Medium Priority Issues (Fixed):

  5. Extracted duplicated URL transformation logic - Created
  shared utility module and updated both files to use it
  6. Fixed cache key strategy - Updated GitHub workflow to
  use content-based hashing instead of base SHA

  Changes Made:

  - 4 JavaScript files: Updated with robust script execution
  detection using fileURLToPath
  - 1 utility module: Created
  /.github/scripts/utils/url-transformer.js for shared logic
  - 2 files: Updated to use the shared URL transformation
  utility
  - 1 workflow file: Improved cache key strategy for better
  cache hit rates
2025-07-28 16:24:24 -05:00
Jason Stirnaman 6a4e8827eb feat(testing): add link validation automation and improvements
- Add GitHub Actions for automated link validation on PRs
- Implement incremental validation with caching (30-day TTL, configurable)
- Add matrix generator for parallel validation strategy
- Create comprehensive TESTING.md documentation
- Add cache manager with configurable TTL via env var or CLI
- Implement smart link extraction and validation
- Add PR comment generator for broken link reports
- Update Cypress tests to use incremental validation
- Consolidate testing docs from CONTRIBUTING.md to TESTING.md

Key improvements:
- Cache-aware validation only checks changed content
- Parallel execution for large changesets
- Detailed PR comments with broken link reports
- Support for LINK_CACHE_TTL_DAYS env var
- Local testing with yarn test:links
- Reduced false positives through intelligent caching
2025-07-28 16:24:24 -05:00