Commit Graph

67 Commits (a02601e345b4724523379e14f00a204f39c49174)

Author SHA1 Message Date
Jason Stirnaman c2093c8212
Feature: Generate documentation in LLM-friendly Markdown (#6555)
* feat(llms): LLM-friendly Markdown, ChatGPT and Claude links.

This enables LLM-friendly documentation for entire sections,
allowing users to copy complete documentation sections with a single click.

Lambda@Edge now generates .md files on-demand with:
- Evaluated Hugo shortcodes
- Proper YAML frontmatter with product metadata
- Clean markdown without UI elements
- Section aggregation (parent + children in single file)

The llms.txt files are now generated automatically during build from
content structure and product metadata in data/products.yml, eliminating
the need for hardcoded files and ensuring maintainability.

**Testing**:
- Automated markdown generation in test setup via cy.exec()
- Implement dynamic content validation that extracts HTML content and
  verifies it appears in markdown version

**Documentation**:
Documents LLM-friendly markdown generation

**Details**:
Add gzip decompression for S3 HTML files in Lambda markdown generator

HTML files stored in S3 are gzip-compressed but the Lambda was attempting
to parse compressed data as UTF-8, causing JSDOM to fail to find article
elements. This resulted in 404 errors for .md and .section.md requests.

- Add zlib gunzip decompression in s3-utils.js fetchHtmlFromS3()
- Detect gzip via ContentEncoding header or magic bytes (0x1f 0x8b)
- Add configurable DEBUG constant for verbose logging
- Add debug logging for buffer sizes and decompression in both files

The decompression adds ~1-5ms per request but is necessary to parse
HTML correctly. CloudFront caching minimizes Lambda invocations.

Await async markdown conversion functions

The convertToMarkdown and convertSectionToMarkdown functions are async
but weren't being awaited, causing the Lambda to return a Promise object
instead of a string. This resulted in CloudFront validation errors:
"The body is not a string, is not an object, or exceeds the maximum size"

**Troubleshooting**:

- Set DEBUG for troubleshooting in lambda

* feat(llms): Add build-time LLM-friendly Markdown generation

Implements static Markdown generation during Hugo build.

**Key Features:**
- Two-phase generation: HTML→MD (memory-bounded), MD→sections (fast)
- Automatic redirect detection via file size check (skips Hugo aliases)
- Product detection using compiled TypeScript product-mappings module
- Token estimation for LLM context planning (4 chars/token heuristic)
- YAML serialization with description sanitization

**Performance:**
- ~105 seconds for 5,000 pages + 500 sections
- ~300MB peak memory (safe for 2GB CircleCI environment)
- 23 files/sec conversion rate with controlled concurrency

**Configuration Parameters:**
- MIN_HTML_SIZE_BYTES (default: 1024) - Skip files below threshold
- CHARS_PER_TOKEN (default: 4) - Token estimation ratio
- Concurrency: 10 workers (CI), 20 workers (local)

**Output:**
- Single pages: public/*/index.md (with frontmatter + content)
- Section bundles: public/*/index.section.md (aggregated child pages)

**Files Changed:**
- scripts/build-llm-markdown.js (new) - Main build script
- scripts/lib/markdown-converter.cjs (renamed from .js) - Core conversion
- scripts/html-to-markdown.js - Updated import path
- package.json - Updated exports for .cjs module

Related: Replaces Lambda@Edge on-demand generation (5s response time)
with build-time static generation for production deployment.

feat(deploy): Add staging deployment workflow and update CI

Integrates LLM markdown generation into deployment workflows with
a complete staging deployment solution.

**CircleCI Updates:**
- Switch from legacy html-to-markdown.js to optimized build:md
- 2x performance improvement (105s vs 200s+ for 5000 pages)
- Better memory management (300MB vs variable)
- Enables section bundle generation (index.section.md files)

**Staging Deployment:**
- New scripts/deploy-staging.sh for local staging deploys
- Complete workflow: Hugo build → markdown gen → S3 upload
- Environment variable driven configuration
- Optional step skipping for faster iteration
- CloudFront cache invalidation support

**NPM Scripts:**
- Added deploy:staging command for convenience
- Wraps deploy-staging.sh script

**Documentation:**
- Updated DOCS-DEPLOYING.md with comprehensive guide
- Merged staging/production workflows with Lambda@Edge docs
- Build-time generation now primary, Lambda@Edge fallback
- Troubleshooting section with common issues
- Environment variable reference
- Performance metrics and optimization tips

**Benefits:**
- Manual staging validation before production
- Consistent markdown generation across environments
- Faster CI builds with optimized script
- Better error handling and progress reporting
- Section aggregation for improved LLM context

**Usage:**
```bash
export STAGING_BUCKET="test2.docs.influxdata.com"
export AWS_REGION="us-east-1"
export STAGING_CF_DISTRIBUTION_ID="E1XXXXXXXXXX"

yarn deploy:staging
```

Related: Completes build-time markdown generation implementation

refactor: Remove Lambda@Edge implementation

Build-time markdown generation has replaced Lambda@Edge on-demand
generation as the primary method. Removed Lambda code and updated
documentation to focus on build-time generation and testing.

Removed:
- deploy/llm-markdown/ directory (Lambda@Edge code)
- Lambda@Edge section from DOCS-DEPLOYING.md

Added:
- Testing and Validation section in DOCS-DEPLOYING.md
- Focus on build-time generation workflow

* feat: Add Rust HTML-to-Markdown prototype

Implements core markdown-converter.cjs functions in Rust for performance comparison.

Performance results:
- Rust: ~257 files/sec (10× faster)
- JavaScript: ~25 files/sec average

Recommendation: Keep JavaScript for now, implement incremental builds first.
Rust migration provides 10× speedup but requires 3-4 weeks integration effort.

Files:
- Cargo.toml: Rust dependencies (html2md, scraper, serde_yaml, clap)
- src/main.rs: Core conversion logic + CLI benchmark tool
- benchmark-comparison.js: Side-by-side performance testing
- README.md: Comprehensive findings and recommendations

* fix(ui): improve dropdown positioning on viewport resize

- Ensure dropdown stays within viewport bounds (min 8px padding)
- Reposition dropdown on window resize and scroll events
- Clean up event listeners when dropdown closes

* chore(deps): add remark and unified packages for markdown processing

Add remark-parse, remark-frontmatter, remark-gfm, and unified for
enhanced markdown processing capabilities.

* fix(edge): add return to prevent trailing-slash redirect for valid extensions

Without the return statement, the Lambda@Edge function would continue
executing after the callback, eventually hitting the trailing-slash
redirect logic. This caused .md files to redirect to URLs with trailing
slashes, which returned 404 from S3.

* fix(md): add built-in product mappings and full URL support

- Add URL_PATTERN_MAP and PRODUCT_NAME_MAP constants directly in the
  CommonJS module (ESM product-mappings.js cannot be require()'d)
- Update generateFrontmatter() to accept baseUrl parameter and construct
  full URLs for the frontmatter url field
- Update generateSectionFrontmatter() similarly for section pages
- Update all call sites to pass baseUrl parameter

This fixes empty product fields and relative URLs in generated markdown
frontmatter when served via Lambda@Edge.

* feat(md): add environment flag for base URL control

Add -e, --env flag to html-to-markdown.js to control the base URL
in generated markdown frontmatter. This matches Hugo's -e flag behavior
and allows generating markdown with staging or production URLs.

Also update build-llm-markdown.js with similar environment support.

* feat(md): add Rust markdown converter and improve validation

- Add Rust-based HTML-to-Markdown converter with NAPI-RS bindings
- Update Cypress markdown validation tests
- Update deploy-staging.sh with force upload flag

* deploy-staging.sh:
  - Defaults STAGING_URL to https://test2.docs.influxdata.com
  if not set
  - Exports it so yarn build:md -e staging can use it
  - Displays it in the summary

* Delete scripts/prototypes/rust-markdown/benchmark-comparison.js

* Delete scripts/prototypes directory

* fix(llms): Include full URL for section page Markdown and list of child pages

* feat(llms): clarify format selector text for AI use case

Update button and dropdown text to make the AI/LLM purpose clearer:
- Button: "Copy page for AI" / "Copy section for AI"
- Sublabel: "Clean Markdown optimized for AI assistants"
- Section sublabel: "{N} pages combined as clean Markdown for AI assistants"

Cypress tests updated and passing (13/13).

---------

Co-authored-by: Scott Anderson <scott@influxdata.com>
2025-12-01 12:32:28 -06:00
Jason Stirnaman 8ca4f9d2bb
Chore: update dependencies and fix frontmatter (#6556)
* chore(deps): update @vvago/vale to v3.12.0

Updates Vale style linter wrapper from v3.4.2 to v3.12.0.
This eliminates all dependency warnings related to deprecated
packages (fstream, rimraf, glob, inflight) in the Vale
dependency tree.

* fix(frontmatter): resolve frontmatter violations revealed by updates

Fixes frontmatter validation errors revealed after dependency updates:
- Remove duplicate 'related' and 'aliases' fields
- Fix invalid alias formatting (missing space after dash)
- Consolidate frontmatter structure to match schema requirements

Files updated:
- content/influxdb/v1/guides/query_data.md
- content/influxdb/v2/reference/cli/influx/auth/_index.md
- content/influxdb/v2/reference/cli/influx/bucket/_index.md
- content/influxdb/v2/reference/cli/influxd/recovery/user/_index.md
- content/influxdb3/cloud-dedicated/process-data/visualize/chronograf.md
- content/influxdb3/cloud-serverless/process-data/visualize/chronograf.md
- content/influxdb3/clustered/install/set-up-cluster/configure-cluster/directly.md
- content/influxdb3/clustered/process-data/visualize/chronograf.md
- content/platform/monitoring/influxdata-platform/monitoring-dashboards.md
2025-11-20 15:27:40 -06:00
Jason Stirnaman f084fe3a31
Ask AI group filters (#6529)
* feat(askai): add Kapa.ai source group filtering for InfluxDB v3

- Add ai_source_group_ids field to all InfluxDB v3 products in data/products.yml
- Add getProductSourceGroupIds() function to retrieve source group IDs from product data
- Enables filtered AI responses using Kapa source groups for documentation pages
- Follows existing pattern for dynamic product configuration
- Implement version-specific config support (__v1, __v2 suffixes)
- Append version hints to example questions for InfluxDB database products only
- Make example questions generic (remove product-specific names)
- Tools (Telegraf, Chronograf, Kapacitor, Flux, Explorer) display questions without version hints
- Pre-fills chat input with [version: /path/] for InfluxDB database products
- Users can easily edit or remove the pre-filled text
- Works for manual opens (Cmd+K) and programmatic opens
- Converts module to TypeScript

* refactor(ask-ai): change version format to 'My version: <product name>'

Use human-readable product names instead of URL paths for better UX.
Example: 'My version: InfluxDB 3 Core' instead of '[version: /influxdb3/core/]'

* fix(ask-ai): restore working Kapa.open() pre-fill implementation

- Replace textarea detection with direct Kapa.open() call
- Add Kapa preinitialization code
- Use click handler on .ask-ai-open button with capture phase
- Handle conversation reset event to re-fill version context
- Remove console logging for cleaner production code

* fix(ask-ai): remove parentheses from example questions for consistency

Make example question format match the pre-fill format:
- Before: 'question (My version: product)'
- After: 'question My version: product'

This ensures users don't think there's a difference between the two formats.

* fix(askai): add Explorer product mapping for Ask AI widget

- Add influxdb3_explorer mapping to getCurrentProductData()
- Add explorer context to getContext() function
- Ensures Explorer pages use correct ai_sample_questions from products.yml
- Reorder Explorer questions with 'install and run' first

This fixes the issue where Explorer Ask AI widget was showing wrong
example questions by properly loading the influxdb3_explorer config.

* test(page-context): add comprehensive e2e tests for product mappings

Add Cypress tests to validate page-context.js correctly identifies:
- Product context values for all InfluxDB products
- Product data from products.yml configuration
- Version information
- AI sample questions and source group IDs
- Placeholder host values

Tests cover:
- InfluxDB 3 (Core, Enterprise, Explorer, Cloud variants, Clustered)
- InfluxDB v2 and v1
- InfluxDB Cloud (TSM)
- Tools (Telegraf, Chronograf, Kapacitor, Flux)

Validates the fix for Explorer Ask AI showing correct example questions.

Related to #jts-askai-group-filters branch work.

* feat(test): add --no-mapping flag to e2e test runner

Allow running functionality tests without requiring content file paths.
The --no-mapping flag skips content-to-URL mapping, making it easier
to run tests that don't depend on specific content files.

Usage:
  # With content mapping (for content-specific tests)
  node run-e2e-specs.js content/influxdb3/core/_index.md

  # Without content mapping (for functionality tests)
  node run-e2e-specs.js --spec cypress/e2e/page-context.cy.js --no-mapping

Benefits:
- Simplifies running functionality tests like page-context.cy.js
- Reduces test startup time by skipping unnecessary file mapping
- Makes test commands clearer about their purpose

The page-context test was updated to work correctly with this flag.

* deps: update caniuse and related hook files

* test: Add a `--no-mapping` flag to run tests without specific content files (i.e., test contains all the URLs it needs)

* chore(ask-ai): Format example questions

* test(page-context): add comprehensive e2e tests for all products in products.yml

- Expanded test suite from 6 to 27 tests covering all products
- Added tests for InfluxDB 3 products (Explorer, Core, Enterprise, Cloud Serverless, Cloud Dedicated, Clustered)
- Added tests for InfluxDB v2 and Cloud (TSM)
- Added tests for InfluxDB v1 and Enterprise v1
- Added tests for other products (Telegraf, Chronograf, Kapacitor, Flux)
- Validates page mappings in page-context.js
- Validates AI sample questions configuration in products.yml
- All 27 tests passing

* fix(page-context): correct enterprise_influxdb URL pattern matching

- Changed pattern from /enterprise_v1/ to /enterprise_influxdb/
- Fixes Ask AI example questions not showing correctly for Enterprise v1
- Pattern now matches actual URL structure /enterprise_influxdb/v1/
- All 27 e2e tests passing

* test(page-context): add UI validation for Ask AI widget configuration

- Added 4 tests checking Kapa widget script data attributes
- Tests verify data-modal-example-questions contains correct product-specific questions
- Validates Explorer, Core, Enterprise, and Enterprise v1 configurations
- All 31 tests passing (27 existing + 4 new UI tests)

* feat(ask-ai): add help in Ask AI widget placeholder

- InfluxDB placeholder recommends specifying product and version
- Fix page-context.js to use products.influxdb_cloud instead of products.cloud
- Add UI tests verifying version-specific naming in Kapa widget script tags

* feat(ask-ai): Tailors placeholder for each version/product. Disables "Viewing <product>" in disclaimer note.
2025-11-18 16:43:02 -06:00
dependabot[bot] 260284d625
chore(deps): bump js-yaml from 4.1.0 to 4.1.1
Bumps [js-yaml](https://github.com/nodeca/js-yaml) from 4.1.0 to 4.1.1.
- [Changelog](https://github.com/nodeca/js-yaml/blob/master/CHANGELOG.md)
- [Commits](https://github.com/nodeca/js-yaml/compare/4.1.0...4.1.1)

---
updated-dependencies:
- dependency-name: js-yaml
  dependency-version: 4.1.1
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-11-14 18:05:18 +00:00
Jason Stirnaman e8a48e3482
chore(docs): Redesign docs CLI tools for creating and editing content, add content/create.md tutorial page for the How to creat… (#6506)
* chore(docs): Add content/create.md tutorial page for the How to create your own documentation tutorial

chore(scripts): docs:create and docs:edit scripts for content creation and editing

Major improvements to docs:create UX for both Claude Code and external tool integration:

**New `docs` CLI command:**
- Add scripts/docs-cli.js - main CLI with subcommand routing
- Add bin field to package.json for `docs` command
- Usage: `docs create` and `docs edit` (cleaner than yarn commands)

**Smart piping detection:**
- Auto-detect when stdout is piped (\!process.stdout.isTTY)
- When piping: automatically output prompt text (no flag needed)
- When interactive: output prompt file path
- --print-prompt flag now optional (auto-enabled when piping)

**Specify products:**
- simplify link following behavior - treat relative paths as local files, all HTTP/HTTPS as external
- stdin now requires --products flag with product keys
- --products now accepts keys from products.yml (influxdb3_core, telegraf, etc.)

Examples:
  --products influxdb3_core
  --products influxdb3_core,influxdb3_enterprise
  --products telegraf

**Usage examples:**
  # Inside Claude Code - automatic execution
  docs create drafts/new-feature.md

  # Pipe to external AI - prompt auto-detected
  docs create FILE --products X | claude -p
  docs create FILE --products X | copilot -p

  # Pipe from stdin
  echo 'content' | docs create --products X | claude -p

Benefits:
- Cleaner syntax (no yarn --silent needed)
- No manual --print-prompt flag when piping
- Consistent with industry tools (git, npm, etc.)
- Backward compatible with yarn commands

WIP: docs:create usage examples

- Redesign of the docs CLI tools for creating and editing content
- Cleaner interface works better for piping output to agents and downstream utilities
- Updates README.md and other authoring docs

This repository includes a `docs` CLI tool for common documentation workflows:

```sh
npx docs create drafts/new-feature.md --products influxdb3_core

npx docs edit https://docs.influxdata.com/influxdb3/core/admin/

npx docs placeholders content/influxdb3/core/admin/upgrade.md

npx docs --help
```

**Run test cases:**

```sh
npx docs test
```

* Update content/create.md

* Update content/create.md

* Update content/create.md

* Update content/create.md

* Update scripts/templates/chatgpt-prompt.md

* Update DOCS-SHORTCODES.md
2025-11-03 10:18:15 -06:00
Jason Stirnaman e93e78be0a feat(influxdb): Version detector shortcode triggers a modal
Creates an interactive InfluxDB version detector component in TypeScript and a shortcode that generates a button to trigger
the version detector modal.
The shortcode takes a parameter that displays a predefined set of links for results.
- Support URL pattern matching and ping header analysis
- Add questionnaire-based product identification logic
- Adds the shortcode in a note in /influxdb3/core/visualize-data/grafana/
- Set up TypeScript configuration for the project
  - Configure automatic TypeScript compilation in pre-commit hooks
- Add to Grafana documentation pages
- Remove last remnants of old Cypress link checker
- Add Cypress tests, but many are still broken

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Apply suggestions from code review

Co-authored-by: Scott Anderson <sanderson@users.noreply.github.com>

Update layouts/shortcodes/influxdb-version-detector.html

Co-authored-by: Scott Anderson <sanderson@users.noreply.github.com>

Update assets/js/influxdb-version-detector.ts

Co-authored-by: Scott Anderson <sanderson@users.noreply.github.com>

Update assets/styles/components/_influxdb-version-detector.scss

Co-authored-by: Scott Anderson <sanderson@users.noreply.github.com>

Fixes:
- Fix Hugo template to include product names in detector config
- Change elimination scores from -100 to -1000 for proper filtering
- Add scoring logic for generic "InfluxDB" product (OSS v2.x)
- Exclude generic "InfluxDB" from results (too vague)
- Add comprehensive test scenario checklist to Cypress tests
- Free license now correctly excludes Enterprise, Clustered, Dedicated
- Self-hosted now correctly excludes all Cloud products
- SQL language now correctly excludes v1 and v2 products
- Results now show only specific products (OSS 1.x, OSS 2.x, etc.)
Changes:
- When users answer "I'm not sure" to all questions, show a helpful
message directing them to the reference table instead of showing
a weak ranking with low confidence.
- Detect when all questionnaire answers are "unknown"
- Display custom message explaining lack of information
- Auto-expand reference table for easy product identification
- Hide ranked results when insufficient information provided
- Make product names clickable in the quick reference table to allow
users to quickly navigate to product documentation after identifying
their InfluxDB version.
2025-09-30 19:01:21 -05:00
dependabot[bot] 2b7aa74814
chore(deps): bump axios from 1.11.0 to 1.12.0
Bumps [axios](https://github.com/axios/axios) from 1.11.0 to 1.12.0.
- [Release notes](https://github.com/axios/axios/releases)
- [Changelog](https://github.com/axios/axios/blob/v1.x/CHANGELOG.md)
- [Commits](https://github.com/axios/axios/compare/v1.11.0...v1.12.0)

---
updated-dependencies:
- dependency-name: axios
  dependency-version: 1.12.0
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-09-12 21:50:51 +00:00
Jason Stirnaman ef1c598ad2 chore: yarn upgrade 2025-09-02 14:16:54 -05:00
dependabot[bot] 73bb35d4de
chore(deps): bump mermaid from 11.9.0 to 11.10.0
Bumps [mermaid](https://github.com/mermaid-js/mermaid) from 11.9.0 to 11.10.0.
- [Release notes](https://github.com/mermaid-js/mermaid/releases)
- [Commits](https://github.com/mermaid-js/mermaid/compare/mermaid@11.9.0...mermaid@11.10.0)

---
updated-dependencies:
- dependency-name: mermaid
  dependency-version: 11.10.0
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-08-20 14:42:57 +00:00
dependabot[bot] 2e12fa79c3
chore(deps): bump tmp from 0.2.3 to 0.2.4
Bumps [tmp](https://github.com/raszi/node-tmp) from 0.2.3 to 0.2.4.
- [Changelog](https://github.com/raszi/node-tmp/blob/master/CHANGELOG.md)
- [Commits](https://github.com/raszi/node-tmp/compare/v0.2.3...v0.2.4)

---
updated-dependencies:
- dependency-name: tmp
  dependency-version: 0.2.4
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-08-06 17:14:39 +00:00
Jason Stirnaman e868d1d3f5 chore(dependencies) 2025-07-30 10:16:23 -05:00
dependabot[bot] 57f52ed863
chore(deps): bump axios from 1.10.0 to 1.11.0
Bumps [axios](https://github.com/axios/axios) from 1.10.0 to 1.11.0.
- [Release notes](https://github.com/axios/axios/releases)
- [Changelog](https://github.com/axios/axios/blob/v1.x/CHANGELOG.md)
- [Commits](https://github.com/axios/axios/compare/v1.10.0...v1.11.0)

---
updated-dependencies:
- dependency-name: axios
  dependency-version: 1.11.0
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-07-25 16:55:42 +00:00
dependabot[bot] b2de7e7e51
chore(deps): bump form-data from 4.0.3 to 4.0.4
Bumps [form-data](https://github.com/form-data/form-data) from 4.0.3 to 4.0.4.
- [Release notes](https://github.com/form-data/form-data/releases)
- [Changelog](https://github.com/form-data/form-data/blob/master/CHANGELOG.md)
- [Commits](https://github.com/form-data/form-data/compare/v4.0.3...v4.0.4)

---
updated-dependencies:
- dependency-name: form-data
  dependency-version: 4.0.4
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-07-22 05:41:30 +00:00
Jason Stirnaman d31a5a0139 chore(vale): Upgrade vale, includes an updated brace-expansion with the vuln. patch 2025-06-17 17:10:37 -05:00
Jason Stirnaman 1dce052e56 fix(JS): Rename CommonJS scripts to .cjs extension, keep type: module as the project default. Update and fix ESLint configuration.
- Renames JavaScript files in flux-build-scripts and api-docs/openapi/plugins to .cjs file extension to declare them as CommonJS module syntax.
2025-05-19 11:34:42 -05:00
Jason Stirnaman 4cfff239f3 End-to-end testing, CI script, and JavaScript QoL improvements:
- **Environment variable formatting** - Updated environment variable configuration from array format to object format to comply with Lefthook schema validation requirements.
- **Unified link testing** - Consolidated multiple product-specific link testing commands into a single `e2e-links` command that processes all staged Markdown and HTML files across content directories.
- **Package script integration** - Modified commands to use centralized yarn scripts instead of direct execution, improving maintainability and consistency.
- **Source information extraction** - Enhanced to correctly extract and report source information from frontmatter.
- **URL and source mapping** - Improved handling of URL to source path mapping for better reporting.
- **Ignored anchor links configuration** - Added proper exclusion of behavior-triggering anchor links (like tab navigation) to prevent false positives.
- **Request options correction** - Fixed Cypress request options to ensure `failOnStatusCode` is properly set when `retryOnStatusCodeFailure` is enabled.
- **Improved error reporting** - Enhanced error reporting with more context about broken links.
- **New test scripts added** - Added centralized testing scripts for link checking and codeblock validation.
- **Product-specific test commands** - Added commands for each product version (InfluxDB v2, v3 Core, Enterprise, Cloud, etc.).
- **API docs testing** - Added specialized commands for testing API documentation links.
- **Comprehensive test runners** - Added commands to run all tests of a specific type (`test:links:all`, `test:codeblocks:all`).
- Fix Docker build command and update CONTRIBUTING.

chore(js): JavaScript QoL improvements:

- Refactor main.js with a componentRegistry object and clear initialization of components and globals
- Add a standard index.js with all necessary exports.
- Update javascript.html to use the index.js
- Remove jQuery script tag from header javascript.html (remains in footer)
- Update package file to improve module discovery.
- Improve Hugo and ESLint config for module discovery and ES6 syntax
2025-05-19 09:50:33 -05:00
Jason Stirnaman 9755033970 chore(ci): closes #5887 Improve and automate pre-commit link-checking 2025-03-13 14:24:36 -05:00
Jason Stirnaman 38c9b6d5ef fix(clustered): set up clustered for air-gapped environments, add instructions for copying and installing the kubit CLI image to the environment, and for Helm, copy the kubit operator images. Closes https://github.com/influxdata/DAR/issues/484 2025-03-11 12:38:33 -05:00
Jason Stirnaman 0b7b73b8f7 fix(ci): Remove Nightwatch dependency 2025-02-25 17:09:32 -06:00
Jason Stirnaman 4314589c07 feat(v3): WIP: Revise API specs for Core and Enterprise
- Adds basic support for core and enterprise in getswagger.sh
- Adds custom OpenAPI info for Core and Enterprise
- Validates as OpenAPI 3.0 (using Spectral)
    - operationId
    - tags
- Revises use of Legacy, v2
- TODO: need to check and validate in UI, adjust tags if nec.
- Add and remove components
- Update parameters
- Add examples
- Add tests for Core
2025-02-25 17:08:05 -06:00
meelahme 8b5c130555 Fix dependency issues 2025-02-19 14:20:07 -08:00
Jason Stirnaman b5cda5ce15 hotfix(api): update header back link for new product structure 2025-02-19 16:06:25 -06:00
meelahme 80e95f7afc Upgrade serialize-javascript to patch XSS vulnerability 2025-02-18 12:37:19 -08:00
Jason Stirnaman 3abf1c506d chore: resolve conflicts 2025-02-14 09:39:44 -06:00
Jason Stirnaman cd3ad9e0a1 chore: Update and resolve yarn.lock 2025-02-12 13:52:06 -06:00
Paul Dix 338fcfec8b Add plugin documentation for Core and Enterprise 2025-02-12 13:52:04 -06:00
Jason Stirnaman 9947c72495 chore(e2e): Add Cypress for link checking and end-to-end tests. Fix broken links revealed by tests.
- Adds Cypress and a few basic tests for the global topnav, the home page, and link-checking.
- For link-checking, pass a comma-delimited list of URLs in an exported cypress_test_subjects environment variable. For examples, see the convenience commands in package.json
2025-02-12 13:48:30 -06:00
Jason Stirnaman c173edce68 Initial Kapa.ai chat integration.
Continue refactoring JavaScript into a component pattern and ESM.
Replaces some jQuery with native DOM API.

chore(ai): reference documentation and instructions for training AI

chore(ai): implement Kapa AI chat widget
- Move script tag to HTML template to make it obvious.
- Cleanup javascript to make it more component-like
- Set Kapa attributes, support setting userid

chore(js): add JS dependencies, previously referenced in script tags, to package.json for JS builds.

fix(api): indents

chore(js): package Mermaid diagram library

chore(js): refactor JS for AIChat and Theme as examples of using the component pattern for HTML/CSS/JS

chore(js): Use the new local-storage API in refactored module code and in code not yet ported. Cleanup syntax in local-storage and make functions available from window.LocalStorageAPI.

fix(js): theme.js name-change

chore(js): fix ai-chat.js file name

fix(js): refactor:
- componentNames are snakecase in HTML
- replace DOM selection method and jQuery eventhandler assignment
- remove old theme.js references

chore(ai): configure chat window overlay, size, and position:
- removes overlay and scroll lock
- positions chat to the right and bottom
- expands sample question width to 12 cols

chore(ai): edit disclaimer

fix(ai): size and position

chore(js): make ai-chat specific to configuration and and setting userid (for testing and future use).

fix(js): copy referrerHost variable to v3-wayfinding instead of relying on influxdb-url to assign it.

chore(ai): add a footer div at page bottom to contain modal triggers for custom-time and ask-ai. Still needs some CSS help. Moves tooltip text from CSS to HTML data attribute.

chore(ai): dynamically load AI script tag after DOMContentLoaded to avoid race conditions. Call initialization from the modal trigger module and pass the show trigger function to the onload handler.

fix(ai): fix modal triggers to viewport

fix(modal-triggers): stack the triggers into a single column.

restyle footer widgets

updated time selector modal to use correct storage term

minor style update

WIP(ai-chat): get product data

chore(js): Factor out pageContext module from influxdb-url.js

chore(js): Refactor helpers.js out of inflluxdb-url.js

WIP: refactor influxdburl - minimal changes for module conversions

feat(ai): Custom AI chat example questions product and version.
Ask AI example questions:
- Adds support for customizing example Ask AI questions per product or version.
- Configure questions in site `data/products.yml`; otherwise, it uses default questions from `ask-ai.js`

Context, page, and product data:
- Adds sample URLs for remaining versions in influxdb_urls
- `page-context.js` consolidates and exports constants for page context (protocol, host, path, referrer) and path-to-data mappings for product and influxdb_url site data

Module refactor:
- Refactors some JavaScript into ES6 modules, and refactors some of those further into a Component pattern--just vanilla JS and no shadow DOM stuff. The Component pattern that uses data attributes to "bind" JavaScript modules with CSS and HTML is a popular approach in modern web development. This pattern enhances modularity, reusability, and maintainability by associating behavior (JavaScript), structure (HTML), and style (CSS) through the use of data attributes.
- `assets/main.js` is the entrypoint
- Passes pageParams from the Hugo page to modules that import `@params`.
- Moves most external dependencies out of `script` tags and into package.json to be managed with `yarn`.
- Adds `eslint`.
- For modules that aren't yet components, wraps execution statements inside an `initialize()` function and calls the function from `main.js` on `DOMContentLoaded`.
- For components, if the page contains the `data-component="<component-name>"`, the matching element is passed to the component function on `DOMContentLoaded`.
- I tried to avoid changing logic where it wasn't necessary.

Update DOC_GPT_PROFILE.md

customize ai chat modal styles

fix(influxdb-url): Rename to cloud_dedicated in influxdb_urls.yml, remove newly added placeholder URL and use the extant default, refactor
- Rename  to  in influxdb_urls.yml
- Fix influxdb-url.js and data provision in local-storage.js to use the new name, mapping it to  to retain the existing local storage key

chore(api-lib): Use local-storage import instead of window global

chore(js): cleanup

fix(js): Ensure feature-callout initializes on page load

fix(theme): Load preferred theme before making the page visible. Execute a predefined function by specifying the function name in data-theme-callback

fix(search-toggle): Restores toggling the search field when sidebar is collapsed. Moves the event handler to a new search-button component

fix(ai): Fix custom attribute assignment. Rename property to ai_example_questions

Include the word `Bearer` or `Token`, a space, and your **token** value (all case-sensitive). Fix TOC links.
Fixes #5781

fix(api-docs): Update API reference directories and generation script for influxdb3 URL paths, update links and names in reference content

fix(api-ref): Update getswagger.sh destination paths to use the new directory structure when fetching spec files. Update the redocly  plugin module path.

hotfix: fix hlevel bug in children shortcode

Remove underline from custom time widget

add color to custom time widget styling
2025-02-12 13:48:29 -06:00
Jason Stirnaman a15f8c8154 chore: Update and resolve yarn.lock 2025-02-12 10:49:00 -06:00
Jason Stirnaman ef106dd3a1 chore(e2e): Add Cypress for link checking and end-to-end tests. Fix broken links revealed by tests.
- Adds Cypress and a few basic tests for the global topnav, the home page, and link-checking.
- For link-checking, pass a comma-delimited list of URLs in an exported cypress_test_subjects environment variable. For examples, see the convenience commands in package.json
2025-02-03 17:52:04 -06:00
Jason Stirnaman f9f81ae8f9 Initial Kapa.ai chat integration.
Continue refactoring JavaScript into a component pattern and ESM.
Replaces some jQuery with native DOM API.

chore(ai): reference documentation and instructions for training AI

chore(ai): implement Kapa AI chat widget
- Move script tag to HTML template to make it obvious.
- Cleanup javascript to make it more component-like
- Set Kapa attributes, support setting userid

chore(js): add JS dependencies, previously referenced in script tags, to package.json for JS builds.

fix(api): indents

chore(js): package Mermaid diagram library

chore(js): refactor JS for AIChat and Theme as examples of using the component pattern for HTML/CSS/JS

chore(js): Use the new local-storage API in refactored module code and in code not yet ported. Cleanup syntax in local-storage and make functions available from window.LocalStorageAPI.

fix(js): theme.js name-change

chore(js): fix ai-chat.js file name

fix(js): refactor:
- componentNames are snakecase in HTML
- replace DOM selection method and jQuery eventhandler assignment
- remove old theme.js references

chore(ai): configure chat window overlay, size, and position:
- removes overlay and scroll lock
- positions chat to the right and bottom
- expands sample question width to 12 cols

chore(ai): edit disclaimer

fix(ai): size and position

chore(js): make ai-chat specific to configuration and and setting userid (for testing and future use).

fix(js): copy referrerHost variable to v3-wayfinding instead of relying on influxdb-url to assign it.

chore(ai): add a footer div at page bottom to contain modal triggers for custom-time and ask-ai. Still needs some CSS help. Moves tooltip text from CSS to HTML data attribute.

chore(ai): dynamically load AI script tag after DOMContentLoaded to avoid race conditions. Call initialization from the modal trigger module and pass the show trigger function to the onload handler.

fix(ai): fix modal triggers to viewport

fix(modal-triggers): stack the triggers into a single column.

restyle footer widgets

updated time selector modal to use correct storage term

minor style update

WIP(ai-chat): get product data

chore(js): Factor out pageContext module from influxdb-url.js

chore(js): Refactor helpers.js out of inflluxdb-url.js

WIP: refactor influxdburl - minimal changes for module conversions

feat(ai): Custom AI chat example questions product and version.
Ask AI example questions:
- Adds support for customizing example Ask AI questions per product or version.
- Configure questions in site `data/products.yml`; otherwise, it uses default questions from `ask-ai.js`

Context, page, and product data:
- Adds sample URLs for remaining versions in influxdb_urls
- `page-context.js` consolidates and exports constants for page context (protocol, host, path, referrer) and path-to-data mappings for product and influxdb_url site data

Module refactor:
- Refactors some JavaScript into ES6 modules, and refactors some of those further into a Component pattern--just vanilla JS and no shadow DOM stuff. The Component pattern that uses data attributes to "bind" JavaScript modules with CSS and HTML is a popular approach in modern web development. This pattern enhances modularity, reusability, and maintainability by associating behavior (JavaScript), structure (HTML), and style (CSS) through the use of data attributes.
- `assets/main.js` is the entrypoint
- Passes pageParams from the Hugo page to modules that import `@params`.
- Moves most external dependencies out of `script` tags and into package.json to be managed with `yarn`.
- Adds `eslint`.
- For modules that aren't yet components, wraps execution statements inside an `initialize()` function and calls the function from `main.js` on `DOMContentLoaded`.
- For components, if the page contains the `data-component="<component-name>"`, the matching element is passed to the component function on `DOMContentLoaded`.
- I tried to avoid changing logic where it wasn't necessary.

Update DOC_GPT_PROFILE.md

customize ai chat modal styles

fix(influxdb-url): Rename to cloud_dedicated in influxdb_urls.yml, remove newly added placeholder URL and use the extant default, refactor
- Rename  to  in influxdb_urls.yml
- Fix influxdb-url.js and data provision in local-storage.js to use the new name, mapping it to  to retain the existing local storage key

chore(api-lib): Use local-storage import instead of window global

chore(js): cleanup

fix(js): Ensure feature-callout initializes on page load

fix(theme): Load preferred theme before making the page visible. Execute a predefined function by specifying the function name in data-theme-callback

fix(search-toggle): Restores toggling the search field when sidebar is collapsed. Moves the event handler to a new search-button component

fix(ai): Fix custom attribute assignment. Rename property to ai_example_questions

Include the word `Bearer` or `Token`, a space, and your **token** value (all case-sensitive). Fix TOC links.
Fixes #5781

fix(api-docs): Update API reference directories and generation script for influxdb3 URL paths, update links and names in reference content

fix(api-ref): Update getswagger.sh destination paths to use the new directory structure when fetching spec files. Update the redocly  plugin module path.

hotfix: fix hlevel bug in children shortcode

Remove underline from custom time widget

add color to custom time widget styling
2025-02-03 08:31:55 -06:00
Paul Dix b78162d24c Add plugin documentation for Core and Enterprise 2025-01-19 18:07:40 -05:00
Scott Anderson e924c0a82d
Add new date_bin_wallclock SQL functions (#5691)
* add new date_bin_wallclock sql fns

* Apply suggestions from code review

Co-authored-by: Jason Stirnaman <jstirnaman@influxdata.com>

* updates to address PR feedback

* Apply suggestions from code review

Co-authored-by: Chunchun Ye <14298407+appletreeisyellow@users.noreply.github.com>

* updates to address PR feedback

* fixed typos

---------

Co-authored-by: Jason Stirnaman <jstirnaman@influxdata.com>
Co-authored-by: Chunchun Ye <14298407+appletreeisyellow@users.noreply.github.com>
2024-12-05 17:10:19 -07:00
Scott Anderson 240873cd14
InfluxQL TZ clause and function (#5689)
* document influxql tz support, related to influxdb/influxdb_iox#12681

* updated deps
2024-11-27 11:58:37 -07:00
dependabot[bot] 3f825e8777
chore(deps): bump micromatch from 4.0.7 to 4.0.8 (#5599)
Bumps [micromatch](https://github.com/micromatch/micromatch) from 4.0.7 to 4.0.8.
- [Release notes](https://github.com/micromatch/micromatch/releases)
- [Changelog](https://github.com/micromatch/micromatch/blob/master/CHANGELOG.md)
- [Commits](https://github.com/micromatch/micromatch/compare/4.0.7...4.0.8)

---
updated-dependencies:
- dependency-name: micromatch
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-09-12 18:00:21 -05:00
Scott Anderson de4e42ba28
View partition documentation and update partition time templates (#5574)
* updates partitioning information, closes #5561

* upgraded yarn deps

* updated partition time template examples

* Apply suggestions from code review

Co-authored-by: Jason Stirnaman <jstirnaman@influxdata.com>

* unskip influxctl tests

---------

Co-authored-by: Jason Stirnaman <jstirnaman@influxdata.com>
2024-08-22 17:06:44 -06:00
Jason Stirnaman 946b7a1f83 chore(ci): Update axios for security patch. 2024-08-20 14:28:58 -05:00
Scott Anderson 78b99259bc hotfix: updated npm packages 2024-08-20 09:12:51 -06:00
Jason Stirnaman 37a7a7c6e7 chore(ci): Improve pre-commit hook, test harness, and linter
- Pre-commit hook:
       - Redo test harness so commands can run in parallel
       - Cleanup Vale command args
       - Replace lint-staged and husky with Lefthook.
    - Pre-push hook:
       - Scan for package vulnerabilities
    - Tests
      - Run tests on demand, in parallel, or debugged using docker compose up, run, exec, etc.
      - Each test container (service) runs the prepare-content script before running tests.
      - Return to using Python instead of Docker to provide environment variables so they don't get exposed in Docker inspect.
      - Bind mount influxctl config.toml and .env.test files
      - Bind mount scripts for faster builds while iterating on test scripts.
    - Vale:
      - Override Acronyms style def--ignore acronym if it's linked.
2024-07-23 15:08:28 -05:00
Jennifer Moore 59d8958ebc
docs(clustered): initial license onboarding guide (#5508)
* docs(clustered): initial license onboarding guide

* refactor: run `deno fmt`

* fix(clustered): make room for documents in between existing ones

* fix(clustered): introduce sensible deterministic order for reference documents

* refactor: separate license installation from license reference

* fix: markdown typo, confusing verbiage

* Licensing content and structure updates (#5520)

* WIP licensing content

* changes to licensing content and structure

* Update content/influxdb/clustered/admin/licensing.md

Co-authored-by: Jennifer Moore <mail@jenniferplusplus.com>

* updated TOC link to license recovery section

* Apply suggestions from code review

Co-authored-by: Jason Stirnaman <jstirnaman@influxdata.com>

* updated license recovery content

---------

Co-authored-by: Jennifer Moore <mail@jenniferplusplus.com>
Co-authored-by: Jason Stirnaman <jstirnaman@influxdata.com>

* ported edits to new feature branch

---------

Co-authored-by: Scott Anderson <sanderson@users.noreply.github.com>
Co-authored-by: wayne warren <wayne.warren.s@gmail.com>
Co-authored-by: Scott Anderson <scott@influxdata.com>
Co-authored-by: Jason Stirnaman <jstirnaman@influxdata.com>
2024-07-11 17:03:38 -06:00
Jason Stirnaman 3bde9061e7 chore: Remove prettier from Markdown linting--to many whitespace conflicts with code blocks 2024-06-24 15:57:26 -05:00
Jason Stirnaman 5c74f013a1 chore(ci): Improve CI lint and test runners
- Reconfigures prettier linting.
- Adds .editorconfig to help with consistent editor settings
- Refactors test runs:
  - Removes test configuration from compose.yaml (not suited for this use case).
  - Splits test runner into test content setup and pytest that can be run separately or together (and with other test runners in the future).
  - Configuration is in Dockerfiles and command line (`.lintstagedrc.mjs`)
- Updates CONTRIBUTING.md
- Updates client library write examples in cloud-dedicated and clustered.
2024-06-21 18:41:07 -05:00
Jason Stirnaman 2e9d94fc90 chore(ci): configure lint-staged to run vale with custom vocabs
- Configures lint-staged to run vale on staged files and use a custom config with custom vocab for specific directories.
- .md files in other directories use the project-level .vale.ini
- Adds the write-good package for style checking
2024-05-29 16:46:51 -05:00
Jason Stirnaman 12b1583b68 chore: add test data and improve test runner:
- Pulls python client lib and CLI from Github repo instead of package repo so we can test the latest release. Specifies the last release tag.
- Adds CSV and JSON home sensor data samples to /static/downloads.
- Test image build bind mounts /static/downloads into /usr/src/app/test/tmp/data. Allows code samples to reference sample data as `./data/[file]`.
- Moves run-tests.sh to setup directory.
- Dockerfile cleanup. Add jq for inspection/troubleshooting.
2024-05-29 13:31:31 -05:00
dependabot[bot] 9ec7d65414
chore(deps): bump tar from 6.2.0 to 6.2.1 (#5416)
Bumps [tar](https://github.com/isaacs/node-tar) from 6.2.0 to 6.2.1.
- [Release notes](https://github.com/isaacs/node-tar/releases)
- [Changelog](https://github.com/isaacs/node-tar/blob/main/CHANGELOG.md)
- [Commits](https://github.com/isaacs/node-tar/compare/v6.2.0...v6.2.1)

---
updated-dependencies:
- dependency-name: tar
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-04-11 09:42:22 -06:00
dependabot[bot] 944c628051
chore(deps): bump follow-redirects from 1.15.5 to 1.15.6 (#5366)
Bumps [follow-redirects](https://github.com/follow-redirects/follow-redirects) from 1.15.5 to 1.15.6.
- [Release notes](https://github.com/follow-redirects/follow-redirects/releases)
- [Commits](https://github.com/follow-redirects/follow-redirects/compare/v1.15.5...v1.15.6)

---
updated-dependencies:
- dependency-name: follow-redirects
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Scott Anderson <sanderson@users.noreply.github.com>
2024-03-21 10:19:34 -06:00
Andreas Deininger 0488345ab7
Fix errors when running latest hugo version v0.123.8 (#5356)
Co-authored-by: Scott Anderson <sanderson@users.noreply.github.com>
2024-03-11 17:31:33 -06:00
Andreas Deininger 476a73e95e
Fixing typos (#5315)
* Fix typos

* Bump hugo to latest version v0.122.0

---------

Co-authored-by: Scott Anderson <sanderson@users.noreply.github.com>
2024-02-05 09:51:51 -07:00
Andreas Deininger 57e4929746
Bump hugo to latest version v0.121.2 (#5293)
* Bump hugo to latest version v0.121.2

* Fix deprecation warnings emitted from hugo

---------

Co-authored-by: Jason Stirnaman <stirnamanj@gmail.com>
Co-authored-by: Scott Anderson <sanderson@users.noreply.github.com>
2024-01-17 09:15:00 -07:00
dependabot[bot] 5fcec0f89a
chore(deps): bump follow-redirects from 1.15.3 to 1.15.4 (#5298)
Bumps [follow-redirects](https://github.com/follow-redirects/follow-redirects) from 1.15.3 to 1.15.4.
- [Release notes](https://github.com/follow-redirects/follow-redirects/releases)
- [Commits](https://github.com/follow-redirects/follow-redirects/compare/v1.15.3...v1.15.4)

---
updated-dependencies:
- dependency-name: follow-redirects
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-01-11 08:50:09 -07:00