The server-rendered TOC path returned early without setting up the
IntersectionObserver, so the is-active class was never applied to
TOC links during scroll. Extract entry IDs from pre-rendered links
and pass them to setupScrollHighlighting.
- Switch operation anchors from {method}-{path} to
operation/{operationId} (PascalCase), matching Redocly convention
and existing content links like #operation/PutPluginFile.
Updated in Hugo templates, all-endpoints list, and JS TOC.
- Remove nested bordered containers from API operations. Operations
separated by top-border dividers instead of card wrappers. Param
lists, schema properties, and responses no longer wrapped in
bordered boxes.
- Replace CSS custom properties (var(--article-text, ...)) with SCSS
theme variables ($article-text, $article-heading, $nav-border, etc).
The CSS vars were never defined in this theme, so all text fell back
to hardcoded dark-gray values invisible in dark mode.
- Delete entire dark mode override section — now handled automatically
by Hugo's per-theme SCSS compilation.
- Add border-bottom divider to operation endpoint line for visual
separation from the operation heading.
- Update PLAN.md to mark Task 7 as completed.
Remove all RapiDoc-specific code from styles and JavaScript:
- Delete rapi-doc::part() CSS selectors from _api-layout.scss
- Delete dead auth modal styles from _api-security-schemes.scss
- Delete api-auth-input.ts component (RapiDoc "Try it" integration)
- Delete static/css/rapidoc-custom.css (unused)
- Remove setupRapiDocNavigation() from api-toc.ts
- Update comments to remove RapiDoc references
Rename _api-hugo-native.scss to _api-operations.scss since
Hugo-native is now the standard (not a POC).
Complete Tasks 1-5 of the Hugo-native API migration:
- Promote Hugo-native templates from POC to production location
- Remove RapiDoc templates (rapidoc.html, rapidoc-tag.html, rapidoc-mini.html)
- Remove RapiDoc JS components (api-rapidoc.ts, rapidoc-mini.ts)
- Clean up generation scripts (~200 lines of dead code removed)
- Rewrite Cypress tests for standard HTML (no shadow DOM)
Templates moved from layouts/partials/api/hugo-native/ to parent.
All verifications pass (Hugo build, TypeScript compilation).
Fix button cutoff on mobile screens by applying CSS fixes to RapiDoc's
nested shadow DOMs. Changes:
- Add getAllShadowRoots() to recursively find nested shadow roots
(RapiDoc contains api-request elements with their own shadow DOMs)
- Apply button fixes to all shadow roots, not just the top-level one
- Add responsive CSS for endpoint cards to prevent horizontal overflow
- Remove debug console.log statements
Tested at viewport sizes 400x770 and 430x932.
* fix(api): refine .IsSection check to prevent breaking endpoint pages
The previous commit added an .IsSection check that correctly fixed
section index pages but inadvertently broke endpoint pages that Hugo
classifies as sections.
This commit refines the conditional to check for the absence of the
staticFilePath parameter, which distinguishes true section index pages
from endpoint pages that need the RapiDoc renderer:
- Section pages (no staticFilePath): Show children listing
- Endpoint pages (with staticFilePath): Use RapiDoc renderer
Fixes the regression introduced in commit 59daf17.
* chore: update Scalar references to RapiDoc in comments
Update code comments and documentation to accurately reflect that
the API documentation uses RapiDoc, not Scalar:
- api-docs/scripts/generate-openapi-articles.ts: Update JSDoc comments
- api-docs/scripts/dist/generate-openapi-articles.js: Update compiled version
- assets/js/components/api-toc.ts: Update implementation comment
- .claude/agents/ts-component-dev.md: Update component reference
- .claude/skills/hugo-template-dev/SKILL.md: Update component reference
The implementation uses RapiDoc throughout, so these comments needed
to be corrected for accuracy.
---------
Co-authored-by: Claude <noreply@anthropic.com>
Auth credentials:
- Switch from in-memory to sessionStorage for credentials
- Credentials persist across page navigations within browser tab
- Auto-clear when tab closes (no long-term storage)
- Pre-fill form fields with saved credentials on page load
- Update status text and button based on credential state
Styling fixes:
- Add right padding to code blocks so Copy button doesn't overlap content
- Make long URLs wrap instead of requiring horizontal scroll
- Hide TOC sidebar when no headings exist (e.g., quick-start page)
- Remove "Use RapiDoc's navigation..." message from TOC
API docs:
- Remove x-codeSamples from write_lp endpoints (Core and Enterprise)
- Add schema descriptions for line protocol request body
- Convert collapsible auth panel to popover UI triggered by
"Set credentials" button positioned above RapiDoc
- Filter auth schemes based on API path:
- /api/v3/* endpoints: Bearer only
- /api/v2/* endpoints: Bearer + Token
- /write, /query (v1): All 4 schemes
- Fix "Authentication Not Required" bug by adding global security
field to path-specific OpenAPI specs
- Change RapiDoc --blue CSS variable to green so status text and
type links don't look like clickable links
- Add explicit button styling for Apply/Clear in popover
- Hide RapiDoc's built-in auth section (use custom popover instead)
- Create api-auth-input.ts TypeScript component
- Store credentials in sessionStorage (secure, cleared on tab close)
- Register component in main.js
- Add auth form styling with dark theme support
- Add component wrapper to rapidoc-mini.html template
- Fix SCSS dark theme (use CSS selectors, not non-existent mixin)
EXPERIMENTAL FINDINGS (Task 4):
Tested RapiDoc's allow-authentication attribute with both render-style
options to determine if it can show credential input on operation pages.
Results:
1. render-style='read' + allow-authentication='true':
- Auth section (#auth) exists in shadow DOM with input fields
- BUT filtered out by match-paths (not visible to users)
- Only matched operation shown, not full spec
2. render-style='focused' + allow-authentication='true':
- Auth section completely removed from shadow DOM
- Shows broken links to non-existent #auth section
- Lists security schemes but no input fields
CONCLUSION:
RapiDoc's built-in auth UI is incompatible with match-paths filtering.
When filtering to single operations, the auth section is either hidden
or removed entirely.
RECOMMENDATION:
Implement custom auth input component (Task 5) using:
- Custom TypeScript component above RapiDoc
- sessionStorage for credential persistence
- Integration with RapiDoc's Try it feature
Code includes detailed inline documentation of findings for future
reference when implementing Task 5.
- Restore original RapiDoc match-paths format (method /path) for proper filtering
- Restrict operation tags to primary tag in tag-specific specs to prevent duplicates
- Rename Token tag to Auth token for clarity in Core and Enterprise specs
- Remove Table tag from cache operations (distinct_cache, last_cache)
- Add build script combining API docs, Hugo, and Markdown generation
- Skip summary rendering for conceptual pages
- Add isConceptual check to hide operations in nav for conceptual pages
The feature is shippable, but needs a few small fixes and we'll need to update or alias all API docs links for Core and Ent3.
- Add rapidoc-mini.ts TypeScript component with CDN loading and theme sync
- Add api-operation layout for standalone operation pages
- Add rapidoc-mini.html partial for reusable RapiDoc rendering
- Add rapidoc-custom.css for RapiDoc style overrides
- Register rapidoc-mini component in main.js
- Add article data for cloud-dedicated and clustered products
- Update API reference Cypress tests
Now that API navigation is integrated into Hugo's menu system:
- Remove api-nav.ts component (no longer needed)
- Remove api/sidebar-nav.html partial (replaced by api-menu-items.html)
- Remove api-nav component registration from main.js
- Update sidebar.html to pass siteData to nested-menu for API nav
- Fix anchor ID handling for special characters (like / in operation IDs)
- Add api-nav.ts for sidebar navigation collapse/expand
- Add api-scalar.ts for Scalar API renderer integration
- Add api-tabs.ts for tab switching functionality
- Add api-toc.ts for table of contents generation
- Register components in main.js
* feat(llms): LLM-friendly Markdown, ChatGPT and Claude links.
This enables LLM-friendly documentation for entire sections,
allowing users to copy complete documentation sections with a single click.
Lambda@Edge now generates .md files on-demand with:
- Evaluated Hugo shortcodes
- Proper YAML frontmatter with product metadata
- Clean markdown without UI elements
- Section aggregation (parent + children in single file)
The llms.txt files are now generated automatically during build from
content structure and product metadata in data/products.yml, eliminating
the need for hardcoded files and ensuring maintainability.
**Testing**:
- Automated markdown generation in test setup via cy.exec()
- Implement dynamic content validation that extracts HTML content and
verifies it appears in markdown version
**Documentation**:
Documents LLM-friendly markdown generation
**Details**:
Add gzip decompression for S3 HTML files in Lambda markdown generator
HTML files stored in S3 are gzip-compressed but the Lambda was attempting
to parse compressed data as UTF-8, causing JSDOM to fail to find article
elements. This resulted in 404 errors for .md and .section.md requests.
- Add zlib gunzip decompression in s3-utils.js fetchHtmlFromS3()
- Detect gzip via ContentEncoding header or magic bytes (0x1f 0x8b)
- Add configurable DEBUG constant for verbose logging
- Add debug logging for buffer sizes and decompression in both files
The decompression adds ~1-5ms per request but is necessary to parse
HTML correctly. CloudFront caching minimizes Lambda invocations.
Await async markdown conversion functions
The convertToMarkdown and convertSectionToMarkdown functions are async
but weren't being awaited, causing the Lambda to return a Promise object
instead of a string. This resulted in CloudFront validation errors:
"The body is not a string, is not an object, or exceeds the maximum size"
**Troubleshooting**:
- Set DEBUG for troubleshooting in lambda
* feat(llms): Add build-time LLM-friendly Markdown generation
Implements static Markdown generation during Hugo build.
**Key Features:**
- Two-phase generation: HTML→MD (memory-bounded), MD→sections (fast)
- Automatic redirect detection via file size check (skips Hugo aliases)
- Product detection using compiled TypeScript product-mappings module
- Token estimation for LLM context planning (4 chars/token heuristic)
- YAML serialization with description sanitization
**Performance:**
- ~105 seconds for 5,000 pages + 500 sections
- ~300MB peak memory (safe for 2GB CircleCI environment)
- 23 files/sec conversion rate with controlled concurrency
**Configuration Parameters:**
- MIN_HTML_SIZE_BYTES (default: 1024) - Skip files below threshold
- CHARS_PER_TOKEN (default: 4) - Token estimation ratio
- Concurrency: 10 workers (CI), 20 workers (local)
**Output:**
- Single pages: public/*/index.md (with frontmatter + content)
- Section bundles: public/*/index.section.md (aggregated child pages)
**Files Changed:**
- scripts/build-llm-markdown.js (new) - Main build script
- scripts/lib/markdown-converter.cjs (renamed from .js) - Core conversion
- scripts/html-to-markdown.js - Updated import path
- package.json - Updated exports for .cjs module
Related: Replaces Lambda@Edge on-demand generation (5s response time)
with build-time static generation for production deployment.
feat(deploy): Add staging deployment workflow and update CI
Integrates LLM markdown generation into deployment workflows with
a complete staging deployment solution.
**CircleCI Updates:**
- Switch from legacy html-to-markdown.js to optimized build:md
- 2x performance improvement (105s vs 200s+ for 5000 pages)
- Better memory management (300MB vs variable)
- Enables section bundle generation (index.section.md files)
**Staging Deployment:**
- New scripts/deploy-staging.sh for local staging deploys
- Complete workflow: Hugo build → markdown gen → S3 upload
- Environment variable driven configuration
- Optional step skipping for faster iteration
- CloudFront cache invalidation support
**NPM Scripts:**
- Added deploy:staging command for convenience
- Wraps deploy-staging.sh script
**Documentation:**
- Updated DOCS-DEPLOYING.md with comprehensive guide
- Merged staging/production workflows with Lambda@Edge docs
- Build-time generation now primary, Lambda@Edge fallback
- Troubleshooting section with common issues
- Environment variable reference
- Performance metrics and optimization tips
**Benefits:**
- Manual staging validation before production
- Consistent markdown generation across environments
- Faster CI builds with optimized script
- Better error handling and progress reporting
- Section aggregation for improved LLM context
**Usage:**
```bash
export STAGING_BUCKET="test2.docs.influxdata.com"
export AWS_REGION="us-east-1"
export STAGING_CF_DISTRIBUTION_ID="E1XXXXXXXXXX"
yarn deploy:staging
```
Related: Completes build-time markdown generation implementation
refactor: Remove Lambda@Edge implementation
Build-time markdown generation has replaced Lambda@Edge on-demand
generation as the primary method. Removed Lambda code and updated
documentation to focus on build-time generation and testing.
Removed:
- deploy/llm-markdown/ directory (Lambda@Edge code)
- Lambda@Edge section from DOCS-DEPLOYING.md
Added:
- Testing and Validation section in DOCS-DEPLOYING.md
- Focus on build-time generation workflow
* feat: Add Rust HTML-to-Markdown prototype
Implements core markdown-converter.cjs functions in Rust for performance comparison.
Performance results:
- Rust: ~257 files/sec (10× faster)
- JavaScript: ~25 files/sec average
Recommendation: Keep JavaScript for now, implement incremental builds first.
Rust migration provides 10× speedup but requires 3-4 weeks integration effort.
Files:
- Cargo.toml: Rust dependencies (html2md, scraper, serde_yaml, clap)
- src/main.rs: Core conversion logic + CLI benchmark tool
- benchmark-comparison.js: Side-by-side performance testing
- README.md: Comprehensive findings and recommendations
* fix(ui): improve dropdown positioning on viewport resize
- Ensure dropdown stays within viewport bounds (min 8px padding)
- Reposition dropdown on window resize and scroll events
- Clean up event listeners when dropdown closes
* chore(deps): add remark and unified packages for markdown processing
Add remark-parse, remark-frontmatter, remark-gfm, and unified for
enhanced markdown processing capabilities.
* fix(edge): add return to prevent trailing-slash redirect for valid extensions
Without the return statement, the Lambda@Edge function would continue
executing after the callback, eventually hitting the trailing-slash
redirect logic. This caused .md files to redirect to URLs with trailing
slashes, which returned 404 from S3.
* fix(md): add built-in product mappings and full URL support
- Add URL_PATTERN_MAP and PRODUCT_NAME_MAP constants directly in the
CommonJS module (ESM product-mappings.js cannot be require()'d)
- Update generateFrontmatter() to accept baseUrl parameter and construct
full URLs for the frontmatter url field
- Update generateSectionFrontmatter() similarly for section pages
- Update all call sites to pass baseUrl parameter
This fixes empty product fields and relative URLs in generated markdown
frontmatter when served via Lambda@Edge.
* feat(md): add environment flag for base URL control
Add -e, --env flag to html-to-markdown.js to control the base URL
in generated markdown frontmatter. This matches Hugo's -e flag behavior
and allows generating markdown with staging or production URLs.
Also update build-llm-markdown.js with similar environment support.
* feat(md): add Rust markdown converter and improve validation
- Add Rust-based HTML-to-Markdown converter with NAPI-RS bindings
- Update Cypress markdown validation tests
- Update deploy-staging.sh with force upload flag
* deploy-staging.sh:
- Defaults STAGING_URL to https://test2.docs.influxdata.com
if not set
- Exports it so yarn build:md -e staging can use it
- Displays it in the summary
* Delete scripts/prototypes/rust-markdown/benchmark-comparison.js
* Delete scripts/prototypes directory
* fix(llms): Include full URL for section page Markdown and list of child pages
* feat(llms): clarify format selector text for AI use case
Update button and dropdown text to make the AI/LLM purpose clearer:
- Button: "Copy page for AI" / "Copy section for AI"
- Sublabel: "Clean Markdown optimized for AI assistants"
- Section sublabel: "{N} pages combined as clean Markdown for AI assistants"
Cypress tests updated and passing (13/13).
---------
Co-authored-by: Scott Anderson <scott@influxdata.com>
1. Refactor JavaScript from search.html into a new DocSearch Component Module. The component needs to include: - Asynchronous loading of Algolia docsearch.min.js - Proper Algolia search facets:
- Nested arrays for AND conditions: In Algolia, filters in the same array are combined with OR, while arrays in an array are combined with AND
- Space after colon: Algolia's parser expects this specific format
2. Update Search HTML Template: refactor the search.html to use the new component:
3. Register the Component in main.js: update main.js to import and register our new DocSearch component:
Improve SearchInteractions event handler coordination with DocSearch:
- Improve SearchInteractions to dynamically detect the dropdown added by Algolia DocSearch and ensure that blur and focus event handlers are registered after the dropdown loads.
Benefits of This Approach:
Asynchronous Loading: DocSearch.js is now loaded asynchronously, improving page load performance
Search functionality is encapsulated as a component
Clean Separation: HTML template only contains configuration data, not implementation
Event Communication: Components can react to DocSearch initialization via events
SearchInteractions dynamically detects dropdown: Instead of waiting for an event or using setTimeout, we actively observe DOM changes to detect when the dropdown is created.
Nested Observation: A second observer monitors changes to the dropdown itself, allowing code to respond to style changes.
Better Event Handling: The blur handler checks if the related target is inside the search components, preventing unwanted dropdown closing.
Resource Cleanup: The cleanup function disconnects all observers, preventing memory leaks.
Additional Interaction Control: Adds an event listener to the dropdown to prevent unwanted blur events when interacting with search results.
- Fixes dynamic loading of Mermaid.js
- Adds MutationObserver to handle light/dark theme switching.
- Sets the background-color for edgeLabel p elements to override a white background that obscures text in the dark theme.
Jason Stirnaman
Scott Anderson
Jason Stirnaman