8.5 KiB

Raw Permalink Blame History

Deploying InfluxData Documentation

This guide covers deploying the docs-v2 site to staging and production environments, as well as LLM markdown generation.

Staging Deployment
Production Deployment
LLM Markdown Generation
Testing and Validation
Troubleshooting

Staging Deployment

Staging deployments are manual and run locally with your AWS credentials.

Prerequisites

AWS Credentials - Configure AWS CLI with appropriate permissions:
```
aws configure
```
s3deploy - Install the s3deploy binary:
```
./deploy/ci-install-s3deploy.sh
```

Environment Variables - Set required variables:

export STAGING_BUCKET="test2.docs.influxdata.com"
export AWS_REGION="us-east-1"
export STAGING_CF_DISTRIBUTION_ID="E1XXXXXXXXXX"  # Optional

Deploy to Staging

Use the staging deployment script:

yarn deploy:staging

Or run the script directly:

./scripts/deploy-staging.sh

What the Script Does

Builds Hugo site with staging configuration (config/staging/hugo.yml)
Generates LLM-friendly Markdown (yarn build:md)
Uploads to S3 using s3deploy
Invalidates CloudFront cache (if STAGING_CF_DISTRIBUTION_ID is set)

Optional Environment Variables

Skip specific steps for faster iteration:

# Skip Hugo build (use existing public/)
export SKIP_BUILD=true

# Skip markdown generation
export SKIP_MARKDOWN=true

# Build only (no S3 upload)
export SKIP_DEPLOY=true

Example: Test Markdown Generation Only

SKIP_DEPLOY=true ./scripts/deploy-staging.sh

Production Deployment

Production deployments are automatic via CircleCI when merging to master.

Workflow

Build Job (.circleci/config.yml):
- Installs dependencies
- Builds Hugo site with production config
- Generates LLM-friendly Markdown (yarn build:md)
- Persists workspace for deploy job
Deploy Job:
- Attaches workspace
- Uploads to S3 using s3deploy
- Invalidates CloudFront cache
- Posts success notification to Slack

Environment Variables (CircleCI)

Production deployment requires the following environment variables set in CircleCI:

BUCKET - Production S3 bucket name
REGION - AWS region
CF_DISTRIBUTION_ID - CloudFront distribution ID
SLACK_WEBHOOK_URL - Slack notification webhook

Trigger Production Deploy

git push origin master

CircleCI will automatically build and deploy.

LLM Markdown Generation

Both staging and production deployments generate LLM-friendly Markdown files at build time.

Output Files

The build generates two types of markdown files in public/:

Single-page markdown (index.md)
- Individual page content with frontmatter
- Contains: title, description, URL, product, version, token estimate
Section bundles (index.section.md)
- Aggregated section with all child pages
- Includes child page list in frontmatter
- Optimized for LLM context windows

Generation Script

# Generate all markdown
yarn build:md

# Generate for specific path
node scripts/build-llm-markdown.js --path influxdb3/core/get-started

# Limit number of files (for testing)
node scripts/build-llm-markdown.js --limit 100

Configuration

Edit scripts/build-llm-markdown.js to adjust:

// Skip files smaller than this (Hugo alias redirects)
const MIN_HTML_SIZE_BYTES = 1024;

// Token estimation ratio
const CHARS_PER_TOKEN = 4;

// Concurrency (workers)
const CONCURRENCY = process.env.CI ? 10 : 20;

Performance

Speed: ~105 seconds for 5,000 pages + 500 sections
Memory: ~300MB peak (safe for 2GB CircleCI)
Rate: ~23 files/second with memory-bounded parallelism

Making Deployment Changes

During Initial Implementation

If making changes that affect yarn build commands or .circleci/config.yml:

Read the surrounding context in the CI file
Notice flags, such as --destination workspace/public on the Hugo build
Ask: "Does the build script need to know about environment details--for example, do paths differ between production and staging?"

Recommended Prompt for Future Similar Work

"This script will run in CI. Let me read the CI configuration to understand the build environment and directory structure before finalizing the implementation."

Summary of Recommendations

Strategy	Implementation	Effort
Read CI config before implementing	Process/habit change	Low
Test on feature branch first	Push and watch CI before merging	Low
Add CI validation step	Add file count check in config.yml	Low

Testing and Validation

Local Testing

Test markdown generation locally before deploying:

# Prerequisites
yarn install
yarn build:ts
npx hugo --quiet

# Generate markdown for testing
yarn build:md

# Generate markdown for specific path
node scripts/build-llm-markdown.js --path influxdb3/core/get-started --limit 10

# Run validation tests
node cypress/support/run-e2e-specs.js \
  --spec "cypress/e2e/content/markdown-content-validation.cy.js"

Validation Checks

The Cypress tests validate:

✅ No raw Hugo shortcodes ({{< >}} or {{% %}})
✅ No HTML comments
✅ Proper YAML frontmatter with required fields
✅ UI elements removed (feedback forms, navigation)
✅ GitHub-style callouts (Note, Warning, etc.)
✅ Properly formatted tables, lists, and code blocks
✅ Product context metadata
✅ Clean link formatting

See DOCS-TESTING.md for comprehensive testing documentation.

Troubleshooting

s3deploy Not Found

Install the s3deploy binary:

./deploy/ci-install-s3deploy.sh

Verify installation:

s3deploy -version

Missing Environment Variables

Check required variables are set:

echo $STAGING_BUCKET
echo $AWS_REGION

Set them if missing:

export STAGING_BUCKET="test2.docs.influxdata.com"
export AWS_REGION="us-east-1"

AWS Permission Errors

Ensure your AWS credentials have the required permissions:

s3:PutObject - Upload files to S3
s3:DeleteObject - Delete old files from S3
cloudfront:CreateInvalidation - Invalidate cache

Check your AWS profile:

aws sts get-caller-identity

Hugo Build Fails

Check for:

Missing dependencies (yarn install)
TypeScript compilation errors (yarn build:ts)
Invalid Hugo configuration

Build Hugo separately to isolate the issue:

yarn hugo --environment staging

Markdown Generation Fails