Commit Graph

5 Commits (v3/inspect-partitions)

Author SHA1 Message Date
Jason Stirnaman 9e276df194
Optimize writes: Use Telegraf to pre-process data before writing (#5282)
* chore(v3): preprocess writes (Telegraf, etc)

* feat: optimize writes using Telegraf

* WIP: dedup plugin

* WIP: Dedup

* chore(v3): Writes: Preprocess data using Telegraf: Dedup example

(#5155)
- Add example using processors.dedup in Telegraf.
- The plugin or configuration doesn't currently work as expected. Investigating...

* chore(v3): Write data: pre-process data using Telegraf

- Add Telegraf examples that pre-process data to avoid schema conflicts and reduce payload (bandwith, memory, etc.) when writing data.
- For Serverless. Will port to Dedicated and Clustered after approval.
- Doesn't include Dedup yet - I'm waiting on help to make it work (see branch https://github.com/influxdata/docs-v2/compare/issue5155-preprocess-with-dedup)

* Update content/influxdb/cloud-serverless/write-data/best-practices/optimize-writes.md

Co-authored-by: Scott Anderson <sanderson@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: Scott Anderson <sanderson@users.noreply.github.com>

* chore(test): Test harness setup, use golang base image

- Use golang base image for ease of Go environment setup. Installing Python separately is easier than Go.
- Add flag to bypass python warning about using system python and pip.
- Remove redundant env.products from pytest.ini; we already load it into environment variables.

* chore(v3): Optimize writes: Use Telegraf

Closes Telegraf guide: how to use the Converter processor to coerce data before writing #5155

- Adds use cases to Dedicated and Clustered docs
- Adds example for Execd processor plugin to all
- Clean up and pass tests
- Remove shared code sample

* fix(v3): Optimize writes: Placeholders and indent fixes

* Update content/influxdb/cloud-dedicated/write-data/best-practices/optimize-writes.md

Co-authored-by: Scott Anderson <sanderson@users.noreply.github.com>

* Update content/influxdb/cloud-serverless/write-data/best-practices/optimize-writes.md

Co-authored-by: Scott Anderson <sanderson@users.noreply.github.com>

* Update content/influxdb/clustered/write-data/best-practices/optimize-writes.md

Co-authored-by: Scott Anderson <sanderson@users.noreply.github.com>

---------

Co-authored-by: Scott Anderson <sanderson@users.noreply.github.com>
2023-12-22 11:23:49 -06:00
Jason Stirnaman 237b01877b
chore(qol): Test python and shell code samples. WIP - use for local testing only, not ready for CI. (#5154)
- Adds a Docker image, scripts, and configuration that install and run pytest-codeblocks on Python and shell code samples in Markdown.
- Copied from https://github.com/jstirnaman/docs-codeblocks/
- TODO: The script copies files into a tmp directory configured as a bind mount for the Docker container. We might want to replace this with the experimental `watch` feature.
- TODO: Use a secret to configure the token. As is, values subbed from the dotenv file are exposed in your Docker container history.
2023-09-26 14:56:21 -05:00
Scott Anderson 471ee4561d
InfluxDB Clustered documentation (#5126)
* WIP base changes for clustered docs

* WIP clustered docs

* Add new influxdb/host shortcode and implement it in 3.0 docs (#5077)

* add new influxdb/host shortcode and implement it in 3.0 docs

* remove oss- cloud-only shortcodes from serverless

* Apply suggestions from code review

Co-authored-by: Jason Stirnaman <stirnamanj@gmail.com>

* updated urls js to PR suggestion

* Updated JavaScript, templates, and styles for Clustered URLs (#5079)

* updated js, templates, and styles for clustered urls

* Apply suggestions from code review

Co-authored-by: Jason Stirnaman <stirnamanj@gmail.com>

---------

Co-authored-by: Jason Stirnaman <stirnamanj@gmail.com>

---------

Co-authored-by: Jason Stirnaman <stirnamanj@gmail.com>

* restructure product dropdown template to be more extensible

* fixed more page template bugs

* fixed references to cloud in clustered

* updated docsearch templates

* added early access flagging and cta-link shortcode

* minor content updates in clustered

* updated staging config

* fixed typo in clustered description

* ported influxctl 2.0.1 to clustered

* ported get started changes to clustered

* ported 3.0 admin docs to clustered

* port null tag content to clustered

* ported influxctl note to clustered

* ported query reorg changes to clustered

* updated early access to limited availability, updated clustered landing content

* ported new content to clustered

* ported new content to clustered

* updated cta on clustered landing page

* Updated notifications and added InfluxDB Clustered announcement notification (#5125)

* updated notifications, added clustered announcement notification

* updated cta in clustered notification

* updated influxctl profile configs

* update clustered search attributes

* updated learn more link in clustered notification

* Apply suggestions from code review

* fixed typos

* fixed typos

---------

Co-authored-by: Jason Stirnaman <stirnamanj@gmail.com>
2023-09-06 06:21:47 -06:00
Scott Anderson 4df36bacc9 hotfix: minor updates to dedicated and serveriless optimize writes 2023-06-21 10:06:12 -06:00
Scott Anderson 7246eb647b
Port "Optimize writes" guide to Serverless and Dedicated (#4973)
* port optimize writes guide to serverless and dedicated

* updated dedicated lp guide
2023-06-21 09:55:11 -06:00