docs-v2/shared/text
Jason Stirnaman 9e276df194
Optimize writes: Use Telegraf to pre-process data before writing (#5282)
* chore(v3): preprocess writes (Telegraf, etc)

* feat: optimize writes using Telegraf

* WIP: dedup plugin

* WIP: Dedup

* chore(v3): Writes: Preprocess data using Telegraf: Dedup example

(#5155)
- Add example using processors.dedup in Telegraf.
- The plugin or configuration doesn't currently work as expected. Investigating...

* chore(v3): Write data: pre-process data using Telegraf

- Add Telegraf examples that pre-process data to avoid schema conflicts and reduce payload (bandwith, memory, etc.) when writing data.
- For Serverless. Will port to Dedicated and Clustered after approval.
- Doesn't include Dedup yet - I'm waiting on help to make it work (see branch https://github.com/influxdata/docs-v2/compare/issue5155-preprocess-with-dedup)

* Update content/influxdb/cloud-serverless/write-data/best-practices/optimize-writes.md

Co-authored-by: Scott Anderson <sanderson@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: Scott Anderson <sanderson@users.noreply.github.com>

* chore(test): Test harness setup, use golang base image

- Use golang base image for ease of Go environment setup. Installing Python separately is easier than Go.
- Add flag to bypass python warning about using system python and pip.
- Remove redundant env.products from pytest.ini; we already load it into environment variables.

* chore(v3): Optimize writes: Use Telegraf

Closes Telegraf guide: how to use the Converter processor to coerce data before writing #5155

- Adds use cases to Dedicated and Clustered docs
- Adds example for Execd processor plugin to all
- Clean up and pass tests
- Remove shared code sample

* fix(v3): Optimize writes: Placeholders and indent fixes

* Update content/influxdb/cloud-dedicated/write-data/best-practices/optimize-writes.md

Co-authored-by: Scott Anderson <sanderson@users.noreply.github.com>

* Update content/influxdb/cloud-serverless/write-data/best-practices/optimize-writes.md

Co-authored-by: Scott Anderson <sanderson@users.noreply.github.com>

* Update content/influxdb/clustered/write-data/best-practices/optimize-writes.md

Co-authored-by: Scott Anderson <sanderson@users.noreply.github.com>

---------

Co-authored-by: Scott Anderson <sanderson@users.noreply.github.com>
2023-12-22 11:23:49 -06:00
..
api Optimize writes: Use Telegraf to pre-process data before writing (#5282) 2023-12-22 11:23:49 -06:00
bucket-schema hotfix: fix explicit schema file examples, closes #4563 2022-10-14 14:10:39 -06:00
flux Flux v0.123 (#2880) 2021-07-21 13:53:25 -06:00