External plugins (addresses #1863) (#3059)

* Create external plugins section

* add external info

* Update WIP

* Update content/telegraf/v1.19/concepts/glossary.md

Co-authored-by: kelseiv <47797004+kelseiv@users.noreply.github.com>

* Update content/telegraf/v1.19/external_plugins/_index.md

Co-authored-by: kelseiv <47797004+kelseiv@users.noreply.github.com>

* Update content/telegraf/v1.19/external_plugins/_index.md

Co-authored-by: kelseiv <47797004+kelseiv@users.noreply.github.com>

* Address PR feedback

* Update content/telegraf/v1.19/external_plugins/shim.md

Co-authored-by: kelseiv <47797004+kelseiv@users.noreply.github.com>

* Update content/telegraf/v1.19/external_plugins/shim.md

Co-authored-by: kelseiv <47797004+kelseiv@users.noreply.github.com>

* Update shim.md

* Additional edits

Co-authored-by: kelseiv <47797004+kelseiv@users.noreply.github.com>

content/telegraf/v1.19/concepts/_index.md

@@ -6,6 +6,6 @@ menu:
      name: Concepts
      weight: 30
 ---
-This section discusses key concepts about Telegraf, the plug-in driven server agent component of the InfluxData time series platform. Topics covered include metrics, aggregator and processor plugins, and a glossary of important terms.
+This section discusses key concepts about Telegraf, the plugin driven server agent component of the InfluxData time series platform. Topics covered include metrics, aggregator and processor plugins, and a glossary of important terms.
 
 {{< children >}}

content/telegraf/v1.19/concepts/glossary.md

@@ -42,6 +42,9 @@ Each collection interval, every input plugin will sleep for a random time betwee
 
 Related entries: [collection interval](/telegraf/v1.15/concepts/glossary/#collection-interval), [input plugin](/telegraf/v1.15/concepts/glossary/#input-plugin)
 
+## external plugin
+
+Programs built outside of Telegraf that run through the `execd` plugin. Provides flexibility to add functionality that doesn't exist in internal Telegraf plugins.
 ## flush interval
 
 The global interval for flushing data from each output plugin to its destination.

content/telegraf/v1.19/external_plugins/_index.md

@@ -0,0 +1,21 @@
+---
+title: External plugins
+description:
+menu:
+  telegraf_1_19:
+     name: External plugins
+     weight: 50
+---
+
+[External plugins](/EXTERNAL_PLUGINS.md) are external programs that are built outside
+of Telegraf that can run through an `execd` plugin. These external plugins allow for
+more flexibility compared to internal Telegraf plugins. Benefits to using external plugins include:
+- Access to libraries not written in Go
+- Using licensed software (not available to open source community)
+- Including large dependencies that would otherwise bloat Telegraf
+- Using your external plugin immediately without waiting for the Telegraf team to publish
+- Easily convert plugins between internal and external using the [shim](/plugins/common/shim)
+
+
+
+{{< children hlevel="h2" >}}

content/telegraf/v1.19/external_plugins/shim.md

@@ -0,0 +1,60 @@
+---
+title: Use the `execd` shim
+description:
+menu:
+  telegraf_1_19:
+     name: Use the `execd` shim
+     weight: 50
+     parent: External plugins
+---
+
+The shim makes it easy to extract an internal input,
+processor, or output plugin from the main Telegraf repo out to a stand-alone
+repo. This allows anyone to build and run it as a separate app using one of the
+`execd` plugins:
+- [inputs.execd](https://github.com/influxdata/telegraf/blob/master/plugins/inputs/execd)
+- [processors.execd](https://github.com/influxdata/telegraf/blob/master//plugins/processors/execd)
+- [outputs.execd](https://github.com/influxdata/telegraf/blob/master//plugins/outputs/execd)
+
+## Extract a plugin using the shim wrapper
+
+1. Move the project to an external repo. We recommend preserving the path
+  structure: for example, if your plugin was located at
+  `plugins/inputs/cpu` in the Telegraf repo, move it to `plugins/inputs/cpu`
+  in the new repo.
+2. Copy [main.go](https://github.com/influxdata/telegraf/blob/master/plugins/common/shim/example/cmd/main.go) into your project under the `cmd` folder.
+  This serves as the entry point to the plugin when run as a stand-alone program.
+  {{% note %}}
+  The shim isn't designed to run multiple plugins at the same time, so include only one plugin per repo.
+  {{% /note %}}
+3. Edit the `main.go` file to import your plugin. For example,`_ "github.com/me/my-plugin-telegraf/plugins/inputs/cpu"`. See an example of where to edit `main.go` [here](https://github.com/influxdata/telegraf/blob/7de9c5ff279e10edf7fe3fdd596f3b33902c912b/plugins/common/shim/example/cmd/main.go#L9).
+4. Add a [plugin.conf](https://github.com/influxdata/telegraf/blob/master/plugins/common/shim/example/cmd/plugin.conf) for configuration
+  specific to your plugin.
+  {{% note %}}
+  This config file must be separate from the rest of the config for Telegraf, and must not be in a shared directory with other Telegraf configs.
+  {{% /note %}}
+
+## Test and run your plugin
+
+1. Build the `cmd/main.go` using the following command with your plugin name: `go build -o plugin-name cmd/main.go`
+1. Test the binary:
+2. If you're building a processor or output, first feed valid metrics in on `STDIN`. Skip this step if you're building an input.
+3. Test out the binary by running it (for example, `./project-name -config plugin.conf`).
+  Metrics will be written to `STDOUT`. You might need to hit enter or wait for your poll duration to elapse to see data.
+4. Press `Ctrl-C` to end your test.
+5. Configure Telegraf to call your new plugin binary. For an input, this would
+  look something like:
+
+```toml
+[[inputs.execd]]
+  command = ["/path/to/rand", "-config", "/path/to/plugin.conf"]
+  signal = "none"
+```
+
+Refer to the `execd` plugin documentation for more information.
+
+## Publish your plugin
+
+Publishing your plugin to GitHub and open a Pull Request
+back to the Telegraf repo letting us know about the availability of your
+[external plugin](https://github.com/influxdata/telegraf/blob/master/EXTERNAL_PLUGINS.md).

content/telegraf/v1.19/external_plugins/write_external_plugin.md

@@ -0,0 +1,30 @@
+---
+title: Write an external plugins
+description:
+menu:
+  telegraf_1_19:
+     name: Write an external plugin
+     weight: 50
+     parent: External plugins
+---
+Set up your plugin to use it with `execd`.
+
+{{% note %}}
+For listed [external plugins](/EXTERNAL_PLUGINS.md), the author of the external plugin is also responsible for the maintenance
+and feature development of external plugins.
+{{% /note %}}
+
+1. Write your Telegraf plugin. Follow InfluxData's best practices:
+   - [Input plugins](https://github.com/influxdata/telegraf/blob/master/docs/INPUTS.md)
+   - [Processor plugins](https://github.com/influxdata/telegraf/blob/master/docs/PROCESSORS.md)
+   - [Aggregator plugins](https://github.com/influxdata/telegraf/blob/master/docs/AGGREGATORS.md)
+   - [Output plugins](https://github.com/influxdata/telegraf/blob/master/docs/OUTPUTS.md)
+2. If your plugin is written in Go, follow the steps for the [Execd Go Shim](/telegraf/latest/external_plugins/shim).
+3. Add usage and development instructions in the homepage of your repository for running your plugin with its respective `execd` plugin. Refer to [openvpn](https://github.com/danielnelson/telegraf-execd-openvpn#usage) and [awsalarms](https://github.com/vipinvkmenon/awsalarms#installation) for examples.
+Include the following steps:
+     - How to download the release package for your platform or how to clone the binary for your external plugin
+     - Commands to build your binary
+     - Location to edit your `telegraf.conf`
+     - Configuration to run your external plugin with [inputs.execd](https://github.com/influxdata/telegraf/blob/master/plugins/inputs/execd),
+     [processors.execd](/plugins/processors/execd) or [outputs.execd](https://github.com/influxdata/telegraf/blob/master/plugins/outputs/execd)
+4. Submit your plugin by opening a PR to add your external plugin to the [/EXTERNAL_PLUGINS.md](https://github.com/influxdata/telegraf/blob/master/EXTERNAL_PLUGINS.md) list. Include the plugin name, a link to the plugin repository and a short description of the plugin.

content/telegraf/v1.19/plugins.md

@@ -22,7 +22,7 @@ aliases:
 ---
 
 Telegraf is a plugin-driven agent that collects, processes, aggregates, and writes metrics.
-It supports four categories of plugins including input, output, aggregator, and processor.
+It supports four categories of plugins including input, output, aggregator, processor, and external.
 
 {{< telegraf/filters >}}
 
@@ -32,6 +32,7 @@ It supports four categories of plugins including input, output, aggregator, and
 - [Output plugins](#output-plugins)
 - [Aggregator plugins](#aggregator-plugins)
 - [Processor plugins](#processor-plugins)
+- [External plugins](#external-plugins)
 
 ## Input plugins
 Telegraf input plugins are used with the InfluxData time series platform to collect
@@ -53,3 +54,8 @@ Telegraf aggregator plugins create aggregate metrics (for example, mean, min, ma
 Telegraf output plugins transform, decorate, and filter metrics.
 
 {{< telegraf/plugins type="processor" >}}
+
+## External plugins
+External plugins are external programs that are built outside of Telegraf that can run through an `execd` plugin.
+
+{{< telegraf/plugins type="external" >}}

data/telegraf_plugins.yml

@@ -69,6 +69,7 @@ input:
       metrics using one of the supported [input data formats](/{{< latest "telegraf" >}}/data_formats/input).
     introduced: 1.10.0
     tags: [linux, macos, windows, cloud, messaging]
+    external: true
 
   - name: Apache Aurora
     id: aurora
@@ -2513,6 +2514,7 @@ processor:
       The GeoIP processor plugin looks up IP addresses in the [MaxMind GeoLite2 database](https://dev.maxmind.com/geoip/geoip2/geolite2/) and adds the respective ISO country code, city name, latitude and longitude.
     introduced: 1.18.0
     tags: [linux, macos, windows]
+    external: true
 
   - name: Network Interface Name
     id: ifname