Influxdata
/
docs-v2
mirror of https://github.com/influxdata/docs-v2.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity

Merge branch 'master' of github.com:influxdata/docs-v2

pull/4383/head
Scott Anderson 2022-08-23 14:46:22 -06:00
parent 1b09626de0 7b00acc24e
commit 688181cea9
7 changed files with 216 additions and 287 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

54
content/influxdb/cloud/reference/json-parsing.md Normal file
View File

@ -0,0 +1,54 @@
---
title: JSON parsing examples
description: >
Review example JSON parsing rules for native subscriptions.
menu:
influxdb_cloud_ref:
name: JSON parsing examples
weight: 8
influxdb/v2.0/tags: [mqtt]
related:
---
Use the following examples to help you set up parsing rules for [native subscriptions](/influxdb/cloud/write-data/no-code/native-subscriptions).
## Example simple MQTT message in JSON format
```js
{
"device_type":"temperature_sensor",
"device_id":2036,
"model_id":"KN24683",
"temperature":25.0,
"time":1653998899010000000,
"error_state":"in_error"
}
```
JSON paths start with a “$.” In the above example, all of the values are at the root level of the JSON, so the JSON paths for these elements are very simple:
- Measurement: $.temperature_sensor
- Timestamp: $.time
- Tag: $.device_id
- Field 1: $.temperature
- Field 2: $.error_state
## Example nested MQTT message in JSON format
```js
{
"device_information": {
"device_type":"temperature_sensor",
"device_id":2036,
"model_id":"KN24683"
},
"temperature":25.0,
"time":165411795400000000,
"error_state":"in_error"
}
```
In this example, the JSON path to the measurement would be `$.device_information.device_type`
The JSON path to the tag would be `$device_information.device_id`.

146
content/influxdb/cloud/write-data/no-code/native-subscriptions.md Normal file
View File

@ -0,0 +1,146 @@
---
title: Set up native MQTT subscriptions
weight: 102
aliases:
- /influxdb/cloud/write-data/no-code/load-data/
description: >
Use native subscriptions to ingest data.
menu:
influxdb_cloud:
name: Native MQTT subscriptions
parent: Write data
influxdb/v2.0/tags: [mqtt, native subscription]
---
{{% note %}}
This feature is available with usage-based plans only. For more information, see [InfluxDB Cloud Plans](/influxdb/cloud/account-management/pricing-plans/).
{{% /note %}}
To ingest MQTT (Message Queuing Telemetry Transport) data into InfluxDB, do the following to set up a native subscription:
1. [Subscribe to an MQTT topic](#subscribe-to-an-mqtt-topic) in InfluxDB Cloud by configuring an MQTT broker, and specifying the topic(s) to subscribe to.
2. [Define parsing rules](#define-parsing-rules) for JSON or regex data (line protocol requires no configuration).
{{% note %}}
For troubleshooting help, see [Troubleshoot MQTT ingest errors](/influxdb/cloud/write-data/troubleshoot/#troubleshoot-mqtt-ingest-errors/).
{{% /note %}}
#### Subscribe to an MQTT topic
1. In the navigation menu on the left, click **Load Data** > **Native Subscriptions**.
{{< nav-icon "data" >}}
2. Click **+ Create Subscription**.
3. On the **Setting Up - MQTT Subscriber** page, under **Connect to Broker**, enter the following:
- Subscription Name
- Description (optional)
- Protocol
- Hostname or IP address (hostname or URL of the MQTT broker)
- Port (TCP/IP port number the MQTT broker uses)
- Security details. Choose one of the following:
- **None**
- **Basic** (username/password)
- **Certificate**
4. Under **Subscribe to a topic**, in the **Topic** field, enter the MQTT topic name to subscribe to. Note, MQTT brokers typically support wildcard subscriptions with the wildcard characters `+` and `#`.
- To subscribe to all topics in a directory, use `+`. For example, if an `iotdevices` directory includes two directories called `asia` and `europe`, to subscribe to a `sensor` topic in either directory, use `iotdevices/+/sensors` to subscribe to `iotdevices/asia/sensors`, and `iotdevices/europe/sensors`.
- To subscribe to all topics in a directory, use `#`. For example, `iotdevices/#` subscribes to all topics in the `iotdevices` directory. For more information about MQTT subscriptions and wildcards, see [the MQTT specification for Topic Names and Topic Filters](https://docs.oasis-open.org/mqtt/mqtt/v5.0/os/mqtt-v5.0-os.html#_Toc3901241).
5. Under **Write Destination**, select an existing InfluxDB bucket to write data to or click **+ Create bucket**. For more information, see [Create a bucket](/influxdb/cloud/organizations/buckets/create-bucket/).
6. Click **Save Subscription** to save and start running the MQTT subscription.
#### Define parsing rules
{{% note %}}
JSON parsing is faster and more efficient than string parsing. We recommend using JSON parsing when your MQTT messages are in JSON format. For examples, see [JSON parsing examples](/influxdb/cloud/reference/json-parsing/).
{{% /note %}}
- Under **Define Data Parsing Rules**, select one of the following MQTT data formats:
- **Line protocol** (no configuration required)
- **JSON**. To define parsing rules to ingest JSON data, click the **JSON** tab below.
- **String**. To define parsing rules to ingest String data, click the **String** tab below.
{{< tabs-wrapper >}}
{{% tabs %}}
[Line protocol](#)
[JSON](#)
[String](#)
{{% /tabs %}}
<!-------------------------------- BEGIN Line protocol -------------------------------->
{{% tab-content %}}
Use line protocol to write data into InfluxDB. Line protocol doesn't require any parsing or configuration.
- Select a **Timepstamp precision** from the dropdown menu:
- **MS**: Milliseconds
- **S**: Seconds
- **US**: Microseconds
- **NS**: Nanoseconds
{{% /tab-content %}}
<!-------------------------------- BEGIN JSON -------------------------------->
{{% tab-content %}}
To associate **JSON** key/value pairs with **InfluxDB elements** (measurements, timestamps, fields, or tags) using parsing rules, complete the following steps:
{{% expand "Example JSON" %}}
```
{
"device_type":"temperature_sensor",
"device_id":2036,
"model_id":"KN24683",
"temperature":25.0,
"time":1653998899010000000,
"error_state":"in_error"
}
```
{{% /expand %}}
1. On the **Setting Up - MQTT Connector** page, under **Data Format**, do the following:
1. (Optional) In the **JSON path to timestamp** field, specify the path in the MQTT message to the JSON key that holds the timestamp: for the example above, `"time":1653998899010000000`. Otherwise, InfluxDB automatically assigns a timestamp when messages are ingested into InfluxDB.
***Important***: *Configure the timestamp format that **matches the format** in your messages.*
2. Configure the JSON parsing rules:
1. Under **Measurement**, enter the **JSON path** (start with `$.`) to assign the InfluxDB measurement key. For the above example, enter `$.device_type`.
2. Select the **Data Type** for the measurement.
3. Specify the JSON paths to tag and field names as needed, and then select the data type for the tag or field. At least one field is required. For the above example, add fields with the JSON paths `$.temperature` and `$.error_state` and a tag with the path `$.error_state`.
Note that JSON paths with arrays are supported, for example, `$.device_information.errors_encountered[0].error_number`.
{{% /tab-content %}}
<!-------------------------------- BEGIN String -------------------------------->
{{% tab-content %}}
To associate **String** key/value pairs with **InfluxDB elements** (measurements, timestamps, fields, or tags), complete the following steps:
1. On the **Setting Up - MQTT Connector** page, under **Data Format**, do the following:
1. (Optional) In the **Regex pattern to find timestamp** field, enter the regex (regular expression) to find the timestamp in the MQTT message. Otherwise, InfluxDB automatically assigns a timestamp when messages are ingested into InfluxDB.
***Note***: *Parsing rules only support finding **one value at a time**.*
For example, if the timestamp string is `time=1653998899010000000`, use a regex to find the string you're looking for and capture the timestamp:
- `time=([\s\S]*?)\n` (captures value after `=` until the EOL (end of line) is reached)
- `time=([\s\S]*?),` (captures value after `=` until comma is reached)
***Important***: *Configure the timestamp format that **matches the format** in your messages.*
2. Under **Measurement**, if the string is `device_type=temperature_sensor`, use regex to find the measurement name. For example:
- `device_type=([\s\S]*?)\n` captures the value after the `=` until the EOL (end of line) is reached), in this case the value would be `temperature_sensor`.
3. Select the **Data Type** for the measurement.
4. Enter **Tag** and **Field**. At least one field is required. For tag and field names, use the regex to find the tag or field name, and what to capture. For example:
- `device_id=\d\d\d\d-([0-9][0-9][0-9][0-9])` (matches on the `device_id=` and also matches on the first four digits of the device id, and then captures the four digits.
5. Select the **Data Type** for the tag or field.
{{% /tab-content %}}
{{< /tabs-wrapper >}}

136
content/influxdb/v2.4/write-data/no-code/load-data.md
View File

@ -87,140 +87,8 @@ Load CSV or line protocol data by uploading a file or pasting the data manually
{{% cloud-only %}}
### Set up a Native MQTT subscription
### Set up an MQTT native subscription
To ingest MQTT (Message Queuing Telemetry Transport) data into InfluxDB, do the following to set up a native subscription:
1. [Subscribe to an MQTT topic](#subscribe-to-an-mqtt-topic) in InfluxDB Cloud by configuring an MQTT broker, and specifying the topic(s) to subscribe to.
2. [Define parsing rules](#define-parsing-rules) for JSON or regex data (line protocol requires no configuration).
{{% note %}}
For troubleshooting help, see [Troubleshoot MQTT ingest errors](/influxdb/cloud/write-data/troubleshoot/#troubleshoot-mqtt-ingest-errors/)
{{% /note %}}
#### Subscribe to an MQTT topic
1. In the navigation menu on the left, click **Load Data** > **Native Subscriptions**.
{{< nav-icon "data" >}}
2. Click **+ Create Subscription**.
3. On the **Setting Up - MQTT Subscriber** page, under **Connect to Broker**, enter the following:
- Subscription Name
- Description
- Protocol
- Hostname or IP address (hostname or URL of the MQTT broker)
- Port (TCP/IP port number the MQTT broker uses)
- Security details. Choose one of the following:
- **None**
- **Basic** (username/password)
- **Certificate**
4. Under **Subscribe to a topic**, in the **Topic** field, enter the MQTT topic name to subscribe to. Note, MQTT brokers typically support wildcard subscriptions with the wildcard characters `+` and `#`.
- To subscribe to all topics in a directory, use `+`. For example, if an `iotdevices` directory includes two directories called `asia` and `europe`, to subscribe to a `sensor` topic in either directory, use `iotdevices/+/sensors` to subscribe to `iotdevices/asia/sensors`, and `iotdevices/europe/sensors`.
- To subscribe to all topics in a directory, use `#`. For example, `iotdevices/#` subscribes to all topics in the `iotdevices` directory. For more information about MQTT subscriptions and wildcards, see [the MQTT specification for Topic Names and Topic Filters](https://docs.oasis-open.org/mqtt/mqtt/v5.0/os/mqtt-v5.0-os.html#_Toc3901241).
5. Under **Write Destination**, select an existing InfluxDB bucket to write data to or click **+ Create bucket**. For more information, see [Create a bucket](/influxdb/cloud/organizations/buckets/create-bucket/).
#### Define parsing rules
{{% note %}}
JSON parsing is faster and more efficient than string parsing. We recommend using JSON parsing when your MQTT messages are in JSON format.
{{% /note %}}
- Under **Define Data Parsing Rules**, select one of the following MQTT data formats:
- **Line protocol** (no configuration required)
- **JSON**. To define parsing rules to ingest JSON data, click the **JSON** tab below.
- **String**. To define parsing rules to ingest String data, click the **String** tab below.
{{< tabs-wrapper >}}
{{% tabs %}}
[Line protocol](#)
[JSON](#)
[String](#)
{{% /tabs %}}
<!-------------------------------- BEGIN Line protocol -------------------------------->
{{% tab-content %}}
Use line protocol to write data into InfluxDB. Line protocol doesn't require any parsing or configuration.
Select a **Timepstamp precision** from the dropdown menu:
- **MS**: Milliseconds
- **S**: Seconds
- **US**: Microseconds
- **NS**: Nanoseconds
{{% /tab-content %}}
<!-------------------------------- BEGIN JSON -------------------------------->
{{% tab-content %}}
Associate **JSON** key/value pairs with **InfluxDB elements** (measurements, timestamps, fields, or tags) using parsing rules.
{{% expand "Example JSON" %}}
```
{
"device_type":"temperature_sensor",
"device_id":2036,
"model_id":"KN24683",
"temperature":25.0,
"time":1653998899010000000,
"error_state":"in_error"
}
```
{{% /expand %}}
1. On the **Setting Up - MQTT Connector** page, under **Data Format**, do the following:
1. (Optional) In the **JSON path to timestamp** field, specify the path in the MQTT message to the JSON key that holds the timestamp: for the example above, `"time":1653998899010000000`. Otherwise, InfluxDB automatically assigns a timestamp when messages are ingested into InfluxDB.
{{% warn %}}
**Important**: Configure the timestamp format that matches the format in your messages.
{{% /warn %}}
2. Configure the JSON parsing rules:
1. Under **Measurement**, enter the **JSON path** (start with `$.`) to assign the InfluxDB measurement key. For the above example, enter `$.device_type`.
3. Select the **Data Type** for the measurement.
4. Specify the JSON paths to tag and field names as needed, and then select the data type for the tag or field. At least one field is required. For the above example, add fields with the JSON paths `$.temperature` and `$.error_state` and a tag with the path `$.error_state`.
Note that JSON paths with arrays are supported, for example, `$.device_information.errors_encountered[0].error_number`.
{{% /tab-content %}}
<!-------------------------------- BEGIN String -------------------------------->
{{% tab-content %}}
Associate **String** key/value pairs with **InfluxDB elements** (measurements, timestamps, fields, or tags).
1. On the **Setting Up - MQTT Connector** page, under **Data Format**, do the following:
1. (Optional) In the **Regex pattern to find timestamp** field, enter the regex (regular expression) to find the timestamp in the MQTT message. Otherwise, InfluxDB automatically assigns a timestamp when messages are ingested into InfluxDB.
{{% note %}}
**Note**: Parsing rules only support finding one value at a time.
{{% /note %}}
For example, if the timestamp string is `time=1653998899010000000`, use a regex to find the string you're looking for and capture the timestamp:
- `time=([\s\S]*?)\n` (captures value after `=` until the EOL (end of line) is reached)
- `time=([\s\S]*?),` (captures value after `=` until comma is reached)
{{% warn %}}
**Important**: Configure the timestamp format that matches the format in your messages.
{{% /warn %}}
2. Under **Measurement**, if the string is `device_type=temperature_sensor` use regex to find the measurement name. For example:
- `device_type=([\s\S]*?)\n` captures the value after the `=` until the EOL (end of line) is reached), in this case the value would be `temperature_sensor`.
3. Select the **Data Type** for the measurement.
4. Enter **Tag** and **Field**. At least one field is required. For tag and field names, use the regex to find the tag or field name, and what to capture. For example:
- `device_id=\d\d\d\d-([0-9][0-9][0-9][0-9])` (matches on the `device_id=` and also matches on the first four digits of the device id, and then captures the four digits.
5. Select the **Data Type** for the tag or field.
{{% /tab-content %}}
{{< /tabs-wrapper >}}
For more details about setting up native subscriptions, see [Set up native subscriptions](/influxdb/cloud/write-data/no-code/native-subscriptions)]
{{% /cloud-only %}}

15
content/kapacitor/v1.5/event_handlers/_index.md
View File

@ -4,6 +4,7 @@ description: >
Kapacitor event handlers provide ways to integrate Kapacitor alert messages with logging, specific URLs, and many third-party applications.
aliases:
- /kapacitor/v1.5/working/event-handler-setup/
- /kapacitor/v1.5/event_handlers/talk/
menu:
kapacitor_1_5_ref:
name: Event handlers
@ -26,19 +27,19 @@ syntax for officially supported Kapacitor event handlers.
[Exec](/kapacitor/v1.5/event_handlers/exec/)
[Hipchat](/kapacitor/v1.5/event_handlers/hipchat/)
[Kafka](/kapacitor/v1.5/event_handlers/kafka/)
[Log](/kapacitor/v1.5/event_handlers/log/)
[Microsoft Teams](/kapacitor/v1.5/event_handlers/microsoftteams/)
[Log](/kapacitor/v1.5/event_handlers/log/)
[Microsoft Teams](/kapacitor/v1.5/event_handlers/microsoftteams/)
[MQTT](/kapacitor/v1.5/event_handlers/mqtt/)
[Opsgenie](/kapacitor/v1.5/event_handlers/opsgenie/v2/)
[Pagerduty](/kapacitor/v1.5/event_handlers/pagerduty/v2/)
[Post](/kapacitor/v1.5/event_handlers/post/)
[Publish](/kapacitor/v1.5/event_handlers/publish/)
[Pushover](/kapacitor/v1.5/event_handlers/pushover/)
[Sensu](/kapacitor/v1.5/event_handlers/sensu/)
[ServiceNow](/kapacitor/v1.5/event_handlers/servicenow/)
[Publish](/kapacitor/v1.5/event_handlers/publish/)
[Pushover](/kapacitor/v1.5/event_handlers/pushover/)
[Sensu](/kapacitor/v1.5/event_handlers/sensu/)
[ServiceNow](/kapacitor/v1.5/event_handlers/servicenow/)
[Slack](/kapacitor/v1.5/event_handlers/slack/)
[Snmptrap](/kapacitor/v1.5/event_handlers/snmptrap/)
[Talk](/kapacitor/v1.5/event_handlers/talk/)
[Talk](/kapacitor/v1.5/event_handlers/talk/) (deprecated)
[TCP](/kapacitor/v1.5/event_handlers/tcp/)
[Telegram](/kapacitor/v1.5/event_handlers/telegram/)
[Victorops](/kapacitor/v1.5/event_handlers/victorops/)

5
content/kapacitor/v1.5/event_handlers/talk.md
View File

@ -8,6 +8,11 @@ menu:
weight: 1800
parent: Event handlers
---
{{% warn %}}
#### Deprecated
The Talk event handler is deprecated in Kapacitor 1.5.
{{% /warn %}}
[Talk](https://jianliao.com/site) is a service that aggregates information into
a centralized hub.

1
content/kapacitor/v1.6/event_handlers/_index.md
View File

@ -4,6 +4,7 @@ description: >
Kapacitor event handlers provide ways to integrate Kapacitor alert messages with logging, specific URLs, and many third-party applications.
aliases:
- /kapacitor/v1.6/working/event-handler-setup/
- /kapacitor/v1.6/event_handlers/talk/
menu:
kapacitor_1_6_ref:
name: Event handlers

146
content/kapacitor/v1.6/event_handlers/talk.md
View File

@ -1,146 +0,0 @@
---
title: Talk event handler
list_title: Talk
description: >
The Talk event handler allows you to send Kapacitor alerts to Talk. This page includes configuration options and usage examples.
menu:
kapacitor_1_6_ref:
name: Talk
weight: 1800
parent: Event handlers
---
[Talk](https://jianliao.com/site) is a service that aggregates information into
a centralized hub.
Kapacitor can be configured to send alert messages to Talk.
## Configuration
Configuration as well as default [option](#options) values for the Talk event
handler are set in your `kapacitor.conf`.
Below is an example configuration:
```toml
[talk]
enabled = true
url = "https://jianliao.com/v2/services/webhook/uuid"
author_name = "Kapacitor"
```
#### `enabled`
Set to `true` to enable the Talk event handler.
#### `url`
The Talk webhook URL.
#### `author_name`
The default authorName.
## Options
The following Talk event handler options can be set in a
[handler file](/kapacitor/v1.6/event_handlers/#create-a-topic-handler-with-a-handler-file) or when using
`.talk()` in a TICKscript.
| Name | Type | Description |
| ---- | ---- | ----------- |
| Title | string | Message title. |
| Text | string | Message text. |
### Example: handler file
```yaml
id: handler-id
topic: topic-name
kind: talk
options:
title: 'Message Title'
text: 'This is the text included in the message.'
```
### Example: TICKscript
```js
|alert()
// ...
.talk()
.title('Message Title')
.text('This is the text included in the message.')
```
## Talk Setup
Create a new incoming webhook to allow Kapacitor to send alerts to Talk.
1. [Sign into your Talk account](https:/account.jianliao.com/signin).
2. Under the "Team" tab, click “Integrations”.
3. Select “Customize service” and click the Incoming Webhook “Add” button.
4. Choose the topic to connect with and click “Confirm Add” button.
5. Once the service is created, you’ll see the “Generate Webhook url”.
6. Place the generated Webhook URL as the `url` in the `[talk]` section of your
`kapacitor.conf`.
## Using the Talk event handler
With the Talk event handler enabled and configured in your `kapacitor.conf`,
use the `.talk()` attribute in your TICKscripts to send alerts to Talk or define
a Talk handler that subscribes to a topic and sends published alerts to Talk.
### Send alerts to Talk from a TICKscript
The following TICKscript sends the message, "Hey, check your CPU", to Talk
whenever idle CPU usage drops below 10% using the `.talk()` event handler.
_**talk-cpu-alert.tick**_
```js
stream
|from()
.measurement('cpu')
|alert()
.crit(lambda: "usage_idle" < 10)
.message('Hey, check your CPU')
.talk()
.title('Alert from Kapacitor')
```
### Send alerts to Talk from a defined handler
The following setup sends an alert to the `cpu` topic with the message,
"Hey, check your CPU".
A Talk handler is added that subscribes to the `cpu` topic and publishes all
alert messages to Talk.
Create a TICKscript that publishes alert messages to a topic.
The TICKscript below sends an alert message to the `cpu` topic any time idle CPU
usage drops below 10%.
_**cpu\_alert.tick**_
```js
stream
|from()
.measurement('cpu')
|alert()
.crit(lambda: "usage_idle" < 10)
.message('Hey, check your CPU')
.topic('cpu')
```
Add and enable the TICKscript:
```bash
kapacitor define cpu_alert -tick cpu_alert.tick
kapacitor enable cpu_alert
```
Create a handler file that subscribes to the `cpu` topic and uses the Talk event
handler to send alerts to Talk.
_**talk\_cpu\_handler.yaml**_
```yaml
id: talk-cpu-alert
topic: cpu
kind: talk
options:
title: Alert from Kapacitor
```
Add the handler:
```bash
kapacitor define-topic-handler talk_cpu_handler.yaml
```