Influxdata
/
docs-v2
mirror of https://github.com/influxdata/docs-v2.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity

added how to guides on: 1. assigning more than four states to data 2.… (#4087) 

* added how to guides on: 1. assigning more than four states to data 2. selecting specific hours from data 3. monitoring state changes across task executions

* Update content/resources/how-to-guides/_index.md

Co-authored-by: Scott Anderson <sanderson@users.noreply.github.com>

* Update content/resources/how-to-guides/_index.md

Co-authored-by: Scott Anderson <sanderson@users.noreply.github.com>

* Update content/resources/how-to-guides/assigning-more-than-four-states.md

Co-authored-by: Scott Anderson <sanderson@users.noreply.github.com>

* Update content/resources/how-to-guides/state-changes-across-task-executions.md

Co-authored-by: Scott Anderson <sanderson@users.noreply.github.com>

* Update content/resources/how-to-guides/state-changes-across-task-executions.md

Co-authored-by: Scott Anderson <sanderson@users.noreply.github.com>

* Update content/resources/how-to-guides/assigning-more-than-four-states.md

Co-authored-by: Scott Anderson <sanderson@users.noreply.github.com>

* Update content/resources/how-to-guides/state-changes-across-task-executions.md

Co-authored-by: Scott Anderson <sanderson@users.noreply.github.com>

* Update content/resources/how-to-guides/state-changes-across-task-executions.md

Co-authored-by: Scott Anderson <sanderson@users.noreply.github.com>

* Update content/resources/how-to-guides/assigning-more-than-four-states.md

Co-authored-by: Scott Anderson <sanderson@users.noreply.github.com>

* Update content/resources/how-to-guides/assigning-more-than-four-states.md

Co-authored-by: Scott Anderson <sanderson@users.noreply.github.com>

* Update content/resources/how-to-guides/assigning-more-than-four-states.md

Co-authored-by: Scott Anderson <sanderson@users.noreply.github.com>

* Update content/resources/how-to-guides/assigning-more-than-four-states.md

Co-authored-by: Scott Anderson <sanderson@users.noreply.github.com>

* Update content/resources/how-to-guides/assigning-more-than-four-states.md

Co-authored-by: Scott Anderson <sanderson@users.noreply.github.com>

* Update content/resources/how-to-guides/assigning-more-than-four-states.md

Co-authored-by: Scott Anderson <sanderson@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: Scott Anderson <sanderson@users.noreply.github.com>

* fix suggestions: header corrections, details on packages imported

* Apply suggestions from code review

Co-authored-by: Scott Anderson <sanderson@users.noreply.github.com>

Co-authored-by: Anais Dotis-Georgiou <anais@Anaiss-MacBook-Pro.local>
Co-authored-by: Scott Anderson <sanderson@users.noreply.github.com>
pull/4151/head
Anaisdg 2022-06-22 10:22:11 -05:00 committed by GitHub
parent 546b5b1123
commit 2c05f032b9
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
4 changed files with 383 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

14
content/resources/how-to-guides/_index.md Normal file
View File

@ -0,0 +1,14 @@
---
title: InfluxData how-to guides
seotitle: InfluxDB and InfluxData how-to guides
description: >
How-to guides related to InfluxDB and other InfluxData products.
menu:
resources:
name: How-to guides
weight: 1
---
Use the following how-to guides to learn more about InfluxDB and other InfluxData products.
{{< children >}}

159
content/resources/how-to-guides/assigning-more-than-four-states.md Normal file
View File

@ -0,0 +1,159 @@
---
title: Assign custom states to data
description: >
Learn how overcome a limitation of the `monitor.stateChanges()` function and assign custom states to your data.
menu:
resources:
parent: How-to guides
weight: 101
---
## Problem
You may want to use the [`monitor` package](/flux/v0.x/stdlib/influxdata/influxdb/monitor/) and take advantage of functions like [monitor.stateChangesOnly()](flux/v0.x/stdlib/influxdata/influxdb/monitor/statechangesonly/). However, `monitor.stateChangesOnly()` only allows you to monitor four states: "crit", "warn", "ok", and "info". What if you want to be able to assign and monitor state changes across custom states or more than four states?
## Solution
Define your own custom `stateChangesOnly()` function. Use the function from the source code here and alter it to accommodate more than four levels. Here we account for six different levels instead of just four.
```js
import "dict"
import "experimental"
stateChangesOnly = (tables=<-) => {
levelInts =
[
"customLevel1": 1,
"customLevel2": 2,
"customLevel3": 3,
"customLevel4": 4,
"customLevel5": 5,
"customLevel6": 6,
]
return
tables
|> map(fn: (r) => ({r with level_value: dict.get(dict: levelInts, key: r._level, default: 0)}))
|> duplicate(column: "_level", as: "____temp_level____")
|> drop(columns: ["_level"])
|> rename(columns: {"____temp_level____": "_level"})
|> sort(columns: ["_source_timestamp", "_time"], desc: false)
|> difference(columns: ["level_value"])
|> filter(fn: (r) => r.level_value != 0)
|> drop(columns: ["level_value"])
|> experimental.group(mode: "extend", columns: ["_level"])
}
```
Construct some example data with [`array.from()`](/flux/v0.x/stdlib/array/from/) and map custom levels to it:
```js
array.from(
rows: [
{_value: 0.0},
{_value: 3.0},
{_value: 5.0},
{_value: 7.0},
{_value: 7.5},
{_value: 9.0},
{_value: 11.0},
],
)
|> map(
fn: (r) =>
({r with _level:
if r._value <= 2.0 then
"customLevel2"
else if r._value <= 4.0 and r._value > 2.0 then
"customLevel3"
else if r._value <= 6.0 and r._value > 4.0 then
"customLevel4"
else if r._value <= 8.0 and r._value > 6.0 then
"customLevel5"
else
"customLevel6",
}),
)
```
Where the example data looks like:
| _value | _level |
| ------ | ------------ |
| 0.0 | customLevel2 |
| 3.0 | customLevel3 |
| 5.0 | customLevel4 |
| 7.0 | customLevel5 |
| 7.5 | customLevel5 |
| 9.0 | customLevel6 |
| 11.0 | customLevel6 |
Now apply our custom `stateChangesOnly()` function:
```js
import "array"
import "dict"
import "experimental"
stateChangesOnly = (tables=<-) => {
levelInts =
[
"customLevel1": 1,
"customLevel2": 2,
"customLevel3": 3,
"customLevel4": 4,
"customLevel5": 5,
"customLevel6": 6,
]
return
tables
|> map(fn: (r) => ({r with level_value: dict.get(dict: levelInts, key: r._level, default: 0)}))
|> duplicate(column: "_level", as: "____temp_level____")
|> drop(columns: ["_level"])
|> rename(columns: {"____temp_level____": "_level"})
|> sort(columns: ["_source_timestamp", "_time"], desc: false)
|> difference(columns: ["level_value"])
|> filter(fn: (r) => r.level_value != 0)
|> drop(columns: ["level_value"])
|> experimental.group(mode: "extend", columns: ["_level"])
}
data =
array.from(
rows: [
{_value: 0.0},
{_value: 3.0},
{_value: 5.0},
{_value: 7.0},
{_value: 7.5},
{_value: 9.0},
{_value: 11.0},
],
)
|> map(
fn: (r) =>
({r with _level:
if r._value <= 2.0 then
"customLevel2"
else if r._value <= 4.0 and r._value > 2.0 then
"customLevel3"
else if r._value <= 6.0 and r._value > 4.0 then
"customLevel4"
else if r._value <= 8.0 and r._value > 6.0 then
"customLevel5"
else
"customLevel6",
}),
)
data
|> stateChangesOnly()
```
This returns:
| _value | _level |
| ------ | ------------ |
| 3.0 | customLevel3 |
| 5.0 | customLevel4 |
| 7.0 | customLevel5 |
| 9.0 | customLevel6 |

40
content/resources/how-to-guides/select-hours-from-data.md Normal file
View File

@ -0,0 +1,40 @@
---
title: Select data from specific hours
description: >
Learn how to select data from specific hours of the day.
menu:
resources:
parent: How-to guides
weight: 102
---
## Problem
You may want to select data from specific hours of the day. For example, you may only want data within normal business hours (9am - 5pm).
## Solution 1
Use [hourSelection()](/flux/v0.x/stdlib/universe/hourselection/) to filter data by a specific hour range in each day.
```js
import "date"
from(bucket: "example-bucket")
|> range(start: -7d)
|> filter(fn: (r) => r["_measurement"] == "example-measurement")
|> filter(fn: (r) => r["_field"] == "example-field")
|> hourSelection(start: 9, stop: 17)
```
## Solution 2
Use [date.hour()](/flux/v0.x/stdlib/date/hour/) to evaluate hours in a `filter()` predicate.
```js
import "date"
from(bucket: "example-bucket")
|> range(start: -7d)
|> filter(fn: (r) => r["_measurement"] == "example-measurement")
|> filter(fn: (r) => r["_field"] == "example-field")
|> filter(fn: (r) => date.hour(t: r["_time"]) > 9 and date.hour(t: r["_time"]) < 17)
This solution also applies if you to select data from certain seconds in a minute, minutes in an hour, days in the month, months in the year, etc. Use the [Flux `date` package](/flux/v0.x/stdlib/date/) to assign integer representations to your data and filter for your desired schedule.

170
content/resources/how-to-guides/state-changes-across-task-executions.md Normal file
View File

@ -0,0 +1,170 @@
---
title: Track state changes across task executions
description: >
Learn how to monitor state changes across task executions, so you don't miss changes across subsequent task runs.
menu:
resources:
parent: How-to guides
weight: 103
---
## Problem
It's common to use [InfluxDB tasks](/influxdb/cloud/process-data/) to evaluate and assign states to your time series data and then detect changes in those states. Tasks process data in batches, but what happens if there is a state change across the batch boundary? The task won't recognize it without knowing the final state of the previous task execution. This guide walks through creating a task that assigns a state to rows and then uses results from the previous task execution to detect any state changes across the batch boundary so you don’t miss any state changes.
## Solution
Explicitly assign levels to your data based on thresholds.
### Solution Advantages
This is the easiest solution to understand if you have never written a task with the [`monitor` package](/flux/v0.x/stdlib/influxdata/influxdb/monitor/).
### Solution Disadvantages
You have to explicitly define your thresholds, which potentially requires more code.
### Solution Overview
Create a task where you:
1. Boilerplate. Import packages and define task options.
2. Query your data.
3. Assign states to your data based on thresholds. Store this data in a variable, i.e. “states”.
4. Write the “states” to a bucket.
5. Find the latest value from the previous task run and store it in a variable “last_state_previous_task”.
6. Union “states” and “last_state_previous_task”. Store this data in a variable “unioned_states”.
7. Discover state changes in “unioned_states”. Store this data in a variable “state_changes”.
8. Notify on state changes that span across the last two tasks to catch any state changes that occur across task executions.
### Solution Explained
1. Import packages and define task options and secrets. Import the following packages:
- [Flux Telegram package](/flux/v0.x/stdlib/contrib/sranka/telegram/): This package
- [Flux InfluxDB secrets package](/flux/v0.x/stdlib/influxdata/influxdb/secrets/): This package contains the [secrets.get()](/flux/v0.x/stdlib/influxdata/influxdb/secrets/get/) function which allows you to retrieve secrets from the InfluxDB secret store. Learn how to [manage secrets](/influxdb/v2.2/security/secrets/) in InfluxDB to use this package.
- [Flux InfluxDB monitoring package](https://docs.influxdata.com/flux/v0.x/stdlib/influxdata/influxdb/monitor/): This package contains functions and tools for monitoring your data.
```js
import "contrib/sranka/telegram"
import "influxdata/influxdb/secrets"
import "influxdata/influxdb/monitor"
option task = {name: "State changes across tasks", every: 30m, offset: 5m}
telegram_token = secrets.get(key: "telegram_token")
telegram_channel_ID = secrets.get(key: "telegram_channel_ID")
```
2. Query the data you want to monitor.
```js
data = from(bucket: "example-bucket")
// Query for data from the last successful task run or from the 1 every duration ago.
// This ensures that you won’t miss any data.
|> range(start: tasks.lastSuccess(orTime: -task.every))
|> filter(fn: (r) => r._measurement == "example-measurement")
|> filter(fn: (r) => r.tagKey1 == "example-tag-value")
|> filter(fn: (r) => r._field == "example-field")
```
Where `data` might look like:
| _measurement | tagKey1 | _field | _value | _time |
| :------------------ | :---------------- | :------------ | -----: | :------------------- |
| example-measurement | example-tag-value | example-field | 30.0 | 2022-01-01T00:00:00Z |
| example-measurement | example-tag-value | example-field | 50.0 | 2022-01-01T00:00:00Z |
3. Assign states to your data based on thresholds. Store this data in a variable, i.e. “states”. To simplify this example, there are only two states: "ok" and "crit." Store states in the `_level` column (required by the `monitor` package).
```js
states =
data
|> map(fn: (r) => ({r with _level: if r._value > 40.0 then "crit" else "ok"}))
```
Where `states` might look like:
| _measurement | tagKey1 | _field | _value | _level | _time |
| :------------------ | :---------------- | :------------ | -----: | :----- | :------------------- |
| example-measurement | example-tag-value | example-field | 30.0 | ok | 2022-01-01T00:00:00Z |
| example-measurement | example-tag-value | example-field | 50.0 | crit | 2022-01-01T00:01:00Z |
4. Write “states” back to InfluxDB. You can write the data to a new measurement or to a new bucket. To write the data to a new measurement, use [`set()`](/flux/v0.x/stdlib/universe/set/) to update the value of the `_measurement` column in your “states” data.
```js
states
// (Optional) Change the measurement name to write the data to a new measurement
|> set(key: "_measurement", value: "new-measurement")
|> to(bucket : "example-bucket")
```
5. Find the latest value from the previous task run and store it in a variable “last_state_previous_task”,
```js
last_state_previous_task =
from(bucket: "example-bucket")
|> range(start: date.sub(d: task.every, from: tasks.lastSuccess(orTime: -task.every))
|> filter(fn: (r) => r._measurement == "example-measurement")
|> filter(fn: (r) => r.tagKey == "example-tag-value")
|> filter(fn: (r) => r._field == "example-field")
|> last()
```
Where `last_state_previous_task` might look like:
| _measurement | tagKey1 | _field | _value | _level | _time |
| :------------------ | :---------------- | :------------ | -----: | :----- | :------------------- |
| example-measurement | example-tag-value | example-field | 55.0 | crit | 2021-12-31T23:59:00Z |
6. Union “states” and “last_state_previous_task”. Store this data in a variable “unioned_states”. Use [`sort()`](/flux/v0.x/stdlib/universe/sort/) to ensure rows are ordered by time.
```js
unioned_states =
union(tables: [states, last_state_previous_task])
|> sort(columns: ["_time"], desc: true)
```
Where `unioned_states` might look like:
| _measurement | tagKey1 | _field | _value | _level | _time |
| :------------------ | :---------------- | :------------ | -----: | :----- | :------------------- |
| example-measurement | example-tag-value | example-field | 55.0 | crit | 2021-12-31T23:59:00Z |
| example-measurement | example-tag-value | example-field | 30.0 | ok | 2022-01-01T00:00:00Z |
| example-measurement | example-tag-value | example-field | 50.0 | crit | 2022-01-01T00:01:00Z |
7. Use [`monitor.stateChangesOnly()`](/flux/v0.x/stdlib/influxdata/influxdb/monitor/statechangesonly/) to return only rows where the state changed in “unioned_states”. Store this data in a variable, “state_changes”.
```js
state_changes =
unioned_states
|> monitor.stateChangesOnly()
```
Where `state_changes` might look like:
| _measurement | tagKey1 | _field | _value | _level | _time |
| :------------------ | :---------------- | :------------ | -----: | :----- | :------------------- |
| example-measurement | example-tag-value | example-field | 30.0 | ok | 2022-01-01T00:00:00Z |
| example-measurement | example-tag-value | example-field | 50.0 | crit | 2022-01-01T00:01:00Z |
8. Notify on state changes that span across the last two tasks to catch any state changes that occur across task executions.
```js
state_changes =
data
|> map(
fn: (r) =>
({
_value:
telegram.message(
token: telegram_token,
channel: telegram_channel_ID,
text: "state change at ${r._value} at ${r._time}",
),
}),
)
```
Using the unioned data, the following alerts would be sent to Telegram:
- `state change at 30.0 at 2022-01-01T00:00:00Z`
- `state change at 50.0 at 2022-01-01T00:01:00Z`