Influxdata
/
docs-v2
mirror of https://github.com/influxdata/docs-v2.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity

migrated flux content into influxdb 1.8

pull/1387/head
Scott Anderson 2020-08-21 16:45:01 -06:00
parent 1e32122a3b
commit 9d8cbfb0f4
38 changed files with 6172 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

37
content/influxdb/v1.8/flux/_index.md Normal file
View File

@ -0,0 +1,37 @@
---
title: Flux data scripting language
description: >
Flux is a functional data scripting language designed for querying, analyzing, and acting on time series data.
menu:
influxdb_1_8:
name: Flux
weight: 80
---
Flux is a functional data scripting language designed for querying, analyzing, and acting on time series data.
Its takes the power of [InfluxQL](/{{< latest "influxdb" "v1" >}}/query_language/spec/) and the functionality of [TICKscript](/{{< latest "kapacitor" >}}/tick/introduction/) and combines them into a single, unified syntax.
> Flux v0.65 is a technical preview included with [InfluxDB v1.8](/influxdb/v1.8).
> It is still in active development and many functions provided by InfluxQL and TICKscript
> have yet to be implemented.
## Flux design principles
Flux is designed to be usable, readable, flexible, composable, testable, contributable, and shareable.
Its syntax is largely inspired by [2018's most popular scripting language](https://insights.stackoverflow.com/survey/2018#technology),
Javascript, and takes a functional approach to data exploration and processing.
The following example illustrates pulling data from a bucket (similar to an InfluxQL database) for the last five minutes,
filtering that data by the `cpu` measurement and the `cpu=cpu-total` tag, windowing the data in 1 minute intervals,
and calculating the average of each window:
```js
from(bucket:"telegraf/autogen")
|> range(start:-1h)
|> filter(fn:(r) =>
r._measurement == "cpu" and
r.cpu == "cpu-total"
)
|> aggregateWindow(every: 1m, fn: mean)
```
{{< children >}}

419
content/influxdb/v1.8/flux/flux-vs-influxql.md Normal file
View File

@ -0,0 +1,419 @@
---
title: Flux vs InfluxQL
description:
menu:
influxdb_1_8:
name: Flux vs InfluxQL
parent: Flux
weight: 5
---
Flux is an alternative to [InfluxQL](/influxdb/v1.8/query_language/) and other SQL-like query languages for querying and analyzing data.
Flux uses functional language patterns making it incredibly powerful, flexible, and able to overcome many of the limitations of InfluxQL.
This article outlines many of the tasks possible with Flux but not InfluxQL and provides information about Flux and InfluxQL parity.
- [Possible with Flux](#possible-with-flux)
- [InfluxQL and Flux parity](#influxql-and-flux-parity)
## Possible with Flux
- [Joins](#joins)
- [Math across measurements](#math-across-measurements)
- [Sort by tags](#sort-by-tags)
- [Group by any column](#group-by-any-column)
- [Window by calendar months and years](#window-by-calendar-months-and-years)
- [Work with multiple data sources](#work-with-multiple-data-sources)
- [DatePart-like queries](#datepart-like-queries)
- [Pivot](#pivot)
- [Histograms](#histograms)
- [Covariance](#covariance)
- [Cast booleans to integers](#cast-booleans-to-integers)
- [String manipulation and data shaping](#string-manipulation-and-data-shaping)
- [Work with geo-temporal data](#work-with-geo-temporal-data)
### Joins
InfluxQL has never supported joins. They can be accomplished using [TICKscript](/kapacitor/latest/tick/introduction/),
but even TICKscript's join capabilities are limited.
Flux's [`join()` function](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/transformations/join/) allows you
to join data **from any bucket, any measurement, and on any columns** as long as
each data set includes the columns on which they are to be joined.
This opens the door for really powerful and useful operations.
```js
dataStream1 = from(bucket: "bucket1")
|> range(start: -1h)
|> filter(fn: (r) =>
r._measurement == "network" and
r._field == "bytes-transferred"
)
dataStream2 = from(bucket: "bucket1")
|> range(start: -1h)
|> filter(fn: (r) =>
r._measurement == "httpd" and
r._field == "requests-per-sec"
)
join(
tables: {d1:dataStream1, d2:dataStream2},
on: ["_time", "_stop", "_start", "host"]
)
```
---
_For an in-depth walkthrough of using the `join()` function, see [How to join data with Flux](/influxdb/v1.8/flux/guides/join)._
---
### Math across measurements
Being able to perform cross-measurement joins also allows you to run calculations using
data from separate measurements – a highly requested feature from the InfluxData community.
The example below takes two data streams from separate measurements, `mem` and `processes`,
joins them, then calculates the average amount of memory used per running process:
```js
// Memory used (in bytes)
memUsed = from(bucket: "telegraf/autogen")
|> range(start: -1h)
|> filter(fn: (r) =>
r._measurement == "mem" and
r._field == "used"
)
// Total processes running
procTotal = from(bucket: "telegraf/autogen")
|> range(start: -1h)
|> filter(fn: (r) =>
r._measurement == "processes" and
r._field == "total"
)
// Join memory used with total processes and calculate
// the average memory (in MB) used for running processes.
join(
tables: {mem:memUsed, proc:procTotal},
on: ["_time", "_stop", "_start", "host"]
)
|> map(fn: (r) => ({
_time: r._time,
_value: (r._value_mem / r._value_proc) / 1000000
})
)
```
### Sort by tags
InfluxQL's sorting capabilities are very limited, allowing you only to control the
sort order of `time` using the `ORDER BY time` clause.
Flux's [`sort()` function](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/transformations/sort) sorts records based on list of columns.
Depending on the column type, records are sorted lexicographically, numerically, or chronologically.
```js
from(bucket:"telegraf/autogen")
|> range(start:-12h)
|> filter(fn: (r) =>
r._measurement == "system" and
r._field == "uptime"
)
|> sort(columns:["region", "host", "_value"])
```
### Group by any column
InfluxQL lets you group by tags or by time intervals, but nothing else.
Flux lets you group by any column in the dataset, including `_value`.
Use the Flux [`group()` function](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/transformations/group/)
to define which columns to group data by.
```js
from(bucket:"telegraf/autogen")
|> range(start:-12h)
|> filter(fn: (r) => r._measurement == "system" and r._field == "uptime" )
|> group(columns:["host", "_value"])
```
### Window by calendar months and years
InfluxQL does not support windowing data by calendar months and years due to their varied lengths.
Flux supports calendar month and year duration units (`1mo`, `1y`) and lets you
window and aggregate data by calendar month and year.
```js
from(bucket:"telegraf/autogen")
|> range(start:-1y)
|> filter(fn: (r) => r._measurement == "mem" and r._field == "used_percent" )
|> aggregateWindow(every: 1mo, fn: mean)
```
### Work with multiple data sources
InfluxQL can only query data stored in InfluxDB.
Flux can query data from other data sources such as CSV, PostgreSQL, MySQL, Google BigTable, and more.
Join that data with data in InfluxDB to enrich query results.
- [Flux CSV package](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/csv/)
- [Flux SQL package](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/sql/)
- [Flux BigTable package](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/experimental/bigtable/)
<!-- -->
```js
import "csv"
import "sql"
csvData = csv.from(csv: rawCSV)
sqlData = sql.from(
driverName: "postgres",
dataSourceName: "postgresql://user:password@localhost",
query:"SELECT * FROM example_table"
)
data = from(bucket: "telegraf/autogen")
|> range(start: -24h)
|> filter(fn: (r) => r._measurement == "sensor")
auxData = join(tables: {csv: csvData, sql: sqlData}, on: ["sensor_id"])
enrichedData = join(tables: {data: data, aux: auxData}, on: ["sensor_id"])
enrichedData
|> yeild(name: "enriched_data")
```
---
_For an in-depth walkthrough of querying SQL data, see [Query SQL data sources](/influxdb/v1.8/flux/guides/sql)._
---
### DatePart-like queries
InfluxQL doesn't support DatePart-like queries that only return results during specified hours of the day.
The Flux [`hourSelection` function](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/transformations/hourselection/)
returns only data with time values in a specified hour range.
```js
from(bucket: "telegraf/autogen")
|> range(start: -1h)
|> filter(fn: (r) =>
r._measurement == "cpu" and
r.cpu == "cpu-total"
)
|> hourSelection(start: 9, stop: 17)
```
### Pivot
Pivoting data tables has never been supported in InfluxQL.
The Flux [`pivot()` function](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/transformations/pivot) provides the ability
to pivot data tables by specifying `rowKey`, `columnKey`, and `valueColumn` parameters.
```js
from(bucket: "telegraf/autogen")
|> range(start: -1h)
|> filter(fn: (r) =>
r._measurement == "cpu" and
r.cpu == "cpu-total"
)
|> pivot(
rowKey:["_time"],
columnKey: ["_field"],
valueColumn: "_value"
)
```
### Histograms
The ability to generate histograms has been a highly requested feature for InfluxQL, but has never been supported.
Flux's [`histogram()` function](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/transformations/histogram) uses input
data to generate a cumulative histogram with support for other histogram types coming in the future.
```js
from(bucket: "telegraf/autogen")
|> range(start: -1h)
|> filter(fn: (r) =>
r._measurement == "mem" and
r._field == "used_percent"
)
|> histogram(
buckets: [10, 20, 30, 40, 50, 60, 70, 80, 90, 100]
)
```
---
_For an example of using Flux to create a cumulative histogram, see [Create histograms](/influxdb/v1.8/flux/guides/histograms)._
---
### Covariance
Flux provides functions for simple covariance calculation.
The [`covariance()` function](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/transformations/aggregates/covariance)
calculates the covariance between two columns and the [`cov()` function](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/transformations/aggregates/cov)
calculates the covariance between two data streams.
###### Covariance between two columns
```js
from(bucket: "telegraf/autogen")
|> range(start:-5m)
|> covariance(columns: ["x", "y"])
```
###### Covariance between two streams of data
```js
table1 = from(bucket: "telegraf/autogen")
|> range(start: -15m)
|> filter(fn: (r) =>
r._measurement == "measurement_1"
)
table2 = from(bucket: "telegraf/autogen")
|> range(start: -15m)
|> filter(fn: (r) =>
r._measurement == "measurement_2"
)
cov(x: table1, y: table2, on: ["_time", "_field"])
```
### Cast booleans to integers
InfluxQL supports type casting, but only for numeric data types (floats to integers and vice versa).
[Flux type conversion functions](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/transformations/type-conversions/)
provide much broader support for type conversions and let you perform some long-requested
operations like casting a boolean values to integers.
##### Cast boolean field values to integers
```js
from(bucket: "telegraf/autogen")
|> range(start: -1h)
|> filter(fn: (r) =>
r._measurement == "m" and
r._field == "bool_field"
)
|> toInt()
```
### String manipulation and data shaping
InfluxQL doesn't support string manipulation when querying data.
The [Flux Strings package](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/strings/) is a collection of functions that operate on string data.
When combined with the [`map()` function](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/transformations/map/),
functions in the string package allow for operations like string sanitization and normalization.
```js
import "strings"
from(bucket: "telegraf/autogen")
|> range(start: -1h)
|> filter(fn: (r) =>
r._measurement == "weather" and
r._field == "temp"
)
|> map(fn: (r) => ({
r with
location: strings.toTitle(v: r.location),
sensor: strings.replaceAll(v: r.sensor, t: " ", u: "-"),
status: strings.substring(v: r.status, start: 0, end: 8)
}))
```
### Work with geo-temporal data
InfluxQL doesn't provide functionality for working with geo-temporal data.
The [Flux Geo package](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/experimental/geo/) is a collection of functions that
let you shape, filter, and group geo-temporal data.
```js
import "experimental/geo"
from(bucket: "geo/autogen")
|> range(start: -1w)
|> filter(fn: (r) => r._measurement == "taxi")
|> geo.shapeData(latField: "latitude", lonField: "longitude", level: 20)
|> geo.filterRows(
region: {lat: 40.69335938, lon: -73.30078125, radius: 20.0},
strict: true
)
|> geo.asTracks(groupBy: ["fare-id"])
```
## InfluxQL and Flux parity
Flux is working towards complete parity with InfluxQL and new functions are being added to that end.
The table below shows InfluxQL statements, clauses, and functions along with their equivalent Flux functions.
_For a complete list of Flux functions, [view all Flux functions](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/all-functions)._
### InfluxQL and Flux parity
| InfluxQL | Flux Functions |
| -------- | -------------- |
| [SELECT](/influxdb/v1.8/query_language/explore-data/#the-basic-select-statement) | [filter()](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/transformations/filter/) |
| [WHERE](/influxdb/v1.8/query_language/explore-data/#the-where-clause) | [filter()](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/transformations/filter/), [range()](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/transformations/range/) |
| [GROUP BY](/influxdb/v1.8/query_language/explore-data/#the-group-by-clause) | [group()](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/transformations/group/) |
| [INTO](/influxdb/v1.8/query_language/explore-data/#the-into-clause) | [to()](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/outputs/to/) <span><a style="color:orange" href="#footnote">*</a></span> |
| [ORDER BY](/influxdb/v1.8/query_language/explore-data/#order-by-time-desc) | [sort()](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/transformations/sort/) |
| [LIMIT](/influxdb/v1.8/query_language/explore-data/#the-limit-clause) | [limit()](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/transformations/limit/) |
| [SLIMIT](/influxdb/v1.8/query_language/explore-data/#the-slimit-clause) | -- |
| [OFFSET](/influxdb/v1.8/query_language/explore-data/#the-offset-clause) | -- |
| [SOFFSET](/influxdb/v1.8/query_language/explore-data/#the-soffset-clause) | -- |
| [SHOW DATABASES](/influxdb/v1.8/query_language/explore-schema/#show-databases) | [buckets()](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/inputs/buckets/) |
| [SHOW MEASUREMENTS](/influxdb/v1.8/query_language/explore-schema/#show-measurements) | [v1.measurements](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/influxdb-v1/measurements) |
| [SHOW FIELD KEYS](/influxdb/v1.8/query_language/explore-schema/#show-field-keys) | [keys()](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/transformations/keys/) |
| [SHOW RETENTION POLICIES](/influxdb/v1.8/query_language/explore-schema/#show-retention-policies) | [buckets()](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/inputs/buckets/) |
| [SHOW TAG KEYS](/influxdb/v1.8/query_language/explore-schema/#show-tag-keys) | [v1.tagKeys()](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/influxdb-v1/tagkeys), [v1.measurementTagKeys()](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/influxdb-v1/measurementtagkeys) |
| [SHOW TAG VALUES](/influxdb/v1.8/query_language/explore-schema/#show-tag-values) | [v1.tagValues()](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/influxdb-v1/tagvalues), [v1.measurementTagValues()](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/influxdb-v1/measurementtagvalues) |
| [SHOW SERIES](/influxdb/v1.8/query_language/explore-schema/#show-series) | -- |
| [CREATE DATABASE](/influxdb/v1.8/query_language/database_management/#create-database) | -- |
| [DROP DATABASE](/influxdb/v1.8/query_language/database_management/#delete-a-database-with-drop-database) | -- |
| [DROP SERIES](/influxdb/v1.8/query_language/database_management/#drop-series-from-the-index-with-drop-serie) | -- |
| [DELETE](/influxdb/v1.8/query_language/database_management/#delete-series-with-delete) | -- |
| [DROP MEASUREMENT](/influxdb/v1.8/query_language/database_management/#delete-measurements-with-drop-measurement) | -- |
| [DROP SHARD](/influxdb/v1.8/query_language/database_management/#delete-a-shard-with-drop-shard) | -- |
| [CREATE RETENTION POLICY](/influxdb/v1.8/query_language/database_management/#create-retention-policies-with-create-retention-policy) | -- |
| [ALTER RETENTION POLICY](/influxdb/v1.8/query_language/database_management/#modify-retention-policies-with-alter-retention-policy) | -- |
| [DROP RETENTION POLICY](/influxdb/v1.8/query_language/database_management/#delete-retention-policies-with-drop-retention-policy) | -- |
| [COUNT](/influxdb/v1.8/query_language/functions#count) | [count()](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/transformations/aggregates/count/) |
| [DISTINCT](/influxdb/v1.8/query_language/functions#distinct) | [distinct()](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/transformations/selectors/distinct/) |
| [INTEGRAL](/influxdb/v1.8/query_language/functions#integral) | [integral()](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/transformations/aggregates/integral/) |
| [MEAN](/influxdb/v1.8/query_language/functions#mean) | [mean()](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/transformations/aggregates/mean/) |
| [MEDIAN](/influxdb/v1.8/query_language/functions#median) | [median()](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/transformations/aggregates/median/) |
| [MODE](/influxdb/v1.8/query_language/functions#mode) | [mode()](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/transformations/aggregates/mode/) |
| [SPREAD](/influxdb/v1.8/query_language/functions#spread) | [spread()](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/transformations/aggregates/spread/) |
| [STDDEV](/influxdb/v1.8/query_language/functions#stddev) | [stddev()](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/transformations/aggregates/stddev/) |
| [SUM](/influxdb/v1.8/query_language/functions#sum) | [sum()](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/transformations/aggregates/sum/) |
| [BOTTOM](/influxdb/v1.8/query_language/functions#bottom) | [bottom()](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/transformations/selectors/bottom/) |
| [FIRST](/influxdb/v1.8/query_language/functions#first) | [first()](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/transformations/selectors/first/) |
| [LAST](/influxdb/v1.8/query_language/functions#last) | [last()](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/transformations/selectors/last/) |
| [MAX](/influxdb/v1.8/query_language/functions#max) | [max()](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/transformations/selectors/max/) |
| [MIN](/influxdb/v1.8/query_language/functions#min) | [min()](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/transformations/selectors/min/) |
| [PERCENTILE](/influxdb/v1.8/query_language/functions#percentile) | [quantile()](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/transformations/aggregates/quantile/) |
| [SAMPLE](/influxdb/v1.8/query_language/functions#sample) | [sample()](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/transformations/selectors/sample/) |
| [TOP](/influxdb/v1.8/query_language/functions#top) | [top()](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/transformations/selectors/top/) |
| [ABS](/influxdb/v1.8/query_language/functions#abs) | [math.abs()](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/math/abs/) |
| [ACOS](/influxdb/v1.8/query_language/functions#acos) | [math.acos()](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/math/acos/) |
| [ASIN](/influxdb/v1.8/query_language/functions#asin) | [math.asin()](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/math/asin/) |
| [ATAN](/influxdb/v1.8/query_language/functions#atan) | [math.atan()](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/math/atan/) |
| [ATAN2](/influxdb/v1.8/query_language/functions#atan2) | [math.atan2()](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/math/atan2/) |
| [CEIL](/influxdb/v1.8/query_language/functions#ceil) | [math.ceil()](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/math/ceil/) |
| [COS](/influxdb/v1.8/query_language/functions#cos) | [math.cos()](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/math/cos/) |
| [CUMULATIVE_SUM](/influxdb/v1.8/query_language/functions#cumulative-sum) | [cumulativeSum()](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/transformations/cumulativesum/) |
| [DERIVATIVE](/influxdb/v1.8/query_language/functions#derivative) | [derivative()](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/transformations/derivative/) |
| [DIFFERENCE](/influxdb/v1.8/query_language/functions#difference) | [difference()](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/transformations/difference/) |
| [ELAPSED](/influxdb/v1.8/query_language/functions#elapsed) | [elapsed()](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/transformations/elapsed/) |
| [EXP](/influxdb/v1.8/query_language/functions#exp) | [math.exp()](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/math/exp/) |
| [FLOOR](/influxdb/v1.8/query_language/functions#floor) | [math.floor()](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/math/floor/) |
| [HISTOGRAM](/influxdb/v1.8/query_language/functions#histogram) | [histogram()](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/transformations/histogram/) |
| [LN](/influxdb/v1.8/query_language/functions#ln) | [math.log()](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/math/log/) |
| [LOG](/influxdb/v1.8/query_language/functions#log) | [math.logb()](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/math/logb/) |
| [LOG2](/influxdb/v1.8/query_language/functions#log2) | [math.log2()](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/math/log2/) |
| [LOG10](/influxdb/v1.8/query_language/functions#logt10) | [math.log10()](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/math/log10/) |
| [MOVING_AVERAGE](/influxdb/v1.8/query_language/functions#moving-average) | [movingAverage()](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/transformations/movingaverage/) |
| [NON_NEGATIVE_DERIVATIVE](/influxdb/v1.8/query_language/functions#non-negative-derivative) | [derivative(nonNegative:true)](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/transformations/derivative/) |
| [NON_NEGATIVE_DIFFERENCE](/influxdb/v1.8/query_language/functions#non-negative-difference) | [difference(nonNegative:true)](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/transformations/derivative/) |
| [POW](/influxdb/v1.8/query_language/functions#pow) | [math.pow()](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/math/pow/) |
| [ROUND](/influxdb/v1.8/query_language/functions#round) | [math.round()](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/math/round/) |
| [SIN](/influxdb/v1.8/query_language/functions#sin) | [math.sin()](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/math/sin/) |
| [SQRT](/influxdb/v1.8/query_language/functions#sqrt) | [math.sqrt()](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/math/sqrt/) |
| [TAN](/influxdb/v1.8/query_language/functions#tan) | [math.tan()](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/math/tan/) |
| [HOLT_WINTERS](/influxdb/v1.8/query_language/functions#holt-winters) | [holtWinters()](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/transformations/holtwinters/) |
| [CHANDE_MOMENTUM_OSCILLATOR](/influxdb/v1.8/query_language/functions#chande-momentum-oscillator) | [chandeMomentumOscillator()](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/transformations/chandemomentumoscillator/) |
| [EXPONENTIAL_MOVING_AVERAGE](/influxdb/v1.8/query_language/functions#exponential-moving-average) | [exponentialMovingAverage()](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/transformations/exponentialmovingaverage/) |
| [DOUBLE_EXPONENTIAL_MOVING_AVERAGE](/influxdb/v1.8/query_language/functions#double-exponential-moving-average) | [doubleEMA()](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/transformations/doubleema/) |
| [KAUFMANS_EFFICIENCY_RATIO](/influxdb/v1.8/query_language/functions#kaufmans-efficiency-ratio) | [kaufmansER()](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/transformations/kaufmanser/) |
| [KAUFMANS_ADAPTIVE_MOVING_AVERAGE](/influxdb/v1.8/query_language/functions#kaufmans-adaptive-moving-average) | [kaufmansAMA()](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/transformations/kaufmansama/) |
| [TRIPLE_EXPONENTIAL_MOVING_AVERAGE](/influxdb/v1.8/query_language/functions#triple-exponential-moving-average) | [tripleEMA()](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/transformations/tripleema/) |
| [TRIPLE_EXPONENTIAL_DERIVATIVE](/influxdb/v1.8/query_language/functions#triple-exponential-derivative) | [tripleExponentialDerivative()](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/transformations/tripleexponentialderivative/) |
| [RELATIVE_STRENGTH_INDEX](/influxdb/v1.8/query_language/functions#relative-strength-index) | [relativeStrengthIndex()](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/transformations/relativestrengthindex/) |
_<span style="font-size:.9rem" id="footnote"><span style="color:orange">*</span> The <code>to()</code> function only writes to InfluxDB 2.0.</span>_

120
content/influxdb/v1.8/flux/get-started/_index.md Normal file
View File

@ -0,0 +1,120 @@
---
title: Get started with Flux
description: >
Get started with Flux, InfluxData's new functional data scripting language.
This step-by-step guide will walk you through the basics and get you on your way.
menu:
influxdb_1_8:
name: Get started with Flux
identifier: get-started
parent: Flux
weight: 2
---
Flux is InfluxData's new functional data scripting language designed for querying,
analyzing, and acting on data.
This multi-part getting started guide walks through important concepts related to Flux.
It covers querying time series data from InfluxDB using Flux, and introduces Flux syntax and functions.
## What you will need
##### InfluxDB v1.8+
Flux v0.65 is built into InfluxDB v1.8 and can be used to query data stored in InfluxDB.
InfluxDB v1.8's `influx` CLI also includes a `-type=` option that, when set to `flux`, will start an
interactive Flux Read-Eval-Print-Loop (REPL) allowing you to write and run Flux queries from the command line.
---
_For information about downloading and installing InfluxDB, see [InfluxDB installation](/{{< latest "influxdb" "v1" >}}/introduction/installation)._
---
##### Chronograf v1.8+
**Not required but strongly recommended**.
Chronograf v1.8's Data Explorer provides a user interface (UI) for writing Flux scripts and visualizing results.
Dashboards in Chronograf v1.8+ also support Flux queries.
---
_For information about downloading and installing Chronograf, see [Chronograf installation](/{{< latest "chronograf" >}}/introduction/installation)._
---
## Key concepts
Flux introduces important new concepts you should understand as you get started.
### Buckets
Flux introduces "buckets," a new data storage concept for InfluxDB.
A **bucket** is a named location where data is stored that has a retention policy.
It's similar to an InfluxDB v1.x "database," but is a combination of both a database and a retention policy.
When using multiple retention policies, each retention policy is treated as is its own bucket.
Flux's [`from()` function](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/inputs/from), which defines an InfluxDB data source, requires a `bucket` parameter.
When using Flux with InfluxDB v1.x, use the following bucket naming convention which combines
the database name and retention policy into a single bucket name:
###### InfluxDB v1.x bucket naming convention
```js
// Pattern
from(bucket:"<database>/<retention-policy>")
// Example
from(bucket:"telegraf/autogen")
```
### Pipe-forward operator
Flux uses pipe-forward operators (`|>`) extensively to chain operations together.
After each function or operation, Flux returns a table or collection of tables containing data.
The pipe-forward operator pipes those tables into the next function or operation where
they are further processed or manipulated.
### Tables
Flux structures all data in tables.
When data is streamed from data sources, Flux formats it as annotated comma-separated values (CSV), representing tables.
Functions then manipulate or process them and output new tables.
This makes it easy to chain together functions to build sophisticated queries.
#### Group keys
Every table has a **group key** which describes the contents of the table.
It's a list of columns for which every row in the table will have the same value.
Columns with unique values in each row are **not** part of the group key.
As functions process and transform data, each modifies the group keys of output tables.
Understanding how tables and group keys are modified by functions is key to properly
shaping your data for the desired output.
###### Example group key
```js
[_start, _stop, _field, _measurement, host]
```
Note that `_time` and `_value` are excluded from the example group key because they
are unique to each row.
## Tools for working with Flux
You have multiple [options for writing and running Flux queries](/{{< latest "influxdb" "v2" >}}/reference/flux/guides/executing-queries),
but as you're getting started, we recommend using the following:
### 1. Chronograf's Data Explorer
Chronograf's Data Explorer makes it easy to write your first Flux script and visualize the results.
To use Chronograf's Flux UI, open the **Data Explorer** and to the right of the source
dropdown above the graph placeholder, select **Flux** as the source type.
This will provide **Schema**, **Script**, and **Functions** panes.
The Schema pane allows you to explore your data.
The Script pane is where you write your Flux script.
The Functions pane provides a list of functions available in your Flux queries.
### 2. influx CLI
The `influx` CLI is an interactive shell for querying InfluxDB.
With InfluxDB v1.8+, use the `-type=flux` option to open a Flux REPL where you write and run Flux queries.
```bash
influx -type=flux
```
<div class="page-nav-btns">
<a class="btn next" href="/influxdb/v1.8/flux/get-started/query-influxdb/">Query InfluxDB with Flux</a>
</div>

130
content/influxdb/v1.8/flux/get-started/query-influxdb.md Normal file
View File

@ -0,0 +1,130 @@
---
title: Query InfluxDB with Flux
description: Learn the basics of using Flux to query data from InfluxDB.
menu:
influxdb_1_8:
name: Query InfluxDB
parent: get-started
weight: 1
---
This guide walks through the basics of using Flux to query data from InfluxDB.
_**If you haven't already, make sure to install InfluxDB v1.8+, [enable Flux](/influxdb/v1.8/flux/installation),
and choose a [tool for writing Flux queries](/influxdb/v1.8/flux/get-started#tools-for-working-with-flux).**_
Every Flux query needs the following:
1. [A data source](#1-define-your-data-source)
2. [A time range](#2-specify-a-time-range)
3. [Data filters](#3-filter-your-data)
## 1. Define your data source
Flux's [`from()`](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/inputs/from) function defines an InfluxDB data source.
It requires a [`bucket`](/influxdb/v1.8/flux/get-started/#buckets) parameter.
For this example, use `telegraf/autogen`, a combination of the default database and retention policy provided by the TICK stack.
```js
from(bucket:"telegraf/autogen")
```
## 2. Specify a time range
Flux requires a time range when querying time series data.
"Unbounded" queries are very resource-intensive and as a protective measure,
Flux will not query the database without a specified range.
Use the pipe-forward operator (`|>`) to pipe data from your data source into the [`range()`](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/transformations/range)
function, which specifies a time range for your query.
It accepts two properties: `start` and `stop`.
Ranges can be **relative** using negative [durations](/flux/v0.65/language/lexical-elements#duration-literals)
or **absolute** using [timestamps](/flux/v0.65/language/lexical-elements#date-and-time-literals).
###### Example relative time ranges
```js
// Relative time range with start only. Stop defaults to now.
from(bucket:"telegraf/autogen")
|> range(start: -1h)
// Relative time range with start and stop
from(bucket:"telegraf/autogen")
|> range(start: -1h, stop: -10m)
```
> Relative ranges are relative to "now."
###### Example absolute time range
```js
from(bucket:"telegraf/autogen")
|> range(start: 2018-11-05T23:30:00Z, stop: 2018-11-06T00:00:00Z)
```
#### Use the following:
For this guide, use the relative time range, `-15m`, to limit query results to data from the last 15 minutes:
```js
from(bucket:"telegraf/autogen")
|> range(start: -15m)
```
## 3. Filter your data
Pass your ranged data into the `filter()` function to narrow results based on data attributes or columns.
The `filter()` function has one parameter, `fn`, which expects an anonymous function
with logic that filters data based on columns or attributes.
Flux's anonymous function syntax is very similar to Javascript's.
Records or rows are passed into the `filter()` function as an object (`r`).
The anonymous function takes the object and evaluates it to see if it matches the defined filters.
Use the `AND` relational operator to chain multiple filters.
```js
// Pattern
(r) => (r.objectProperty comparisonOperator comparisonExpression)
// Example with single filter
(r) => (r._measurement == "cpu")
// Example with multiple filters
(r) => (r._measurement == "cpu") and (r._field != "usage_system" )
```
#### Use the following:
For this example, filter by the `cpu` measurement, the `usage_system` field, and the `cpu-total` tag value:
```js
from(bucket:"telegraf/autogen")
|> range(start: -15m)
|> filter(fn: (r) =>
r._measurement == "cpu" and
r._field == "usage_system" and
r.cpu == "cpu-total"
)
```
## 4. Yield your queried data
Use Flux's `yield()` function to output the filtered tables as the result of the query.
```js
from(bucket:"telegraf/autogen")
|> range(start: -15m)
|> filter(fn: (r) =>
r._measurement == "cpu" and
r._field == "usage_system" and
r.cpu == "cpu-total"
)
|> yield()
```
> Chronograf and the `influx` CLI automatically assume a `yield()` function at
> the end of each script in order to output and visualize the data.
> Best practice is to include a `yield()` function, but it is not always necessary.
## Congratulations!
You have now queried data from InfluxDB using Flux.
The query shown here is a barebones example.
Flux queries can be extended in many ways to form powerful scripts.
<div class="page-nav-btns">
<a class="btn prev" href="/influxdb/v1.8/flux/get-started/">Get started with Flux</a>
<a class="btn next" href="/influxdb/v1.8/flux/get-started/transform-data/">Transform your data</a>
</div>

234
content/influxdb/v1.8/flux/get-started/syntax-basics.md Normal file
View File

@ -0,0 +1,234 @@
---
title: Flux syntax basics
description: An introduction to the basic elements of the Flux syntax with real-world application examples.
menu:
influxdb_1_8:
name: Syntax basics
parent: get-started
weight: 3
---
Flux, at its core, is a scripting language designed specifically for working with data.
This guide walks through a handful of simple expressions and how they are handled in Flux.
## Use the influx CLI
Use the `influx` CLI in "Flux mode" as you follow this guide.
When started with `-type=flux`, the `influx` CLI is an interactive read-eval-print-loop (REPL) that supports Flux syntax.
##### Start in the influx CLI in Flux mode
```bash
influx -type=flux
```
> If using the [InfluxData Sandbox](/platform/install-and-deploy/deploying/sandbox-install), use the `./sandbox enter`
> command to enter the `influxdb` container, where you can start the `influx` CLI in Flux mode.
> You will also need to specify the `host` as `influxdb` to connect to InfluxDB over the Docker network.
>
```bash
./sandbox enter influxdb
root@9bfc3c08579c:/# influx -host influxdb -type=flux
```
## Basic Flux syntax
The code blocks below provide commands that illustrate the basic syntax of Flux.
Run these commands in the `influx` CLI's Flux REPL.
### Simple expressions
Flux is a scripting language that supports basic expressions.
For example, simple addition:
```js
> 1 + 1
2
```
### Variables
Assign an expression to a variable using the assignment operator, `=`.
```js
> s = "this is a string"
> i = 1 // an integer
> f = 2.0 // a floating point number
```
Type the name of a variable to print its value:
```js
> s
this is a string
> i
1
> f
2
```
### Objects
Flux also supports objects. Each value in an object can be a different data type.
```js
> o = {name:"Jim", age: 42, "favorite color": "red"}
```
Use **dot notation** to access a properties of an object:
```js
> o.name
Jim
> o.age
42
```
Or **bracket notation**:
```js
> o["name"]
Jim
> o["age"]
42
> o["favorite color"]
red
```
{{% note %}}
Use bracket notation to reference object properties with special or
white space characters in the property key.
{{% /note %}}
### Lists
Flux supports lists. List values must be the same type.
```js
> n = 4
> l = [1,2,3,n]
> l
[1, 2, 3, 4]
```
### Functions
Flux uses functions for most of its heavy lifting.
Below is a simple function that squares a number, `n`.
```js
> square = (n) => n * n
> square(n:3)
9
```
> Flux does not support positional arguments or parameters.
> Parameters must always be named when calling a function.
### Pipe-forward operator
Flux uses the pipe-forward operator (`|>`) extensively to chain operations together.
After each function or operation, Flux returns a table or collection of tables containing data.
The pipe-forward operator pipes those tables into the next function where they are further processed or manipulated.
```js
data |> someFunction() |> anotherFunction()
```
## Real-world application of basic syntax
This likely seems familiar if you've already been through through the other [getting started guides](/influxdb/v1.8/flux/get-started).
Flux's syntax is inspired by Javascript and other functional scripting languages.
As you begin to apply these basic principles in real-world use cases such as creating data stream variables,
custom functions, etc., the power of Flux and its ability to query and process data will become apparent.
The examples below provide both multi-line and single-line versions of each input command.
Carriage returns in Flux aren't necessary, but do help with readability.
Both single- and multi-line commands can be copied and pasted into the `influx` CLI running in Flux mode.
{{< tabs-wrapper >}}
{{% tabs %}}
[Multi-line inputs](#)
[Single-line inputs](#)
{{% /tabs %}}
{{% tab-content %}}
### Define data stream variables
A common use case for variable assignments in Flux is creating variables for one
or more input data streams.
```js
timeRange = -1h
cpuUsageUser =
from(bucket:"telegraf/autogen")
|> range(start: timeRange)
|> filter(fn: (r) =>
r._measurement == "cpu" and
r._field == "usage_user" and
r.cpu == "cpu-total"
)
memUsagePercent =
from(bucket:"telegraf/autogen")
|> range(start: timeRange)
|> filter(fn: (r) =>
r._measurement == "mem" and
r._field == "used_percent"
)
```
These variables can be used in other functions, such as `join()`, while keeping the syntax minimal and flexible.
### Define custom functions
Create a function that returns the `N` number rows in the input stream with the highest `_value`s.
To do this, pass the input stream (`tables`) and the number of results to return (`n`) into a custom function.
Then using Flux's `sort()` and `limit()` functions to find the top `n` results in the data set.
```js
topN = (tables=<-, n) =>
tables
|> sort(desc: true)
|> limit(n: n)
```
_More information about creating custom functions is available in the [Custom functions](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/custom-functions) documentation._
Using this new custom function `topN` and the `cpuUsageUser` data stream variable defined above,
find the top five data points and yield the results.
```js
cpuUsageUser
|> topN(n:5)
|> yield()
```
{{% /tab-content %}}
{{% tab-content %}}
### Define data stream variables
A common use case for variable assignments in Flux is creating variables for multiple filtered input data streams.
```js
timeRange = -1h
cpuUsageUser = from(bucket:"telegraf/autogen") |> range(start: timeRange) |> filter(fn: (r) => r._measurement == "cpu" and r._field == "usage_user" and r.cpu == "cpu-total")
memUsagePercent = from(bucket:"telegraf/autogen") |> range(start: timeRange) |> filter(fn: (r) => r._measurement == "mem" and r._field == "used_percent")
```
These variables can be used in other functions, such as `join()`, while keeping the syntax minimal and flexible.
### Define custom functions
Let's create a function that returns the `N` number rows in the input data stream with the highest `_value`s.
To do this, pass the input stream (`tables`) and the number of results to return (`n`) into a custom function.
Then using Flux's `sort()` and `limit()` functions to find the top `n` results in the data set.
```js
topN = (tables=<-, n) => tables |> sort(desc: true) |> limit(n: n)
```
_More information about creating custom functions is available in the [Custom functions](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/custom-functions) documentation._
Using the `cpuUsageUser` data stream variable defined [above](#define-data-stream-variables),
find the top five data points with the custom `topN` function and yield the results.
```js
cpuUsageUser |> topN(n:5) |> yield()
```
{{% /tab-content %}}
{{< /tabs-wrapper >}}
This query will return the five data points with the highest user CPU usage over the last hour.
<div class="page-nav-btns">
<a class="btn prev" href="/influxdb/v1.8/flux/get-started/transform-data/">Transform your data</a>
</div>

180
content/influxdb/v1.8/flux/get-started/transform-data.md Normal file
View File

@ -0,0 +1,180 @@
---
title: Transform data with Flux
description: Learn the basics of using Flux to transform data queried from InfluxDB.
menu:
influxdb_1_8:
name: Transform your data
parent: get-started
weight: 2
---
When [querying data from InfluxDB](/influxdb/v1.8/flux/get-started/query-influxdb),
you often need to transform that data in some way.
Common examples are aggregating data into averages, downsampling data, etc.
This guide demonstrates using [Flux functions](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib) to transform your data.
It walks through creating a Flux script that partitions data into windows of time,
averages the `_value`s in each window, and outputs the averages as a new table.
It's important to understand how the "shape" of your data changes through each of these operations.
## Query data
Use the query built in the previous [Query data from InfluxDB](/influxdb/v1.8/flux/get-started/query-influxdb)
guide, but update the range to pull data from the last hour:
```js
from(bucket:"telegraf/autogen")
|> range(start: -1h)
|> filter(fn: (r) =>
r._measurement == "cpu" and
r._field == "usage_system" and
r.cpu == "cpu-total"
)
```
## Flux functions
Flux provides a number of functions that perform specific operations, transformations, and tasks.
You can also [create custom functions](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/custom-functions) in your Flux queries.
_Functions are covered in detail in the [Flux functions](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib) documentation._
A common type of function used when transforming data queried from InfluxDB is an aggregate function.
Aggregate functions take a set of `_value`s in a table, aggregate them, and transform
them into a new value.
This example uses the [`mean()` function](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/transformations/aggregates/mean)
to average values within time windows.
> The following example walks through the steps required to window and aggregate data,
> but there is a [`aggregateWindow()` helper function](#helper-functions) that does it for you.
> It's just good to understand the steps in the process.
## Window your data
Flux's [`window()` function](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/transformations/window) partitions records based on a time value.
Use the `every` parameter to define a duration of time for each window.
{{% note %}}
#### Calendar months and years
`every` supports all [valid duration units](/flux/v0.65/language/types/#duration-types),
including **calendar months (`1mo`)** and **years (`1y`)**.
{{% /note %}}
For this example, window data in five minute intervals (`5m`).
```js
from(bucket:"telegraf/autogen")
|> range(start: -1h)
|> filter(fn: (r) =>
r._measurement == "cpu" and
r._field == "usage_system" and
r.cpu == "cpu-total"
)
|> window(every: 5m)
```
As data is gathered into windows of time, each window is output as its own table.
When visualized, each table is assigned a unique color.
![Windowed data tables](/img/flux/flux-windowed-data.png)
## Aggregate windowed data
Flux aggregate functions take the `_value`s in each table and aggregate them in some way.
Use the [`mean()` function](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/transformations/aggregates/mean) to average the `_value`s of each table.
```js
from(bucket:"telegraf/autogen")
|> range(start: -1h)
|> filter(fn: (r) =>
r._measurement == "cpu" and
r._field == "usage_system" and
r.cpu == "cpu-total"
)
|> window(every: 5m)
|> mean()
```
As rows in each window are aggregated, their output table contains only a single row with the aggregate value.
Windowed tables are all still separate and, when visualized, will appear as single, unconnected points.
![Windowed aggregate data](/img/flux/flux-windowed-aggregates.png)
## Add times to your aggregates
As values are aggregated, the resulting tables do not have a `_time` column because
the records used for the aggregation all have different timestamps.
Aggregate functions don't infer what time should be used for the aggregate value.
Therefore the `_time` column is dropped.
A `_time` column is required in the [next operation](#unwindow-aggregate-tables).
To add one, use the [`duplicate()` function](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/transformations/duplicate)
to duplicate the `_stop` column as the `_time` column for each windowed table.
```js
from(bucket:"telegraf/autogen")
|> range(start: -1h)
|> filter(fn: (r) =>
r._measurement == "cpu" and
r._field == "usage_system" and
r.cpu == "cpu-total"
)
|> window(every: 5m)
|> mean()
|> duplicate(column: "_stop", as: "_time")
```
## Unwindow aggregate tables
Use the `window()` function with the `every: inf` parameter to gather all points
into a single, infinite window.
```js
from(bucket:"telegraf/autogen")
|> range(start: -1h)
|> filter(fn: (r) =>
r._measurement == "cpu" and
r._field == "usage_system" and
r.cpu == "cpu-total"
)
|> window(every: 5m)
|> mean()
|> duplicate(column: "_stop", as: "_time")
|> window(every: inf)
```
Once ungrouped and combined into a single table, the aggregate data points will appear connected in your visualization.
![Unwindowed aggregate data](/img/flux/flux-windowed-aggregates-ungrouped.png)
## Helper functions
This may seem like a lot of coding just to build a query that aggregates data, however going through the
process helps to understand how data changes "shape" as it is passed through each function.
Flux provides (and allows you to create) "helper" functions that abstract many of these steps.
The same operation performed in this guide can be accomplished using the
[`aggregateWindow()` function](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/transformations/aggregates/aggregatewindow).
```js
from(bucket:"telegraf/autogen")
|> range(start: -1h)
|> filter(fn: (r) =>
r._measurement == "cpu" and
r._field == "usage_system" and
r.cpu == "cpu-total"
)
|> aggregateWindow(every: 5m, fn: mean)
```
## Congratulations!
You have now constructed a Flux query that uses Flux functions to transform your data.
There are many more ways to manipulate your data using both Flux's primitive functions
and your own custom functions, but this is a good introduction into the basic syntax and query structure.
---
_For a deeper dive into windowing and aggregating data with example data output for each transformation,
view the [Windowing and aggregating data](/influxdb/v1.8/flux/guides/window-aggregate) guide._
---
<div class="page-nav-btns">
<a class="btn prev" href="/influxdb/v1.8/flux/get-started/query-influxdb/">Query InfluxDB</a>
<a class="btn next" href="/influxdb/v1.8/flux/get-started/syntax-basics/">Syntax basics</a>
</div>

35
content/influxdb/v1.8/flux/guides/_index.md Normal file
View File

@ -0,0 +1,35 @@
---
title: Query data with Flux
description: Guides that walk through both common and complex queries and use cases for Flux.
weight: 3
menu:
influxdb_1_8:
name: Query with Flux
parent: Flux
---
The following guides walk through both common and complex queries and use cases for Flux.
{{% note %}}
#### Example data variable
Many of the examples provided in the following guides use a `data` variable,
which represents a basic query that filters data by measurement and field.
`data` is defined as:
```js
data = from(bucket: "db/rp")
|> range(start: -1h)
|> filter(fn: (r) =>
r._measurement == "example-measurement" and
r._field == "example-field"
)
```
{{% /note %}}
## Flux query guides
{{< children type="anchored-list" pages="all" >}}
---
{{< children pages="all" readmore="true" hr="true" >}}

208
content/influxdb/v1.8/flux/guides/calculate-percentages.md Normal file
View File

@ -0,0 +1,208 @@
---
title: Calculate percentages with Flux
list_title: Calculate percentages
description: >
Use `pivot()` or `join()` and the `map()` function to align operand values into rows and calculate a percentage.
menu:
influxdb_1_8:
name: Calculate percentages
identifier: flux-calc-perc
parent: Query with Flux
weight: 6
list_query_example: percentages
---
Calculating percentages from queried data is a common use case for time series data.
To calculate a percentage in Flux, operands must be in each row.
Use `map()` to re-map values in the row and calculate a percentage.
**To calculate percentages**
1. Use [`from()`](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/inputs/from/),
[`range()`](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/transformations/range/) and
[`filter()`](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/transformations/filter/) to query operands.
2. Use [`pivot()` or `join()`](/influxdb/v1.8/flux/guides/mathematic-operations/#pivot-vs-join)
to align operand values into rows.
3. Use [`map()`](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/transformations/map/)
to divide the numerator operand value by the denominator operand value and multiply by 100.
{{% note %}}
The following examples use `pivot()` to align operands into rows because
`pivot()` works in most cases and is more performant than `join()`.
_See [Pivot vs join](/influxdb/v1.8/flux/guides/mathematic-operations/#pivot-vs-join)._
{{% /note %}}
```js
from(bucket: "db/rp")
|> range(start: -1h)
|> filter(fn: (r) => r._measurement == "m1" and r._field =~ /field[1-2]/ )
|> pivot(rowKey:["_time"], columnKey: ["_field"], valueColumn: "_value")
|> map(fn: (r) => ({ r with _value: r.field1 / r.field2 * 100.0 }))
```
## GPU monitoring example
The following example queries data from the gpu-monitor bucket and calculates the
percentage of GPU memory used over time.
Data includes the following:
- **`gpu` measurement**
- **`mem_used` field**: used GPU memory in bytes
- **`mem_total` field**: total GPU memory in bytes
### Query mem_used and mem_total fields
```js
from(bucket: "gpu-monitor")
|> range(start: 2020-01-01T00:00:00Z)
|> filter(fn: (r) => r._measurement == "gpu" and r._field =~ /mem_/)
```
###### Returns the following stream of tables:
| _time | _measurement | _field | _value |
|:----- |:------------:|:------: | ------: |
| 2020-01-01T00:00:00Z | gpu | mem_used | 2517924577 |
| 2020-01-01T00:00:10Z | gpu | mem_used | 2695091978 |
| 2020-01-01T00:00:20Z | gpu | mem_used | 2576980377 |
| 2020-01-01T00:00:30Z | gpu | mem_used | 3006477107 |
| 2020-01-01T00:00:40Z | gpu | mem_used | 3543348019 |
| 2020-01-01T00:00:50Z | gpu | mem_used | 4402341478 |
<p style="margin:-2.5rem 0;"></p>
| _time | _measurement | _field | _value |
|:----- |:------------:|:------: | ------: |
| 2020-01-01T00:00:00Z | gpu | mem_total | 8589934592 |
| 2020-01-01T00:00:10Z | gpu | mem_total | 8589934592 |
| 2020-01-01T00:00:20Z | gpu | mem_total | 8589934592 |
| 2020-01-01T00:00:30Z | gpu | mem_total | 8589934592 |
| 2020-01-01T00:00:40Z | gpu | mem_total | 8589934592 |
| 2020-01-01T00:00:50Z | gpu | mem_total | 8589934592 |
### Pivot fields into columns
Use `pivot()` to pivot the `mem_used` and `mem_total` fields into columns.
Output includes `mem_used` and `mem_total` columns with values for each corresponding `_time`.
```js
// ...
|> pivot(rowKey:["_time"], columnKey: ["_field"], valueColumn: "_value")
```
###### Returns the following:
| _time | _measurement | mem_used | mem_total |
|:----- |:------------:| --------: | ---------: |
| 2020-01-01T00:00:00Z | gpu | 2517924577 | 8589934592 |
| 2020-01-01T00:00:10Z | gpu | 2695091978 | 8589934592 |
| 2020-01-01T00:00:20Z | gpu | 2576980377 | 8589934592 |
| 2020-01-01T00:00:30Z | gpu | 3006477107 | 8589934592 |
| 2020-01-01T00:00:40Z | gpu | 3543348019 | 8589934592 |
| 2020-01-01T00:00:50Z | gpu | 4402341478 | 8589934592 |
### Map new values
Each row now contains the values necessary to calculate a percentage.
Use `map()` to re-map values in each row.
Divide `mem_used` by `mem_total` and multiply by 100 to return the percentage.
{{% note %}}
To return a precise float percentage value that includes decimal points, the example
below casts integer field values to floats and multiplies by a float value (`100.0`).
{{% /note %}}
```js
// ...
|> map(fn: (r) => ({
_time: r._time,
_measurement: r._measurement,
_field: "mem_used_percent",
_value: float(v: r.mem_used) / float(v: r.mem_total) * 100.0
}))
```
##### Query results:
| _time | _measurement | _field | _value |
|:----- |:------------:|:------: | ------: |
| 2020-01-01T00:00:00Z | gpu | mem_used_percent | 29.31 |
| 2020-01-01T00:00:10Z | gpu | mem_used_percent | 31.37 |
| 2020-01-01T00:00:20Z | gpu | mem_used_percent | 30.00 |
| 2020-01-01T00:00:30Z | gpu | mem_used_percent | 35.00 |
| 2020-01-01T00:00:40Z | gpu | mem_used_percent | 41.25 |
| 2020-01-01T00:00:50Z | gpu | mem_used_percent | 51.25 |
### Full query
```js
from(bucket: "gpu-monitor")
|> range(start: 2020-01-01T00:00:00Z)
|> filter(fn: (r) => r._measurement == "gpu" and r._field =~ /mem_/ )
|> pivot(rowKey:["_time"], columnKey: ["_field"], valueColumn: "_value")
|> map(fn: (r) => ({
_time: r._time,
_measurement: r._measurement,
_field: "mem_used_percent",
_value: float(v: r.mem_used) / float(v: r.mem_total) * 100.0
}))
```
## Examples
#### Calculate percentages using multiple fields
```js
from(bucket: "db/rp")
|> range(start: -1h)
|> filter(fn: (r) => r._measurement == "example-measurement")
|> filter(fn: (r) =>
r._field == "used_system" or
r._field == "used_user" or
r._field == "total"
)
|> pivot(rowKey:["_time"], columnKey: ["_field"], valueColumn: "_value")
|> map(fn: (r) => ({ r with
_value: float(v: r.used_system + r.used_user) / float(v: r.total) * 100.0
}))
```
#### Calculate percentages using multiple measurements
1. Ensure measurements are in the same [bucket](/influxdb/v1.8/flux/get-started/#buckets).
2. Use `filter()` to include data from both measurements.
3. Use `group()` to ungroup data and return a single table.
4. Use `pivot()` to pivot fields into columns.
5. Use `map()` to re-map rows and perform the percentage calculation.
<!-- -->
```js
from(bucket: "db/rp")
|> range(start: -1h)
|> filter(fn: (r) =>
(r._measurement == "m1" or r._measurement == "m2") and
(r._field == "field1" or r._field == "field2")
)
|> group()
|> pivot(rowKey:["_time"], columnKey: ["_field"], valueColumn: "_value")
|> map(fn: (r) => ({ r with _value: r.field1 / r.field2 * 100.0 }))
```
#### Calculate percentages using multiple data sources
```js
import "sql"
import "influxdata/influxdb/secrets"
pgUser = secrets.get(key: "POSTGRES_USER")
pgPass = secrets.get(key: "POSTGRES_PASSWORD")
pgHost = secrets.get(key: "POSTGRES_HOST")
t1 = sql.from(
driverName: "postgres",
dataSourceName: "postgresql://${pgUser}:${pgPass}@${pgHost}",
query:"SELECT id, name, available FROM exampleTable"
)
t2 = from(bucket: "db/rp")
|> range(start: -1h)
|> filter(fn: (r) =>
r._measurement == "example-measurement" and
r._field == "example-field"
)
join(tables: {t1: t1, t2: t2}, on: ["id"])
|> map(fn: (r) => ({ r with _value: r._value_t2 / r.available_t1 * 100.0 }))
```

195
content/influxdb/v1.8/flux/guides/conditional-logic.md Normal file
View File

@ -0,0 +1,195 @@
---
title: Query using conditional logic
seotitle: Query using conditional logic in Flux
list_title: Conditional logic
description: >
This guide describes how to use Flux conditional expressions, such as `if`,
`else`, and `then`, to query and transform data. **Flux evaluates statements from left to right and stops evaluating once a condition matches.**
menu:
influxdb_1_8:
name: Conditional logic
parent: Query with Flux
weight: 20
list_code_example: |
```js
if color == "green" then "008000" else "ffffff"
```
---
Flux provides `if`, `then`, and `else` conditional expressions that allow for powerful and flexible Flux queries.
##### Conditional expression syntax
```js
// Pattern
if <condition> then <action> else <alternative-action>
// Example
if color == "green" then "008000" else "ffffff"
```
Conditional expressions are most useful in the following contexts:
- When defining variables.
- When using functions that operate on a single row at a time (
[`filter()`](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/transformations/filter/),
[`map()`](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/transformations/map/),
[`reduce()`](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/transformations/aggregates/reduce) ).
## Evaluating conditional expressions
Flux evaluates statements in order and stops evaluating once a condition matches.
For example, given the following statement:
```js
if r._value > 95.0000001 and r._value <= 100.0 then "critical"
else if r._value > 85.0000001 and r._value <= 95.0 then "warning"
else if r._value > 70.0000001 and r._value <= 85.0 then "high"
else "normal"
```
When `r._value` is 96, the output is "critical" and the remaining conditions are not evaluated.
## Examples
- [Conditionally set the value of a variable](#conditionally-set-the-value-of-a-variable)
- [Create conditional filters](#create-conditional-filters)
- [Conditionally transform column values with map()](#conditionally-transform-column-values-with-map)
- [Conditionally increment a count with reduce()](#conditionally-increment-a-count-with-reduce)
### Conditionally set the value of a variable
The following example sets the `overdue` variable based on the
`dueDate` variable's relation to `now()`.
```js
dueDate = 2019-05-01
overdue = if dueDate < now() then true else false
```
### Create conditional filters
The following example uses an example `metric` variable to change how the query filters data.
`metric` has three possible values:
- Memory
- CPU
- Disk
```js
metric = "Memory"
from(bucket: "telegraf/autogen")
|> range(start: -1h)
|> filter(fn: (r) =>
if v.metric == "Memory"
then r._measurement == "mem" and r._field == "used_percent"
else if v.metric == "CPU"
then r._measurement == "cpu" and r._field == "usage_user"
else if v.metric == "Disk"
then r._measurement == "disk" and r._field == "used_percent"
else r._measurement != ""
)
```
### Conditionally transform column values with map()
The following example uses the [`map()` function](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/transformations/map/)
to conditionally transform column values.
It sets the `level` column to a specific string based on `_value` column.
{{< code-tabs-wrapper >}}
{{% code-tabs %}}
[No Comments](#)
[Comments](#)
{{% /code-tabs %}}
{{% code-tab-content %}}
```js
from(bucket: "telegraf/autogen")
|> range(start: -5m)
|> filter(fn: (r) => r._measurement == "mem" and r._field == "used_percent" )
|> map(fn: (r) => ({
r with
level:
if r._value >= 95.0000001 and r._value <= 100.0 then "critical"
else if r._value >= 85.0000001 and r._value <= 95.0 then "warning"
else if r._value >= 70.0000001 and r._value <= 85.0 then "high"
else "normal"
})
)
```
{{% /code-tab-content %}}
{{% code-tab-content %}}
```js
from(bucket: "telegraf/autogen")
|> range(start: -5m)
|> filter(fn: (r) => r._measurement == "mem" and r._field == "used_percent" )
|> map(fn: (r) => ({
// Retain all existing columns in the mapped row
r with
// Set the level column value based on the _value column
level:
if r._value >= 95.0000001 and r._value <= 100.0 then "critical"
else if r._value >= 85.0000001 and r._value <= 95.0 then "warning"
else if r._value >= 70.0000001 and r._value <= 85.0 then "high"
else "normal"
})
)
```
{{% /code-tab-content %}}
{{< /code-tabs-wrapper >}}
### Conditionally increment a count with reduce()
The following example uses the [`aggregateWindow()`](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/transformations/aggregates/aggregatewindow/)
and [`reduce()`](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/transformations/aggregates/reduce/)
functions to count the number of records in every five minute window that exceed a defined threshold.
{{< code-tabs-wrapper >}}
{{% code-tabs %}}
[No Comments](#)
[Comments](#)
{{% /code-tabs %}}
{{% code-tab-content %}}
```js
threshold = 65.0
from(bucket: "telegraf/autogen")
|> range(start: -1h)
|> filter(fn: (r) => r._measurement == "mem" and r._field == "used_percent" )
|> aggregateWindow(
every: 5m,
fn: (column, tables=<-) => tables |> reduce(
identity: {above_threshold_count: 0.0},
fn: (r, accumulator) => ({
above_threshold_count:
if r._value >= threshold then accumulator.above_threshold_count + 1.0
else accumulator.above_threshold_count + 0.0
})
)
)
```
{{% /code-tab-content %}}
{{% code-tab-content %}}
```js
threshold = 65.0
from(bucket: "telegraf/autogen")
|> range(start: -1h)
|> filter(fn: (r) => r._measurement == "mem" and r._field == "used_percent" )
// Aggregate data into 5 minute windows using a custom reduce() function
|> aggregateWindow(
every: 5m,
// Use a custom function in the fn parameter.
// The aggregateWindow fn parameter requires 'column' and 'tables' parameters.
fn: (column, tables=<-) => tables |> reduce(
identity: {above_threshold_count: 0.0},
fn: (r, accumulator) => ({
// Conditionally increment above_threshold_count if
// r.value exceeds the threshold
above_threshold_count:
if r._value >= threshold then accumulator.above_threshold_count + 1.0
else accumulator.above_threshold_count + 0.0
})
)
)
```
{{% /code-tab-content %}}
{{< /code-tabs-wrapper >}}

67
content/influxdb/v1.8/flux/guides/cumulativesum.md Normal file
View File

@ -0,0 +1,67 @@
---
title: Query cumulative sum
seotitle: Query cumulative sum in Flux
list_title: Cumulative sum
description: >
Use the `cumulativeSum()` function to calculate a running total of values.
weight: 10
menu:
influxdb_1_8:
parent: Query with Flux
name: Cumulative sum
list_query_example: cumulative_sum
---
Use the [`cumulativeSum()` function](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/transformations/cumulativesum/)
to calculate a running total of values.
`cumulativeSum` sums the values of subsequent records and returns each row updated with the summed total.
{{< flex >}}
{{% flex-content "half" %}}
**Given the following input table:**
| _time | _value |
| ----- |:------:|
| 0001 | 1 |
| 0002 | 2 |
| 0003 | 1 |
| 0004 | 3 |
{{% /flex-content %}}
{{% flex-content "half" %}}
**`cumulativeSum()` returns:**
| _time | _value |
| ----- |:------:|
| 0001 | 1 |
| 0002 | 3 |
| 0003 | 4 |
| 0004 | 7 |
{{% /flex-content %}}
{{< /flex >}}
{{% note %}}
The examples below use the [example data variable](/influxdb/v1.8/flux/guides/#example-data-variable).
{{% /note %}}
##### Calculate the running total of values
```js
data
|> cumulativeSum()
```
## Use cumulativeSum() with aggregateWindow()
[`aggregateWindow()`](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/transformations/aggregates/aggregatewindow/)
segments data into windows of time, aggregates data in each window into a single
point, then removes the time-based segmentation.
It is primarily used to downsample data.
`aggregateWindow()` expects an aggregate function that returns a single row for each time window.
To use `cumulativeSum()` with `aggregateWindow`, use `sum` in `aggregateWindow()`,
then calculate the running total of the aggregate values with `cumulativeSum()`.
<!-- -->
```js
data
|> aggregateWindow(every: 5m, fn: sum)
|> cumulativeSum()
```

141
content/influxdb/v1.8/flux/guides/execute-queries.md Normal file
View File

@ -0,0 +1,141 @@
---
title: Execute Flux queries
description: Use the InfluxDB CLI, API, and the Chronograf Data Explorer to execute Flux queries.
menu:
influxdb_1_8:
name: Execute Flux queries
parent: Query with Flux
weight: 1
aliases:
- /influxdb/v1.8/flux/guides/executing-queries/
---
There are multiple ways to execute Flux queries with InfluxDB and Chronograf v1.8+.
This guide covers the different options:
1. [Chronograf's Data Explorer](#chronograf-s-data-explorer)
2. [Influx CLI](#influx-cli)
3. [InfluxDB API](#influxdb-api)
> Before attempting these methods, make sure Flux is enabled by setting
> `flux-enabled = true` in the `[http]` section of your InfluxDB configuration file.
## Chronograf's Data Explorer
Chronograf v1.8+ supports Flux in its Data Explorer.
Flux queries can be built, executed, and visualized from within the Chronograf user interface.
## Influx CLI
InfluxDB v1.8+'s `influx` CLI includes a `-type` option which allows you specify
what type of interactive session to start.
`-type=flux` will start an interactive read-eval-print-loop (REPL) that supports Flux.
{{% note %}}
If [authentication is enabled](/influxdb/latest/administration/authentication_and_authorization)
on your InfluxDB instance, use the `-username` flag to provide your InfluxDB username and
the `-password` flag to provide your password.
{{% /note %}}
##### Enter an interactive Flux REPL
{{< code-tabs-wrapper >}}
{{% code-tabs %}}
[No Auth](#)
[Auth Enabled](#)
{{% /code-tabs %}}
{{% code-tab-content %}}
```bash
influx -type=flux
```
{{% /code-tab-content %}}
{{% code-tab-content %}}
```bash
influx -type=flux -username myuser -password PasSw0rd
```
{{% /code-tab-content %}}
{{< /code-tabs-wrapper >}}
Any Flux query can be executed within the REPL.
### Submit a Flux query via parameter
Flux queries can also be passed to the Flux REPL as a parameter using the `influx` CLI's `-type=flux` option and the `-execute` parameter.
The accompanying string is executed as a Flux query and results are output in your terminal.
{{< code-tabs-wrapper >}}
{{% code-tabs %}}
[No Auth](#)
[Auth Enabled](#)
{{% /code-tabs %}}
{{% code-tab-content %}}
```bash
influx -type=flux -execute '<flux query>'
```
{{% /code-tab-content %}}
{{% code-tab-content %}}
```bash
influx -type=flux -username myuser -password PasSw0rd -execute '<flux query>'
```
{{% /code-tab-content %}}
{{< /code-tabs-wrapper >}}
### Submit a Flux query via via STDIN
Flux queries an be piped into the `influx` CLI via STDIN.
Query results are otuput in your terminal.
{{< code-tabs-wrapper >}}
{{% code-tabs %}}
[No Auth](#)
[Auth Enabled](#)
{{% /code-tabs %}}
{{% code-tab-content %}}
```bash
echo '<flux query>' | influx -type=flux
```
{{% /code-tab-content %}}
{{% code-tab-content %}}
```bash
echo '<flux query>' | influx -type=flux -username myuser -password PasSw0rd
```
{{% /code-tab-content %}}
{{< /code-tabs-wrapper >}}
## InfluxDB API
Flux can be used to query InfluxDB through InfluxDB's `/api/v2/query` endpoint.
Queried data is returned in annotated CSV format.
In your request, set the following:
- `Accept` header to `application/csv`
- `Content-type` header to `application/vnd.flux`
- If [authentication is enabled](/influxdb/latest/administration/authentication_and_authorization)
on your InfluxDB instance, `Authorization` header to `Token <username>:<password>`
This allows you to POST the Flux query in plain text and receive the annotated CSV response.
Below is an example `curl` command that queries InfluxDB using Flux:
{{< code-tabs-wrapper >}}
{{% code-tabs %}}
[No Auth](#)
[Auth Enabled](#)
{{% /code-tabs %}}
{{% code-tab-content %}}
```bash
curl -XPOST localhost:8086/api/v2/query -sS \
-H 'Accept:application/csv' \
-H 'Content-type:application/vnd.flux' \
-d 'from(bucket:"telegraf")
|> range(start:-5m)
|> filter(fn:(r) => r._measurement == "cpu")'
```
{{% /code-tab-content %}}
{{% code-tab-content %}}
```bash
curl -XPOST localhost:8086/api/v2/query -sS \
-H 'Accept:application/csv' \
-H 'Content-type:application/vnd.flux' \
-H 'Authorization: Token <username>:<password>' \
-d 'from(bucket:"telegraf")
|> range(start:-5m)
|> filter(fn:(r) => r._measurement == "cpu")'
```
{{% /code-tab-content %}}
{{< /code-tabs-wrapper >}}

80
content/influxdb/v1.8/flux/guides/exists.md Normal file
View File

@ -0,0 +1,80 @@
---
title: Check if a value exists
seotitle: Use Flux to check if a value exists
list_title: Exists
description: >
Use the Flux `exists` operator to check if an object contains a key or if that
key's value is `null`.
menu:
influxdb_1_8:
name: Exists
parent: Query with Flux
weight: 20
list_code_example: |
##### Filter null values
```js
data
|> filter(fn: (r) => exists r._value)
```
---
Use the Flux `exists` operator to check if an object contains a key or if that
key's value is `null`.
```js
p = {firstName: "John", lastName: "Doe", age: 42}
exists p.firstName
// Returns true
exists p.height
// Returns false
```
If you're just getting started with Flux queries, check out the following:
- [Get started with Flux](/influxdb/v1.8/flux/get-started/) for a conceptual overview of Flux and parts of a Flux query.
- [Execute queries](/influxdb/v1.8/flux/guides/execute-queries/) to discover a variety of ways to run your queries.
Use `exists` with row functions (
[`filter()`](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/transformations/filter/),
[`map()`](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/transformations/map/),
[`reduce()`](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/transformations/aggregates/reduce/))
to check if a row includes a column or if the value for that column is `null`.
#### Filter null values
```js
from(bucket: "db/rp")
|> range(start: -5m)
|> filter(fn: (r) => exists r._value)
```
#### Map values based on existence
```js
from(bucket: "default")
|> range(start: -30s)
|> map(fn: (r) => ({
r with
human_readable:
if exists r._value then "${r._field} is ${string(v:r._value)}."
else "${r._field} has no value."
}))
```
#### Ignore null values in a custom aggregate function
```js
customSumProduct = (tables=<-) =>
tables
|> reduce(
identity: {sum: 0.0, product: 1.0},
fn: (r, accumulator) => ({
r with
sum:
if exists r._value then r._value + accumulator.sum
else accumulator.sum,
product:
if exists r._value then r.value * accumulator.product
else accumulator.product
})
)
```

110
content/influxdb/v1.8/flux/guides/fill.md Normal file
View File

@ -0,0 +1,110 @@
---
title: Fill null values in data
seotitle: Fill null values in data
list_title: Fill
description: >
Use the `fill()` function to replace _null_ values.
weight: 10
menu:
influxdb_1_8:
parent: Query with Flux
name: Fill
list_query_example: fill_null
---
Use the [`fill()` function](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/transformations/fill/)
to replace _null_ values with:
- [the previous non-null value](#fill-with-the-previous-value)
- [a specified value](#fill-with-a-specified-value)
<!-- -->
```js
data
|> fill(usePrevious: true)
// OR
data
|> fill(value: 0.0)
```
{{% note %}}
#### Fill empty windows of time
The `fill()` function **does not** fill empty windows of time.
It only replaces _null_ values in existing data.
Filling empty windows of time requires time interpolation
_(see [influxdata/flux#2428](https://github.com/influxdata/flux/issues/2428))_.
{{% /note %}}
## Fill with the previous value
To fill _null_ values with the previous **non-null** value, set the `usePrevious` parameter to `true`.
{{% note %}}
Values remain _null_ if there is no previous non-null value in the table.
{{% /note %}}
```js
data
|> fill(usePrevious: true)
```
{{< flex >}}
{{% flex-content %}}
**Given the following input:**
| _time | _value |
|:----- | ------:|
| 2020-01-01T00:01:00Z | null |
| 2020-01-01T00:02:00Z | 0.8 |
| 2020-01-01T00:03:00Z | null |
| 2020-01-01T00:04:00Z | null |
| 2020-01-01T00:05:00Z | 1.4 |
{{% /flex-content %}}
{{% flex-content %}}
**`fill(usePrevious: true)` returns:**
| _time | _value |
|:----- | ------:|
| 2020-01-01T00:01:00Z | null |
| 2020-01-01T00:02:00Z | 0.8 |
| 2020-01-01T00:03:00Z | 0.8 |
| 2020-01-01T00:04:00Z | 0.8 |
| 2020-01-01T00:05:00Z | 1.4 |
{{% /flex-content %}}
{{< /flex >}}
## Fill with a specified value
To fill _null_ values with a specified value, use the `value` parameter to specify the fill value.
_The fill value must match the [data type](/flux/v0.65/language/types/#basic-types)
of the [column](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/transformations/fill/#column)._
```js
data
|> fill(value: 0.0)
```
{{< flex >}}
{{% flex-content %}}
**Given the following input:**
| _time | _value |
|:----- | ------:|
| 2020-01-01T00:01:00Z | null |
| 2020-01-01T00:02:00Z | 0.8 |
| 2020-01-01T00:03:00Z | null |
| 2020-01-01T00:04:00Z | null |
| 2020-01-01T00:05:00Z | 1.4 |
{{% /flex-content %}}
{{% flex-content %}}
**`fill(value: 0.0)` returns:**
| _time | _value |
|:----- | ------:|
| 2020-01-01T00:01:00Z | 0.0 |
| 2020-01-01T00:02:00Z | 0.8 |
| 2020-01-01T00:03:00Z | 0.0 |
| 2020-01-01T00:04:00Z | 0.0 |
| 2020-01-01T00:05:00Z | 1.4 |
{{% /flex-content %}}
{{< /flex >}}

147
content/influxdb/v1.8/flux/guides/first-last.md Normal file
View File

@ -0,0 +1,147 @@
---
title: Query first and last values
seotitle: Query first and last values in Flux
list_title: First and last
description: >
Use the `first()` or `last()` functions to return the first or last point in an input table.
weight: 10
menu:
influxdb_1_8:
parent: Query with Flux
name: First & last
list_query_example: first_last
---
Use the [`first()`](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/transformations/selectors/first/) or
[`last()`](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/transformations/selectors/last/) functions
to return the first or last record in an input table.
```js
data
|> first()
// OR
data
|> last()
```
{{% note %}}
By default, InfluxDB returns results sorted by time, however you can use the
[`sort()` function](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/transformations/sort/)
to change how results are sorted.
`first()` and `last()` respect the sort order of input data and return records
based on the order they are received in.
{{% /note %}}
### first
`first()` returns the first non-null record in an input table.
{{< flex >}}
{{% flex-content %}}
**Given the following input:**
| _time | _value |
|:----- | ------:|
| 2020-01-01T00:01:00Z | 1.0 |
| 2020-01-01T00:02:00Z | 1.0 |
| 2020-01-01T00:03:00Z | 2.0 |
| 2020-01-01T00:04:00Z | 3.0 |
{{% /flex-content %}}
{{% flex-content %}}
**The following function returns:**
```js
|> first()
```
| _time | _value |
|:----- | ------:|
| 2020-01-01T00:01:00Z | 1.0 |
{{% /flex-content %}}
{{< /flex >}}
### last
`last()` returns the last non-null record in an input table.
{{< flex >}}
{{% flex-content %}}
**Given the following input:**
| _time | _value |
|:----- | ------:|
| 2020-01-01T00:01:00Z | 1.0 |
| 2020-01-01T00:02:00Z | 1.0 |
| 2020-01-01T00:03:00Z | 2.0 |
| 2020-01-01T00:04:00Z | 3.0 |
{{% /flex-content %}}
{{% flex-content %}}
**The following function returns:**
```js
|> last()
```
| _time | _value |
|:----- | ------:|
| 2020-01-01T00:04:00Z | 3.0 |
{{% /flex-content %}}
{{< /flex >}}
## Use first() or last() with aggregateWindow()
Use `first()` and `last()` with [`aggregateWindow()`](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/transformations/aggregates/aggregatewindow/)
to select the first or last records in time-based groups.
`aggregateWindow()` segments data into windows of time, aggregates data in each window into a single
point using aggregate or selector functions, and then removes the time-based segmentation.
{{< flex >}}
{{% flex-content %}}
**Given the following input:**
| _time | _value |
|:----- | ------:|
| 2020-01-01T00:00:00Z | 10 |
| 2020-01-01T00:00:15Z | 12 |
| 2020-01-01T00:00:45Z | 9 |
| 2020-01-01T00:01:05Z | 9 |
| 2020-01-01T00:01:10Z | 15 |
| 2020-01-01T00:02:30Z | 11 |
{{% /flex-content %}}
{{% flex-content %}}
**The following function returns:**
{{< code-tabs-wrapper >}}
{{% code-tabs %}}
[first](#)
[last](#)
{{% /code-tabs %}}
{{% code-tab-content %}}
```js
|> aggregateWindow(
every: 1h,
fn: first
)
```
| _time | _value |
|:----- | ------:|
| 2020-01-01T00:00:59Z | 10 |
| 2020-01-01T00:01:59Z | 9 |
| 2020-01-01T00:02:59Z | 11 |
{{% /code-tab-content %}}
{{% code-tab-content %}}
```js
|> aggregateWindow(
every: 1h,
fn: last
)
```
| _time | _value |
|:----- | ------:|
| 2020-01-01T00:00:59Z | 9 |
| 2020-01-01T00:01:59Z | 15 |
| 2020-01-01T00:02:59Z | 11 |
{{% /code-tab-content %}}
{{< /code-tabs-wrapper >}}
{{%/flex-content %}}
{{< /flex >}}

150
content/influxdb/v1.8/flux/guides/flux-in-dashboards.md Normal file
View File

@ -0,0 +1,150 @@
---
title: Use Flux in Chronograf dashboards
description: >
This guide walks through using Flux queries in Chronograf dashboard cells,
what template variables are available, and how to use them.
menu:
influxdb_1_8:
name: Use Flux in dashboards
parent: Query with Flux
weight: 30
---
[Chronograf](/chronograf/latest/) is the web user interface for managing for the
InfluxData platform that lest you create and customize dashboards that visualize your data.
Visualized data is retrieved using either an InfluxQL or Flux query.
This guide walks through using Flux queries in Chronograf dashboard cells.
## Using Flux in dashboard cells
---
_**Chronograf v1.8+** and **InfluxDB v1.8 with [Flux enabled](/influxdb/v1.8/flux/installation)**
are required to use Flux in dashboards._
---
To use Flux in a dashboard cell, either create a new cell or edit an existing cell
by clicking the **pencil** icon in the top right corner of the cell.
To the right of the **Source dropdown** above the graph preview, select **Flux** as the source type.
![Flux in Chronograf dashboard cells](/img/flux/flux-dashboard-cell.png)
> The Flux source type is only available if your data source has
> [Flux enabled](/influxdb/v1.8/flux/installation).
This will provide **Schema**, **Script**, and **Functions** panes.
### Schema pane
The Schema pane allows you to explore your data and add filters for specific
measurements, fields, and tags to your Flux script.
<img src="/img/flux/flux-dashboard-add-filter.png" style="max-width:316px" title="Add a filter from the Schema panel">
### Script pane
The Script pane is where you write your Flux script.
In its default state, the **Script** pane includes an optional [Script Wizard](/chronograf/v1.8/guides/querying-data/#explore-data-with-flux)
that uses selected options to build a Flux query for you.
The generated query includes all the relevant functions and [template variables](#template-variables-in-flux)
required to return your desired data.
### Functions pane
The Functions pane provides a list of functions available in your Flux queries.
Clicking on a function will add it to the end of the script in the Script pane.
Hovering over a function provides documentation for the function as well as links
to deep documentation.
### Dynamic sources
Chronograf can be configured with multiple data sources.
The **Sources dropdown** allows you to select a specific data source to connect to,
but a **Dynamic Source** options is also available.
With a dynamic source, the cell will query data from whatever data source to which
Chronograf is currently connected.
Connections are managed under Chronograf's **Configuration** tab.
### View raw data
As you're building your Flux scripts, each function processes or transforms your
data is ways specific to the function.
It can be helpful to view the actual data in order to see how it is being shaped.
The **View Raw Data** toggle above the data visualization switches between graphed
data and raw data shown in table form.
![View raw data](/img/flux/flux-dashboard-view-raw.png)
_The **View Raw Data** toggle is only available when using Flux._
## Template variables in Flux
Chronograf [template variables](/chronograf/latest/guides/dashboard-template-variables/)
allow you to alter specific components of cells’ queries using elements provided in the
Chronograf user interface.
In your Flux query, reference template variables just as you would reference defined Flux variables.
The following example uses Chronograf's [predefined template variables](#predefined-template-variables),
`dashboardTime`, `upperDashboardTime`, and `autoInterval`:
```js
from(bucket: "telegraf/autogen")
|> filter(fn: (r) => r._measurement == "cpu")
|> range(
start: dashboardTime,
stop: upperDashboardTime
)
window(every: autoInterval)
```
### Predefined template variables
#### dashboardTime
The `dashboardTime` template variable represents the lower time bound of ranged data.
It's value is controlled by the time dropdown in your dashboard.
It should be used to define the `start` parameter of the `range()` function.
```js
dataSet
|> range(
start: dashboardTime
)
```
#### upperDashboardTime
The `upperDashboardTime` template variable represents the upper time bound of ranged data.
It's value is modified by the time dropdown in your dashboard when using an absolute time range.
It should be used to define the `stop` parameter of the `range()` function.
```js
dataSet
|> range(
start: dashboardTime,
stop: upperDashboardTime
)
```
> As a best practice, always set the `stop` parameter of the `range()` function to `upperDashboardTime` in cell queries.
> Without it, `stop` defaults to "now" and the absolute upper range bound selected in the time dropdown is not honored,
> potentially causing unnecessary load on InfluxDB.
#### autoInterval
The `autoInterval` template variable represents the refresh interval of the dashboard
and is controlled by the refresh interval dropdown.
It's typically used to align window intervals created in
[windowing and aggregation](/influxdb/v1.8/flux/guides/window-aggregate) operations with dashboard refreshes.
```js
dataSet
|> range(
start: dashboardTime,
stop: upperDashboardTime
)
|> aggregateWindow(
every: autoInterval,
fn: mean
)
```
### Custom template variables
<dt>
Chronograf does not yet support the use of custom template variables in Flux queries.
</dt>
## Using Flux and InfluxQL
Within individual dashboard cells, the use of Flux and InfluxQL is mutually exclusive.
However, a dashboard may consist of different cells, each using Flux or InfluxQL.

89
content/influxdb/v1.8/flux/guides/geo/_index.md Normal file
View File

@ -0,0 +1,89 @@
---
title: Work with geo-temporal data
list_title: Geo-temporal data
description: >
Use the Flux Geo package to filter geo-temporal data and group by geographic location or track.
menu:
influxdb_1_8:
name: Geo-temporal data
parent: Query with Flux
weight: 20
list_code_example: |
```js
import "experimental/geo"
sampleGeoData
|> geo.filterRows(region: {lat: 30.04, lon: 31.23, radius: 200.0})
|> geo.groupByArea(newColumn: "geoArea", level: 5)
```
---
Use the [Flux Geo package](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/experimental/geo) to
filter geo-temporal data and group by geographic location or track.
{{% warn %}}
The Geo package is experimental and subject to change at any time.
By using it, you agree to the [risks of experimental functions](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/experimental/#use-experimental-functions-at-your-own-risk).
{{% /warn %}}
**To work with geo-temporal data:**
1. Import the `experimental/geo` package.
```js
import "experimental/geo"
```
2. Load geo-temporal data. _See below for [sample geo-temporal data](#sample-data)._
3. Do one or more of the following:
- [Shape data to work with the Geo package](#shape-data-to-work-with-the-geo-package)
- [Filter data by region](#filter-geo-temporal-data-by-region) (using strict or non-strict filters)
- [Group data by area or by track](#group-geo-temporal-data)
{{< children >}}
---
## Sample data
Many of the examples in this section use a `sampleGeoData` variable that represents
a sample set of geo-temporal data.
The [Bird Migration Sample Data](https://github.com/influxdata/influxdb2-sample-data/tree/master/bird-migration-data)
available on GitHub provides sample geo-temporal data that meets the
[requirements of the Flux Geo package](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/experimental/geo/#geo-schema-requirements).
### Load annotated CSV sample data
Use the [experimental `csv.from()` function](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/experimental/csv/from/)
to load the sample bird migration annotated CSV data from GitHub:
```js
import `experimental/csv`
sampleGeoData = csv.from(
url: "https://github.com/influxdata/influxdb2-sample-data/blob/master/bird-migration-data/bird-migration.csv"
)
```
{{% note %}}
`csv.from(url: ...)` downloads sample data each time you execute the query **(~1.3 MB)**.
If bandwidth is a concern, use the [`to()` function](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/outputs/to/)
to write the data to a bucket, and then query the bucket with [`from()`](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/inputs/from/).
{{% /note %}}
### Write sample data to InfluxDB with line protocol
Use `curl` and the `influx write` command to write bird migration line protocol to InfluxDB.
Replace `db/rp` with your destination bucket:
```sh
curl https://raw.githubusercontent.com/influxdata/influxdb2-sample-data/master/bird-migration-data/bird-migration.line --output ./tmp-data
influx write -b db/rp @./tmp-data
rm -f ./tmp-data
```
Use Flux to query the bird migration data and assign it to the `sampleGeoData` variable:
```js
sampleGeoData = from(bucket: "db/rp")
|> range(start: 2019-01-01T00:00:00Z, stop: 2019-12-31T23:59:59Z)
|> filter(fn: (r) => r._measurement == "migration")
```

130
content/influxdb/v1.8/flux/guides/geo/filter-by-region.md Normal file
View File

@ -0,0 +1,130 @@
---
title: Filter geo-temporal data by region
description: >
Use the `geo.filterRows` function to filter geo-temporal data by box-shaped, circular, or polygonal geographic regions.
menu:
influxdb_1_8:
name: Filter by region
parent: Geo-temporal data
weight: 302
related:
- /{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/experimental/geo/
- /{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/experimental/geo/filterrows/
list_code_example: |
```js
import "experimental/geo"
sampleGeoData
|> geo.filterRows(
region: {lat: 30.04, lon: 31.23, radius: 200.0},
strict: true
)
```
---
Use the [`geo.filterRows` function](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/experimental/geo/filterrows/)
to filter geo-temporal data by geographic region:
1. [Define a geographic region](#define-a-geographic-region)
2. [Use strict or non-strict filtering](#strict-and-non-strict-filtering)
The following example uses the [sample bird migration data](/influxdb/v1.8/flux/guides/geo/#sample-data)
and queries data points **within 200km of Cairo, Egypt**:
```js
import "experimental/geo"
sampleGeoData
|> geo.filterRows(
region: {lat: 30.04, lon: 31.23, radius: 200.0},
strict: true
)
```
## Define a geographic region
Many functions in the Geo package filter data based on geographic region.
Define a geographic region using one of the the following shapes:
- [box](#box)
- [circle](#circle)
- [polygon](#polygon)
### box
Define a box-shaped region by specifying an object containing the following properties:
- **minLat:** minimum latitude in decimal degrees (WGS 84) _(Float)_
- **maxLat:** maximum latitude in decimal degrees (WGS 84) _(Float)_
- **minLon:** minimum longitude in decimal degrees (WGS 84) _(Float)_
- **maxLon:** maximum longitude in decimal degrees (WGS 84) _(Float)_
##### Example box-shaped region
```js
{
minLat: 40.51757813,
maxLat: 40.86914063,
minLon: -73.65234375,
maxLon: -72.94921875
}
```
### circle
Define a circular region by specifying an object containing the following properties:
- **lat**: latitude of the circle center in decimal degrees (WGS 84) _(Float)_
- **lon**: longitude of the circle center in decimal degrees (WGS 84) _(Float)_
- **radius**: radius of the circle in kilometers (km) _(Float)_
##### Example circular region
```js
{
lat: 40.69335938,
lon: -73.30078125,
radius: 20.0
}
```
### polygon
Define a polygonal region with an object containing the latitude and longitude for
each point in the polygon:
- **points**: points that define the custom polygon _(Array of objects)_
Define each point with an object containing the following properties:
- **lat**: latitude in decimal degrees (WGS 84) _(Float)_
- **lon**: longitude in decimal degrees (WGS 84) _(Float)_
##### Example polygonal region
```js
{
points: [
{lat: 40.671659, lon: -73.936631},
{lat: 40.706543, lon: -73.749177},
{lat: 40.791333, lon: -73.880327}
]
}
```
## Strict and non-strict filtering
In most cases, the specified geographic region does not perfectly align with S2 grid cells.
- **Non-strict filtering** returns points that may be outside of the specified region but
inside S2 grid cells partially covered by the region.
- **Strict filtering** returns only points inside the specified region.
_Strict filtering is less performant, but more accurate than non-strict filtering._
<span class="key-geo-cell"></span> S2 grid cell
<span class="key-geo-region"></span> Filter region
<span class="key-geo-point"></span> Returned point
{{< flex >}}
{{% flex-content %}}
**Strict filtering**
{{< svg "/static/svgs/geo-strict.svg" >}}
{{% /flex-content %}}
{{% flex-content %}}
**Non-strict filtering**
{{< svg "/static/svgs/geo-non-strict.svg" >}}
{{% /flex-content %}}
{{< /flex >}}

74
content/influxdb/v1.8/flux/guides/geo/group-geo-data.md Normal file
View File

@ -0,0 +1,74 @@
---
title: Group geo-temporal data
description: >
Use the `geo.groupByArea()` to group geo-temporal data by area and `geo.asTracks()`
to group data into tracks or routes.
menu:
influxdb_1_8:
parent: Geo-temporal data
weight: 302
related:
- /{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/experimental/geo/
- /{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/experimental/geo/groupbyarea/
- /{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/experimental/geo/astracks/
list_code_example: |
```js
import "experimental/geo"
sampleGeoData
|> geo.groupByArea(newColumn: "geoArea", level: 5)
|> geo.asTracks(groupBy: ["id"],sortBy: ["_time"])
```
---
Use the `geo.groupByArea()` to group geo-temporal data by area and `geo.asTracks()`
to group data into tracks or routes.
- [Group data by area](#group-data-by-area)
- [Group data into tracks or routes](#group-data-by-track-or-route)
### Group data by area
Use the [`geo.groupByArea()` function](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/experimental/geo/groupbyarea/)
to group geo-temporal data points by geographic area.
Areas are determined by [S2 grid cells](https://s2geometry.io/devguide/s2cell_hierarchy.html#s2cellid-numbering)
- Specify a new column to store the unique area identifier for each point with the `newColumn` parameter.
- Specify the [S2 cell level](https://s2geometry.io/resources/s2cell_statistics)
to use when calculating geographic areas with the `level` parameter.
The following example uses the [sample bird migration data](/influxdb/v1.8/flux/guides/geo/#sample-data)
to query data points within 200km of Cairo, Egypt and group them by geographic area:
```js
import "experimental/geo"
sampleGeoData
|> geo.filterRows(region: {lat: 30.04, lon: 31.23, radius: 200.0})
|> geo.groupByArea(
newColumn: "geoArea",
level: 5
)
```
### Group data by track or route
Use [`geo.asTracks()` function](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/experimental/geo/astracks/)
to group data points into tracks or routes and order them by time or other columns.
Data must contain a unique identifier for each track. For example: `id` or `tid`.
- Specify columns that uniquely identify each track or route with the `groupBy` parameter.
- Specify which columns to sort by with the `sortBy` parameter. Default is `["_time"]`.
The following example uses the [sample bird migration data](/influxdb/v1.8/flux/guides/geo/#sample-data)
to query data points within 200km of Cairo, Egypt and group them into routes unique
to each bird:
```js
import "experimental/geo"
sampleGeoData
|> geo.filterRows(region: {lat: 30.04, lon: 31.23, radius: 200.0})
|> geo.asTracks(
groupBy: ["id"],
sortBy: ["_time"]
)
```

129
content/influxdb/v1.8/flux/guides/geo/shape-geo-data.md Normal file
View File

@ -0,0 +1,129 @@
---
title: Shape data to work with the Geo package
description: >
Functions in the Flux Geo package require **lat** and **lon** fields and an **s2_cell_id** tag.
Rename latitude and longitude fields and generate S2 cell ID tokens.
menu:
influxdb_1_8:
name: Shape geo-temporal data
parent: Geo-temporal data
weight: 301
related:
- /{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/experimental/geo/
- /{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/experimental/geo/shapedata/
- /{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/experimental/geo/s2cellidtoken/
list_code_example: |
```js
import "experimental/geo"
sampleGeoData
|> map(fn: (r) => ({ r with
_field:
if r._field == "latitude" then "lat"
else if r._field == "longitude" then "lon"
else r._field
}))
|> map(fn: (r) => ({ r with
s2_cell_id: geo.s2CellIDToken(point: {lon: r.lon, lat: r.lat}, level: 10)
}))
```
---
Functions in the Geo package require the following data schema:
- an **s2_cell_id** tag containing the [S2 Cell ID](https://s2geometry.io/devguide/s2cell_hierarchy.html#s2cellid-numbering)
**as a token**
- a **`lat` field** field containing the **latitude in decimal degrees** (WGS 84)
- a **`lon` field** field containing the **longitude in decimal degrees** (WGS 84)
## Shape geo-temporal data
If your data already contains latitude and longitude fields, use the
[`geo.shapeData()`function](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/experimental/geo/shapedata/)
to rename the fields to match the requirements of the Geo package, pivot the data
into row-wise sets, and generate S2 cell ID tokens for each point.
```js
import "experimental/geo"
from(bucket: "example-bucket")
|> range(start: -1h)
|> filter(fn: (r) => r._measurement == "example-measurement")
|> geo.shapeData(
latField: "latitude",
lonField: "longitude",
level: 10
)
```
## Generate S2 cell ID tokens
The Geo package uses the [S2 Geometry Library](https://s2geometry.io/) to represent
geographic coordinates on a three-dimensional sphere.
The sphere is divided into [cells](https://s2geometry.io/devguide/s2cell_hierarchy),
each with a unique 64-bit identifier (S2 cell ID).
Grid and S2 cell ID accuracy are defined by a [level](https://s2geometry.io/resources/s2cell_statistics).
{{% note %}}
To filter more quickly, use higher S2 Cell ID levels,
but know that that higher levels increase [series cardinality](/influxdb/v1.8/concepts/glossary/#series-cardinality).
{{% /note %}}
The Geo package requires S2 cell IDs as tokens.
To generate add S2 cell IDs tokens to your data, use one of the following options:
- [Generate S2 cell ID tokens with Telegraf](#generate-s2-cell-id-tokens-with-telegraf)
- [Generate S2 cell ID tokens language-specific libraries](#generate-s2-cell-id-tokens-language-specific-libraries)
- [Generate S2 cell ID tokens with Flux](#generate-s2-cell-id-tokens-with-flux)
### Generate S2 cell ID tokens with Telegraf
Enable the [Telegraf S2 Geo (`s2geo`) processor](https://github.com/influxdata/telegraf/tree/master/plugins/processors/s2geo)
to generate S2 cell ID tokens at a specified `cell_level` using `lat` and `lon` field values.
Add the `processors.s2geo` configuration to your Telegraf configuration file (`telegraf.conf`):
```toml
[[processors.s2geo]]
## The name of the lat and lon fields containing WGS-84 latitude and
## longitude in decimal degrees.
lat_field = "lat"
lon_field = "lon"
## New tag to create
tag_key = "s2_cell_id"
## Cell level (see https://s2geometry.io/resources/s2cell_statistics.html)
cell_level = 9
```
Telegraf stores the S2 cell ID token in the `s2_cell_id` tag.
### Generate S2 cell ID tokens language-specific libraries
Many programming languages offer S2 Libraries with methods for generating S2 cell ID tokens.
Use latitude and longitude with the `s2.CellID.ToToken` endpoint of the S2 Geometry
Library to generate `s2_cell_id` tags. For example:
- **Go:** [s2.CellID.ToToken()](https://godoc.org/github.com/golang/geo/s2#CellID.ToToken)
- **Python:** [s2sphere.CellId.to_token()](https://s2sphere.readthedocs.io/en/latest/api.html#s2sphere.CellId)
- **JavaScript:** [s2.cellid.toToken()](https://github.com/mapbox/node-s2/blob/master/API.md#cellidtotoken---string)
### Generate S2 cell ID tokens with Flux
Use the [`geo.s2CellIDToken()` function](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/experimental/geo/s2cellidtoken/)
with existing longitude (`lon`) and latitude (`lat`) field values to generate and add the S2 cell ID token.
First, use the [`geo.toRows()` function](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/experimental/geo/torows/)
to pivot **lat** and **lon** fields into row-wise sets:
```js
import "experimental/geo"
from(bucket: "example-bucket")
|> range(start: -1h)
|> filter(fn: (r) => r._measurement == "example-measurement")
|> geo.toRows()
|> map(fn: (r) => ({ r with
s2_cell_id: geo.s2CellIDToken(point: {lon: r.lon, lat: r.lat}, level: 10)
}))
```
{{% note %}}
The [`geo.shapeData()`function](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/experimental/geo/shapedata/)
generates S2 cell ID tokens as well.
{{% /note %}}

674
content/influxdb/v1.8/flux/guides/group-data.md Normal file
View File

@ -0,0 +1,674 @@
---
title: Group data in InfluxDB with Flux
list_title: Group
description: >
Use the `group()` function to group data with common values in specific columns.
menu:
influxdb_1_8:
name: Group
parent: Query with Flux
weight: 2
aliases:
- /influxdb/v1.8/flux/guides/grouping-data/
list_query_example: group
---
With Flux, you can group data by any column in your queried data set.
"Grouping" partitions data into tables in which each row shares a common value for specified columns.
This guide walks through grouping data in Flux and provides examples of how data is shaped in the process.
If you're just getting started with Flux queries, check out the following:
- [Get started with Flux](/influxdb/v1.8/flux/get-started/) for a conceptual overview of Flux and parts of a Flux query.
- [Execute queries](/influxdb/v1.8/flux/guides/execute-queries/) to discover a variety of ways to run your queries.
## Group keys
Every table has a **group key** – a list of columns which for which every row in the table has the same value.
###### Example group key
```js
[_start, _stop, _field, _measurement, host]
```
Grouping data in Flux is essentially defining the group key of output tables.
Understanding how modifying group keys shapes output data is key to successfully
grouping and transforming data into your desired output.
## group() Function
Flux's [`group()` function](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/transformations/group) defines the
group key for output tables, i.e. grouping records based on values for specific columns.
###### group() example
```js
dataStream
|> group(columns: ["cpu", "host"])
```
###### Resulting group key
```js
[cpu, host]
```
The `group()` function has the following parameters:
### columns
The list of columns to include or exclude (depending on the [mode](#mode)) in the grouping operation.
### mode
The method used to define the group and resulting group key.
Possible values include `by` and `except`.
## Example grouping operations
To illustrate how grouping works, define a `dataSet` variable that queries System
CPU usage from the `db/rp` bucket.
Filter the `cpu` tag so it only returns results for each numbered CPU core.
### Data set
CPU used by system operations for all numbered CPU cores.
It uses a regular expression to filter only numbered cores.
```js
dataSet = from(bucket: "db/rp")
|> range(start: -2m)
|> filter(fn: (r) =>
r._field == "usage_system" and
r.cpu =~ /cpu[0-9*]/
)
|> drop(columns: ["host"])
```
{{% note %}}
This example drops the `host` column from the returned data since the CPU data
is only tracked for a single host and it simplifies the output tables.
Don't drop the `host` column if monitoring multiple hosts.
{{% /note %}}
{{% truncate %}}
```
Table: keys: [_start, _stop, _field, _measurement, cpu]
_start:time _stop:time _field:string _measurement:string cpu:string _time:time _value:float
------------------------------ ------------------------------ ---------------------- ---------------------- ---------------------- ------------------------------ ----------------------------
2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z usage_system cpu cpu0 2018-11-05T21:34:00.000000000Z 7.892107892107892
2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z usage_system cpu cpu0 2018-11-05T21:34:10.000000000Z 7.2
2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z usage_system cpu cpu0 2018-11-05T21:34:20.000000000Z 7.4
2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z usage_system cpu cpu0 2018-11-05T21:34:30.000000000Z 5.5
2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z usage_system cpu cpu0 2018-11-05T21:34:40.000000000Z 7.4
2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z usage_system cpu cpu0 2018-11-05T21:34:50.000000000Z 7.5
2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z usage_system cpu cpu0 2018-11-05T21:35:00.000000000Z 10.3
2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z usage_system cpu cpu0 2018-11-05T21:35:10.000000000Z 9.2
2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z usage_system cpu cpu0 2018-11-05T21:35:20.000000000Z 8.4
2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z usage_system cpu cpu0 2018-11-05T21:35:30.000000000Z 8.5
2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z usage_system cpu cpu0 2018-11-05T21:35:40.000000000Z 8.6
2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z usage_system cpu cpu0 2018-11-05T21:35:50.000000000Z 10.2
2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z usage_system cpu cpu0 2018-11-05T21:36:00.000000000Z 10.6
Table: keys: [_start, _stop, _field, _measurement, cpu]
_start:time _stop:time _field:string _measurement:string cpu:string _time:time _value:float
------------------------------ ------------------------------ ---------------------- ---------------------- ---------------------- ------------------------------ ----------------------------
2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z usage_system cpu cpu1 2018-11-05T21:34:00.000000000Z 0.7992007992007992
2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z usage_system cpu cpu1 2018-11-05T21:34:10.000000000Z 0.7
2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z usage_system cpu cpu1 2018-11-05T21:34:20.000000000Z 0.7
2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z usage_system cpu cpu1 2018-11-05T21:34:30.000000000Z 0.4
2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z usage_system cpu cpu1 2018-11-05T21:34:40.000000000Z 0.7
2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z usage_system cpu cpu1 2018-11-05T21:34:50.000000000Z 0.7
2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z usage_system cpu cpu1 2018-11-05T21:35:00.000000000Z 1.4
2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z usage_system cpu cpu1 2018-11-05T21:35:10.000000000Z 1.2
2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z usage_system cpu cpu1 2018-11-05T21:35:20.000000000Z 0.8
2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z usage_system cpu cpu1 2018-11-05T21:35:30.000000000Z 0.8991008991008991
2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z usage_system cpu cpu1 2018-11-05T21:35:40.000000000Z 0.8008008008008008
2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z usage_system cpu cpu1 2018-11-05T21:35:50.000000000Z 0.999000999000999
2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z usage_system cpu cpu1 2018-11-05T21:36:00.000000000Z 1.1022044088176353
Table: keys: [_start, _stop, _field, _measurement, cpu]
_start:time _stop:time _field:string _measurement:string cpu:string _time:time _value:float
------------------------------ ------------------------------ ---------------------- ---------------------- ---------------------- ------------------------------ ----------------------------
2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z usage_system cpu cpu2 2018-11-05T21:34:00.000000000Z 4.1
2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z usage_system cpu cpu2 2018-11-05T21:34:10.000000000Z 3.6
2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z usage_system cpu cpu2 2018-11-05T21:34:20.000000000Z 3.5
2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z usage_system cpu cpu2 2018-11-05T21:34:30.000000000Z 2.6
2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z usage_system cpu cpu2 2018-11-05T21:34:40.000000000Z 4.5
2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z usage_system cpu cpu2 2018-11-05T21:34:50.000000000Z 4.895104895104895
2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z usage_system cpu cpu2 2018-11-05T21:35:00.000000000Z 6.906906906906907
2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z usage_system cpu cpu2 2018-11-05T21:35:10.000000000Z 5.7
2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z usage_system cpu cpu2 2018-11-05T21:35:20.000000000Z 5.1
2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z usage_system cpu cpu2 2018-11-05T21:35:30.000000000Z 4.7
2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z usage_system cpu cpu2 2018-11-05T21:35:40.000000000Z 5.1
2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z usage_system cpu cpu2 2018-11-05T21:35:50.000000000Z 5.9
2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z usage_system cpu cpu2 2018-11-05T21:36:00.000000000Z 6.4935064935064934
Table: keys: [_start, _stop, _field, _measurement, cpu]
_start:time _stop:time _field:string _measurement:string cpu:string _time:time _value:float
------------------------------ ------------------------------ ---------------------- ---------------------- ---------------------- ------------------------------ ----------------------------
2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z usage_system cpu cpu3 2018-11-05T21:34:00.000000000Z 0.5005005005005005
2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z usage_system cpu cpu3 2018-11-05T21:34:10.000000000Z 0.5
2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z usage_system cpu cpu3 2018-11-05T21:34:20.000000000Z 0.5
2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z usage_system cpu cpu3 2018-11-05T21:34:30.000000000Z 0.3
2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z usage_system cpu cpu3 2018-11-05T21:34:40.000000000Z 0.6
2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z usage_system cpu cpu3 2018-11-05T21:34:50.000000000Z 0.6
2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z usage_system cpu cpu3 2018-11-05T21:35:00.000000000Z 1.3986013986013985
2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z usage_system cpu cpu3 2018-11-05T21:35:10.000000000Z 0.9
2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z usage_system cpu cpu3 2018-11-05T21:35:20.000000000Z 0.5005005005005005
2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z usage_system cpu cpu3 2018-11-05T21:35:30.000000000Z 0.7
2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z usage_system cpu cpu3 2018-11-05T21:35:40.000000000Z 0.6
2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z usage_system cpu cpu3 2018-11-05T21:35:50.000000000Z 0.8
2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z usage_system cpu cpu3 2018-11-05T21:36:00.000000000Z 0.9
```
{{% /truncate %}}
**Note that the group key is output with each table: `Table: keys: <group-key>`.**
![Group example data set](/img/grouping-data-set.png)
### Group by CPU
Group the `dataSet` stream by the `cpu` column.
```js
dataSet
|> group(columns: ["cpu"])
```
This won't actually change the structure of the data since it already has `cpu`
in the group key and is therefore grouped by `cpu`.
However, notice that it does change the group key:
{{% truncate %}}
###### Group by CPU output tables
```
Table: keys: [cpu]
cpu:string _stop:time _time:time _value:float _field:string _measurement:string _start:time
---------------------- ------------------------------ ------------------------------ ---------------------------- ---------------------- ---------------------- ------------------------------
cpu0 2018-11-05T21:36:00.000000000Z 2018-11-05T21:34:00.000000000Z 7.892107892107892 usage_system cpu 2018-11-05T21:34:00.000000000Z
cpu0 2018-11-05T21:36:00.000000000Z 2018-11-05T21:34:10.000000000Z 7.2 usage_system cpu 2018-11-05T21:34:00.000000000Z
cpu0 2018-11-05T21:36:00.000000000Z 2018-11-05T21:34:20.000000000Z 7.4 usage_system cpu 2018-11-05T21:34:00.000000000Z
cpu0 2018-11-05T21:36:00.000000000Z 2018-11-05T21:34:30.000000000Z 5.5 usage_system cpu 2018-11-05T21:34:00.000000000Z
cpu0 2018-11-05T21:36:00.000000000Z 2018-11-05T21:34:40.000000000Z 7.4 usage_system cpu 2018-11-05T21:34:00.000000000Z
cpu0 2018-11-05T21:36:00.000000000Z 2018-11-05T21:34:50.000000000Z 7.5 usage_system cpu 2018-11-05T21:34:00.000000000Z
cpu0 2018-11-05T21:36:00.000000000Z 2018-11-05T21:35:00.000000000Z 10.3 usage_system cpu 2018-11-05T21:34:00.000000000Z
cpu0 2018-11-05T21:36:00.000000000Z 2018-11-05T21:35:10.000000000Z 9.2 usage_system cpu 2018-11-05T21:34:00.000000000Z
cpu0 2018-11-05T21:36:00.000000000Z 2018-11-05T21:35:20.000000000Z 8.4 usage_system cpu 2018-11-05T21:34:00.000000000Z
cpu0 2018-11-05T21:36:00.000000000Z 2018-11-05T21:35:30.000000000Z 8.5 usage_system cpu 2018-11-05T21:34:00.000000000Z
cpu0 2018-11-05T21:36:00.000000000Z 2018-11-05T21:35:40.000000000Z 8.6 usage_system cpu 2018-11-05T21:34:00.000000000Z
cpu0 2018-11-05T21:36:00.000000000Z 2018-11-05T21:35:50.000000000Z 10.2 usage_system cpu 2018-11-05T21:34:00.000000000Z
cpu0 2018-11-05T21:36:00.000000000Z 2018-11-05T21:36:00.000000000Z 10.6 usage_system cpu 2018-11-05T21:34:00.000000000Z
Table: keys: [cpu]
cpu:string _stop:time _time:time _value:float _field:string _measurement:string _start:time
---------------------- ------------------------------ ------------------------------ ---------------------------- ---------------------- ---------------------- ------------------------------
cpu1 2018-11-05T21:36:00.000000000Z 2018-11-05T21:34:00.000000000Z 0.7992007992007992 usage_system cpu 2018-11-05T21:34:00.000000000Z
cpu1 2018-11-05T21:36:00.000000000Z 2018-11-05T21:34:10.000000000Z 0.7 usage_system cpu 2018-11-05T21:34:00.000000000Z
cpu1 2018-11-05T21:36:00.000000000Z 2018-11-05T21:34:20.000000000Z 0.7 usage_system cpu 2018-11-05T21:34:00.000000000Z
cpu1 2018-11-05T21:36:00.000000000Z 2018-11-05T21:34:30.000000000Z 0.4 usage_system cpu 2018-11-05T21:34:00.000000000Z
cpu1 2018-11-05T21:36:00.000000000Z 2018-11-05T21:34:40.000000000Z 0.7 usage_system cpu 2018-11-05T21:34:00.000000000Z
cpu1 2018-11-05T21:36:00.000000000Z 2018-11-05T21:34:50.000000000Z 0.7 usage_system cpu 2018-11-05T21:34:00.000000000Z
cpu1 2018-11-05T21:36:00.000000000Z 2018-11-05T21:35:00.000000000Z 1.4 usage_system cpu 2018-11-05T21:34:00.000000000Z
cpu1 2018-11-05T21:36:00.000000000Z 2018-11-05T21:35:10.000000000Z 1.2 usage_system cpu 2018-11-05T21:34:00.000000000Z
cpu1 2018-11-05T21:36:00.000000000Z 2018-11-05T21:35:20.000000000Z 0.8 usage_system cpu 2018-11-05T21:34:00.000000000Z
cpu1 2018-11-05T21:36:00.000000000Z 2018-11-05T21:35:30.000000000Z 0.8991008991008991 usage_system cpu 2018-11-05T21:34:00.000000000Z
cpu1 2018-11-05T21:36:00.000000000Z 2018-11-05T21:35:40.000000000Z 0.8008008008008008 usage_system cpu 2018-11-05T21:34:00.000000000Z
cpu1 2018-11-05T21:36:00.000000000Z 2018-11-05T21:35:50.000000000Z 0.999000999000999 usage_system cpu 2018-11-05T21:34:00.000000000Z
cpu1 2018-11-05T21:36:00.000000000Z 2018-11-05T21:36:00.000000000Z 1.1022044088176353 usage_system cpu 2018-11-05T21:34:00.000000000Z
Table: keys: [cpu]
cpu:string _stop:time _time:time _value:float _field:string _measurement:string _start:time
---------------------- ------------------------------ ------------------------------ ---------------------------- ---------------------- ---------------------- ------------------------------
cpu2 2018-11-05T21:36:00.000000000Z 2018-11-05T21:34:00.000000000Z 4.1 usage_system cpu 2018-11-05T21:34:00.000000000Z
cpu2 2018-11-05T21:36:00.000000000Z 2018-11-05T21:34:10.000000000Z 3.6 usage_system cpu 2018-11-05T21:34:00.000000000Z
cpu2 2018-11-05T21:36:00.000000000Z 2018-11-05T21:34:20.000000000Z 3.5 usage_system cpu 2018-11-05T21:34:00.000000000Z
cpu2 2018-11-05T21:36:00.000000000Z 2018-11-05T21:34:30.000000000Z 2.6 usage_system cpu 2018-11-05T21:34:00.000000000Z
cpu2 2018-11-05T21:36:00.000000000Z 2018-11-05T21:34:40.000000000Z 4.5 usage_system cpu 2018-11-05T21:34:00.000000000Z
cpu2 2018-11-05T21:36:00.000000000Z 2018-11-05T21:34:50.000000000Z 4.895104895104895 usage_system cpu 2018-11-05T21:34:00.000000000Z
cpu2 2018-11-05T21:36:00.000000000Z 2018-11-05T21:35:00.000000000Z 6.906906906906907 usage_system cpu 2018-11-05T21:34:00.000000000Z
cpu2 2018-11-05T21:36:00.000000000Z 2018-11-05T21:35:10.000000000Z 5.7 usage_system cpu 2018-11-05T21:34:00.000000000Z
cpu2 2018-11-05T21:36:00.000000000Z 2018-11-05T21:35:20.000000000Z 5.1 usage_system cpu 2018-11-05T21:34:00.000000000Z
cpu2 2018-11-05T21:36:00.000000000Z 2018-11-05T21:35:30.000000000Z 4.7 usage_system cpu 2018-11-05T21:34:00.000000000Z
cpu2 2018-11-05T21:36:00.000000000Z 2018-11-05T21:35:40.000000000Z 5.1 usage_system cpu 2018-11-05T21:34:00.000000000Z
cpu2 2018-11-05T21:36:00.000000000Z 2018-11-05T21:35:50.000000000Z 5.9 usage_system cpu 2018-11-05T21:34:00.000000000Z
cpu2 2018-11-05T21:36:00.000000000Z 2018-11-05T21:36:00.000000000Z 6.4935064935064934 usage_system cpu 2018-11-05T21:34:00.000000000Z
Table: keys: [cpu]
cpu:string _stop:time _time:time _value:float _field:string _measurement:string _start:time
---------------------- ------------------------------ ------------------------------ ---------------------------- ---------------------- ---------------------- ------------------------------
cpu3 2018-11-05T21:36:00.000000000Z 2018-11-05T21:34:00.000000000Z 0.5005005005005005 usage_system cpu 2018-11-05T21:34:00.000000000Z
cpu3 2018-11-05T21:36:00.000000000Z 2018-11-05T21:34:10.000000000Z 0.5 usage_system cpu 2018-11-05T21:34:00.000000000Z
cpu3 2018-11-05T21:36:00.000000000Z 2018-11-05T21:34:20.000000000Z 0.5 usage_system cpu 2018-11-05T21:34:00.000000000Z
cpu3 2018-11-05T21:36:00.000000000Z 2018-11-05T21:34:30.000000000Z 0.3 usage_system cpu 2018-11-05T21:34:00.000000000Z
cpu3 2018-11-05T21:36:00.000000000Z 2018-11-05T21:34:40.000000000Z 0.6 usage_system cpu 2018-11-05T21:34:00.000000000Z
cpu3 2018-11-05T21:36:00.000000000Z 2018-11-05T21:34:50.000000000Z 0.6 usage_system cpu 2018-11-05T21:34:00.000000000Z
cpu3 2018-11-05T21:36:00.000000000Z 2018-11-05T21:35:00.000000000Z 1.3986013986013985 usage_system cpu 2018-11-05T21:34:00.000000000Z
cpu3 2018-11-05T21:36:00.000000000Z 2018-11-05T21:35:10.000000000Z 0.9 usage_system cpu 2018-11-05T21:34:00.000000000Z
cpu3 2018-11-05T21:36:00.000000000Z 2018-11-05T21:35:20.000000000Z 0.5005005005005005 usage_system cpu 2018-11-05T21:34:00.000000000Z
cpu3 2018-11-05T21:36:00.000000000Z 2018-11-05T21:35:30.000000000Z 0.7 usage_system cpu 2018-11-05T21:34:00.000000000Z
cpu3 2018-11-05T21:36:00.000000000Z 2018-11-05T21:35:40.000000000Z 0.6 usage_system cpu 2018-11-05T21:34:00.000000000Z
cpu3 2018-11-05T21:36:00.000000000Z 2018-11-05T21:35:50.000000000Z 0.8 usage_system cpu 2018-11-05T21:34:00.000000000Z
cpu3 2018-11-05T21:36:00.000000000Z 2018-11-05T21:36:00.000000000Z 0.9 usage_system cpu 2018-11-05T21:34:00.000000000Z
```
{{% /truncate %}}
The visualization remains the same.
![Group by CPU](/img/grouping-data-set.png)
### Group by time
Grouping data by the `_time` column is a good illustration of how grouping changes the structure of your data.
```js
dataSet
|> group(columns: ["_time"])
```
When grouping by `_time`, all records that share a common `_time` value are grouped into individual tables.
So each output table represents a single point in time.
{{% truncate %}}
###### Group by time output tables
```
Table: keys: [_time]
_time:time _start:time _stop:time _value:float _field:string _measurement:string cpu:string
------------------------------ ------------------------------ ------------------------------ ---------------------------- ---------------------- ---------------------- ----------------------
2018-11-05T21:34:00.000000000Z 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z 7.892107892107892 usage_system cpu cpu0
2018-11-05T21:34:00.000000000Z 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z 0.7992007992007992 usage_system cpu cpu1
2018-11-05T21:34:00.000000000Z 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z 4.1 usage_system cpu cpu2
2018-11-05T21:34:00.000000000Z 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z 0.5005005005005005 usage_system cpu cpu3
Table: keys: [_time]
_time:time _start:time _stop:time _value:float _field:string _measurement:string cpu:string
------------------------------ ------------------------------ ------------------------------ ---------------------------- ---------------------- ---------------------- ----------------------
2018-11-05T21:34:10.000000000Z 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z 7.2 usage_system cpu cpu0
2018-11-05T21:34:10.000000000Z 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z 0.7 usage_system cpu cpu1
2018-11-05T21:34:10.000000000Z 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z 3.6 usage_system cpu cpu2
2018-11-05T21:34:10.000000000Z 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z 0.5 usage_system cpu cpu3
Table: keys: [_time]
_time:time _start:time _stop:time _value:float _field:string _measurement:string cpu:string
------------------------------ ------------------------------ ------------------------------ ---------------------------- ---------------------- ---------------------- ----------------------
2018-11-05T21:34:20.000000000Z 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z 7.4 usage_system cpu cpu0
2018-11-05T21:34:20.000000000Z 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z 0.7 usage_system cpu cpu1
2018-11-05T21:34:20.000000000Z 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z 3.5 usage_system cpu cpu2
2018-11-05T21:34:20.000000000Z 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z 0.5 usage_system cpu cpu3
Table: keys: [_time]
_time:time _start:time _stop:time _value:float _field:string _measurement:string cpu:string
------------------------------ ------------------------------ ------------------------------ ---------------------------- ---------------------- ---------------------- ----------------------
2018-11-05T21:34:30.000000000Z 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z 5.5 usage_system cpu cpu0
2018-11-05T21:34:30.000000000Z 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z 0.4 usage_system cpu cpu1
2018-11-05T21:34:30.000000000Z 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z 2.6 usage_system cpu cpu2
2018-11-05T21:34:30.000000000Z 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z 0.3 usage_system cpu cpu3
Table: keys: [_time]
_time:time _start:time _stop:time _value:float _field:string _measurement:string cpu:string
------------------------------ ------------------------------ ------------------------------ ---------------------------- ---------------------- ---------------------- ----------------------
2018-11-05T21:34:40.000000000Z 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z 7.4 usage_system cpu cpu0
2018-11-05T21:34:40.000000000Z 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z 0.7 usage_system cpu cpu1
2018-11-05T21:34:40.000000000Z 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z 4.5 usage_system cpu cpu2
2018-11-05T21:34:40.000000000Z 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z 0.6 usage_system cpu cpu3
Table: keys: [_time]
_time:time _start:time _stop:time _value:float _field:string _measurement:string cpu:string
------------------------------ ------------------------------ ------------------------------ ---------------------------- ---------------------- ---------------------- ----------------------
2018-11-05T21:34:50.000000000Z 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z 7.5 usage_system cpu cpu0
2018-11-05T21:34:50.000000000Z 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z 0.7 usage_system cpu cpu1
2018-11-05T21:34:50.000000000Z 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z 4.895104895104895 usage_system cpu cpu2
2018-11-05T21:34:50.000000000Z 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z 0.6 usage_system cpu cpu3
Table: keys: [_time]
_time:time _start:time _stop:time _value:float _field:string _measurement:string cpu:string
------------------------------ ------------------------------ ------------------------------ ---------------------------- ---------------------- ---------------------- ----------------------
2018-11-05T21:35:00.000000000Z 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z 10.3 usage_system cpu cpu0
2018-11-05T21:35:00.000000000Z 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z 1.4 usage_system cpu cpu1
2018-11-05T21:35:00.000000000Z 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z 6.906906906906907 usage_system cpu cpu2
2018-11-05T21:35:00.000000000Z 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z 1.3986013986013985 usage_system cpu cpu3
Table: keys: [_time]
_time:time _start:time _stop:time _value:float _field:string _measurement:string cpu:string
------------------------------ ------------------------------ ------------------------------ ---------------------------- ---------------------- ---------------------- ----------------------
2018-11-05T21:35:10.000000000Z 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z 9.2 usage_system cpu cpu0
2018-11-05T21:35:10.000000000Z 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z 1.2 usage_system cpu cpu1
2018-11-05T21:35:10.000000000Z 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z 5.7 usage_system cpu cpu2
2018-11-05T21:35:10.000000000Z 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z 0.9 usage_system cpu cpu3
Table: keys: [_time]
_time:time _start:time _stop:time _value:float _field:string _measurement:string cpu:string
------------------------------ ------------------------------ ------------------------------ ---------------------------- ---------------------- ---------------------- ----------------------
2018-11-05T21:35:20.000000000Z 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z 8.4 usage_system cpu cpu0
2018-11-05T21:35:20.000000000Z 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z 0.8 usage_system cpu cpu1
2018-11-05T21:35:20.000000000Z 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z 5.1 usage_system cpu cpu2
2018-11-05T21:35:20.000000000Z 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z 0.5005005005005005 usage_system cpu cpu3
Table: keys: [_time]
_time:time _start:time _stop:time _value:float _field:string _measurement:string cpu:string
------------------------------ ------------------------------ ------------------------------ ---------------------------- ---------------------- ---------------------- ----------------------
2018-11-05T21:35:30.000000000Z 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z 8.5 usage_system cpu cpu0
2018-11-05T21:35:30.000000000Z 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z 0.8991008991008991 usage_system cpu cpu1
2018-11-05T21:35:30.000000000Z 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z 4.7 usage_system cpu cpu2
2018-11-05T21:35:30.000000000Z 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z 0.7 usage_system cpu cpu3
Table: keys: [_time]
_time:time _start:time _stop:time _value:float _field:string _measurement:string cpu:string
------------------------------ ------------------------------ ------------------------------ ---------------------------- ---------------------- ---------------------- ----------------------
2018-11-05T21:35:40.000000000Z 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z 8.6 usage_system cpu cpu0
2018-11-05T21:35:40.000000000Z 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z 0.8008008008008008 usage_system cpu cpu1
2018-11-05T21:35:40.000000000Z 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z 5.1 usage_system cpu cpu2
2018-11-05T21:35:40.000000000Z 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z 0.6 usage_system cpu cpu3
Table: keys: [_time]
_time:time _start:time _stop:time _value:float _field:string _measurement:string cpu:string
------------------------------ ------------------------------ ------------------------------ ---------------------------- ---------------------- ---------------------- ----------------------
2018-11-05T21:35:50.000000000Z 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z 10.2 usage_system cpu cpu0
2018-11-05T21:35:50.000000000Z 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z 0.999000999000999 usage_system cpu cpu1
2018-11-05T21:35:50.000000000Z 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z 5.9 usage_system cpu cpu2
2018-11-05T21:35:50.000000000Z 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z 0.8 usage_system cpu cpu3
Table: keys: [_time]
_time:time _start:time _stop:time _value:float _field:string _measurement:string cpu:string
------------------------------ ------------------------------ ------------------------------ ---------------------------- ---------------------- ---------------------- ----------------------
2018-11-05T21:36:00.000000000Z 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z 10.6 usage_system cpu cpu0
2018-11-05T21:36:00.000000000Z 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z 1.1022044088176353 usage_system cpu cpu1
2018-11-05T21:36:00.000000000Z 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z 6.4935064935064934 usage_system cpu cpu2
2018-11-05T21:36:00.000000000Z 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z 0.9 usage_system cpu cpu3
```
{{% /truncate %}}
Because each timestamp is structured as a separate table, when visualized, all
points that share the same timestamp appear connected.
![Group by time](/img/grouping-by-time.png)
{{% note %}}
With some further processing, you could calculate the average CPU usage across all CPUs per point
of time and group them into a single table, but we won't cover that in this example.
If you're interested in running and visualizing this yourself, here's what the query would look like:
```js
dataSet
|> group(columns: ["_time"])
|> mean()
|> group(columns: ["_value", "_time"], mode: "except")
```
{{% /note %}}
## Group by CPU and time
Group by the `cpu` and `_time` columns.
```js
dataSet
|> group(columns: ["cpu", "_time"])
```
This outputs a table for every unique `cpu` and `_time` combination:
{{% truncate %}}
###### Group by CPU and time output tables
```
Table: keys: [_time, cpu]
_time:time cpu:string _stop:time _value:float _field:string _measurement:string _start:time
------------------------------ ---------------------- ------------------------------ ---------------------------- ---------------------- ---------------------- ------------------------------
2018-11-05T21:34:00.000000000Z cpu0 2018-11-05T21:36:00.000000000Z 7.892107892107892 usage_system cpu 2018-11-05T21:34:00.000000000Z
Table: keys: [_time, cpu]
_time:time cpu:string _stop:time _value:float _field:string _measurement:string _start:time
------------------------------ ---------------------- ------------------------------ ---------------------------- ---------------------- ---------------------- ------------------------------
2018-11-05T21:34:00.000000000Z cpu1 2018-11-05T21:36:00.000000000Z 0.7992007992007992 usage_system cpu 2018-11-05T21:34:00.000000000Z
Table: keys: [_time, cpu]
_time:time cpu:string _stop:time _value:float _field:string _measurement:string _start:time
------------------------------ ---------------------- ------------------------------ ---------------------------- ---------------------- ---------------------- ------------------------------
2018-11-05T21:34:00.000000000Z cpu2 2018-11-05T21:36:00.000000000Z 4.1 usage_system cpu 2018-11-05T21:34:00.000000000Z
Table: keys: [_time, cpu]
_time:time cpu:string _stop:time _value:float _field:string _measurement:string _start:time
------------------------------ ---------------------- ------------------------------ ---------------------------- ---------------------- ---------------------- ------------------------------
2018-11-05T21:34:00.000000000Z cpu3 2018-11-05T21:36:00.000000000Z 0.5005005005005005 usage_system cpu 2018-11-05T21:34:00.000000000Z
Table: keys: [_time, cpu]
_time:time cpu:string _stop:time _value:float _field:string _measurement:string _start:time
------------------------------ ---------------------- ------------------------------ ---------------------------- ---------------------- ---------------------- ------------------------------
2018-11-05T21:34:10.000000000Z cpu0 2018-11-05T21:36:00.000000000Z 7.2 usage_system cpu 2018-11-05T21:34:00.000000000Z
Table: keys: [_time, cpu]
_time:time cpu:string _stop:time _value:float _field:string _measurement:string _start:time
------------------------------ ---------------------- ------------------------------ ---------------------------- ---------------------- ---------------------- ------------------------------
2018-11-05T21:34:10.000000000Z cpu1 2018-11-05T21:36:00.000000000Z 0.7 usage_system cpu 2018-11-05T21:34:00.000000000Z
Table: keys: [_time, cpu]
_time:time cpu:string _stop:time _value:float _field:string _measurement:string _start:time
------------------------------ ---------------------- ------------------------------ ---------------------------- ---------------------- ---------------------- ------------------------------
2018-11-05T21:34:10.000000000Z cpu2 2018-11-05T21:36:00.000000000Z 3.6 usage_system cpu 2018-11-05T21:34:00.000000000Z
Table: keys: [_time, cpu]
_time:time cpu:string _stop:time _value:float _field:string _measurement:string _start:time
------------------------------ ---------------------- ------------------------------ ---------------------------- ---------------------- ---------------------- ------------------------------
2018-11-05T21:34:10.000000000Z cpu3 2018-11-05T21:36:00.000000000Z 0.5 usage_system cpu 2018-11-05T21:34:00.000000000Z
Table: keys: [_time, cpu]
_time:time cpu:string _stop:time _value:float _field:string _measurement:string _start:time
------------------------------ ---------------------- ------------------------------ ---------------------------- ---------------------- ---------------------- ------------------------------
2018-11-05T21:34:20.000000000Z cpu0 2018-11-05T21:36:00.000000000Z 7.4 usage_system cpu 2018-11-05T21:34:00.000000000Z
Table: keys: [_time, cpu]
_time:time cpu:string _stop:time _value:float _field:string _measurement:string _start:time
------------------------------ ---------------------- ------------------------------ ---------------------------- ---------------------- ---------------------- ------------------------------
2018-11-05T21:34:20.000000000Z cpu1 2018-11-05T21:36:00.000000000Z 0.7 usage_system cpu 2018-11-05T21:34:00.000000000Z
Table: keys: [_time, cpu]
_time:time cpu:string _stop:time _value:float _field:string _measurement:string _start:time
------------------------------ ---------------------- ------------------------------ ---------------------------- ---------------------- ---------------------- ------------------------------
2018-11-05T21:34:20.000000000Z cpu2 2018-11-05T21:36:00.000000000Z 3.5 usage_system cpu 2018-11-05T21:34:00.000000000Z
Table: keys: [_time, cpu]
_time:time cpu:string _stop:time _value:float _field:string _measurement:string _start:time
------------------------------ ---------------------- ------------------------------ ---------------------------- ---------------------- ---------------------- ------------------------------
2018-11-05T21:34:20.000000000Z cpu3 2018-11-05T21:36:00.000000000Z 0.5 usage_system cpu 2018-11-05T21:34:00.000000000Z
Table: keys: [_time, cpu]
_time:time cpu:string _stop:time _value:float _field:string _measurement:string _start:time
------------------------------ ---------------------- ------------------------------ ---------------------------- ---------------------- ---------------------- ------------------------------
2018-11-05T21:34:30.000000000Z cpu0 2018-11-05T21:36:00.000000000Z 5.5 usage_system cpu 2018-11-05T21:34:00.000000000Z
Table: keys: [_time, cpu]
_time:time cpu:string _stop:time _value:float _field:string _measurement:string _start:time
------------------------------ ---------------------- ------------------------------ ---------------------------- ---------------------- ---------------------- ------------------------------
2018-11-05T21:34:30.000000000Z cpu1 2018-11-05T21:36:00.000000000Z 0.4 usage_system cpu 2018-11-05T21:34:00.000000000Z
Table: keys: [_time, cpu]
_time:time cpu:string _stop:time _value:float _field:string _measurement:string _start:time
------------------------------ ---------------------- ------------------------------ ---------------------------- ---------------------- ---------------------- ------------------------------
2018-11-05T21:34:30.000000000Z cpu2 2018-11-05T21:36:00.000000000Z 2.6 usage_system cpu 2018-11-05T21:34:00.000000000Z
Table: keys: [_time, cpu]
_time:time cpu:string _stop:time _value:float _field:string _measurement:string _start:time
------------------------------ ---------------------- ------------------------------ ---------------------------- ---------------------- ---------------------- ------------------------------
2018-11-05T21:34:30.000000000Z cpu3 2018-11-05T21:36:00.000000000Z 0.3 usage_system cpu 2018-11-05T21:34:00.000000000Z
Table: keys: [_time, cpu]
_time:time cpu:string _stop:time _value:float _field:string _measurement:string _start:time
------------------------------ ---------------------- ------------------------------ ---------------------------- ---------------------- ---------------------- ------------------------------
2018-11-05T21:34:40.000000000Z cpu0 2018-11-05T21:36:00.000000000Z 7.4 usage_system cpu 2018-11-05T21:34:00.000000000Z
Table: keys: [_time, cpu]
_time:time cpu:string _stop:time _value:float _field:string _measurement:string _start:time
------------------------------ ---------------------- ------------------------------ ---------------------------- ---------------------- ---------------------- ------------------------------
2018-11-05T21:34:40.000000000Z cpu1 2018-11-05T21:36:00.000000000Z 0.7 usage_system cpu 2018-11-05T21:34:00.000000000Z
Table: keys: [_time, cpu]
_time:time cpu:string _stop:time _value:float _field:string _measurement:string _start:time
------------------------------ ---------------------- ------------------------------ ---------------------------- ---------------------- ---------------------- ------------------------------
2018-11-05T21:34:40.000000000Z cpu2 2018-11-05T21:36:00.000000000Z 4.5 usage_system cpu 2018-11-05T21:34:00.000000000Z
Table: keys: [_time, cpu]
_time:time cpu:string _stop:time _value:float _field:string _measurement:string _start:time
------------------------------ ---------------------- ------------------------------ ---------------------------- ---------------------- ---------------------- ------------------------------
2018-11-05T21:34:40.000000000Z cpu3 2018-11-05T21:36:00.000000000Z 0.6 usage_system cpu 2018-11-05T21:34:00.000000000Z
Table: keys: [_time, cpu]
_time:time cpu:string _stop:time _value:float _field:string _measurement:string _start:time
------------------------------ ---------------------- ------------------------------ ---------------------------- ---------------------- ---------------------- ------------------------------
2018-11-05T21:34:50.000000000Z cpu0 2018-11-05T21:36:00.000000000Z 7.5 usage_system cpu 2018-11-05T21:34:00.000000000Z
Table: keys: [_time, cpu]
_time:time cpu:string _stop:time _value:float _field:string _measurement:string _start:time
------------------------------ ---------------------- ------------------------------ ---------------------------- ---------------------- ---------------------- ------------------------------
2018-11-05T21:34:50.000000000Z cpu1 2018-11-05T21:36:00.000000000Z 0.7 usage_system cpu 2018-11-05T21:34:00.000000000Z
Table: keys: [_time, cpu]
_time:time cpu:string _stop:time _value:float _field:string _measurement:string _start:time
------------------------------ ---------------------- ------------------------------ ---------------------------- ---------------------- ---------------------- ------------------------------
2018-11-05T21:34:50.000000000Z cpu2 2018-11-05T21:36:00.000000000Z 4.895104895104895 usage_system cpu 2018-11-05T21:34:00.000000000Z
Table: keys: [_time, cpu]
_time:time cpu:string _stop:time _value:float _field:string _measurement:string _start:time
------------------------------ ---------------------- ------------------------------ ---------------------------- ---------------------- ---------------------- ------------------------------
2018-11-05T21:34:50.000000000Z cpu3 2018-11-05T21:36:00.000000000Z 0.6 usage_system cpu 2018-11-05T21:34:00.000000000Z
Table: keys: [_time, cpu]
_time:time cpu:string _stop:time _value:float _field:string _measurement:string _start:time
------------------------------ ---------------------- ------------------------------ ---------------------------- ---------------------- ---------------------- ------------------------------
2018-11-05T21:35:00.000000000Z cpu0 2018-11-05T21:36:00.000000000Z 10.3 usage_system cpu 2018-11-05T21:34:00.000000000Z
Table: keys: [_time, cpu]
_time:time cpu:string _stop:time _value:float _field:string _measurement:string _start:time
------------------------------ ---------------------- ------------------------------ ---------------------------- ---------------------- ---------------------- ------------------------------
2018-11-05T21:35:00.000000000Z cpu1 2018-11-05T21:36:00.000000000Z 1.4 usage_system cpu 2018-11-05T21:34:00.000000000Z
Table: keys: [_time, cpu]
_time:time cpu:string _stop:time _value:float _field:string _measurement:string _start:time
------------------------------ ---------------------- ------------------------------ ---------------------------- ---------------------- ---------------------- ------------------------------
2018-11-05T21:35:00.000000000Z cpu2 2018-11-05T21:36:00.000000000Z 6.906906906906907 usage_system cpu 2018-11-05T21:34:00.000000000Z
Table: keys: [_time, cpu]
_time:time cpu:string _stop:time _value:float _field:string _measurement:string _start:time
------------------------------ ---------------------- ------------------------------ ---------------------------- ---------------------- ---------------------- ------------------------------
2018-11-05T21:35:00.000000000Z cpu3 2018-11-05T21:36:00.000000000Z 1.3986013986013985 usage_system cpu 2018-11-05T21:34:00.000000000Z
Table: keys: [_time, cpu]
_time:time cpu:string _stop:time _value:float _field:string _measurement:string _start:time
------------------------------ ---------------------- ------------------------------ ---------------------------- ---------------------- ---------------------- ------------------------------
2018-11-05T21:35:10.000000000Z cpu0 2018-11-05T21:36:00.000000000Z 9.2 usage_system cpu 2018-11-05T21:34:00.000000000Z
Table: keys: [_time, cpu]
_time:time cpu:string _stop:time _value:float _field:string _measurement:string _start:time
------------------------------ ---------------------- ------------------------------ ---------------------------- ---------------------- ---------------------- ------------------------------
2018-11-05T21:35:10.000000000Z cpu1 2018-11-05T21:36:00.000000000Z 1.2 usage_system cpu 2018-11-05T21:34:00.000000000Z
Table: keys: [_time, cpu]
_time:time cpu:string _stop:time _value:float _field:string _measurement:string _start:time
------------------------------ ---------------------- ------------------------------ ---------------------------- ---------------------- ---------------------- ------------------------------
2018-11-05T21:35:10.000000000Z cpu2 2018-11-05T21:36:00.000000000Z 5.7 usage_system cpu 2018-11-05T21:34:00.000000000Z
Table: keys: [_time, cpu]
_time:time cpu:string _stop:time _value:float _field:string _measurement:string _start:time
------------------------------ ---------------------- ------------------------------ ---------------------------- ---------------------- ---------------------- ------------------------------
2018-11-05T21:35:10.000000000Z cpu3 2018-11-05T21:36:00.000000000Z 0.9 usage_system cpu 2018-11-05T21:34:00.000000000Z
Table: keys: [_time, cpu]
_time:time cpu:string _stop:time _value:float _field:string _measurement:string _start:time
------------------------------ ---------------------- ------------------------------ ---------------------------- ---------------------- ---------------------- ------------------------------
2018-11-05T21:35:20.000000000Z cpu0 2018-11-05T21:36:00.000000000Z 8.4 usage_system cpu 2018-11-05T21:34:00.000000000Z
Table: keys: [_time, cpu]
_time:time cpu:string _stop:time _value:float _field:string _measurement:string _start:time
------------------------------ ---------------------- ------------------------------ ---------------------------- ---------------------- ---------------------- ------------------------------
2018-11-05T21:35:20.000000000Z cpu1 2018-11-05T21:36:00.000000000Z 0.8 usage_system cpu 2018-11-05T21:34:00.000000000Z
Table: keys: [_time, cpu]
_time:time cpu:string _stop:time _value:float _field:string _measurement:string _start:time
------------------------------ ---------------------- ------------------------------ ---------------------------- ---------------------- ---------------------- ------------------------------
2018-11-05T21:35:20.000000000Z cpu2 2018-11-05T21:36:00.000000000Z 5.1 usage_system cpu 2018-11-05T21:34:00.000000000Z
Table: keys: [_time, cpu]
_time:time cpu:string _stop:time _value:float _field:string _measurement:string _start:time
------------------------------ ---------------------- ------------------------------ ---------------------------- ---------------------- ---------------------- ------------------------------
2018-11-05T21:35:20.000000000Z cpu3 2018-11-05T21:36:00.000000000Z 0.5005005005005005 usage_system cpu 2018-11-05T21:34:00.000000000Z
Table: keys: [_time, cpu]
_time:time cpu:string _stop:time _value:float _field:string _measurement:string _start:time
------------------------------ ---------------------- ------------------------------ ---------------------------- ---------------------- ---------------------- ------------------------------
2018-11-05T21:35:30.000000000Z cpu0 2018-11-05T21:36:00.000000000Z 8.5 usage_system cpu 2018-11-05T21:34:00.000000000Z
Table: keys: [_time, cpu]
_time:time cpu:string _stop:time _value:float _field:string _measurement:string _start:time
------------------------------ ---------------------- ------------------------------ ---------------------------- ---------------------- ---------------------- ------------------------------
2018-11-05T21:35:30.000000000Z cpu1 2018-11-05T21:36:00.000000000Z 0.8991008991008991 usage_system cpu 2018-11-05T21:34:00.000000000Z
Table: keys: [_time, cpu]
_time:time cpu:string _stop:time _value:float _field:string _measurement:string _start:time
------------------------------ ---------------------- ------------------------------ ---------------------------- ---------------------- ---------------------- ------------------------------
2018-11-05T21:35:30.000000000Z cpu2 2018-11-05T21:36:00.000000000Z 4.7 usage_system cpu 2018-11-05T21:34:00.000000000Z
Table: keys: [_time, cpu]
_time:time cpu:string _stop:time _value:float _field:string _measurement:string _start:time
------------------------------ ---------------------- ------------------------------ ---------------------------- ---------------------- ---------------------- ------------------------------
2018-11-05T21:35:30.000000000Z cpu3 2018-11-05T21:36:00.000000000Z 0.7 usage_system cpu 2018-11-05T21:34:00.000000000Z
Table: keys: [_time, cpu]
_time:time cpu:string _stop:time _value:float _field:string _measurement:string _start:time
------------------------------ ---------------------- ------------------------------ ---------------------------- ---------------------- ---------------------- ------------------------------
2018-11-05T21:35:40.000000000Z cpu0 2018-11-05T21:36:00.000000000Z 8.6 usage_system cpu 2018-11-05T21:34:00.000000000Z
Table: keys: [_time, cpu]
_time:time cpu:string _stop:time _value:float _field:string _measurement:string _start:time
------------------------------ ---------------------- ------------------------------ ---------------------------- ---------------------- ---------------------- ------------------------------
2018-11-05T21:35:40.000000000Z cpu1 2018-11-05T21:36:00.000000000Z 0.8008008008008008 usage_system cpu 2018-11-05T21:34:00.000000000Z
Table: keys: [_time, cpu]
_time:time cpu:string _stop:time _value:float _field:string _measurement:string _start:time
------------------------------ ---------------------- ------------------------------ ---------------------------- ---------------------- ---------------------- ------------------------------
2018-11-05T21:35:40.000000000Z cpu2 2018-11-05T21:36:00.000000000Z 5.1 usage_system cpu 2018-11-05T21:34:00.000000000Z
Table: keys: [_time, cpu]
_time:time cpu:string _stop:time _value:float _field:string _measurement:string _start:time
------------------------------ ---------------------- ------------------------------ ---------------------------- ---------------------- ---------------------- ------------------------------
2018-11-05T21:35:40.000000000Z cpu3 2018-11-05T21:36:00.000000000Z 0.6 usage_system cpu 2018-11-05T21:34:00.000000000Z
Table: keys: [_time, cpu]
_time:time cpu:string _stop:time _value:float _field:string _measurement:string _start:time
------------------------------ ---------------------- ------------------------------ ---------------------------- ---------------------- ---------------------- ------------------------------
2018-11-05T21:35:50.000000000Z cpu0 2018-11-05T21:36:00.000000000Z 10.2 usage_system cpu 2018-11-05T21:34:00.000000000Z
Table: keys: [_time, cpu]
_time:time cpu:string _stop:time _value:float _field:string _measurement:string _start:time
------------------------------ ---------------------- ------------------------------ ---------------------------- ---------------------- ---------------------- ------------------------------
2018-11-05T21:35:50.000000000Z cpu1 2018-11-05T21:36:00.000000000Z 0.999000999000999 usage_system cpu 2018-11-05T21:34:00.000000000Z
Table: keys: [_time, cpu]
_time:time cpu:string _stop:time _value:float _field:string _measurement:string _start:time
------------------------------ ---------------------- ------------------------------ ---------------------------- ---------------------- ---------------------- ------------------------------
2018-11-05T21:35:50.000000000Z cpu2 2018-11-05T21:36:00.000000000Z 5.9 usage_system cpu 2018-11-05T21:34:00.000000000Z
Table: keys: [_time, cpu]
_time:time cpu:string _stop:time _value:float _field:string _measurement:string _start:time
------------------------------ ---------------------- ------------------------------ ---------------------------- ---------------------- ---------------------- ------------------------------
2018-11-05T21:35:50.000000000Z cpu3 2018-11-05T21:36:00.000000000Z 0.8 usage_system cpu 2018-11-05T21:34:00.000000000Z
Table: keys: [_time, cpu]
_time:time cpu:string _stop:time _value:float _field:string _measurement:string _start:time
------------------------------ ---------------------- ------------------------------ ---------------------------- ---------------------- ---------------------- ------------------------------
2018-11-05T21:36:00.000000000Z cpu0 2018-11-05T21:36:00.000000000Z 10.6 usage_system cpu 2018-11-05T21:34:00.000000000Z
Table: keys: [_time, cpu]
_time:time cpu:string _stop:time _value:float _field:string _measurement:string _start:time
------------------------------ ---------------------- ------------------------------ ---------------------------- ---------------------- ---------------------- ------------------------------
2018-11-05T21:36:00.000000000Z cpu1 2018-11-05T21:36:00.000000000Z 1.1022044088176353 usage_system cpu 2018-11-05T21:34:00.000000000Z
Table: keys: [_time, cpu]
_time:time cpu:string _stop:time _value:float _field:string _measurement:string _start:time
------------------------------ ---------------------- ------------------------------ ---------------------------- ---------------------- ---------------------- ------------------------------
2018-11-05T21:36:00.000000000Z cpu2 2018-11-05T21:36:00.000000000Z 6.4935064935064934 usage_system cpu 2018-11-05T21:34:00.000000000Z
Table: keys: [_time, cpu]
_time:time cpu:string _stop:time _value:float _field:string _measurement:string _start:time
------------------------------ ---------------------- ------------------------------ ---------------------------- ---------------------- ---------------------- ------------------------------
2018-11-05T21:36:00.000000000Z cpu3 2018-11-05T21:36:00.000000000Z 0.9 usage_system cpu 2018-11-05T21:34:00.000000000Z
```
{{% /truncate %}}
When visualized, tables appear as individual, unconnected points.
![Group by CPU and time](/img/grouping-by-cpu-time.png)
Grouping by `cpu` and `_time` is a good illustration of how grouping works.
## In conclusion
Grouping is a powerful way to shape your data into your desired output format.
It modifies the group keys of output tables, grouping records into tables that
all share common values within specified columns.

140
content/influxdb/v1.8/flux/guides/histograms.md Normal file
View File

@ -0,0 +1,140 @@
---
title: Create histograms with Flux
list_title: Histograms
description: >
Use the `histogram()` function to create cumulative histograms with Flux.
menu:
influxdb_1_8:
name: Histograms
parent: Query with Flux
weight: 10
list_query_example: histogram
---
Histograms provide valuable insight into the distribution of your data.
This guide walks through using Flux's `histogram()` function to transform your data into a **cumulative histogram**.
## histogram() function
The [`histogram()` function](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/transformations/histogram) approximates the
cumulative distribution of a dataset by counting data frequencies for a list of "bins."
A **bin** is simply a range in which a data point falls.
All data points that are less than or equal to the bound are counted in the bin.
In the histogram output, a column is added (le) that represents the upper bounds of of each bin.
Bin counts are cumulative.
```js
from(bucket:"telegraf/autogen")
|> range(start: -5m)
|> filter(fn: (r) =>
r._measurement == "mem" and
r._field == "used_percent"
)
|> histogram(bins: [0.0, 10.0, 20.0, 30.0])
```
> Values output by the `histogram` function represent points of data aggregated over time.
> Since values do not represent single points in time, there is no `_time` column in the output table.
## Bin helper functions
Flux provides two helper functions for generating histogram bins.
Each generates and outputs an array of floats designed to be used in the `histogram()` function's `bins` parameter.
### linearBins()
The [`linearBins()` function](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/misc/linearbins) generates a list of linearly separated floats.
```js
linearBins(start: 0.0, width: 10.0, count: 10)
// Generated list: [0, 10, 20, 30, 40, 50, 60, 70, 80, 90, +Inf]
```
### logarithmicBins()
The [`logarithmicBins()` function](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/misc/logarithmicbins) generates a list of exponentially separated floats.
```js
logarithmicBins(start: 1.0, factor: 2.0, count: 10, infinty: true)
// Generated list: [1, 2, 4, 8, 16, 32, 64, 128, 256, 512, +Inf]
```
## Examples
### Generating a histogram with linear bins
```js
from(bucket:"telegraf/autogen")
|> range(start: -5m)
|> filter(fn: (r) =>
r._measurement == "mem" and
r._field == "used_percent"
)
|> histogram(
bins: linearBins(
start:65.5,
width: 0.5,
count: 20,
infinity:false
)
)
```
###### Output table
```
Table: keys: [_start, _stop, _field, _measurement, host]
_start:time _stop:time _field:string _measurement:string host:string le:float _value:float
------------------------------ ------------------------------ ---------------------- ---------------------- ------------------------ ---------------------------- ----------------------------
2018-11-07T22:19:58.423658000Z 2018-11-07T22:24:58.423658000Z used_percent mem Scotts-MacBook-Pro.local 65.5 5
2018-11-07T22:19:58.423658000Z 2018-11-07T22:24:58.423658000Z used_percent mem Scotts-MacBook-Pro.local 66 6
2018-11-07T22:19:58.423658000Z 2018-11-07T22:24:58.423658000Z used_percent mem Scotts-MacBook-Pro.local 66.5 8
2018-11-07T22:19:58.423658000Z 2018-11-07T22:24:58.423658000Z used_percent mem Scotts-MacBook-Pro.local 67 9
2018-11-07T22:19:58.423658000Z 2018-11-07T22:24:58.423658000Z used_percent mem Scotts-MacBook-Pro.local 67.5 9
2018-11-07T22:19:58.423658000Z 2018-11-07T22:24:58.423658000Z used_percent mem Scotts-MacBook-Pro.local 68 10
2018-11-07T22:19:58.423658000Z 2018-11-07T22:24:58.423658000Z used_percent mem Scotts-MacBook-Pro.local 68.5 12
2018-11-07T22:19:58.423658000Z 2018-11-07T22:24:58.423658000Z used_percent mem Scotts-MacBook-Pro.local 69 12
2018-11-07T22:19:58.423658000Z 2018-11-07T22:24:58.423658000Z used_percent mem Scotts-MacBook-Pro.local 69.5 15
2018-11-07T22:19:58.423658000Z 2018-11-07T22:24:58.423658000Z used_percent mem Scotts-MacBook-Pro.local 70 23
2018-11-07T22:19:58.423658000Z 2018-11-07T22:24:58.423658000Z used_percent mem Scotts-MacBook-Pro.local 70.5 30
2018-11-07T22:19:58.423658000Z 2018-11-07T22:24:58.423658000Z used_percent mem Scotts-MacBook-Pro.local 71 30
2018-11-07T22:19:58.423658000Z 2018-11-07T22:24:58.423658000Z used_percent mem Scotts-MacBook-Pro.local 71.5 30
2018-11-07T22:19:58.423658000Z 2018-11-07T22:24:58.423658000Z used_percent mem Scotts-MacBook-Pro.local 72 30
2018-11-07T22:19:58.423658000Z 2018-11-07T22:24:58.423658000Z used_percent mem Scotts-MacBook-Pro.local 72.5 30
2018-11-07T22:19:58.423658000Z 2018-11-07T22:24:58.423658000Z used_percent mem Scotts-MacBook-Pro.local 73 30
2018-11-07T22:19:58.423658000Z 2018-11-07T22:24:58.423658000Z used_percent mem Scotts-MacBook-Pro.local 73.5 30
2018-11-07T22:19:58.423658000Z 2018-11-07T22:24:58.423658000Z used_percent mem Scotts-MacBook-Pro.local 74 30
2018-11-07T22:19:58.423658000Z 2018-11-07T22:24:58.423658000Z used_percent mem Scotts-MacBook-Pro.local 74.5 30
2018-11-07T22:19:58.423658000Z 2018-11-07T22:24:58.423658000Z used_percent mem Scotts-MacBook-Pro.local 75 30
```
### Generating a histogram with logarithmic bins
```js
from(bucket:"telegraf/autogen")
|> range(start: -5m)
|> filter(fn: (r) =>
r._measurement == "mem" and
r._field == "used_percent"
)
|> histogram(
bins: logarithmicBins(
start:0.5,
factor: 2.0,
count: 10,
infinity:false
)
)
```
###### Output table
```
Table: keys: [_start, _stop, _field, _measurement, host]
_start:time _stop:time _field:string _measurement:string host:string le:float _value:float
------------------------------ ------------------------------ ---------------------- ---------------------- ------------------------ ---------------------------- ----------------------------
2018-11-07T22:23:36.860664000Z 2018-11-07T22:28:36.860664000Z used_percent mem Scotts-MacBook-Pro.local 0.5 0
2018-11-07T22:23:36.860664000Z 2018-11-07T22:28:36.860664000Z used_percent mem Scotts-MacBook-Pro.local 1 0
2018-11-07T22:23:36.860664000Z 2018-11-07T22:28:36.860664000Z used_percent mem Scotts-MacBook-Pro.local 2 0
2018-11-07T22:23:36.860664000Z 2018-11-07T22:28:36.860664000Z used_percent mem Scotts-MacBook-Pro.local 4 0
2018-11-07T22:23:36.860664000Z 2018-11-07T22:28:36.860664000Z used_percent mem Scotts-MacBook-Pro.local 8 0
2018-11-07T22:23:36.860664000Z 2018-11-07T22:28:36.860664000Z used_percent mem Scotts-MacBook-Pro.local 16 0
2018-11-07T22:23:36.860664000Z 2018-11-07T22:28:36.860664000Z used_percent mem Scotts-MacBook-Pro.local 32 0
2018-11-07T22:23:36.860664000Z 2018-11-07T22:28:36.860664000Z used_percent mem Scotts-MacBook-Pro.local 64 2
2018-11-07T22:23:36.860664000Z 2018-11-07T22:28:36.860664000Z used_percent mem Scotts-MacBook-Pro.local 128 30
2018-11-07T22:23:36.860664000Z 2018-11-07T22:28:36.860664000Z used_percent mem Scotts-MacBook-Pro.local 256 30
```

54
content/influxdb/v1.8/flux/guides/increase.md Normal file
View File

@ -0,0 +1,54 @@
---
title: Calculate the increase
seotitle: Calculate the increase in Flux
list_title: Increase
description: >
Use the `increase()` function to track increases across multiple columns in a table.
This function is especially useful when tracking changes in counter values that
wrap over time or periodically reset.
weight: 10
menu:
influxdb_1_8:
parent: Query with Flux
name: Increase
list_query_example: increase
---
Use the [`increase()` function](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/transformations/aggregates/increase/)
to track increases across multiple columns in a table.
This function is especially useful when tracking changes in counter values that
wrap over time or periodically reset.
```js
data
|> increase()
```
`increase()` returns a cumulative sum of **non-negative** differences between rows in a table.
For example:
{{< flex >}}
{{% flex-content %}}
**Given the following input:**
| _time | _value |
|:----- | ------:|
| 2020-01-01T00:01:00Z | 1 |
| 2020-01-01T00:02:00Z | 2 |
| 2020-01-01T00:03:00Z | 8 |
| 2020-01-01T00:04:00Z | 10 |
| 2020-01-01T00:05:00Z | 0 |
| 2020-01-01T00:06:00Z | 4 |
{{% /flex-content %}}
{{% flex-content %}}
**`increase()` returns:**
| _time | _value |
|:----- | ------:|
| 2020-01-01T00:02:00Z | 1 |
| 2020-01-01T00:03:00Z | 7 |
| 2020-01-01T00:04:00Z | 9 |
| 2020-01-01T00:05:00Z | 9 |
| 2020-01-01T00:06:00Z | 13 |
{{% /flex-content %}}
{{< /flex >}}

310
content/influxdb/v1.8/flux/guides/join.md Normal file
View File

@ -0,0 +1,310 @@
---
title: Join data with Flux
seotitle: Join data in InfluxDB with Flux
list_title: Join
description: This guide walks through joining data with Flux and outlines how it shapes your data in the process.
menu:
influxdb_1_8:
name: Join
parent: Query with Flux
weight: 10
list_query_example: join
---
The [`join()` function](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/transformations/join) merges two or more
input streams, whose values are equal on a set of common columns, into a single output stream.
Flux allows you to join on any columns common between two data streams and opens the door
for operations such as cross-measurement joins and math across measurements.
To illustrate a join operation, use data captured by Telegraf and and stored in
InfluxDB - memory usage and processes.
In this guide, we'll join two data streams, one representing memory usage and the other representing the
total number of running processes, then calculate the average memory usage per running process.
If you're just getting started with Flux queries, check out the following:
- [Get started with Flux](/influxdb/v1.8/flux/get-started/) for a conceptual overview of Flux and parts of a Flux query.
- [Execute queries](/influxdb/v1.8/flux/guides/execute-queries/) to discover a variety of ways to run your queries.
## Define stream variables
In order to perform a join, you must have two streams of data.
Assign a variable to each data stream.
### Memory used variable
Define a `memUsed` variable that filters on the `mem` measurement and the `used` field.
This returns the amount of memory (in bytes) used.
###### memUsed stream definition
```js
memUsed = from(bucket: "db/rp")
|> range(start: -5m)
|> filter(fn: (r) =>
r._measurement == "mem" and
r._field == "used"
)
```
{{% truncate %}}
###### memUsed data output
```
Table: keys: [_start, _stop, _field, _measurement, host]
_start:time _stop:time _field:string _measurement:string host:string _time:time _value:int
------------------------------ ------------------------------ ---------------------- ---------------------- ------------------------ ------------------------------ --------------------------
2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z used mem host1.local 2018-11-06T05:50:00.000000000Z 10956333056
2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z used mem host1.local 2018-11-06T05:50:10.000000000Z 11014008832
2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z used mem host1.local 2018-11-06T05:50:20.000000000Z 11373428736
2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z used mem host1.local 2018-11-06T05:50:30.000000000Z 11001421824
2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z used mem host1.local 2018-11-06T05:50:40.000000000Z 10985852928
2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z used mem host1.local 2018-11-06T05:50:50.000000000Z 10992279552
2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z used mem host1.local 2018-11-06T05:51:00.000000000Z 11053568000
2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z used mem host1.local 2018-11-06T05:51:10.000000000Z 11092242432
2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z used mem host1.local 2018-11-06T05:51:20.000000000Z 11612774400
2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z used mem host1.local 2018-11-06T05:51:30.000000000Z 11131961344
2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z used mem host1.local 2018-11-06T05:51:40.000000000Z 11124805632
2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z used mem host1.local 2018-11-06T05:51:50.000000000Z 11332464640
2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z used mem host1.local 2018-11-06T05:52:00.000000000Z 11176923136
2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z used mem host1.local 2018-11-06T05:52:10.000000000Z 11181068288
2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z used mem host1.local 2018-11-06T05:52:20.000000000Z 11182579712
2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z used mem host1.local 2018-11-06T05:52:30.000000000Z 11238862848
2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z used mem host1.local 2018-11-06T05:52:40.000000000Z 11275296768
2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z used mem host1.local 2018-11-06T05:52:50.000000000Z 11225411584
2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z used mem host1.local 2018-11-06T05:53:00.000000000Z 11252690944
2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z used mem host1.local 2018-11-06T05:53:10.000000000Z 11227029504
2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z used mem host1.local 2018-11-06T05:53:20.000000000Z 11201646592
2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z used mem host1.local 2018-11-06T05:53:30.000000000Z 11227897856
2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z used mem host1.local 2018-11-06T05:53:40.000000000Z 11330428928
2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z used mem host1.local 2018-11-06T05:53:50.000000000Z 11347976192
2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z used mem host1.local 2018-11-06T05:54:00.000000000Z 11368271872
2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z used mem host1.local 2018-11-06T05:54:10.000000000Z 11269623808
2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z used mem host1.local 2018-11-06T05:54:20.000000000Z 11295637504
2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z used mem host1.local 2018-11-06T05:54:30.000000000Z 11354423296
2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z used mem host1.local 2018-11-06T05:54:40.000000000Z 11379687424
2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z used mem host1.local 2018-11-06T05:54:50.000000000Z 11248926720
2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z used mem host1.local 2018-11-06T05:55:00.000000000Z 11292524544
```
{{% /truncate %}}
### Total processes variable
Define a `procTotal` variable that filters on the `processes` measurement and the `total` field.
This returns the number of running processes.
###### procTotal stream definition
```js
procTotal = from(bucket: "db/rp")
|> range(start: -5m)
|> filter(fn: (r) =>
r._measurement == "processes" and
r._field == "total"
)
```
{{% truncate %}}
###### procTotal data output
```
Table: keys: [_start, _stop, _field, _measurement, host]
_start:time _stop:time _field:string _measurement:string host:string _time:time _value:int
------------------------------ ------------------------------ ---------------------- ---------------------- ------------------------ ------------------------------ --------------------------
2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z total processes host1.local 2018-11-06T05:50:00.000000000Z 470
2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z total processes host1.local 2018-11-06T05:50:10.000000000Z 470
2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z total processes host1.local 2018-11-06T05:50:20.000000000Z 471
2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z total processes host1.local 2018-11-06T05:50:30.000000000Z 470
2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z total processes host1.local 2018-11-06T05:50:40.000000000Z 469
2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z total processes host1.local 2018-11-06T05:50:50.000000000Z 471
2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z total processes host1.local 2018-11-06T05:51:00.000000000Z 470
2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z total processes host1.local 2018-11-06T05:51:10.000000000Z 470
2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z total processes host1.local 2018-11-06T05:51:20.000000000Z 470
2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z total processes host1.local 2018-11-06T05:51:30.000000000Z 470
2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z total processes host1.local 2018-11-06T05:51:40.000000000Z 469
2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z total processes host1.local 2018-11-06T05:51:50.000000000Z 471
2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z total processes host1.local 2018-11-06T05:52:00.000000000Z 471
2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z total processes host1.local 2018-11-06T05:52:10.000000000Z 470
2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z total processes host1.local 2018-11-06T05:52:20.000000000Z 470
2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z total processes host1.local 2018-11-06T05:52:30.000000000Z 471
2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z total processes host1.local 2018-11-06T05:52:40.000000000Z 472
2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z total processes host1.local 2018-11-06T05:52:50.000000000Z 471
2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z total processes host1.local 2018-11-06T05:53:00.000000000Z 470
2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z total processes host1.local 2018-11-06T05:53:10.000000000Z 470
2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z total processes host1.local 2018-11-06T05:53:20.000000000Z 470
2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z total processes host1.local 2018-11-06T05:53:30.000000000Z 471
2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z total processes host1.local 2018-11-06T05:53:40.000000000Z 471
2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z total processes host1.local 2018-11-06T05:53:50.000000000Z 471
2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z total processes host1.local 2018-11-06T05:54:00.000000000Z 471
2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z total processes host1.local 2018-11-06T05:54:10.000000000Z 470
2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z total processes host1.local 2018-11-06T05:54:20.000000000Z 471
2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z total processes host1.local 2018-11-06T05:54:30.000000000Z 473
2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z total processes host1.local 2018-11-06T05:54:40.000000000Z 471
2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z total processes host1.local 2018-11-06T05:54:50.000000000Z 471
2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z total processes host1.local 2018-11-06T05:55:00.000000000Z 471
```
{{% /truncate %}}
## Join the two data streams
With the two data streams defined, use the `join()` function to join them together.
`join()` requires two parameters:
##### `tables`
A map of tables to join with keys by which they will be aliased.
In the example below, `mem` is the alias for `memUsed` and `proc` is the alias for `procTotal`.
##### `on`
An array of strings defining the columns on which the tables will be joined.
_**Both tables must have all columns specified in this list.**_
```js
join(
tables: {mem:memUsed, proc:procTotal},
on: ["_time", "_stop", "_start", "host"]
)
```
{{% truncate %}}
###### Joined output table
```
Table: keys: [_field_mem, _field_proc, _measurement_mem, _measurement_proc, _start, _stop, host]
_field_mem:string _field_proc:string _measurement_mem:string _measurement_proc:string _start:time _stop:time host:string _time:time _value_mem:int _value_proc:int
---------------------- ---------------------- ----------------------- ------------------------ ------------------------------ ------------------------------ ------------------------ ------------------------------ -------------------------- --------------------------
used total mem processes 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z Scotts-MacBook-Pro.local 2018-11-06T05:50:00.000000000Z 10956333056 470
used total mem processes 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z Scotts-MacBook-Pro.local 2018-11-06T05:50:10.000000000Z 11014008832 470
used total mem processes 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z Scotts-MacBook-Pro.local 2018-11-06T05:50:20.000000000Z 11373428736 471
used total mem processes 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z Scotts-MacBook-Pro.local 2018-11-06T05:50:30.000000000Z 11001421824 470
used total mem processes 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z Scotts-MacBook-Pro.local 2018-11-06T05:50:40.000000000Z 10985852928 469
used total mem processes 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z Scotts-MacBook-Pro.local 2018-11-06T05:50:50.000000000Z 10992279552 471
used total mem processes 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z Scotts-MacBook-Pro.local 2018-11-06T05:51:00.000000000Z 11053568000 470
used total mem processes 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z Scotts-MacBook-Pro.local 2018-11-06T05:51:10.000000000Z 11092242432 470
used total mem processes 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z Scotts-MacBook-Pro.local 2018-11-06T05:51:20.000000000Z 11612774400 470
used total mem processes 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z Scotts-MacBook-Pro.local 2018-11-06T05:51:30.000000000Z 11131961344 470
used total mem processes 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z Scotts-MacBook-Pro.local 2018-11-06T05:51:40.000000000Z 11124805632 469
used total mem processes 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z Scotts-MacBook-Pro.local 2018-11-06T05:51:50.000000000Z 11332464640 471
used total mem processes 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z Scotts-MacBook-Pro.local 2018-11-06T05:52:00.000000000Z 11176923136 471
used total mem processes 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z Scotts-MacBook-Pro.local 2018-11-06T05:52:10.000000000Z 11181068288 470
used total mem processes 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z Scotts-MacBook-Pro.local 2018-11-06T05:52:20.000000000Z 11182579712 470
used total mem processes 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z Scotts-MacBook-Pro.local 2018-11-06T05:52:30.000000000Z 11238862848 471
used total mem processes 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z Scotts-MacBook-Pro.local 2018-11-06T05:52:40.000000000Z 11275296768 472
used total mem processes 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z Scotts-MacBook-Pro.local 2018-11-06T05:52:50.000000000Z 11225411584 471
used total mem processes 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z Scotts-MacBook-Pro.local 2018-11-06T05:53:00.000000000Z 11252690944 470
used total mem processes 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z Scotts-MacBook-Pro.local 2018-11-06T05:53:10.000000000Z 11227029504 470
used total mem processes 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z Scotts-MacBook-Pro.local 2018-11-06T05:53:20.000000000Z 11201646592 470
used total mem processes 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z Scotts-MacBook-Pro.local 2018-11-06T05:53:30.000000000Z 11227897856 471
used total mem processes 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z Scotts-MacBook-Pro.local 2018-11-06T05:53:40.000000000Z 11330428928 471
used total mem processes 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z Scotts-MacBook-Pro.local 2018-11-06T05:53:50.000000000Z 11347976192 471
used total mem processes 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z Scotts-MacBook-Pro.local 2018-11-06T05:54:00.000000000Z 11368271872 471
used total mem processes 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z Scotts-MacBook-Pro.local 2018-11-06T05:54:10.000000000Z 11269623808 470
used total mem processes 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z Scotts-MacBook-Pro.local 2018-11-06T05:54:20.000000000Z 11295637504 471
used total mem processes 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z Scotts-MacBook-Pro.local 2018-11-06T05:54:30.000000000Z 11354423296 473
used total mem processes 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z Scotts-MacBook-Pro.local 2018-11-06T05:54:40.000000000Z 11379687424 471
used total mem processes 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z Scotts-MacBook-Pro.local 2018-11-06T05:54:50.000000000Z 11248926720 471
used total mem processes 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z Scotts-MacBook-Pro.local 2018-11-06T05:55:00.000000000Z 11292524544 471
```
{{% /truncate %}}
Notice the output table includes the following columns:
- `_field_mem`
- `_field_proc`
- `_measurement_mem`
- `_measurement_proc`
- `_value_mem`
- `_value_proc`
These represent the columns with values unique to the two input tables.
## Calculate and create a new table
With the two streams of data joined into a single table, use the
[`map()` function](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/transformations/map)
to build a new table by mapping the existing `_time` column to a new `_time`
column and dividing `_value_mem` by `_value_proc` and mapping it to a
new `_value` column.
```js
join(tables: {mem:memUsed, proc:procTotal}, on: ["_time", "_stop", "_start", "host"])
|> map(fn: (r) => ({
_time: r._time,
_value: r._value_mem / r._value_proc
})
)
```
{{% truncate %}}
###### Mapped table
```
Table: keys: [_field_mem, _field_proc, _measurement_mem, _measurement_proc, _start, _stop, host]
_field_mem:string _field_proc:string _measurement_mem:string _measurement_proc:string _start:time _stop:time host:string _time:time _value:int
---------------------- ---------------------- ----------------------- ------------------------ ------------------------------ ------------------------------ ------------------------ ------------------------------ --------------------------
used total mem processes 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z Scotts-MacBook-Pro.local 2018-11-06T05:50:00.000000000Z 23311346
used total mem processes 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z Scotts-MacBook-Pro.local 2018-11-06T05:50:10.000000000Z 23434061
used total mem processes 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z Scotts-MacBook-Pro.local 2018-11-06T05:50:20.000000000Z 24147407
used total mem processes 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z Scotts-MacBook-Pro.local 2018-11-06T05:50:30.000000000Z 23407280
used total mem processes 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z Scotts-MacBook-Pro.local 2018-11-06T05:50:40.000000000Z 23423993
used total mem processes 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z Scotts-MacBook-Pro.local 2018-11-06T05:50:50.000000000Z 23338173
used total mem processes 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z Scotts-MacBook-Pro.local 2018-11-06T05:51:00.000000000Z 23518229
used total mem processes 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z Scotts-MacBook-Pro.local 2018-11-06T05:51:10.000000000Z 23600515
used total mem processes 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z Scotts-MacBook-Pro.local 2018-11-06T05:51:20.000000000Z 24708030
used total mem processes 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z Scotts-MacBook-Pro.local 2018-11-06T05:51:30.000000000Z 23685024
used total mem processes 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z Scotts-MacBook-Pro.local 2018-11-06T05:51:40.000000000Z 23720267
used total mem processes 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z Scotts-MacBook-Pro.local 2018-11-06T05:51:50.000000000Z 24060434
used total mem processes 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z Scotts-MacBook-Pro.local 2018-11-06T05:52:00.000000000Z 23730197
used total mem processes 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z Scotts-MacBook-Pro.local 2018-11-06T05:52:10.000000000Z 23789506
used total mem processes 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z Scotts-MacBook-Pro.local 2018-11-06T05:52:20.000000000Z 23792722
used total mem processes 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z Scotts-MacBook-Pro.local 2018-11-06T05:52:30.000000000Z 23861704
used total mem processes 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z Scotts-MacBook-Pro.local 2018-11-06T05:52:40.000000000Z 23888340
used total mem processes 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z Scotts-MacBook-Pro.local 2018-11-06T05:52:50.000000000Z 23833145
used total mem processes 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z Scotts-MacBook-Pro.local 2018-11-06T05:53:00.000000000Z 23941895
used total mem processes 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z Scotts-MacBook-Pro.local 2018-11-06T05:53:10.000000000Z 23887296
used total mem processes 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z Scotts-MacBook-Pro.local 2018-11-06T05:53:20.000000000Z 23833290
used total mem processes 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z Scotts-MacBook-Pro.local 2018-11-06T05:53:30.000000000Z 23838424
used total mem processes 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z Scotts-MacBook-Pro.local 2018-11-06T05:53:40.000000000Z 24056112
used total mem processes 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z Scotts-MacBook-Pro.local 2018-11-06T05:53:50.000000000Z 24093367
used total mem processes 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z Scotts-MacBook-Pro.local 2018-11-06T05:54:00.000000000Z 24136458
used total mem processes 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z Scotts-MacBook-Pro.local 2018-11-06T05:54:10.000000000Z 23977922
used total mem processes 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z Scotts-MacBook-Pro.local 2018-11-06T05:54:20.000000000Z 23982245
used total mem processes 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z Scotts-MacBook-Pro.local 2018-11-06T05:54:30.000000000Z 24005123
used total mem processes 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z Scotts-MacBook-Pro.local 2018-11-06T05:54:40.000000000Z 24160695
used total mem processes 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z Scotts-MacBook-Pro.local 2018-11-06T05:54:50.000000000Z 23883071
used total mem processes 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z Scotts-MacBook-Pro.local 2018-11-06T05:55:00.000000000Z 23975635
```
{{% /truncate %}}
This table represents the average amount of memory in bytes per running process.
## Real world example
The following function calculates the batch sizes written to an InfluxDB cluster by joining
fields from `httpd` and `write` measurements in order to compare `pointReq` and `writeReq`.
The results are grouped by cluster ID so you can make comparisons across clusters.
```js
batchSize = (cluster_id, start=-1m, interval=10s) => {
httpd = from(bucket:"telegraf")
|> range(start:start)
|> filter(fn:(r) =>
r._measurement == "influxdb_httpd" and
r._field == "writeReq" and
r.cluster_id == cluster_id
)
|> aggregateWindow(every: interval, fn: mean)
|> derivative(nonNegative:true,unit:60s)
write = from(bucket:"telegraf")
|> range(start:start)
|> filter(fn:(r) =>
r._measurement == "influxdb_write" and
r._field == "pointReq" and
r.cluster_id == cluster_id
)
|> aggregateWindow(every: interval, fn: max)
|> derivative(nonNegative:true,unit:60s)
return join(
tables:{httpd:httpd, write:write},
on:["_time","_stop","_start","host"]
)
|> map(fn:(r) => ({
_time: r._time,
_value: r._value_httpd / r._value_write,
}))
|> group(columns: cluster_id)
}
batchSize(cluster_id: "enter cluster id here")
```

187
content/influxdb/v1.8/flux/guides/manipulate-timestamps.md Normal file
View File

@ -0,0 +1,187 @@
---
title: Manipulate timestamps with Flux
list_title: Manipulate timestamps
description: >
Use Flux to process and manipulate timestamps.
menu:
influxdb_1_8:
name: Manipulate timestamps
parent: Query with Flux
weight: 20
---
Every point stored in InfluxDB has an associated timestamp.
Use Flux to process and manipulate timestamps to suit your needs.
- [Convert timestamp format](#convert-timestamp-format)
- [Calculate the duration between two timestamps](#calculate-the-duration-between-two-timestamps)
- [Retrieve the current time](#retrieve-the-current-time)
- [Normalize irregular timestamps](#normalize-irregular-timestamps)
- [Use timestamps and durations together](#use-timestamps-and-durations-together)
{{% note %}}
If you're just getting started with Flux queries, check out the following:
- [Get started with Flux](/v2.0/query-data/get-started/) for a conceptual overview of Flux and parts of a Flux query.
- [Execute queries](/v2.0/query-data/execute-queries/) to discover a variety of ways to run your queries.
{{% /note %}}
## Convert timestamp format
- [Unix nanosecond to RFC3339](#unix-nanosecond-to-rfc3339)
- [RFC3339 to Unix nanosecond](#rfc3339-to-unix-nanosecond)
### Unix nanosecond to RFC3339
Use the [`time()` function](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/transformations/type-conversions/time/)
to convert a [Unix **nanosecond** timestamp](/v2.0/reference/glossary/#unix-timestamp)
to an [RFC3339 timestamp](/v2.0/reference/glossary/#rfc3339-timestamp).
```js
time(v: 1568808000000000000)
// Returns 2019-09-18T12:00:00.000000000Z
```
### RFC3339 to Unix nanosecond
Use the [`uint()` function](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/transformations/type-conversions/uint/)
to convert an RFC3339 timestamp to a Unix nanosecond timestamp.
```js
uint(v: 2019-09-18T12:00:00.000000000Z)
// Returns 1568808000000000000
```
## Calculate the duration between two timestamps
Flux doesn't support mathematical operations using [time type](/flux/v0.65/language/types/#time-types) values.
To calculate the duration between two timestamps:
1. Use the `uint()` function to convert each timestamp to a Unix nanosecond timestamp.
2. Subtract one Unix nanosecond timestamp from the other.
3. Use the `duration()` function to convert the result into a duration.
```js
time1 = uint(v: 2019-09-17T21:12:05Z)
time2 = uint(v: 2019-09-18T22:16:35Z)
duration(v: time2 - time1)
// Returns 25h4m30s
```
{{% note %}}
Flux doesn't support duration column types.
To store a duration in a column, use the [`string()` function](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/transformations/type-conversions/string/)
to convert the duration to a string.
{{% /note %}}
## Retrieve the current time
- [Current UTC time](#current-utc-time)
- [Current system time](#current-system-time)
### Current UTC time
Use the [`now()` function](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/misc/now/) to
return the current UTC time in RFC3339 format.
```js
now()
```
{{% note %}}
`now()` is cached at runtime, so all instances of `now()` in a Flux script
return the same value.
{{% /note %}}
### Current system time
Import the `system` package and use the [`system.time()` function](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/system/time/)
to return the current system time of the host machine in RFC3339 format.
```js
import "system"
system.time()
```
{{% note %}}
`system.time()` returns the time it is executed, so each instance of `system.time()`
in a Flux script returns a unique value.
{{% /note %}}
## Normalize irregular timestamps
To normalize irregular timestamps, truncate all `_time` values to a specified unit
with the [`truncateTimeColumn()` function](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/transformations/truncatetimecolumn/).
This is useful in [`join()`](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/transformations/join/)
and [`pivot()`](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/transformations/pivot/)
operations where points should align by time, but timestamps vary slightly.
```js
data
|> truncateTimeColumn(unit: 1m)
```
{{< flex >}}
{{% flex-content %}}
**Input:**
| _time | _value |
|:----- | ------:|
| 2020-01-01T00:00:49Z | 2.0 |
| 2020-01-01T00:01:01Z | 1.9 |
| 2020-01-01T00:03:22Z | 1.8 |
| 2020-01-01T00:04:04Z | 1.9 |
| 2020-01-01T00:05:38Z | 2.1 |
{{% /flex-content %}}
{{% flex-content %}}
**Output:**
| _time | _value |
|:----- | ------:|
| 2020-01-01T00:00:00Z | 2.0 |
| 2020-01-01T00:01:00Z | 1.9 |
| 2020-01-01T00:03:00Z | 1.8 |
| 2020-01-01T00:04:00Z | 1.9 |
| 2020-01-01T00:05:00Z | 2.1 |
{{% /flex-content %}}
{{< /flex >}}
## Use timestamps and durations together
- [Add a duration to a timestamp](#add-a-duration-to-a-timestamp)
- [Subtract a duration from a timestamp](#subtract-a-duration-from-a-timestamp)
### Add a duration to a timestamp
The [`experimental.addDuration()` function](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/experimental/addduration/)
adds a duration to a specified time and returns the resulting time.
{{% warn %}}
By using `experimental.addDuration()`, you accept the
[risks of experimental functions](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/experimental/#use-experimental-functions-at-your-own-risk).
{{% /warn %}}
```js
import "experimental"
experimental.addDuration(
d: 6h,
to: 2019-09-16T12:00:00Z,
)
// Returns 2019-09-16T18:00:00.000000000Z
```
### Subtract a duration from a timestamp
The [`experimental.subDuration()` function](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/experimental/subduration/)
subtracts a duration from a specified time and returns the resulting time.
{{% warn %}}
By using `experimental.subDuration()`, you accept the
[risks of experimental functions](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/experimental/#use-experimental-functions-at-your-own-risk).
{{% /warn %}}
```js
import "experimental"
experimental.subDuration(
d: 6h,
from: 2019-09-16T12:00:00Z,
)
// Returns 2019-09-16T06:00:00.000000000Z
```

225
content/influxdb/v1.8/flux/guides/mathematic-operations.md Normal file
View File

@ -0,0 +1,225 @@
---
title: Transform data with mathematic operations
seotitle: Transform data with mathematic operations in Flux
list_title: Transform data with math
description: >
Use the `map()` function to remap column values and apply mathematic operations.
menu:
influxdb_1_8:
name: Transform data with math
parent: Query with Flux
weight: 5
list_query_example: map_math
---
Flux supports mathematic expressions in data transformations.
This article describes how to use [Flux arithmetic operators](/flux/v0.65/language/operators/#arithmetic-operators)
to "map" over data and transform values using mathematic operations.
If you're just getting started with Flux queries, check out the following:
- [Get started with Flux](/influxdb/v1.8/flux/get-started/) for a conceptual overview of Flux and parts of a Flux query.
- [Execute queries](/influxdb/v1.8/flux/guides/execute-queries/) to discover a variety of ways to run your queries.
##### Basic mathematic operations
```js
// Examples executed using the Flux REPL
> 9 + 9
18
> 22 - 14
8
> 6 * 5
30
> 21 / 7
3
```
<p style="font-size:.85rem;font-style:italic;margin-top:-2rem;">See <a href="/influxdb/v1.8/flux/guides/execute-queries/#influx-cli">Flux read-eval-print-loop (REPL)</a>.</p>
{{% note %}}
#### Operands must be the same type
Operands in Flux mathematic operations must be the same data type.
For example, integers cannot be used in operations with floats.
Otherwise, you will get an error similar to:
```
Error: type error: float != int
```
To convert operands to the same type, use [type-conversion functions](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/transformations/type-conversions/)
or manually format operands.
The operand data type determines the output data type.
For example:
```js
100 // Parsed as an integer
100.0 // Parsed as a float
// Example evaluations
> 20 / 8
2
> 20.0 / 8.0
2.5
```
{{% /note %}}
## Custom mathematic functions
Flux lets you [create custom functions](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/custom-functions) that use mathematic operations.
View the examples below.
###### Custom multiplication function
```js
multiply = (x, y) => x * y
multiply(x: 10, y: 12)
// Returns 120
```
###### Custom percentage function
```js
percent = (sample, total) => (sample / total) * 100.0
percent(sample: 20.0, total: 80.0)
// Returns 25.0
```
### Transform values in a data stream
To transform multiple values in an input stream, your function needs to:
- [Handle piped-forward data](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/custom-functions/#functions-that-manipulate-piped-forward-data).
- Each operand necessary for the calculation exists in each row _(see [Pivot vs join](#pivot-vs-join) below)_.
- Use the [`map()` function](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/transformations/map) to iterate over each row.
The example `multiplyByX()` function below includes:
- A `tables` parameter that represents the input data stream (`<-`).
- An `x` parameter which is the number by which values in the `_value` column are multiplied.
- A `map()` function that iterates over each row in the input stream.
It uses the `with` operator to preserve existing columns in each row.
It also multiples the `_value` column by `x`.
```js
multiplyByX = (x, tables=<-) =>
tables
|> map(fn: (r) => ({
r with
_value: r._value * x
})
)
data
|> multiplyByX(x: 10)
```
## Examples
### Convert bytes to gigabytes
To convert active memory from bytes to gigabytes (GB), divide the `active` field
in the `mem` measurement by 1,073,741,824.
The `map()` function iterates over each row in the piped-forward data and defines
a new `_value` by dividing the original `_value` by 1073741824.
```js
from(bucket: "db/rp")
|> range(start: -10m)
|> filter(fn: (r) =>
r._measurement == "mem" and
r._field == "active"
)
|> map(fn: (r) => ({
r with
_value: r._value / 1073741824
})
)
```
You could turn that same calculation into a function:
```js
bytesToGB = (tables=<-) =>
tables
|> map(fn: (r) => ({
r with
_value: r._value / 1073741824
})
)
data
|> bytesToGB()
```
#### Include partial gigabytes
Because the original metric (bytes) is an integer, the output of the operation is an integer and does not include partial GBs.
To calculate partial GBs, convert the `_value` column and its values to floats using the
[`float()` function](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/transformations/type-conversions/float)
and format the denominator in the division operation as a float.
```js
bytesToGB = (tables=<-) =>
tables
|> map(fn: (r) => ({
r with
_value: float(v: r._value) / 1073741824.0
})
)
```
### Calculate a percentage
To calculate a percentage, use simple division, then multiply the result by 100.
```js
> 1.0 / 4.0 * 100.0
25.0
```
_For an in-depth look at calculating percentages, see [Calculate percentates](/influxdb/v1.8/flux/guides/calculate-percentages)._
## Pivot vs join
To query and use values in mathematical operations in Flux, operand values must
exists in a single row.
Both `pivot()` and `join()` will do this, but there are important differences between the two:
#### Pivot is more performant
`pivot()` reads and operates on a single stream of data.
`join()` requires two streams of data and the overhead of reading and combining
both streams can be significant, especially for larger data sets.
#### Use join for multiple data sources
Use `join()` when querying data from different buckets or data sources.
##### Pivot fields into columns for mathematic calculations
```js
data
|> pivot(rowKey:["_time"], columnKey: ["_field"], valueColumn: "_value")
|> map(fn: (r) => ({ r with
_value: (r.field1 + r.field2) / r.field3 * 100.0
}))
```
##### Join multiple data sources for mathematic calculations
```js
import "sql"
import "influxdata/influxdb/secrets"
pgUser = secrets.get(key: "POSTGRES_USER")
pgPass = secrets.get(key: "POSTGRES_PASSWORD")
pgHost = secrets.get(key: "POSTGRES_HOST")
t1 = sql.from(
driverName: "postgres",
dataSourceName: "postgresql://${pgUser}:${pgPass}@${pgHost}",
query:"SELECT id, name, available FROM exampleTable"
)
t2 = from(bucket: "db/rp")
|> range(start: -1h)
|> filter(fn: (r) =>
r._measurement == "example-measurement" and
r._field == "example-field"
)
join(tables: {t1: t1, t2: t2}, on: ["id"])
|> map(fn: (r) => ({ r with _value: r._value_t2 / r.available_t1 * 100.0 }))
```

144
content/influxdb/v1.8/flux/guides/median.md Normal file
View File

@ -0,0 +1,144 @@
---
title: Find median values
seotitle: Find median values in Flux
list_title: Median
description: >
Use the `median()` function to return a value representing the `0.5` quantile (50th percentile) or median of input data.
weight: 10
menu:
influxdb_1_8:
parent: Query with Flux
name: Median
list_query_example: median
---
Use the [`median()` function](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/transformations/aggregates/median/)
to return a value representing the `0.5` quantile (50th percentile) or median of input data.
## Select a method for calculating the median
Select one of the following methods to calculate the median:
- [estimate_tdigest](#estimate-tdigest)
- [exact_mean](#exact-mean)
- [exact_selector](#exact-selector)
### estimate_tdigest
**(Default)** An aggregate method that uses a [t-digest data structure](https://github.com/tdunning/t-digest)
to compute an accurate `0.5` quantile estimate on large data sources.
Output tables consist of a single row containing the calculated median.
{{< flex >}}
{{% flex-content %}}
**Given the following input table:**
| _time | _value |
|:----- | ------:|
| 2020-01-01T00:01:00Z | 1.0 |
| 2020-01-01T00:02:00Z | 1.0 |
| 2020-01-01T00:03:00Z | 2.0 |
| 2020-01-01T00:04:00Z | 3.0 |
{{% /flex-content %}}
{{% flex-content %}}
**`estimate_tdigest` returns:**
| _value |
|:------:|
| 1.5 |
{{% /flex-content %}}
{{< /flex >}}
### exact_mean
An aggregate method that takes the average of the two points closest to the `0.5` quantile value.
Output tables consist of a single row containing the calculated median.
{{< flex >}}
{{% flex-content %}}
**Given the following input table:**
| _time | _value |
|:----- | ------:|
| 2020-01-01T00:01:00Z | 1.0 |
| 2020-01-01T00:02:00Z | 1.0 |
| 2020-01-01T00:03:00Z | 2.0 |
| 2020-01-01T00:04:00Z | 3.0 |
{{% /flex-content %}}
{{% flex-content %}}
**`exact_mean` returns:**
| _value |
|:------:|
| 1.5 |
{{% /flex-content %}}
{{< /flex >}}
### exact_selector
A selector method that returns the data point for which at least 50% of points are less than.
Output tables consist of a single row containing the calculated median.
{{< flex >}}
{{% flex-content %}}
**Given the following input table:**
| _time | _value |
|:----- | ------:|
| 2020-01-01T00:01:00Z | 1.0 |
| 2020-01-01T00:02:00Z | 1.0 |
| 2020-01-01T00:03:00Z | 2.0 |
| 2020-01-01T00:04:00Z | 3.0 |
{{% /flex-content %}}
{{% flex-content %}}
**`exact_selector` returns:**
| _time | _value |
|:----- | ------:|
| 2020-01-01T00:02:00Z | 1.0 |
{{% /flex-content %}}
{{< /flex >}}
{{% note %}}
The examples below use the [example data variable](/influxdb/v1.8/flux/guides/#example-data-variable).
{{% /note %}}
## Find the value that represents the median
Use the default method, `"estimate_tdigest"`, to return all rows in a table that
contain values in the 50th percentile of data in the table.
```js
data
|> median()
```
## Find the average of values closest to the median
Use the `exact_mean` method to return a single row per input table containing the
average of the two values closest to the mathematical median of data in the table.
```js
data
|> median(method: "exact_mean")
```
## Find the point with the median value
Use the `exact_selector` method to return a single row per input table containing the
value that 50% of values in the table are less than.
```js
data
|> median(method: "exact_selector")
```
## Use median() with aggregateWindow()
[`aggregateWindow()`](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/transformations/aggregates/aggregatewindow/)
segments data into windows of time, aggregates data in each window into a single
point, and then removes the time-based segmentation.
It is primarily used to downsample data.
To specify the [median calculation method](#select-a-method-for-calculating-the-median) in `aggregateWindow()`, use the
[full function syntax](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/transformations/aggregates/aggregatewindow/#specify-parameters-of-the-aggregate-function):
```js
data
|> aggregateWindow(
every: 5m,
fn: (tables=<-, column) => tables |> median(method: "exact_selector")
)
```

149
content/influxdb/v1.8/flux/guides/monitor-states.md Normal file
View File

@ -0,0 +1,149 @@
---
title: Monitor states
seotitle: Monitor states and state changes in your events and metrics with Flux.
description: Flux provides several functions to help monitor states and state changes in your data.
menu:
influxdb_1_8:
name: Monitor states
parent: Query with Flux
weight: 20
---
Flux helps you monitor states in your metrics and events:
- [Find how long a state persists](#find-how-long-a-state-persists)
- [Count the number of consecutive states](#count-the-number-of-consecutive-states)
- [Detect state changes](#example-query-to-count-machine-state)
If you're just getting started with Flux queries, check out the following:
- [Get started with Flux](/influxdb/v1.8/flux/get-started/) for a conceptual overview of Flux.
- [Execute queries](/influxdb/v1.8/flux/guides/executing-queries/) to discover a variety of ways to run your queries.
## Find how long a state persists
1. Use the [`stateDuration()`](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/transformations/stateduration/) function to calculate how long a column value has remained the same value (or state). Include the following information:
- **Column to search:** any tag key, tag value, field key, field value, or measurement.
- **Value:** the value (or state) to search for in the specified column.
- **State duration column:** a new column to store the state duration─the length of time that the specified value persists.
- **Unit:** the unit of time (`1s` (by default), `1m`, `1h`) used to increment the state duration.
<!-- -->
```js
|> stateDuration(
fn: (r) =>
r._column_to_search == "value_to_search_for",
column: "state_duration",
unit: 1s
)
```
2. Use `stateDuration()` to search each point for the specified value:
- For the first point that evaluates `true`, the state duration is set to `0`. For each consecutive point that evaluates `true`, the state duration increases by the time interval between each consecutive point (in specified units).
- If the state is `false`, the state duration is reset to `-1`.
### Example query with stateDuration()
The following query searches the `doors` bucket over the past 5 minutes to find how many seconds a door has been `closed`.
```js
from(bucket: "doors")
|> range(start: -5m)
|> stateDuration(
fn: (r) =>
r._value == "closed",
column: "door_closed",
unit: 1s
)
```
In this example, `door_closed` is the **State duration** column. If you write data to the `doors` bucket every minute, the state duration increases by `60s` for each consecutive point where `_value` is `closed`. If `_value` is not `closed`, the state duration is reset to `0`.
#### Query results
Results for the example query above may look like this (for simplicity, we've omitted the measurement, tag, and field columns):
```bash
_time _value door_closed
2019-10-26T17:39:16Z closed 0
2019-10-26T17:40:16Z closed 60
2019-10-26T17:41:16Z closed 120
2019-10-26T17:42:16Z open -1
2019-10-26T17:43:16Z closed 0
2019-10-26T17:44:27Z closed 60
```
## Count the number of consecutive states
1. Use the `stateCount()` function and include the following information:
- **Column to search:** any tag key, tag value, field key, field value, or measurement.
- **Value:** to search for in the specified column.
- **State count column:** a new column to store the state count─the number of consecutive records in which the specified value exists.
<!-- -->
```js
|> stateCount
(fn: (r) =>
r._column_to_search == "value_to_search_for",
column: "state_count"
)
```
2. Use `stateCount()` to search each point for the specified value:
- For the first point that evaluates `true`, the state count is set to `1`. For each consecutive point that evaluates `true`, the state count increases by 1.
- If the state is `false`, the state count is reset to `-1`.
### Example query with stateCount()
The following query searches the `doors` bucket over the past 5 minutes and
calculates how many points have `closed` as their `_value`.
```js
from(bucket: "doors")
|> range(start: -5m)
|> stateDuration(
fn: (r) =>
r._value == "closed",
column: "door_closed")
```
This example stores the **state count** in the `door_closed` column.
If you write data to the `doors` bucket every minute, the state count increases
by `1` for each consecutive point where `_value` is `closed`.
If `_value` is not `closed`, the state count is reset to `-1`.
#### Query results
Results for the example query above may look like this (for simplicity, we've omitted the measurement, tag, and field columns):
```bash
_time _value door_closed
2019-10-26T17:39:16Z closed 1
2019-10-26T17:40:16Z closed 2
2019-10-26T17:41:16Z closed 3
2019-10-26T17:42:16Z open -1
2019-10-26T17:43:16Z closed 1
2019-10-26T17:44:27Z closed 2
```
#### Example query to count machine state
The following query checks the machine state every minute (idle, assigned, or busy).
InfluxDB searches the `servers` bucket over the past hour and counts records with a machine state of `idle`, `assigned` or `busy`.
```js
from(bucket: "servers")
|> range(start: -1h)
|> filter(fn: (r) =>
r.machine_state == "idle" or
r.machine_state == "assigned" or
r.machine_state == "busy"
)
|> stateCount(fn: (r) => r.machine_state == "busy", column: "_count")
|> stateCount(fn: (r) => r.machine_state == "assigned", column: "_count")
|> stateCount(fn: (r) => r.machine_state == "idle", column: "_count")
```

113
content/influxdb/v1.8/flux/guides/moving-average.md Normal file
View File

@ -0,0 +1,113 @@
---
title: Calculate the moving average
seotitle: Calculate the moving average in Flux
list_title: Moving Average
description: >
Use the `movingAverage()` or `timedMovingAverage()` functions to return the moving average of data.
weight: 10
menu:
influxdb_1_8:
parent: Query with Flux
name: Moving Average
list_query_example: moving_average
---
Use the [`movingAverage()`](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/transformations/aggregates/movingaverage/)
or [`timedMovingAverage()`](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/transformations/aggregates/timedmovingaverage/)
functions to return the moving average of data.
```js
data
|> movingAverage(n: 5)
// OR
data
|> timedMovingAverage(every: 5m, period: 10m)
```
### movingAverage()
For each row in a table, `movingAverage()` returns the average of the current value and
**previous** values where `n` is the total number of values used to calculate the average.
If `n = 3`:
| Row # | Calculation |
|:-----:|:----------- |
| 1 | _Insufficient number of rows_ |
| 2 | _Insufficient number of rows_ |
| 3 | (Row1 + Row2 + Row3) / 3 |
| 4 | (Row2 + Row3 + Row4) / 3 |
| 5 | (Row3 + Row4 + Row5) / 3 |
{{< flex >}}
{{% flex-content %}}
**Given the following input:**
| _time | _value |
|:----- | ------:|
| 2020-01-01T00:01:00Z | 1.0 |
| 2020-01-01T00:02:00Z | 1.2 |
| 2020-01-01T00:03:00Z | 1.8 |
| 2020-01-01T00:04:00Z | 0.9 |
| 2020-01-01T00:05:00Z | 1.4 |
| 2020-01-01T00:06:00Z | 2.0 |
{{% /flex-content %}}
{{% flex-content %}}
**The following would return:**
```js
|> movingAverage(n: 3)
```
| _time | _value |
|:----- | ------:|
| 2020-01-01T00:03:00Z | 1.33 |
| 2020-01-01T00:04:00Z | 1.30 |
| 2020-01-01T00:05:00Z | 1.36 |
| 2020-01-01T00:06:00Z | 1.43 |
{{% /flex-content %}}
{{< /flex >}}
### timedMovingAverage()
For each row in a table, `timedMovingAverage()` returns the average of the
current value and all row values in the **previous** `period` (duration).
It returns moving averages at a frequency defined by the `every` parameter.
Each color in the diagram below represents a period of time used to calculate an
average and the time a point representing the average is returned.
If `every = 30m` and `period = 1h`:
{{< svg "/static/svgs/timed-moving-avg.svg" >}}
{{< flex >}}
{{% flex-content %}}
**Given the following input:**
| _time | _value |
|:----- | ------:|
| 2020-01-01T00:01:00Z | 1.0 |
| 2020-01-01T00:02:00Z | 1.2 |
| 2020-01-01T00:03:00Z | 1.8 |
| 2020-01-01T00:04:00Z | 0.9 |
| 2020-01-01T00:05:00Z | 1.4 |
| 2020-01-01T00:06:00Z | 2.0 |
{{% /flex-content %}}
{{% flex-content %}}
**The following would return:**
```js
|> timedMovingAverage(
every: 2m,
period: 4m
)
```
| _time | _value |
|:----- | ------:|
| 2020-01-01T00:02:00Z | 1.000 |
| 2020-01-01T00:04:00Z | 1.333 |
| 2020-01-01T00:06:00Z | 1.325 |
| 2020-01-01T00:06:00Z | 1.150 |
{{% /flex-content %}}
{{< /flex >}}

74
content/influxdb/v1.8/flux/guides/optimize-queries.md Normal file
View File

@ -0,0 +1,74 @@
---
title: Optimize Flux queries
description: >
Optimize your Flux queries to reduce their memory and compute (CPU) requirements.
weight: 30
menu:
influxdb_1_8:
name: Optimize queries
parent: Query with Flux
---
Optimize your Flux queries to reduce their memory and compute (CPU) requirements.
- [Start queries with pushdown functions](#start-queries-with-pushdown-functions)
- [Avoid short window durations](#avoid-short-window-durations)
- [Use "heavy" functions sparingly](#use-heavy-functions-sparingly)
- [Balance time range and data precision](#balance-time-range-and-data-precision)
## Start queries with pushdown functions
Some Flux functions can push their data manipulation down to the underlying
data source rather than storing and manipulating data in memory.
These are known as "pushdown" functions and using them correctly can greatly
reduce the amount of memory necessary to run a query.
#### Pushdown functions
- [range()](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/transformations/range/)
- [filter()](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/transformations/filter/)
- [group()](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/transformations/group/)
Use pushdown functions at the beginning of your query.
Once a non-pushdown function runs, Flux pulls data into memory and runs all
subsequent operations there.
##### Pushdown functions in use
```js
from(bucket: "db/rp")
|> range(start: -1h) //
|> filter(fn: (r) => r.sensor == "abc123") // Pushed to the data source
|> group(columns: ["_field", "host"]) //
|> aggregateWindow(every: 5m, fn: max) //
|> filter(fn: (r) => r._value >= 90.0) // Run in memory
|> top(n: 10) //
```
## Avoid short window durations
Windowing (grouping data based on time intervals) is commonly used to aggregate and downsample data.
Increase performance by avoiding short window durations.
More windows require more compute power to evaluate which window each row should be assigned to.
Reasonable window durations depend on the total time range queried.
## Use "heavy" functions sparingly
The following functions use more memory or CPU than others.
Consider their necessity in your data processing before using them:
- [map()](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/transformations/map/)
- [reduce()](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/transformations/aggregates/reduce/)
- [window()](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/transformations/window/)
- [join()](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/transformations/join/)
- [union()](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/transformations/union/)
- [pivot()](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/transformations/pivot/)
{{% note %}}
We're continually optimizing Flux and this list may not represent its current state.
{{% /note %}}
## Balance time range and data precision
To ensure queries are performant, balance the time range and the precision of your data.
For example, if you query data stored every second and request six months worth of data,
results will include a minimum of ≈15.5 million points.
Flux must store these points in memory to generate a response.
To query data over large periods of time, create a [continuous query](/influxdb/latest/query_language/continuous_queries/)
to downsample data, and then query the downsampled data instead.

161
content/influxdb/v1.8/flux/guides/percentile-quantile.md Normal file
View File

@ -0,0 +1,161 @@
---
title: Find percentile and quantile values
seotitle: Query percentile and quantile values in Flux
list_title: Percentile & quantile
description: >
Use the `quantile()` function to return all values within the `q` quantile or
percentile of input data.
weight: 10
menu:
influxdb_1_8:
parent: Query with Flux
name: Percentile & quantile
list_query_example: quantile
---
Use the [`quantile()` function](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/transformations/aggregates/quantile/)
to return a value representing the `q` quantile or percentile of input data.
## Percentile versus quantile
Percentiles and quantiles are very similar, differing only in the number used to calculate return values.
A percentile is calculated using numbers between `0` and `100`.
A quantile is calculated using numbers between `0.0` and `1.0`.
For example, the **`0.5` quantile** is the same as the **50th percentile**.
## Select a method for calculating the quantile
Select one of the following methods to calculate the quantile:
- [estimate_tdigest](#estimate-tdigest)
- [exact_mean](#exact-mean)
- [exact_selector](#exact-selector)
### estimate_tdigest
**(Default)** An aggregate method that uses a [t-digest data structure](https://github.com/tdunning/t-digest)
to compute a quantile estimate on large data sources.
Output tables consist of a single row containing the calculated quantile.
If calculating the `0.5` quantile or 50th percentile:
{{< flex >}}
{{% flex-content %}}
**Given the following input table:**
| _time | _value |
|:----- | ------:|
| 2020-01-01T00:01:00Z | 1.0 |
| 2020-01-01T00:02:00Z | 1.0 |
| 2020-01-01T00:03:00Z | 2.0 |
| 2020-01-01T00:04:00Z | 3.0 |
{{% /flex-content %}}
{{% flex-content %}}
**`estimate_tdigest` returns:**
| _value |
|:------:|
| 1.5 |
{{% /flex-content %}}
{{< /flex >}}
### exact_mean
An aggregate method that takes the average of the two points closest to the quantile value.
Output tables consist of a single row containing the calculated quantile.
If calculating the `0.5` quantile or 50th percentile:
{{< flex >}}
{{% flex-content %}}
**Given the following input table:**
| _time | _value |
|:----- | ------:|
| 2020-01-01T00:01:00Z | 1.0 |
| 2020-01-01T00:02:00Z | 1.0 |
| 2020-01-01T00:03:00Z | 2.0 |
| 2020-01-01T00:04:00Z | 3.0 |
{{% /flex-content %}}
{{% flex-content %}}
**`exact_mean` returns:**
| _value |
|:------:|
| 1.5 |
{{% /flex-content %}}
{{< /flex >}}
### exact_selector
A selector method that returns the data point for which at least `q` points are less than.
Output tables consist of a single row containing the calculated quantile.
If calculating the `0.5` quantile or 50th percentile:
{{< flex >}}
{{% flex-content %}}
**Given the following input table:**
| _time | _value |
|:----- | ------:|
| 2020-01-01T00:01:00Z | 1.0 |
| 2020-01-01T00:02:00Z | 1.0 |
| 2020-01-01T00:03:00Z | 2.0 |
| 2020-01-01T00:04:00Z | 3.0 |
{{% /flex-content %}}
{{% flex-content %}}
**`exact_selector` returns:**
| _time | _value |
|:----- | ------:|
| 2020-01-01T00:02:00Z | 1.0 |
{{% /flex-content %}}
{{< /flex >}}
{{% note %}}
The examples below use the [example data variable](/influxdb/v1.8/flux/guides/#example-data-variable).
{{% /note %}}
## Find the value representing the 99th percentile
Use the default method, `"estimate_tdigest"`, to return all rows in a table that
contain values in the 99th percentile of data in the table.
```js
data
|> quantile(q: 0.99)
```
## Find the average of values closest to the quantile
Use the `exact_mean` method to return a single row per input table containing the
average of the two values closest to the mathematical quantile of data in the table.
For example, to calculate the `0.99` quantile:
```js
data
|> quantile(q: 0.99, method: "exact_mean")
```
## Find the point with the quantile value
Use the `exact_selector` method to return a single row per input table containing the
value that `q * 100`% of values in the table are less than.
For example, to calculate the `0.99` quantile:
```js
data
|> quantile(q: 0.99, method: "exact_selector")
```
## Use quantile() with aggregateWindow()
[`aggregateWindow()`](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/transformations/aggregates/aggregatewindow/)
segments data into windows of time, aggregates data in each window into a single
point, and then removes the time-based segmentation.
It is primarily used to downsample data.
To specify the [quantile calculation method](#select-a-method-for-calculating-the-quantile) in
`aggregateWindow()`, use the [full function syntax](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/transformations/aggregates/aggregatewindow/#specify-parameters-of-the-aggregate-function):
```js
data
|> aggregateWindow(
every: 5m,
fn: (tables=<-, column) =>
tables
|> quantile(q: 0.99, method: "exact_selector")
)
```

76
content/influxdb/v1.8/flux/guides/query-fields.md Normal file
View File

@ -0,0 +1,76 @@
---
title: Query fields and tags
seotitle: Query fields and tags in InfluxDB using Flux
description: >
Use the `filter()` function to query data based on fields, tags, or any other column value.
`filter()` performs operations similar to the `SELECT` statement and the `WHERE`
clause in InfluxQL and other SQL-like query languages.
weight: 1
menu:
influxdb_1_8:
parent: Query with Flux
list_code_example: |
```js
from(bucket: "db/rp")
|> range(start: -1h)
|> filter(fn: (r) =>
r._measurement == "example-measurement" and
r._field == "example-field" and
r.tag == "example-tag"
)
```
---
Use the [`filter()` function](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/transformations/filter/)
to query data based on fields, tags, or any other column value.
`filter()` performs operations similar to the `SELECT` statement and the `WHERE`
clause in InfluxQL and other SQL-like query languages.
## The filter() function
`filter()` has an `fn` parameter that expects a **predicate function**,
an anonymous function comprised of one or more **predicate expressions**.
The predicate function evaluates each input row.
Rows that evaluate to `true` are **included** in the output data.
Rows that evaluate to `false` are **excluded** from the output data.
```js
// ...
|> filter(fn: (r) => r._measurement == "example-measurement" )
```
The `fn` predicate function requires an `r` argument, which represents each row
as `filter()` iterates over input data.
Key-value pairs in the row object represent columns and their values.
Use **dot notation** or **bracket notation** to reference specific column values in the predicate function.
Use [logical operators](/flux/v0.65/language/operators/#logical-operators)
to chain multiple predicate expressions together.
```js
// Row object
r = {foo: "bar", baz: "quz"}
// Example predicate function
(r) => r.foo == "bar" and r["baz"] == "quz"
// Evaluation results
(r) => true and true
```
## Filter by fields and tags
The combination of [`from()`](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/inputs/from),
[`range()`](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/transformations/range),
and `filter()` represent the most basic Flux query:
1. Use `from()` to define your [bucket](/influxdb/v1.8/flux/get-started/#buckets).
2. Use `range()` to limit query results by time.
3. Use `filter()` to identify what rows of data to output.
```js
from(bucket: "db/rp")
|> range(start: -1h)
|> filter(fn: (r) =>
r._measurement == "example-measurement" and
r._field == "example-field" and
r.tag == "example-tag"
)
```

173
content/influxdb/v1.8/flux/guides/rate.md Normal file
View File

@ -0,0 +1,173 @@
---
title: Calculate the rate of change
seotitle: Calculate the rate of change in Flux
list_title: Rate
description: >
Use the `derivative()` function to calculate the rate of change between subsequent values or the
`aggregate.rate()` function to calculate the average rate of change per window of time.
If time between points varies, these functions normalize points to a common time interval
making values easily comparable.
weight: 10
menu:
influxdb_1_8:
parent: Query with Flux
name: Rate
list_query_example: rate_of_change
---
Use the [`derivative()` function](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/transformations/aggregates/derivative/)
to calculate the rate of change between subsequent values or the
[`aggregate.rate()` function](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/experimental/aggregate/rate/)
to calculate the average rate of change per window of time.
If time between points varies, these functions normalize points to a common time interval
making values easily comparable.
- [Rate of change between subsequent values](#rate-of-change-between-subsequent-values)
- [Average rate of change per window of time](#average-rate-of-change-per-window-of-time)
## Rate of change between subsequent values
Use the [`derivative()` function](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/transformations/aggregates/derivative/)
to calculate the rate of change per unit of time between subsequent _non-null_ values.
```js
data
|> derivative(unit: 1s)
```
By default, `derivative()` returns only positive derivative values and replaces negative values with _null_.
Cacluated values are returned as [floats](/flux/v0.65/language/types/#numeric-types).
{{< flex >}}
{{% flex-content %}}
**Given the following input:**
| _time | _value |
|:----- | ------:|
| 2020-01-01T00:00:00Z | 250 |
| 2020-01-01T00:04:00Z | 160 |
| 2020-01-01T00:12:00Z | 150 |
| 2020-01-01T00:19:00Z | 220 |
| 2020-01-01T00:32:00Z | 200 |
| 2020-01-01T00:51:00Z | 290 |
| 2020-01-01T01:00:00Z | 340 |
{{% /flex-content %}}
{{% flex-content %}}
**`derivative(unit: 1m)` returns:**
| _time | _value |
|:----- | ------:|
| 2020-01-01T00:04:00Z | |
| 2020-01-01T00:12:00Z | |
| 2020-01-01T00:19:00Z | 10.0 |
| 2020-01-01T00:32:00Z | |
| 2020-01-01T00:51:00Z | 4.74 |
| 2020-01-01T01:00:00Z | 5.56 |
{{% /flex-content %}}
{{< /flex >}}
Results represent the rate of change **per minute** between subsequent values with
negative values set to _null_.
### Return negative derivative values
To return negative derivative values, set the `nonNegative` parameter to `false`,
{{< flex >}}
{{% flex-content %}}
**Given the following input:**
| _time | _value |
|:----- | ------:|
| 2020-01-01T00:00:00Z | 250 |
| 2020-01-01T00:04:00Z | 160 |
| 2020-01-01T00:12:00Z | 150 |
| 2020-01-01T00:19:00Z | 220 |
| 2020-01-01T00:32:00Z | 200 |
| 2020-01-01T00:51:00Z | 290 |
| 2020-01-01T01:00:00Z | 340 |
{{% /flex-content %}}
{{% flex-content %}}
**The following returns:**
```js
|> derivative(
unit: 1m,
nonNegative: false
)
```
| _time | _value |
|:----- | ------:|
| 2020-01-01T00:04:00Z | -22.5 |
| 2020-01-01T00:12:00Z | -1.25 |
| 2020-01-01T00:19:00Z | 10.0 |
| 2020-01-01T00:32:00Z | -1.54 |
| 2020-01-01T00:51:00Z | 4.74 |
| 2020-01-01T01:00:00Z | 5.56 |
{{% /flex-content %}}
{{< /flex >}}
Results represent the rate of change **per minute** between subsequent values and
include negative values.
## Average rate of change per window of time
Use the [`aggregate.rate()` function](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/experimental/aggregate/rate/)
to calculate the average rate of change per window of time.
```js
import "experimental/aggregate"
data
|> aggregate.rate(
every: 1m,
unit: 1s,
groupColumns: ["tag1", "tag2"]
)
```
`aggregate.rate()` returns the average rate of change (as a [float](/flux/v0.65/language/types/#numeric-types))
per `unit` for time intervals defined by `every`.
Negative values are replaced with _null_.
{{% note %}}
`aggregate.rate()` does not support `nonNegative: false`.
{{% /note %}}
{{< flex >}}
{{% flex-content %}}
**Given the following input:**
| _time | _value |
|:----- | ------:|
| 2020-01-01T00:00:00Z | 250 |
| 2020-01-01T00:04:00Z | 160 |
| 2020-01-01T00:12:00Z | 150 |
| 2020-01-01T00:19:00Z | 220 |
| 2020-01-01T00:32:00Z | 200 |
| 2020-01-01T00:51:00Z | 290 |
| 2020-01-01T01:00:00Z | 340 |
{{% /flex-content %}}
{{% flex-content %}}
**The following returns:**
```js
|> aggregate.rate(
every: 20m,
unit: 1m
)
```
| _time | _value |
|:----- | ------:|
| 2020-01-01T00:20:00Z | |
| 2020-01-01T00:40:00Z | 10.0 |
| 2020-01-01T01:00:00Z | 4.74 |
| 2020-01-01T01:20:00Z | 5.56 |
{{% /flex-content %}}
{{< /flex >}}
Results represent the **average change rate per minute** of every **20 minute interval**
with negative values set to _null_.
Timestamps represent the right bound of the time window used to average values.

91
content/influxdb/v1.8/flux/guides/regular-expressions.md Normal file
View File

@ -0,0 +1,91 @@
---
title: Use regular expressions in Flux
list_title: Regular expressions
description: This guide walks through using regular expressions in evaluation logic in Flux functions.
menu:
influxdb_1_8:
name: Regular expressions
parent: Query with Flux
weight: 20
list_query_example: regular_expressions
---
Regular expressions (regexes) are incredibly powerful when matching patterns in large collections of data.
With Flux, regular expressions are primarily used for evaluation logic in predicate functions for things
such as filtering rows, dropping and keeping columns, state detection, etc.
This guide shows how to use regular expressions in your Flux scripts.
If you're just getting started with Flux queries, check out the following:
- [Get started with Flux](/influxdb/v1.8/flux/get-started/) for a conceptual overview of Flux and parts of a Flux query.
- [Execute queries](/influxdb/v1.8/flux/guides/execute-queries/) to discover a variety of ways to run your queries.
## Go regular expression syntax
Flux uses Go's [regexp package](https://golang.org/pkg/regexp/) for regular expression search.
The links [below](#helpful-links) provide information about Go's regular expression syntax.
## Regular expression operators
Flux provides two comparison operators for use with regular expressions.
#### `=~`
When the expression on the left **MATCHES** the regular expression on the right, this evaluates to `true`.
#### `!~`
When the expression on the left **DOES NOT MATCH** the regular expression on the right, this evaluates to `true`.
## Regular expressions in Flux
When using regex matching in your Flux scripts, enclose your regular expressions with `/`.
The following is the basic regex comparison syntax:
###### Basic regex comparison syntax
```js
expression =~ /regex/
expression !~ /regex/
```
## Examples
### Use a regex to filter by tag value
The following example filters records by the `cpu` tag.
It only keeps records for which the `cpu` is either `cpu0`, `cpu1`, or `cpu2`.
```js
from(bucket: "db/rp")
|> range(start: -15m)
|> filter(fn: (r) =>
r._measurement == "cpu" and
r._field == "usage_user" and
r.cpu =~ /cpu[0-2]/
)
```
### Use a regex to filter by field key
The following example excludes records that do not have `_percent` in a field key.
```js
from(bucket: "db/rp")
|> range(start: -15m)
|> filter(fn: (r) =>
r._measurement == "mem" and
r._field =~ /_percent/
)
```
### Drop columns matching a regex
The following example drops columns whose names do not being with `_`.
```js
from(bucket: "db/rp")
|> range(start: -15m)
|> filter(fn: (r) => r._measurement == "mem")
|> drop(fn: (column) => column !~ /_.*/)
```
## Helpful links
##### Syntax documentation
[regexp Syntax GoDoc](https://godoc.org/regexp/syntax)
[RE2 Syntax Overview](https://github.com/google/re2/wiki/Syntax)
##### Go regex testers
[Regex Tester - Golang](https://regex-golang.appspot.com/assets/html/index.html)
[Regex101](https://regex101.com/)

260
content/influxdb/v1.8/flux/guides/scalar-values.md Normal file
View File

@ -0,0 +1,260 @@
---
title: Extract scalar values in Flux
list_title: Extract scalar values
description: >
Use Flux stream and table functions to extract scalar values from Flux query output.
This lets you, for example, dynamically set variables using query results.
menu:
influxdb_1_8:
name: Extract scalar values
parent: Query with Flux
weight: 20
list_code_example: |
```js
scalarValue = {
_record =
data
|> tableFind(fn: key => true)
|> getRecord(idx: 0)
return _record._value
}
```
---
Use Flux [stream and table functions](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/transformations/stream-table/)
to extract scalar values from Flux query output.
This lets you, for example, dynamically set variables using query results.
**To extract scalar values from output:**
1. [Extract a table](#extract-a-table).
2. [Extract a column from the table](#extract-a-column-from-the-table)
_**or**_ [extract a row from the table](#extract-a-row-from-the-table).
_The samples on this page use the [sample data provided below](#sample-data)._
{{% warn %}}
#### Current limitations
- The InfluxDB user interface (UI) does not currently support raw scalar output.
Use [`map()`](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/transformations/map/) to add
scalar values to output data.
- The [Flux REPL](/influxdb/v1.8/flux/guides/execute-queries/#influx-cli) does not currently support
Flux stream and table functions (also known as "dynamic queries").
See [#15321](https://github.com/influxdata/influxdb/issues/15231).
{{% /warn %}}
## Extract a table
Flux formats query results as a stream of tables.
To extract a scalar value from a stream of tables, you must first extract a single table.
to extract a single table from the stream of tables.
{{% note %}}
If query results include only one table, it is still formatted as a stream of tables.
You still must extract that table from the stream.
{{% /note %}}
Use [`tableFind()`](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/transformations/stream-table/tablefind/)
to extract the **first** table whose [group key](/influxdb/v1.8/flux/get-started/#group-keys)
values match the `fn` **predicate function**.
The predicate function requires a `key` object, which represents the group key of
each table.
```js
sampleData
|> tableFind(fn: (key) =>
key._field == "temp" and
key.location == "sfo"
)
```
The example above returns a single table:
| _time | location | _field | _value |
|:----- |:--------:|:------:| ------:|
| 2019-11-01T12:00:00Z | sfo | temp | 65.1 |
| 2019-11-01T13:00:00Z | sfo | temp | 66.2 |
| 2019-11-01T14:00:00Z | sfo | temp | 66.3 |
| 2019-11-01T15:00:00Z | sfo | temp | 66.8 |
{{% note %}}
#### Extract the correct table
Flux functions do not guarantee table order and `tableFind()` returns only the
**first** table that matches the `fn` predicate.
To extract the table that includes the data you actually want, be very specific in
your predicate function or filter and transform your data to minimize the number
of tables piped-forward into `tableFind()`.
{{% /note %}}
## Extract a column from the table
Use the [`getColumn()` function](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/transformations/stream-table/getcolumn/)
to output an array of values from a specific column in the extracted table.
```js
sampleData
|> tableFind(fn: (key) =>
key._field == "temp" and
key.location == "sfo"
)
|> getColumn(column: "_value")
// Returns [65.1, 66.2, 66.3, 66.8]
```
### Use extracted column values
Use a variable to store the array of values.
In the example below, `SFOTemps` represents the array of values.
Reference a specific index (integer starting from `0`) in the array to return the
value at that index.
```js
SFOTemps = sampleData
|> tableFind(fn: (key) =>
key._field == "temp" and
key.location == "sfo"
)
|> getColumn(column: "_value")
SFOTemps
// Returns [65.1, 66.2, 66.3, 66.8]
SFOTemps[0]
// Returns 65.1
SFOTemps[2]
// Returns 66.3
```
## Extract a row from the table
Use the [`getRecord()` function](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/transformations/stream-table/getrecord/)
to output data from a single row in the extracted table.
Specify the index of the row to output using the `idx` parameter.
The function outputs an object with key-value pairs for each column.
```js
sampleData
|> tableFind(fn: (key) =>
key._field == "temp" and
key.location == "sfo"
)
|> getRecord(idx: 0)
// Returns {
// _time:2019-11-11T12:00:00Z,
// _field:"temp",
// location:"sfo",
// _value: 65.1
// }
```
### Use an extracted row object
Use a variable to store the extracted row object.
In the example below, `tempInfo` represents the extracted row.
Use [dot notation](/influxdb/v1.8/flux/get-started/syntax-basics/#objects) to reference
keys in the object.
```js
tempInfo = sampleData
|> tableFind(fn: (key) =>
key._field == "temp" and
key.location == "sfo"
)
|> getRecord(idx: 0)
tempInfo
// Returns {
// _time:2019-11-11T12:00:00Z,
// _field:"temp",
// location:"sfo",
// _value: 65.1
// }
tempInfo._time
// Returns 2019-11-11T12:00:00Z
tempInfo.location
// Returns sfo
```
## Example helper functions
Create custom helper functions to extract scalar values from query output.
##### Extract a scalar field value
```js
// Define a helper function to extract field values
getFieldValue = (tables=<-, field) => {
extract = tables
|> tableFind(fn: (key) => key._field == field)
|> getColumn(column: "_value")
return extract[0]
}
// Use the helper function to define a variable
lastJFKTemp = sampleData
|> filter(fn: (r) => r.location == "kjfk")
|> last()
|> getFieldValue(field: "temp")
lastJFKTemp
// Returns 71.2
```
##### Extract scalar row data
```js
// Define a helper function to extract a row as an object
getRow = (tables=<-, field, idx=0) => {
extract = tables
|> tableFind(fn: (key) => true)
|> getRecord(idx: idx)
return extract
}
// Use the helper function to define a variable
lastReported = sampleData
|> last()
|> getRow(idx: 0)
"The last location to report was ${lastReported.location}.
The temperature was ${string(v: lastReported._value)}°F."
// Returns:
// The last location to report was kord.
// The temperature was 38.9°F.
```
---
## Sample data
The following sample data set represents fictional temperature metrics collected
from three locations.
It's formatted in [annotated CSV](https://v2.docs.influxdata.com/v2.0/reference/syntax/annotated-csv/) and imported
into the Flux query using the [`csv.from()` function](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/csv/from/).
Place the following at the beginning of your query to use the sample data:
{{% truncate %}}
```js
import "csv"
sampleData = csv.from(csv: "
#datatype,string,long,dateTime:RFC3339,string,string,double
#group,false,true,false,true,true,false
#default,,,,,,
,result,table,_time,location,_field,_value
,,0,2019-11-01T12:00:00Z,sfo,temp,65.1
,,0,2019-11-01T13:00:00Z,sfo,temp,66.2
,,0,2019-11-01T14:00:00Z,sfo,temp,66.3
,,0,2019-11-01T15:00:00Z,sfo,temp,66.8
,,1,2019-11-01T12:00:00Z,kjfk,temp,69.4
,,1,2019-11-01T13:00:00Z,kjfk,temp,69.9
,,1,2019-11-01T14:00:00Z,kjfk,temp,71.0
,,1,2019-11-01T15:00:00Z,kjfk,temp,71.2
,,2,2019-11-01T12:00:00Z,kord,temp,46.4
,,2,2019-11-01T13:00:00Z,kord,temp,46.3
,,2,2019-11-01T14:00:00Z,kord,temp,42.7
,,2,2019-11-01T15:00:00Z,kord,temp,38.9
")
```
{{% /truncate %}}

68
content/influxdb/v1.8/flux/guides/sort-limit.md Normal file
View File

@ -0,0 +1,68 @@
---
title: Sort and limit data with Flux
seotitle: Sort and limit data in InfluxDB with Flux
list_title: Sort and limit
description: >
Use the `sort()`function to order records within each table by specific columns and the
`limit()` function to limit the number of records in output tables to a fixed number, `n`.
menu:
influxdb_1_8:
name: Sort and limit
parent: Query with Flux
weight: 3
list_query_example: sort_limit
---
Use the [`sort()`function](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/transformations/sort)
to order records within each table by specific columns and the
[`limit()` function](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/transformations/limit)
to limit the number of records in output tables to a fixed number, `n`.
If you're just getting started with Flux queries, check out the following:
- [Get started with Flux](/influxdb/v1.8/flux/get-started/) for a conceptual overview of Flux and parts of a Flux query.
- [Execute queries](/influxdb/v1.8/flux/guides/execute-queries/) to discover a variety of ways to run your queries.
##### Example sorting system uptime
The following example orders system uptime first by region, then host, then value.
```js
from(bucket:"db/rp")
|> range(start:-12h)
|> filter(fn: (r) =>
r._measurement == "system" and
r._field == "uptime"
)
|> sort(columns:["region", "host", "_value"])
```
The [`limit()` function](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/transformations/limit)
limits the number of records in output tables to a fixed number, `n`.
The following example shows up to 10 records from the past hour.
```js
from(bucket:"db/rp")
|> range(start:-1h)
|> limit(n:10)
```
You can use `sort()` and `limit()` together to show the top N records.
The example below returns the 10 top system uptime values sorted first by
region, then host, then value.
```js
from(bucket:"db/rp")
|> range(start:-12h)
|> filter(fn: (r) =>
r._measurement == "system" and
r._field == "uptime"
)
|> sort(columns:["region", "host", "_value"])
|> limit(n:10)
```
You now have created a Flux query that sorts and limits data.
Flux also provides the [`top()`](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/transformations/selectors/top)
and [`bottom()`](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/transformations/selectors/bottom)
functions to perform both of these functions at the same time.

213
content/influxdb/v1.8/flux/guides/sql.md Normal file
View File

@ -0,0 +1,213 @@
---
title: Query SQL data sources
seotitle: Query SQL data sources with InfluxDB
list_title: Query SQL data
description: >
The Flux `sql` package provides functions for working with SQL data sources.
Use `sql.from()` to query SQL databases like PostgreSQL and MySQL
menu:
influxdb_1_8:
parent: Query with Flux
list_title: SQL data
weight: 20
list_code_example: |
```js
import "sql"
sql.from(
driverName: "postgres",
dataSourceName: "postgresql://user:password@localhost",
query: "SELECT * FROM example_table"
)
```
---
The [Flux](/flux/v0.65) `sql` package provides functions for working with SQL data sources.
[`sql.from()`](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/sql/from/) lets you query SQL data sources
like [PostgreSQL](https://www.postgresql.org/), [MySQL](https://www.mysql.com/),
and [SQLite](https://www.sqlite.org/index.html), and use the results with InfluxDB
dashboards, tasks, and other operations.
- [Query a SQL data source](#query-a-sql-data-source)
- [Join SQL data with data in InfluxDB](#join-sql-data-with-data-in-influxdb)
- [Sample sensor data](#sample-sensor-data)
## Query a SQL data source
To query a SQL data source:
1. Import the `sql` package in your Flux query
2. Use the `sql.from()` function to specify the driver, data source name (DSN),
and query used to query data from your SQL data source:
{{< code-tabs-wrapper >}}
{{% code-tabs %}}
[PostgreSQL](#)
[MySQL](#)
[SQLite](#)
{{% /code-tabs %}}
{{% code-tab-content %}}
```js
import "sql"
sql.from(
driverName: "postgres",
dataSourceName: "postgresql://user:password@localhost",
query: "SELECT * FROM example_table"
)
```
{{% /code-tab-content %}}
{{% code-tab-content %}}
```js
import "sql"
sql.from(
driverName: "mysql",
dataSourceName: "user:password@tcp(localhost:3306)/db",
query: "SELECT * FROM example_table"
)
```
{{% /code-tab-content %}}
{{% code-tab-content %}}
```js
// NOTE: InfluxDB OSS and InfluxDB Cloud do not have access to
// the local filesystem and cannot query SQLite data sources.
// Use the Flux REPL to query an SQLite data source.
import "sql"
sql.from(
driverName: "sqlite3",
dataSourceName: "file:/path/to/test.db?cache=shared&mode=ro",
query: "SELECT * FROM example_table"
)
```
{{% /code-tab-content %}}
{{< /code-tabs-wrapper >}}
_See the [`sql.from()` documentation](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/sql/from/) for
information about required function parameters._
## Join SQL data with data in InfluxDB
One of the primary benefits of querying SQL data sources from InfluxDB
is the ability to enrich query results with data stored outside of InfluxDB.
Using the [air sensor sample data](#sample-sensor-data) below, the following query
joins air sensor metrics stored in InfluxDB with sensor information stored in PostgreSQL.
The joined data lets you query and filter results based on sensor information
that isn't stored in InfluxDB.
```js
// Import the "sql" package
import "sql"
// Query data from PostgreSQL
sensorInfo = sql.from(
driverName: "postgres",
dataSourceName: "postgresql://localhost?sslmode=disable",
query: "SELECT * FROM sensors"
)
// Query data from InfluxDB
sensorMetrics = from(bucket: "telegraf/autogen")
|> range(start: -1h)
|> filter(fn: (r) => r._measurement == "airSensors")
// Join InfluxDB query results with PostgreSQL query results
join(tables: {metric: sensorMetrics, info: sensorInfo}, on: ["sensor_id"])
```
---
## Sample sensor data
The [sample data generator](#download-and-run-the-sample-data-generator) and
[sample sensor information](#import-the-sample-sensor-information) simulate a
group of sensors that measure temperature, humidity, and carbon monoxide
in rooms throughout a building.
Each collected data point is stored in InfluxDB with a `sensor_id` tag that identifies
the specific sensor it came from.
Sample sensor information is stored in PostgreSQL.
**Sample data includes:**
- Simulated data collected from each sensor and stored in the `airSensors` measurement in **InfluxDB**:
- temperature
- humidity
- co
- Information about each sensor stored in the `sensors` table in **PostgreSQL**:
- sensor_id
- location
- model_number
- last_inspected
### Import and generate sample sensor data
#### Download and run the sample data generator
`air-sensor-data.rb` is a script that generates air sensor data and stores the data in InfluxDB.
To use `air-sensor-data.rb`:
1. [Create a database](/influxdb/latest/introduction/getting-started/#creating-a-database) to store the data.
2. Download the sample data generator. _This tool requires [Ruby](https://www.ruby-lang.org/en/)._
<a class="btn download" style="color:#fff" href="/downloads/air-sensor-data.rb" download>Download Air Sensor Generator</a>
3. Give `air-sensor-data.rb` executable permissions:
```
chmod +x air-sensor-data.rb
```
4. Start the generator. Specify your database.
```
./air-sensor-data.rb -d database-name
```
The generator begins to write data to InfluxDB and will continue until stopped.
Use `ctrl-c` to stop the generator.
_**Note:** Use the `--help` flag to view other configuration options._
5. Query your target database to ensure the generated data is writing successfully.
The generator doesn't catch errors from write requests, so it will continue running
even if data is not writing to InfluxDB successfully.
```
from(bucket: "database-name/autogen")
|> range(start: -1m)
|> filter(fn: (r) => r._measurement == "airSensors")
```
#### Import the sample sensor information
1. [Download and install PostgreSQL](https://www.postgresql.org/download/).
2. Download the sample sensor information CSV.
<a class="btn download" style="color:#fff" href="/downloads/sample-sensor-info.csv" download>Download Sample Data</a>
3. Use a PostgreSQL client (`psql` or a GUI) to create the `sensors` table:
```
CREATE TABLE sensors (
sensor_id character varying(50),
location character varying(50),
model_number character varying(50),
last_inspected date
);
```
4. Import the downloaded CSV sample data.
_Update the `FROM` file path to the path of the downloaded CSV sample data._
```
COPY sensors(sensor_id,location,model_number,last_inspected)
FROM '/path/to/sample-sensor-info.csv' DELIMITER ',' CSV HEADER;
```
5. Query the table to ensure the data was imported correctly:
```
SELECT * FROM sensors;
```

352
content/influxdb/v1.8/flux/guides/window-aggregate.md Normal file
View File

@ -0,0 +1,352 @@
---
title: Window and aggregate data with Flux
seotitle: Window and aggregate data in InfluxDB with Flux
list_title: Window & aggregate
description: >
This guide walks through windowing and aggregating data with Flux and outlines
how it shapes your data in the process.
menu:
influxdb_1_8:
name: Window & aggregate
parent: Query with Flux
weight: 4
list_query_example: aggregate_window
---
A common operation performed with time series data is grouping data into windows of time,
or "windowing" data, then aggregating windowed values into a new value.
This guide walks through windowing and aggregating data with Flux and demonstrates
how data is shaped in the process.
If you're just getting started with Flux queries, check out the following:
- [Get started with Flux](/influxdb/v1.8/flux/get-started/) for a conceptual overview of Flux and parts of a Flux query.
- [Execute queries](/influxdb/v1.8/flux/guides/execute-queries/) to discover a variety of ways to run your queries.
{{% note %}}
The following example is an in-depth walk-through of the steps required to window and aggregate data.
The [`aggregateWindow()` function](#summing-up) performs these operations for you, but understanding
how data is shaped in the process helps to successfully create your desired output.
{{% /note %}}
## Data set
For the purposes of this guide, define a variable that represents your base data set.
The following example queries the memory usage of the host machine.
```js
dataSet = from(bucket: "db/rp")
|> range(start: -5m)
|> filter(fn: (r) =>
r._measurement == "mem" and
r._field == "used_percent"
)
|> drop(columns: ["host"])
```
{{% note %}}
This example drops the `host` column from the returned data since the memory data
is only tracked for a single host and it simplifies the output tables.
Dropping the `host` column is optional and not recommended if monitoring memory
on multiple hosts.
{{% /note %}}
`dataSet` can now be used to represent your base data, which will look similar to the following:
{{% truncate %}}
```
Table: keys: [_start, _stop, _field, _measurement]
_start:time _stop:time _field:string _measurement:string _time:time _value:float
------------------------------ ------------------------------ ---------------------- ---------------------- ------------------------------ ----------------------------
2018-11-03T17:50:00.000000000Z 2018-11-03T17:55:00.000000000Z used_percent mem 2018-11-03T17:50:00.000000000Z 71.11611366271973
2018-11-03T17:50:00.000000000Z 2018-11-03T17:55:00.000000000Z used_percent mem 2018-11-03T17:50:10.000000000Z 67.39630699157715
2018-11-03T17:50:00.000000000Z 2018-11-03T17:55:00.000000000Z used_percent mem 2018-11-03T17:50:20.000000000Z 64.16666507720947
2018-11-03T17:50:00.000000000Z 2018-11-03T17:55:00.000000000Z used_percent mem 2018-11-03T17:50:30.000000000Z 64.19951915740967
2018-11-03T17:50:00.000000000Z 2018-11-03T17:55:00.000000000Z used_percent mem 2018-11-03T17:50:40.000000000Z 64.2122745513916
2018-11-03T17:50:00.000000000Z 2018-11-03T17:55:00.000000000Z used_percent mem 2018-11-03T17:50:50.000000000Z 64.22209739685059
2018-11-03T17:50:00.000000000Z 2018-11-03T17:55:00.000000000Z used_percent mem 2018-11-03T17:51:00.000000000Z 64.6336555480957
2018-11-03T17:50:00.000000000Z 2018-11-03T17:55:00.000000000Z used_percent mem 2018-11-03T17:51:10.000000000Z 64.16516304016113
2018-11-03T17:50:00.000000000Z 2018-11-03T17:55:00.000000000Z used_percent mem 2018-11-03T17:51:20.000000000Z 64.18349742889404
2018-11-03T17:50:00.000000000Z 2018-11-03T17:55:00.000000000Z used_percent mem 2018-11-03T17:51:30.000000000Z 64.20474052429199
2018-11-03T17:50:00.000000000Z 2018-11-03T17:55:00.000000000Z used_percent mem 2018-11-03T17:51:40.000000000Z 68.65062713623047
2018-11-03T17:50:00.000000000Z 2018-11-03T17:55:00.000000000Z used_percent mem 2018-11-03T17:51:50.000000000Z 67.20139980316162
2018-11-03T17:50:00.000000000Z 2018-11-03T17:55:00.000000000Z used_percent mem 2018-11-03T17:52:00.000000000Z 70.9143877029419
2018-11-03T17:50:00.000000000Z 2018-11-03T17:55:00.000000000Z used_percent mem 2018-11-03T17:52:10.000000000Z 64.14549350738525
2018-11-03T17:50:00.000000000Z 2018-11-03T17:55:00.000000000Z used_percent mem 2018-11-03T17:52:20.000000000Z 64.15379047393799
2018-11-03T17:50:00.000000000Z 2018-11-03T17:55:00.000000000Z used_percent mem 2018-11-03T17:52:30.000000000Z 64.1592264175415
2018-11-03T17:50:00.000000000Z 2018-11-03T17:55:00.000000000Z used_percent mem 2018-11-03T17:52:40.000000000Z 64.18190002441406
2018-11-03T17:50:00.000000000Z 2018-11-03T17:55:00.000000000Z used_percent mem 2018-11-03T17:52:50.000000000Z 64.28837776184082
2018-11-03T17:50:00.000000000Z 2018-11-03T17:55:00.000000000Z used_percent mem 2018-11-03T17:53:00.000000000Z 64.29731845855713
2018-11-03T17:50:00.000000000Z 2018-11-03T17:55:00.000000000Z used_percent mem 2018-11-03T17:53:10.000000000Z 64.36963081359863
2018-11-03T17:50:00.000000000Z 2018-11-03T17:55:00.000000000Z used_percent mem 2018-11-03T17:53:20.000000000Z 64.37397003173828
2018-11-03T17:50:00.000000000Z 2018-11-03T17:55:00.000000000Z used_percent mem 2018-11-03T17:53:30.000000000Z 64.44413661956787
2018-11-03T17:50:00.000000000Z 2018-11-03T17:55:00.000000000Z used_percent mem 2018-11-03T17:53:40.000000000Z 64.42906856536865
2018-11-03T17:50:00.000000000Z 2018-11-03T17:55:00.000000000Z used_percent mem 2018-11-03T17:53:50.000000000Z 64.44573402404785
2018-11-03T17:50:00.000000000Z 2018-11-03T17:55:00.000000000Z used_percent mem 2018-11-03T17:54:00.000000000Z 64.48912620544434
2018-11-03T17:50:00.000000000Z 2018-11-03T17:55:00.000000000Z used_percent mem 2018-11-03T17:54:10.000000000Z 64.49522972106934
2018-11-03T17:50:00.000000000Z 2018-11-03T17:55:00.000000000Z used_percent mem 2018-11-03T17:54:20.000000000Z 64.48652744293213
2018-11-03T17:50:00.000000000Z 2018-11-03T17:55:00.000000000Z used_percent mem 2018-11-03T17:54:30.000000000Z 64.49949741363525
2018-11-03T17:50:00.000000000Z 2018-11-03T17:55:00.000000000Z used_percent mem 2018-11-03T17:54:40.000000000Z 64.4949197769165
2018-11-03T17:50:00.000000000Z 2018-11-03T17:55:00.000000000Z used_percent mem 2018-11-03T17:54:50.000000000Z 64.49787616729736
2018-11-03T17:50:00.000000000Z 2018-11-03T17:55:00.000000000Z used_percent mem 2018-11-03T17:55:00.000000000Z 64.49816226959229
```
{{% /truncate %}}
## Windowing data
Use the [`window()` function](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/transformations/window)
to group your data based on time bounds.
The most common parameter passed with the `window()` is `every` which
defines the duration of time between windows.
Other parameters are available, but for this example, window the base data
set into one minute windows.
```js
dataSet
|> window(every: 1m)
```
{{% note %}}
The `every` parameter supports all [valid duration units](/flux/v0.65/language/types/#duration-types),
including **calendar months (`1mo`)** and **years (`1y`)**.
{{% /note %}}
Each window of time is output in its own table containing all records that fall within the window.
{{% truncate %}}
###### window() output tables
```
Table: keys: [_start, _stop, _field, _measurement]
_start:time _stop:time _field:string _measurement:string _time:time _value:float
------------------------------ ------------------------------ ---------------------- ---------------------- ------------------------------ ----------------------------
2018-11-03T17:50:00.000000000Z 2018-11-03T17:51:00.000000000Z used_percent mem 2018-11-03T17:50:00.000000000Z 71.11611366271973
2018-11-03T17:50:00.000000000Z 2018-11-03T17:51:00.000000000Z used_percent mem 2018-11-03T17:50:10.000000000Z 67.39630699157715
2018-11-03T17:50:00.000000000Z 2018-11-03T17:51:00.000000000Z used_percent mem 2018-11-03T17:50:20.000000000Z 64.16666507720947
2018-11-03T17:50:00.000000000Z 2018-11-03T17:51:00.000000000Z used_percent mem 2018-11-03T17:50:30.000000000Z 64.19951915740967
2018-11-03T17:50:00.000000000Z 2018-11-03T17:51:00.000000000Z used_percent mem 2018-11-03T17:50:40.000000000Z 64.2122745513916
2018-11-03T17:50:00.000000000Z 2018-11-03T17:51:00.000000000Z used_percent mem 2018-11-03T17:50:50.000000000Z 64.22209739685059
Table: keys: [_start, _stop, _field, _measurement]
_start:time _stop:time _field:string _measurement:string _time:time _value:float
------------------------------ ------------------------------ ---------------------- ---------------------- ------------------------------ ----------------------------
2018-11-03T17:51:00.000000000Z 2018-11-03T17:52:00.000000000Z used_percent mem 2018-11-03T17:51:00.000000000Z 64.6336555480957
2018-11-03T17:51:00.000000000Z 2018-11-03T17:52:00.000000000Z used_percent mem 2018-11-03T17:51:10.000000000Z 64.16516304016113
2018-11-03T17:51:00.000000000Z 2018-11-03T17:52:00.000000000Z used_percent mem 2018-11-03T17:51:20.000000000Z 64.18349742889404
2018-11-03T17:51:00.000000000Z 2018-11-03T17:52:00.000000000Z used_percent mem 2018-11-03T17:51:30.000000000Z 64.20474052429199
2018-11-03T17:51:00.000000000Z 2018-11-03T17:52:00.000000000Z used_percent mem 2018-11-03T17:51:40.000000000Z 68.65062713623047
2018-11-03T17:51:00.000000000Z 2018-11-03T17:52:00.000000000Z used_percent mem 2018-11-03T17:51:50.000000000Z 67.20139980316162
Table: keys: [_start, _stop, _field, _measurement]
_start:time _stop:time _field:string _measurement:string _time:time _value:float
------------------------------ ------------------------------ ---------------------- ---------------------- ------------------------------ ----------------------------
2018-11-03T17:52:00.000000000Z 2018-11-03T17:53:00.000000000Z used_percent mem 2018-11-03T17:52:00.000000000Z 70.9143877029419
2018-11-03T17:52:00.000000000Z 2018-11-03T17:53:00.000000000Z used_percent mem 2018-11-03T17:52:10.000000000Z 64.14549350738525
2018-11-03T17:52:00.000000000Z 2018-11-03T17:53:00.000000000Z used_percent mem 2018-11-03T17:52:20.000000000Z 64.15379047393799
2018-11-03T17:52:00.000000000Z 2018-11-03T17:53:00.000000000Z used_percent mem 2018-11-03T17:52:30.000000000Z 64.1592264175415
2018-11-03T17:52:00.000000000Z 2018-11-03T17:53:00.000000000Z used_percent mem 2018-11-03T17:52:40.000000000Z 64.18190002441406
2018-11-03T17:52:00.000000000Z 2018-11-03T17:53:00.000000000Z used_percent mem 2018-11-03T17:52:50.000000000Z 64.28837776184082
Table: keys: [_start, _stop, _field, _measurement]
_start:time _stop:time _field:string _measurement:string _time:time _value:float
------------------------------ ------------------------------ ---------------------- ---------------------- ------------------------------ ----------------------------
2018-11-03T17:53:00.000000000Z 2018-11-03T17:54:00.000000000Z used_percent mem 2018-11-03T17:53:00.000000000Z 64.29731845855713
2018-11-03T17:53:00.000000000Z 2018-11-03T17:54:00.000000000Z used_percent mem 2018-11-03T17:53:10.000000000Z 64.36963081359863
2018-11-03T17:53:00.000000000Z 2018-11-03T17:54:00.000000000Z used_percent mem 2018-11-03T17:53:20.000000000Z 64.37397003173828
2018-11-03T17:53:00.000000000Z 2018-11-03T17:54:00.000000000Z used_percent mem 2018-11-03T17:53:30.000000000Z 64.44413661956787
2018-11-03T17:53:00.000000000Z 2018-11-03T17:54:00.000000000Z used_percent mem 2018-11-03T17:53:40.000000000Z 64.42906856536865
2018-11-03T17:53:00.000000000Z 2018-11-03T17:54:00.000000000Z used_percent mem 2018-11-03T17:53:50.000000000Z 64.44573402404785
Table: keys: [_start, _stop, _field, _measurement]
_start:time _stop:time _field:string _measurement:string _time:time _value:float
------------------------------ ------------------------------ ---------------------- ---------------------- ------------------------------ ----------------------------
2018-11-03T17:54:00.000000000Z 2018-11-03T17:55:00.000000000Z used_percent mem 2018-11-03T17:54:00.000000000Z 64.48912620544434
2018-11-03T17:54:00.000000000Z 2018-11-03T17:55:00.000000000Z used_percent mem 2018-11-03T17:54:10.000000000Z 64.49522972106934
2018-11-03T17:54:00.000000000Z 2018-11-03T17:55:00.000000000Z used_percent mem 2018-11-03T17:54:20.000000000Z 64.48652744293213
2018-11-03T17:54:00.000000000Z 2018-11-03T17:55:00.000000000Z used_percent mem 2018-11-03T17:54:30.000000000Z 64.49949741363525
2018-11-03T17:54:00.000000000Z 2018-11-03T17:55:00.000000000Z used_percent mem 2018-11-03T17:54:40.000000000Z 64.4949197769165
2018-11-03T17:54:00.000000000Z 2018-11-03T17:55:00.000000000Z used_percent mem 2018-11-03T17:54:50.000000000Z 64.49787616729736
Table: keys: [_start, _stop, _field, _measurement]
_start:time _stop:time _field:string _measurement:string _time:time _value:float
------------------------------ ------------------------------ ---------------------- ---------------------- ------------------------------ ----------------------------
2018-11-03T17:55:00.000000000Z 2018-11-03T17:55:00.000000000Z used_percent mem 2018-11-03T17:55:00.000000000Z 64.49816226959229
```
{{% /truncate %}}
When visualized in the InfluxDB UI, each window table is displayed in a different color.
![Windowed data](/img/simple-windowed-data.png)
## Aggregate data
[Aggregate functions](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/transformations/aggregates) take the values
of all rows in a table and use them to perform an aggregate operation.
The result is output as a new value in a single-row table.
Since windowed data is split into separate tables, aggregate operations run against
each table separately and output new tables containing only the aggregated value.
For this example, use the [`mean()` function](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/transformations/aggregates/mean)
to output the average of each window:
```js
dataSet
|> window(every: 1m)
|> mean()
```
{{% truncate %}}
###### mean() output tables
```
Table: keys: [_start, _stop, _field, _measurement]
_start:time _stop:time _field:string _measurement:string _value:float
------------------------------ ------------------------------ ---------------------- ---------------------- ----------------------------
2018-11-03T17:50:00.000000000Z 2018-11-03T17:51:00.000000000Z used_percent mem 65.88549613952637
Table: keys: [_start, _stop, _field, _measurement]
_start:time _stop:time _field:string _measurement:string _value:float
------------------------------ ------------------------------ ---------------------- ---------------------- ----------------------------
2018-11-03T17:51:00.000000000Z 2018-11-03T17:52:00.000000000Z used_percent mem 65.50651391347249
Table: keys: [_start, _stop, _field, _measurement]
_start:time _stop:time _field:string _measurement:string _value:float
------------------------------ ------------------------------ ---------------------- ---------------------- ----------------------------
2018-11-03T17:52:00.000000000Z 2018-11-03T17:53:00.000000000Z used_percent mem 65.30719598134358
Table: keys: [_start, _stop, _field, _measurement]
_start:time _stop:time _field:string _measurement:string _value:float
------------------------------ ------------------------------ ---------------------- ---------------------- ----------------------------
2018-11-03T17:53:00.000000000Z 2018-11-03T17:54:00.000000000Z used_percent mem 64.39330975214641
Table: keys: [_start, _stop, _field, _measurement]
_start:time _stop:time _field:string _measurement:string _value:float
------------------------------ ------------------------------ ---------------------- ---------------------- ----------------------------
2018-11-03T17:54:00.000000000Z 2018-11-03T17:55:00.000000000Z used_percent mem 64.49386278788249
Table: keys: [_start, _stop, _field, _measurement]
_start:time _stop:time _field:string _measurement:string _value:float
------------------------------ ------------------------------ ---------------------- ---------------------- ----------------------------
2018-11-03T17:55:00.000000000Z 2018-11-03T17:55:00.000000000Z used_percent mem 64.49816226959229
```
{{% /truncate %}}
Because each data point is contained in its own table, when visualized,
they appear as single, unconnected points.
![Aggregated windowed data](/img/simple-windowed-aggregate-data.png)
### Recreate the time column
**Notice the `_time` column is not in the [aggregated output tables](#mean-output-tables).**
Because records in each table are aggregated together, their timestamps no longer
apply and the column is removed from the group key and table.
Also notice the `_start` and `_stop` columns still exist.
These represent the lower and upper bounds of the time window.
Many Flux functions rely on the `_time` column.
To further process your data after an aggregate function, you need to re-add `_time`.
Use the [`duplicate()` function](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/transformations/duplicate) to
duplicate either the `_start` or `_stop` column as a new `_time` column.
```js
dataSet
|> window(every: 1m)
|> mean()
|> duplicate(column: "_stop", as: "_time")
```
{{% truncate %}}
###### duplicate() output tables
```
Table: keys: [_start, _stop, _field, _measurement]
_start:time _stop:time _field:string _measurement:string _time:time _value:float
------------------------------ ------------------------------ ---------------------- ---------------------- ------------------------------ ----------------------------
2018-11-03T17:50:00.000000000Z 2018-11-03T17:51:00.000000000Z used_percent mem 2018-11-03T17:51:00.000000000Z 65.88549613952637
Table: keys: [_start, _stop, _field, _measurement]
_start:time _stop:time _field:string _measurement:string _time:time _value:float
------------------------------ ------------------------------ ---------------------- ---------------------- ------------------------------ ----------------------------
2018-11-03T17:51:00.000000000Z 2018-11-03T17:52:00.000000000Z used_percent mem 2018-11-03T17:52:00.000000000Z 65.50651391347249
Table: keys: [_start, _stop, _field, _measurement]
_start:time _stop:time _field:string _measurement:string _time:time _value:float
------------------------------ ------------------------------ ---------------------- ---------------------- ------------------------------ ----------------------------
2018-11-03T17:52:00.000000000Z 2018-11-03T17:53:00.000000000Z used_percent mem 2018-11-03T17:53:00.000000000Z 65.30719598134358
Table: keys: [_start, _stop, _field, _measurement]
_start:time _stop:time _field:string _measurement:string _time:time _value:float
------------------------------ ------------------------------ ---------------------- ---------------------- ------------------------------ ----------------------------
2018-11-03T17:53:00.000000000Z 2018-11-03T17:54:00.000000000Z used_percent mem 2018-11-03T17:54:00.000000000Z 64.39330975214641
Table: keys: [_start, _stop, _field, _measurement]
_start:time _stop:time _field:string _measurement:string _time:time _value:float
------------------------------ ------------------------------ ---------------------- ---------------------- ------------------------------ ----------------------------
2018-11-03T17:54:00.000000000Z 2018-11-03T17:55:00.000000000Z used_percent mem 2018-11-03T17:55:00.000000000Z 64.49386278788249
Table: keys: [_start, _stop, _field, _measurement]
_start:time _stop:time _field:string _measurement:string _time:time _value:float
------------------------------ ------------------------------ ---------------------- ---------------------- ------------------------------ ----------------------------
2018-11-03T17:55:00.000000000Z 2018-11-03T17:55:00.000000000Z used_percent mem 2018-11-03T17:55:00.000000000Z 64.49816226959229
```
{{% /truncate %}}
## "Unwindow" aggregate tables
Keeping aggregate values in separate tables generally isn't the format in which you want your data.
Use the `window()` function to "unwindow" your data into a single infinite (`inf`) window.
```js
dataSet
|> window(every: 1m)
|> mean()
|> duplicate(column: "_stop", as: "_time")
|> window(every: inf)
```
{{% note %}}
Windowing requires a `_time` column which is why it's necessary to
[recreate the `_time` column](#recreate-the-time-column) after an aggregation.
{{% /note %}}
###### Unwindowed output table
```
Table: keys: [_start, _stop, _field, _measurement]
_start:time _stop:time _field:string _measurement:string _time:time _value:float
------------------------------ ------------------------------ ---------------------- ---------------------- ------------------------------ ----------------------------
2018-11-03T17:50:00.000000000Z 2018-11-03T17:55:00.000000000Z used_percent mem 2018-11-03T17:51:00.000000000Z 65.88549613952637
2018-11-03T17:50:00.000000000Z 2018-11-03T17:55:00.000000000Z used_percent mem 2018-11-03T17:52:00.000000000Z 65.50651391347249
2018-11-03T17:50:00.000000000Z 2018-11-03T17:55:00.000000000Z used_percent mem 2018-11-03T17:53:00.000000000Z 65.30719598134358
2018-11-03T17:50:00.000000000Z 2018-11-03T17:55:00.000000000Z used_percent mem 2018-11-03T17:54:00.000000000Z 64.39330975214641
2018-11-03T17:50:00.000000000Z 2018-11-03T17:55:00.000000000Z used_percent mem 2018-11-03T17:55:00.000000000Z 64.49386278788249
2018-11-03T17:50:00.000000000Z 2018-11-03T17:55:00.000000000Z used_percent mem 2018-11-03T17:55:00.000000000Z 64.49816226959229
```
With the aggregate values in a single table, data points in the visualization are connected.
![Unwindowed aggregate data](/img/simple-unwindowed-data.png)
## Summing up
You have now created a Flux query that windows and aggregates data.
The data transformation process outlined in this guide should be used for all aggregation operations.
Flux also provides the [`aggregateWindow()` function](/{{< latest "influxdb" "v2" >}}/reference/flux/stdlib/built-in/transformations/aggregates/aggregatewindow)
which performs all these separate functions for you.
The following Flux query will return the same results:
###### aggregateWindow function
```js
dataSet
|> aggregateWindow(every: 1m, fn: mean)
```

33
content/influxdb/v1.8/flux/installation.md Normal file
View File

@ -0,0 +1,33 @@
---
title: Enable Flux
description: Instructions for enabling Flux in your InfluxDB configuration.
menu:
influxdb_1_8:
name: Enable Flux
parent: Flux
weight: 1
---
Flux is packaged with **InfluxDB v1.8+** and does not require any additional installation,
however it is **disabled by default and needs to be enabled**.
## Enable Flux
Enable Flux by setting the `flux-enabled` option to `true` under the `[http]` section of your `influxdb.conf`:
###### influxdb.conf
```toml
# ...
[http]
# ...
flux-enabled = true
# ...
```
> The default location of your `influxdb.conf` depends on your operating system.
> More information is available in the [Configuring InfluxDB](/influxdb/v1.8/administration/config/#using-the-configuration-file) guide.
When InfluxDB starts, the Flux daemon starts as well and data can be queried using Flux.