updated flux guides with v2 information

pull/22/head
Scott Anderson 2019-01-22 08:24:11 -07:00
parent 11f4402511
commit 1f3708d13d
28 changed files with 98 additions and 80 deletions

View File

@ -11,9 +11,9 @@ menu:
weight: 3
---
With Flux, data can be grouped by any column in your queried data set.
"Grouping" is accomplished by partitioning data into tables where each row shares a common value for specified columns.
This guide walks through grouping data in Flux with examples of how data is shaped in the process.
With Flux, you can group data by any column in your queried data set.
"Grouping" partitions data into tables in which each row shares a common value for specified columns.
This guide walks through grouping data in Flux and provides examples of how data is shaped in the process.
## Group keys
Every table has a **group key** a list of columns which for which every row in the table has the same value.
@ -44,22 +44,13 @@ dataStream
The `group()` function has the following parameters:
### by
An explicit method for defining the group key with an array of strings.
Only columns specified are included in the output group key.
### columns
The list of columns to include or exclude (depending on the [mode](#mode)) in the grouping operation.
### except
An implicit method for defining the group key with an array of strings.
All columns **except** those specified are included in the output group key.
### mode
The method used to define the group and resulting group key.
Possible values include `by` and `except`.
### none
A boolean that removes all grouping and outputs everything as a single table.
---
_For more information, see the [`group()` function](/v2.0/reference/flux/functions/transformations/group)._
---
## Example grouping operations
To illustrate how grouping works, define a `dataSet` variable that queries System
@ -80,9 +71,11 @@ dataSet = from(bucket: "telegraf/autogen")
|> drop(columns: ["host"])
```
> This example drops the `host` column from the returned data since the CPU data
> is only tracked for a single host and it simplifies the output tables.
> Don't drop the `host` column if monitoring multiple hosts.
{{% note %}}
This example drops the `host` column from the returned data since the CPU data
is only tracked for a single host and it simplifies the output tables.
Don't drop the `host` column if monitoring multiple hosts.
{{% /note %}}
{{% truncate %}}
```
@ -158,7 +151,7 @@ Table: keys: [_start, _stop, _field, _measurement, cpu]
**Note that the group key is output with each table: `Table: keys: <group-key>`.**
![Group example data set](/img/flux/grouping-data-set.png)
![Group example data set](/img/grouping-data-set.png)
### Group by CPU
Group the `dataSet` stream by the `cpu` column.
@ -168,7 +161,8 @@ dataSet
|> group(columns: ["cpu"])
```
This won't actually change the structure of the data since it already has `cpu` in the group key and is therefore grouped by `cpu`.
This won't actually change the structure of the data since it already has `cpu`
in the group key and is therefore grouped by `cpu`.
However, notice that it does change the group key:
{{% truncate %}}
@ -246,7 +240,7 @@ Table: keys: [cpu]
The visualization remains the same.
![Group by CPU](/img/flux/grouping-data-set.png)
![Group by CPU](/img/grouping-data-set.png)
### Group by time
Grouping data by the `_time` column is a good illustration of how grouping changes the structure of your data.
@ -369,20 +363,22 @@ Table: keys: [_time]
{{% /truncate %}}
Because each timestamp is a structured as a separate table, when visualized, they appear as individual, unconnected points.
Even though there are multiple records per timestamp, it will only visualize the last record of the table.
Even though there are multiple records per timestamp, it will only visualize the last record of group's table.
![Group by time](/img/flux/grouping-by-time.png)
![Group by time](/img/grouping-by-time.png)
{{% note %}}
With some further processing, you could calculate the average CPU usage across all CPUs per point
of time and group them into a single table, but we won't cover that in this example.
If you're interested in running and visualizing this yourself, here's what the query would look like:
> With some further processing, you could calculate the average CPU usage across all CPUs per point
> of time and group them into a single table, but we won't cover that in this example.
> If you're interested in running and visualizing this yourself, here's what the query would look like:
>
```js
dataSet
|> group(columns: ["_time"])
|> mean()
|> group(columns: ["_value", "_time"], mode: "except")
```
{{% /note %}}
## Group by CPU and time
Group by the `cpu` and `_time` columns.
@ -661,7 +657,7 @@ Table: keys: [_time, cpu]
When visualized, tables appear as individual, unconnected points.
![Group by CPU and time](/img/flux/grouping-by-cpu-time.png)
![Group by CPU and time](/img/grouping-by-cpu-time.png)
Grouping by `cpu` and `_time` is a good illustration of how grouping works.

View File

@ -18,7 +18,7 @@ The [`histogram()` function](/v2.0/reference/flux/functions/transformations/hist
cumulative distribution of a dataset by counting data frequencies for a list of "bins."
A **bin** is simply a range in which a data point falls.
All data points that are less than or equal to the bound are counted in the bin.
In the histogram output, a column is added (le) that represents the upper bounds of of each bin.
In the histogram output, a column is added (`le`) that represents the upper bounds of of each bin.
Bin counts are cumulative.
```js
@ -31,12 +31,14 @@ from(bucket:"telegraf/autogen")
|> histogram(bins: [0.0, 10.0, 20.0, 30.0])
```
> Values output by the `histogram` function represent points of data aggregated over time.
> Since values do not represent single points in time, there is no `_time` column in the output table.
{{% note %}}
Values output by the `histogram` function represent points of data aggregated over time.
Since values do not represent single points in time, there is no `_time` column in the output table.
{{% /note %}}
## Bin helper functions
Flux provides two helper functions for generating histogram bins.
Each generates and outputs an array of floats designed to be used in the `histogram()` function's `bins` parameter.
Each generates an array of floats designed to be used in the `histogram()` function's `bins` parameter.
### linearBins()
The [`linearBins()` function](/v2.0/reference/flux/functions/misc/linearbins) generates a list of linearly separated floats.

View File

@ -10,12 +10,12 @@ menu:
---
The [`join()` function](/v2.0/reference/flux/functions/transformations/join) merges two or more
input streams whose values are equal on a set of common columns into a single output stream.
input streams, whose values are equal on a set of common columns, into a single output stream.
Flux allows you to join on any columns common between two data streams and opens the door
for operations such as cross-measurement joins and math across measurements.
To illustrate a join operation, use data captured by Telegraf and and stored in
InfluxDB with a default TICK stack installation - memory usage and processes.
InfluxDB - memory usage and processes.
In this guide, we'll join two data streams, one representing memory usage and the other representing the
total number of running processes, then calculate the average memory usage per running process.
@ -142,7 +142,7 @@ In the example below, `mem` is the alias for `memUsed` and `proc` is the alias f
##### `on`
An array of strings defining the columns on which the tables will be joined.
_**Both tables must have all columns defined in this list.**_
_**Both tables must have all columns specified in this list.**_
```js
join(
@ -203,9 +203,11 @@ Notice the output table includes the following columns:
These represent the columns with values unique to the two input tables.
## Calculate and create a new table
With the two streams of data joined into a single table, use the [`map()` function](/v2.0/reference/flux/functions/transformations/map)
to build a new table by mapping the existing `_time` column to a new `_time` column and dividing `_value_mem` by `_value_proc`
and mapping it to a new `_value` column.
With the two streams of data joined into a single table, use the
[`map()` function](/v2.0/reference/flux/functions/transformations/map)
to build a new table by mapping the existing `_time` column to a new `_time`
column and dividing `_value_mem` by `_value_proc` and mapping it to a
new `_value` column.
```js
join(tables: {mem:memUsed, proc:procTotal}, on: ["_time", "_stop", "_start", "host"])

View File

@ -10,8 +10,8 @@ menu:
---
Regular expressions (regexes) are incredibly powerful when matching patterns in large collections of data.
With Flux, regular expressions are primarily used for evaluation logic in operations such as filtering rows,
dropping and keeping columns, state detection, etc.
With Flux, regular expressions are primarily used for evaluation logic in predicate functions for things
such as filtering rows, dropping and keeping columns, state detection, etc.
This guide shows how to use regular expressions in your Flux scripts.
## Go regular expression syntax
@ -71,7 +71,7 @@ The following example drops columns whose names do not being with `_`.
from(bucket: "telegraf/autogen")
|> range(start: -15m)
|> filter(fn: (r) => r._measurement == "mem")
|> drop(fn: (col) => col !~ /_.*/)
|> drop(fn: (column) => column !~ /_.*/)
```
## Helpful links

View File

@ -11,7 +11,9 @@ menu:
weight: 6
---
The [`sort()`function](/v2.0/reference/flux/functions/transformations/sort) orders the records within each table. The following example orders system uptime first by region, then host, then value.
The [`sort()`function](/v2.0/reference/flux/functions/transformations/sort)
orders the records within each table.
The following example orders system uptime first by region, then host, then value.
```js
from(bucket:"telegraf/autogen")
@ -23,7 +25,9 @@ from(bucket:"telegraf/autogen")
|> sort(columns:["region", "host", "_value"])
```
The [`limit()` function](/v2.0/reference/flux/functions/transformations/limit) limit the number of records in output tables to a fixed number (n). The following example shows up to 10 records from the past hour.
The [`limit()` function](/v2.0/reference/flux/functions/transformations/limit)
limits the number of records in output tables to a fixed number, `n`.
The following example shows up to 10 records from the past hour.
```js
from(bucket:"telegraf/autogen")
@ -31,7 +35,9 @@ from(bucket:"telegraf/autogen")
|> limit(n:10)
```
You can use `sort()` and `limit()` together to show the top N records. The example below returns the 10 top system uptime values sorted first by region, then host, then value.
You can use `sort()` and `limit()` together to show the top N records.
The example below returns the 10 top system uptime values sorted first by
region, then host, then value.
```js
from(bucket:"telegraf/autogen")
@ -44,4 +50,7 @@ from(bucket:"telegraf/autogen")
|> limit(n:10)
```
You now have created a Flux query that sorts and limits data. Flux also provides the [`top()`](/v2.0/reference/flux/functions/transformations/selectors/top) and [`bottom()`](/v2.0/reference/flux/functions/transformations/selectors/bottom) functions to perform both of these functions at the same time.
You now have created a Flux query that sorts and limits data.
Flux also provides the [`top()`](/v2.0/reference/flux/functions/transformations/selectors/top)
and [`bottom()`](/v2.0/reference/flux/functions/transformations/selectors/bottom)
functions to perform both of these functions at the same time.

View File

@ -16,9 +16,11 @@ or "windowing" data, then aggregating windowed values into a new value.
This guide walks through windowing and aggregating data with Flux and demonstrates
how data is shaped in the process.
> The following example is an in-depth walk through of the steps required to window and aggregate data.
> The [`aggregateWindow()` function](#summing-up) performs these operations for you, but understanding
> how data is shaped in the process helps to successfully create your desired output.
{{% note %}}
The following example is an in-depth walk-through of the steps required to window and aggregate data.
The [`aggregateWindow()` function](#summing-up) performs these operations for you, but understanding
how data is shaped in the process helps to successfully create your desired output.
{{% /note %}}
## Data set
For the purposes of this guide, define a variable that represents your base data set.
@ -34,10 +36,12 @@ dataSet = from(bucket: "telegraf/autogen")
|> drop(columns: ["host"])
```
> This example drops the `host` column from the returned data since the memory data
> is only tracked for a single host and it simplifies the output tables.
> Dropping the `host` column is column is optional and not recommended if monitoring memory
> on multiple hosts.
{{% note %}}
This example drops the `host` column from the returned data since the memory data
is only tracked for a single host and it simplifies the output tables.
Dropping the `host` column is column is optional and not recommended if monitoring memory
on multiple hosts.
{{% /note %}}
`dataSet` can now be used to represent your base data, which will look similar to the following:
@ -81,9 +85,12 @@ Table: keys: [_start, _stop, _field, _measurement]
{{% /truncate %}}
## Windowing data
Use the [`window()` function](/v2.0/reference/flux/functions/transformations/window) to group your data based on time bounds.
The most common parameter passed with the `window()` is `every` which defines the duration of time between windows.
Other parameters are available, but for this example, window the base data set into one minute windows.
Use the [`window()` function](/v2.0/reference/flux/functions/transformations/window)
to group your data based on time bounds.
The most common parameter passed with the `window()` is `every` which
defines the duration of time between windows.
Other parameters are available, but for this example, window the base data
set into one minute windows.
```js
dataSet
@ -157,9 +164,9 @@ Table: keys: [_start, _stop, _field, _measurement]
```
{{% /truncate %}}
When visualized in [Chronograf](/chronograf/latest/), each window table is displayed in a different color.
When visualized in the InfluxDB UI, each window table is displayed in a different color.
![Windowed data](/img/flux/simple-windowed-data.png)
![Windowed data](/img/simple-windowed-data.png)
## Aggregate data
[Aggregate functions](/v2.0/reference/flux/functions/transformations/aggregates) take the values
@ -221,7 +228,7 @@ Table: keys: [_start, _stop, _field, _measurement]
Because each data point is contained in its own table, when visualized,
they appear as single, unconnected points.
![Aggregated windowed data](/img/flux/simple-windowed-aggregate-data.png)
![Aggregated windowed data](/img/simple-windowed-aggregate-data.png)
### Recreate the time column
**Notice the `_time` column is not in the [aggregated output tables](#mean-output-tables).**
@ -232,7 +239,7 @@ Also notice the `_start` and `_stop` columns still exist.
These represent the lower and upper bounds of the time window.
Many Flux functions rely on the `_time` column.
To further process your data after an aggregate function, you need to add `_time` back in.
To further process your data after an aggregate function, you need to re-add `_time`.
Use the [`duplicate()` function](/v2.0/reference/flux/functions/transformations/duplicate) to
duplicate either the `_start` or `_stop` column as a new `_time` column.
@ -295,8 +302,10 @@ dataSet
|> window(every: inf)
```
> Windowing requires a `_time` column which is why it's necessary to
> [recreate the `_time` column](#recreate-the-time-column) after an aggregation.
{{% note %}}
Windowing requires a `_time` column which is why it's necessary to
[recreate the `_time` column](#recreate-the-time-column) after an aggregation.
{{% /note %}}
###### Unwindowed output table
```
@ -313,7 +322,7 @@ Table: keys: [_start, _stop, _field, _measurement]
With the aggregate values in a single table, data points in the visualization are connected.
![Unwindowed aggregate data](/img/flux/simple-unwindowed-data.png)
![Unwindowed aggregate data](/img/simple-unwindowed-data.png)
## Summing up
You have now created a Flux query that windows and aggregates data.

View File

@ -5,7 +5,7 @@ description: >
which is designed for querying, analyzing, and acting on data.
menu:
v2_0_ref:
name: Flux language specification
name: Flux specification
parent: Flux query language
weight: 5
---

View File

@ -3,7 +3,7 @@ title: Assignment and scope
description: An assignment binds an identifier to a variable, option, or function. Every identifier in a program must be assigned.
menu:
v2_0_ref:
parent: Flux language specification
parent: Flux specification
name: Assignment and scope
weight: 20
---

View File

@ -3,7 +3,7 @@ title: Blocks
description: A block is a possibly empty sequence of statements within matching braces ({}).
menu:
v2_0_ref:
parent: Flux language specification
parent: Flux specification
name: Blocks
weight: 30
---

View File

@ -6,7 +6,7 @@ description: >
menu:
v2_0_ref:
name: Built-ins
parent: Flux language specification
parent: Flux specification
weight: 80
---

View File

@ -4,7 +4,7 @@ description: Flux employs a basic data model built from basic data types. The da
menu:
v2_0_ref:
name: Data model
parent: Flux language specification
parent: Flux specification
weight: 1
---

View File

@ -3,7 +3,7 @@ title: Expressions
description: An expression specifies the computation of a value by applying the operators and functions to operands.
menu:
v2_0_ref:
parent: Flux language specification
parent: Flux specification
name: Expressions
weight: 40
---

View File

@ -3,7 +3,7 @@ title: Lexical elements
description: Descriptions of Flux comments, tokens, identifiers, keywords, and other lexical elements.
menu:
v2_0_ref:
parent: Flux language specification
parent: Flux specification
name: Lexical elements
weight: 50
---

View File

@ -3,7 +3,7 @@ title: Notation
description: Notation principles for the Flux functional data scripting language.
menu:
v2_0_ref:
parent: Flux language specification
parent: Flux specification
name: Notation
weight: 60
---

View File

@ -4,7 +4,7 @@ description: Flux supports many types of operators including arithmetic operator
menu:
v2_0_ref:
name: Operators
parent: Flux language specification
parent: Flux specification
weight: 130
---

View File

@ -3,7 +3,7 @@ title: Options
description: placeholder
menu:
v2_0_ref:
parent: Flux language specification
parent: Flux specification
name: Options
weight: 110
---

View File

@ -8,7 +8,7 @@ aliases:
- /v2.0/reference/flux/language/programs
menu:
v2_0_ref:
parent: Flux language specification
parent: Flux specification
name: Packages
weight: 70
---

View File

@ -3,7 +3,7 @@ title: Representation
description: Source code is encoded in UTF-8. The text need not be canonicalized.
menu:
v2_0_ref:
parent: Flux language specification
parent: Flux specification
name: Representation
weight: 80
---

View File

@ -3,7 +3,7 @@ title: Side effects
description: A summary of side effects in the Flux functional data scripting language.
menu:
v2_0_ref:
parent: Flux language specification
parent: Flux specification
name: Side effects
weight: 90
---

View File

@ -3,7 +3,7 @@ title: Statements
description: Statements control execution in the Flux functional data scripting language.
menu:
v2_0_ref:
parent: Flux language specification
parent: Flux specification
name: Statements
weight: 100
---

View File

@ -3,7 +3,7 @@ title: Types
description: A type defines the set of values and operations on those values. Types are never explicitly declared as part of the syntax. Types are always inferred from the usage of the value.
menu:
v2_0_ref:
parent: Flux language specification
parent: Flux specification
name: Types
weight: 110
---

View File

@ -3,7 +3,7 @@ title: Variables
description: Flux variables hold values. A variable can only hold values defined by its type.
menu:
v2_0_ref:
parent: Flux language specification
parent: Flux specification
name: Variables
weight: 120
---

Binary file not shown.

After

Width:  |  Height:  |  Size: 13 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 12 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 23 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 14 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 12 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 19 KiB