Merge pull request #937 from influxdata/flux/get-record-column

Flux 'getRecord' and 'getColumn' functions
pull/966/head
Scott Anderson 2020-04-20 15:33:19 -06:00 committed by GitHub
commit 7acdad37e2
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
35 changed files with 203 additions and 100 deletions

View File

@ -22,9 +22,9 @@ This lets you, for example, dynamically set variables using query results.
**To extract scalar values from output:** **To extract scalar values from output:**
1. [Extract a table](#extract-a-table). 1. [Extract a column from the input stream](#extract-a-column)
2. [Extract a column from the table](#extract-a-column-from-the-table) _**or**_ [extract a row from the input stream](#extract-a-row).
_**or**_ [extract a row from the table](#extract-a-row-from-the-table). 2. Use the returned array or object to reference scalar values.
_The samples on this page use the [sample data provided below](#sample-data)._ _The samples on this page use the [sample data provided below](#sample-data)._
@ -38,61 +38,33 @@ _The samples on this page use the [sample data provided below](#sample-data)._
See [#15321](https://github.com/influxdata/influxdb/issues/15231). See [#15321](https://github.com/influxdata/influxdb/issues/15231).
{{% /warn %}} {{% /warn %}}
## Extract a table ## Table extraction
Flux formats query results as a stream of tables. Flux formats query results as a stream of tables.
To extract a scalar value from a stream of tables, you must first extract a single table. Both [`findColumn()`](/v2.0/reference/flux/stdlib/built-in/transformations/stream-table/findcolumn/)
and [`findRecord()`](/v2.0/reference/flux/stdlib/built-in/transformations/stream-table/findrecord/)
to extract a single table from the stream of tables. extract the first table in a stream of tables whose [group key](/v2.0/reference/glossary/#group-key)
values match the `fn` [predicate function](/v2.0/reference/glossary/#predicate-function).
{{% note %}}
If query results include only one table, it is still formatted as a stream of tables.
You still must extract that table from the stream.
{{% /note %}}
Use [`tableFind()`](/v2.0/reference/flux/stdlib/built-in/transformations/stream-table/tablefind/)
to extract the **first** table whose [group key](/v2.0/reference/glossary/#group-key)
values match the `fn` [predicate function](/v2.0/reference/glossary/#predicate-expression).
The predicate function requires a `key` object, which represents the group key of
each table.
```js
sampleData
|> tableFind(fn: (key) =>
key._field == "temp" and
key.location == "sfo"
)
```
The example above returns a single table:
| _time | location | _field | _value |
|:----- |:--------:|:------:| ------:|
| 2019-11-01T12:00:00Z | sfo | temp | 65.1 |
| 2019-11-01T13:00:00Z | sfo | temp | 66.2 |
| 2019-11-01T14:00:00Z | sfo | temp | 66.3 |
| 2019-11-01T15:00:00Z | sfo | temp | 66.8 |
{{% note %}} {{% note %}}
#### Extract the correct table #### Extract the correct table
Flux functions do not guarantee table order and `tableFind()` returns only the Flux functions do not guarantee table order.
**first** table that matches the `fn` predicate. `findColumn()` and `findRecord` extract only the **first** table that matches the `fn` predicate.
To extract the table that includes the data you actually want, be very specific in To extract the correct table, be very specific in your predicate function or
your predicate function or filter and transform your data to minimize the number filter and transform your data to minimize the number of tables piped-forward into the functions.
of tables piped-forward into `tableFind()`.
{{% /note %}} {{% /note %}}
## Extract a column from the table ## Extract a column
Use the [`getColumn()` function](/v2.0/reference/flux/stdlib/built-in/transformations/stream-table/getcolumn/) Use the [`findColumn()` function](/v2.0/reference/flux/stdlib/built-in/transformations/stream-table/findcolumn/)
to output an array of values from a specific column in the extracted table. to output an array of values from a specific column in the extracted table.
_See [Sample data](#sample-data) below._
```js ```js
sampleData sampleData
|> tableFind(fn: (key) => |> findColumn(
key._field == "temp" and fn: (key) => key._field == "temp" and key.location == "sfo",
key.location == "sfo" column: "_value"
) )
|> getColumn(column: "_value")
// Returns [65.1, 66.2, 66.3, 66.8] // Returns [65.1, 66.2, 66.3, 66.8]
``` ```
@ -103,13 +75,14 @@ In the example below, `SFOTemps` represents the array of values.
Reference a specific index (integer starting from `0`) in the array to return the Reference a specific index (integer starting from `0`) in the array to return the
value at that index. value at that index.
_See [Sample data](#sample-data) below._
```js ```js
SFOTemps = sampleData SFOTemps = sampleData
|> tableFind(fn: (key) => |> findColumn(
key._field == "temp" and fn: (key) => key._field == "temp" and key.location == "sfo",
key.location == "sfo" column: "_value"
) )
|> getColumn(column: "_value")
SFOTemps SFOTemps
// Returns [65.1, 66.2, 66.3, 66.8] // Returns [65.1, 66.2, 66.3, 66.8]
@ -121,19 +94,18 @@ SFOTemps[2]
// Returns 66.3 // Returns 66.3
``` ```
## Extract a row from the table ## Extract a row
Use the [`getRecord()` function](/v2.0/reference/flux/stdlib/built-in/transformations/stream-table/getrecord/) Use the [`findRecord()` function](/v2.0/reference/flux/stdlib/built-in/transformations/stream-table/findrecord/)
to output data from a single row in the extracted table. to output data from a single row in the extracted table.
Specify the index of the row to output using the `idx` parameter. Specify the index of the row to output using the `idx` parameter.
The function outputs an object with key-value pairs for each column. The function outputs an object with key-value pairs for each column.
```js ```js
sampleData sampleData
|> tableFind(fn: (key) => |> findRecord(
key._field == "temp" and fn: (key) => key._field == "temp" and key.location == "sfo",
key.location == "sfo" idx: 0
) )
|> getRecord(idx: 0)
// Returns { // Returns {
// _time:2019-11-11T12:00:00Z, // _time:2019-11-11T12:00:00Z,
@ -151,11 +123,10 @@ keys in the object.
```js ```js
tempInfo = sampleData tempInfo = sampleData
|> tableFind(fn: (key) => |> findRecord(
key._field == "temp" and fn: (key) => key._field == "temp" and key.location == "sfo",
key.location == "sfo" idx: 0
) )
|> getRecord(idx: 0)
tempInfo tempInfo
// Returns { // Returns {
@ -180,8 +151,10 @@ Create custom helper functions to extract scalar values from query output.
// Define a helper function to extract field values // Define a helper function to extract field values
getFieldValue = (tables=<-, field) => { getFieldValue = (tables=<-, field) => {
extract = tables extract = tables
|> tableFind(fn: (key) => key._field == field) |> findColumn(
|> getColumn(column: "_value") fn: (key) => key._field == field,
column: "_value"
)
return extract[0] return extract[0]
} }
@ -200,8 +173,10 @@ lastJFKTemp
// Define a helper function to extract a row as an object // Define a helper function to extract a row as an object
getRow = (tables=<-, field, idx=0) => { getRow = (tables=<-, field, idx=0) => {
extract = tables extract = tables
|> tableFind(fn: (key) => true) |> findRecord(
|> getRecord(idx: idx) fn: (key) => true,
idx: idx
)
return extract return extract
} }

View File

@ -11,7 +11,7 @@ menu:
v2_0_ref: v2_0_ref:
name: columns name: columns
parent: built-in-transformations parent: built-in-transformations
weight: 401 weight: 402
--- ---
The `columns()` function lists the column labels of input tables. The `columns()` function lists the column labels of input tables.

View File

@ -8,7 +8,7 @@ menu:
v2_0_ref: v2_0_ref:
name: cumulativeSum name: cumulativeSum
parent: built-in-transformations parent: built-in-transformations
weight: 401 weight: 402
--- ---
The `cumulativeSum()` function computes a running sum for non-null records in the table. The `cumulativeSum()` function computes a running sum for non-null records in the table.

View File

@ -8,7 +8,7 @@ menu:
v2_0_ref: v2_0_ref:
name: drop name: drop
parent: built-in-transformations parent: built-in-transformations
weight: 401 weight: 402
--- ---
The `drop()` function removes specified columns from a table. The `drop()` function removes specified columns from a table.

View File

@ -8,7 +8,7 @@ menu:
v2_0_ref: v2_0_ref:
name: duplicate name: duplicate
parent: built-in-transformations parent: built-in-transformations
weight: 401 weight: 402
--- ---
The `duplicate()` function duplicates a specified column in a table. The `duplicate()` function duplicates a specified column in a table.

View File

@ -7,7 +7,7 @@ menu:
v2_0_ref: v2_0_ref:
name: elapsed name: elapsed
parent: built-in-transformations parent: built-in-transformations
weight: 401 weight: 402
--- ---
The `elapsed()` function returns the time between subsequent records. The `elapsed()` function returns the time between subsequent records.

View File

@ -8,7 +8,7 @@ menu:
v2_0_ref: v2_0_ref:
name: fill name: fill
parent: built-in-transformations parent: built-in-transformations
weight: 401 weight: 402
--- ---
The `fill()` function replaces all null values in an input stream with a non-null value. The `fill()` function replaces all null values in an input stream with a non-null value.

View File

@ -8,7 +8,7 @@ menu:
v2_0_ref: v2_0_ref:
name: filter name: filter
parent: built-in-transformations parent: built-in-transformations
weight: 401 weight: 402
v2.0/tags: [exists] v2.0/tags: [exists]
--- ---

View File

@ -8,7 +8,7 @@ menu:
v2_0_ref: v2_0_ref:
name: group name: group
parent: built-in-transformations parent: built-in-transformations
weight: 401 weight: 402
--- ---
The `group()` function groups records based on their values for specific columns. The `group()` function groups records based on their values for specific columns.

View File

@ -8,7 +8,7 @@ menu:
v2_0_ref: v2_0_ref:
name: histogram name: histogram
parent: built-in-transformations parent: built-in-transformations
weight: 401 weight: 402
--- ---
The `histogram()` function approximates the cumulative distribution of a dataset by counting data frequencies for a list of bins. The `histogram()` function approximates the cumulative distribution of a dataset by counting data frequencies for a list of bins.

View File

@ -10,7 +10,7 @@ menu:
v2_0_ref: v2_0_ref:
name: hourSelection name: hourSelection
parent: built-in-transformations parent: built-in-transformations
weight: 401 weight: 402
--- ---
The `hourSelection()` function retains all rows with time values in a specified hour range. The `hourSelection()` function retains all rows with time values in a specified hour range.

View File

@ -8,7 +8,7 @@ menu:
v2_0_ref: v2_0_ref:
name: join name: join
parent: built-in-transformations parent: built-in-transformations
weight: 401 weight: 402
related: related:
- /v2.0/reference/flux/stdlib/built-in/transformations/union/ - /v2.0/reference/flux/stdlib/built-in/transformations/union/
--- ---

View File

@ -8,7 +8,7 @@ menu:
v2_0_ref: v2_0_ref:
name: keep name: keep
parent: built-in-transformations parent: built-in-transformations
weight: 401 weight: 402
--- ---
The `keep()` function returns a table containing only the specified columns, ignoring all others. The `keep()` function returns a table containing only the specified columns, ignoring all others.

View File

@ -11,7 +11,7 @@ menu:
v2_0_ref: v2_0_ref:
name: keys name: keys
parent: built-in-transformations parent: built-in-transformations
weight: 401 weight: 402
--- ---
The `keys()` function outputs the group key of input tables. The `keys()` function outputs the group key of input tables.

View File

@ -8,7 +8,7 @@ menu:
v2_0_ref: v2_0_ref:
name: keyValues name: keyValues
parent: built-in-transformations parent: built-in-transformations
weight: 401 weight: 402
--- ---
The `keyValues()` function returns a table with the input table's group key plus two columns, The `keyValues()` function returns a table with the input table's group key plus two columns,

View File

@ -8,7 +8,7 @@ menu:
v2_0_ref: v2_0_ref:
name: limit name: limit
parent: built-in-transformations parent: built-in-transformations
weight: 401 weight: 402
related: related:
- /v2.0/reference/flux/stdlib/built-in/transformations/tail/ - /v2.0/reference/flux/stdlib/built-in/transformations/tail/
- https://docs.influxdata.com/influxdb/latest/query_language/data_exploration/#the-limit-and-slimit-clauses, InfluxQL LIMIT - https://docs.influxdata.com/influxdb/latest/query_language/data_exploration/#the-limit-and-slimit-clauses, InfluxQL LIMIT

View File

@ -8,7 +8,7 @@ menu:
v2_0_ref: v2_0_ref:
name: map name: map
parent: built-in-transformations parent: built-in-transformations
weight: 401 weight: 402
v2.0/tags: [exists] v2.0/tags: [exists]
--- ---

View File

@ -8,7 +8,7 @@ menu:
v2_0_ref: v2_0_ref:
name: pivot name: pivot
parent: built-in-transformations parent: built-in-transformations
weight: 401 weight: 402
--- ---
The `pivot()` function collects values stored vertically (column-wise) in a table The `pivot()` function collects values stored vertically (column-wise) in a table

View File

@ -8,7 +8,7 @@ menu:
v2_0_ref: v2_0_ref:
name: range name: range
parent: built-in-transformations parent: built-in-transformations
weight: 401 weight: 402
--- ---
The `range()` function filters records based on time bounds. The `range()` function filters records based on time bounds.

View File

@ -8,7 +8,7 @@ menu:
v2_0_ref: v2_0_ref:
name: rename name: rename
parent: built-in-transformations parent: built-in-transformations
weight: 401 weight: 402
--- ---
The `rename()` function renames specified columns in a table. The `rename()` function renames specified columns in a table.

View File

@ -8,7 +8,7 @@ menu:
v2_0_ref: v2_0_ref:
name: set name: set
parent: built-in-transformations parent: built-in-transformations
weight: 401 weight: 402
--- ---
The `set()` function assigns a static value to each record in the input table. The `set()` function assigns a static value to each record in the input table.

View File

@ -8,7 +8,7 @@ menu:
v2_0_ref: v2_0_ref:
name: sort name: sort
parent: built-in-transformations parent: built-in-transformations
weight: 401 weight: 402
--- ---
The `sort()` function orders the records within each table. The `sort()` function orders the records within each table.

View File

@ -8,7 +8,7 @@ menu:
v2_0_ref: v2_0_ref:
name: stateCount name: stateCount
parent: built-in-transformations parent: built-in-transformations
weight: 401 weight: 402
--- ---
The `stateCount()` function computes the number of consecutive records in a given state. The `stateCount()` function computes the number of consecutive records in a given state.

View File

@ -8,7 +8,7 @@ menu:
v2_0_ref: v2_0_ref:
name: stateDuration name: stateDuration
parent: built-in-transformations parent: built-in-transformations
weight: 401 weight: 402
--- ---
The `stateDuration()` function computes the duration of a given state. The `stateDuration()` function computes the duration of a given state.

View File

@ -13,12 +13,34 @@ menu:
name: Stream & table name: Stream & table
parent: built-in-transformations parent: built-in-transformations
v2.0/tags: [transformations, built-in, functions, stream, table] v2.0/tags: [transformations, built-in, functions, stream, table]
related:
- /v2.0/query-data/flux/scalar-values/
--- ---
Use stream and table functions to extract a table from a stream of tables and access its Use stream and table functions to extract a table from a stream of tables and access its
columns and records. columns and records.
##### Example stream and table functions {{< children type="functions" >}}
### Example stream and table functions
##### Recommended usage
```js
data = from(bucket:"example-bucket")
|> range(start: -5m)
|> filter(fn:(r) => r._measurement == "cpu")
// Extract the "_value" column from the table
data
|> findColumn(fn: (key) => key._field == "usage_idle", column: "_value")
// Extract the first record from the table
data
|> findRecord(fn: (key) => key._field == "usage_idle", idx: 0)
```
##### Alternate usage
```js ```js
data = from(bucket:"example-bucket") data = from(bucket:"example-bucket")
|> range(start: -5m) |> range(start: -5m)
@ -33,5 +55,3 @@ values = t |> getColumn(column: "_value")
// Extract the first record from the table // Extract the first record from the table
r0 = t |> getRecord(idx: 0) r0 = t |> getRecord(idx: 0)
``` ```
{{< children type="functions" >}}

View File

@ -0,0 +1,54 @@
---
title: findColumn() function
description: >
The `findColumn()` function returns an array of values in a specified column from the
first table in a stream of tables where group key values match the specified predicate.
menu:
v2_0_ref:
name: findColumn
parent: Stream & table
weight: 501
related:
- /v2.0/query-data/flux/scalar-values/
---
The `findColumn()` function returns an array of values in a specified column from the
first table in a stream of tables where the group key values match the specified predicate.
The function returns an empty array if no table is found or if the column label
is not present in the set of columns.
_**Function type:** Stream and table_
```js
findColumn(
fn: (key) => key._field == "fieldName")
column: "_value"
)
```
## Parameters
### fn
A predicate function for matching keys in a table's group key.
Expects a `key` argument that represents a group key in the input stream.
_**Data type:** Function_
### column
Name of the column to extract.
_**Data type:** String_
## Example
```js
vs = from(bucket:"example-bucket")
|> range(start: -5m)
|> filter(fn:(r) => r._measurement == "cpu")
|> findColumn(
fn: (key) => key._field == "usage_idle",
column: "_value"
)
// Use column values
x = vs[0] + vs[1]
```

View File

@ -0,0 +1,54 @@
---
title: findRecord() function
description: >
The `findRecord()` function returns a record at a specified index from the first
table in a stream of tables where the group key values match the specified predicate.
menu:
v2_0_ref:
name: findRecord
parent: Stream & table
weight: 501
related:
- /v2.0/query-data/flux/scalar-values/
---
The `findRecord()` function returns a record at a specified index from the first
table in a stream of tables where the group key values match the specified predicate.
The function returns an empty object if no table is found or if the index is out of bounds.
_**Function type:** Stream and table_
```js
findRecord(
fn: (key) => key._field == "fieldName",
idx: 0
)
```
## Parameters
### fn
A predicate function for matching keys in a table's group key.
Expects a `key` argument that represents a group key in the input stream.
_**Data type:** Function_
### idx
Index of the record to extract.
_**Data type:** Integer_
## Example
```js
r0 = from(bucket:"example-bucket")
|> range(start: -5m)
|> filter(fn:(r) => r._measurement == "cpu")
|> tableFind()
|> findRecord(
fn: (key) => key._field == "usage_idle",
idx: 0
)
// Use record values
x = r0._field + "--" + r0._measurement
```

View File

@ -31,7 +31,7 @@ to extract a single table from a stream of tables.
## Parameters ## Parameters
### column ### column
The name of the column to extract. Name of the column to extract.
_**Data type:** String_ _**Data type:** String_

View File

@ -31,7 +31,7 @@ to extract a single table from a stream of tables.
## Parameters ## Parameters
### idx ### idx
The index of the record to extract. Index of the record to extract.
_**Data type:** Integer_ _**Data type:** Integer_

View File

@ -32,7 +32,7 @@ To learn why, see [Match parameter names](/v2.0/reference/flux/language/data-mod
A predicate function for matching keys in a table's group key. A predicate function for matching keys in a table's group key.
`tableFind` returns the first table that resolves as `true`. `tableFind` returns the first table that resolves as `true`.
It expects a `key` argument which represents a group key in the input stream. Expects a `key` argument that represents a group key in the input stream.
_**Data type:** Function_ _**Data type:** Function_

View File

@ -7,7 +7,7 @@ menu:
v2_0_ref: v2_0_ref:
name: tail name: tail
parent: built-in-transformations parent: built-in-transformations
weight: 401 weight: 402
related: related:
- /v2.0/reference/flux/functions/built-in/transformations/limit/ - /v2.0/reference/flux/functions/built-in/transformations/limit/
--- ---

View File

@ -9,7 +9,7 @@ menu:
v2_0_ref: v2_0_ref:
name: timeShift name: timeShift
parent: built-in-transformations parent: built-in-transformations
weight: 401 weight: 402
--- ---
The `timeShift()` function adds a fixed duration to time columns. The `timeShift()` function adds a fixed duration to time columns.

View File

@ -8,7 +8,7 @@ menu:
v2_0_ref: v2_0_ref:
name: truncateTimeColumn name: truncateTimeColumn
parent: built-in-transformations parent: built-in-transformations
weight: 401 weight: 402
related: related:
- /v2.0/reference/flux/stdlib/date/truncate/ - /v2.0/reference/flux/stdlib/date/truncate/
--- ---

View File

@ -8,7 +8,7 @@ menu:
v2_0_ref: v2_0_ref:
name: union name: union
parent: built-in-transformations parent: built-in-transformations
weight: 401 weight: 402
related: related:
- /v2.0/reference/flux/stdlib/built-in/transformations/join/ - /v2.0/reference/flux/stdlib/built-in/transformations/join/
--- ---

View File

@ -8,7 +8,7 @@ menu:
v2_0_ref: v2_0_ref:
name: window name: window
parent: built-in-transformations parent: built-in-transformations
weight: 401 weight: 402
--- ---
The `window()` function groups records based on a time value. The `window()` function groups records based on a time value.