Scott Anderson
GitHub
commit
ff0a313f8b
|
|
@ -3,6 +3,7 @@
|
|||
.flex-wrapper {
|
||||
display: flex;
|
||||
flex-wrap: wrap;
|
||||
margin: 1.5rem 0;
|
||||
}
|
||||
|
||||
.flex-container {
|
||||
|
|
|
|||
|
|
@ -59,6 +59,19 @@ generating S2 Cell ID tokens. For example:
|
|||
- **Python:** [`s2sphere.CellId.to_token()`](https://s2sphere.readthedocs.io/en/latest/api.html#s2sphere.CellId)
|
||||
- **Javascript:** [`s2.cellid.toToken()`](https://github.com/mapbox/node-s2/blob/master/API.md#cellidtotoken---string)
|
||||
|
||||
### Add S2 Cell IDs to existing geo-temporal data
|
||||
Use the [`geo.shapeData()` function](/v2.0/reference/flux/stdlib/experimental/geo/shapedata/)
|
||||
to add `s2_cell_id` tags to data that includes fields with latitude and longitude values.
|
||||
|
||||
```js
|
||||
//...
|
||||
|> shapeData(
|
||||
latField: "latitude",
|
||||
lonField: "longitude",
|
||||
level: 10
|
||||
)
|
||||
```
|
||||
|
||||
## Region definitions
|
||||
Many functions in the Geo package filter data based on geographic region.
|
||||
Define geographic regions using the following shapes:
|
||||
|
|
|
|||
|
|
@ -35,6 +35,20 @@ geo.filterRows(
|
|||
)
|
||||
```
|
||||
|
||||
{{% note %}}
|
||||
#### s2_cell_id must be part of the group key
|
||||
To filter geo-temporal data with `geo.filterRows()`, `s2_cell_id` must be part
|
||||
of the [group key](/v2.0/reference/glossary/#group-key).
|
||||
To add `s2_cell_id` to the group key, use [`experimental.group`](/v2.0/reference/flux/stdlib/experimental/group):
|
||||
|
||||
```js
|
||||
import "experimental"
|
||||
|
||||
// ...
|
||||
|> experimental.group(columns: ["s2_cell_id"], mode: "extend")
|
||||
```
|
||||
{{% /note %}}
|
||||
|
||||
### Strict and non-strict filtering
|
||||
In most cases, the specified geographic region does not perfectly align with S2 grid cells.
|
||||
|
||||
|
|
|
|||
|
|
@ -39,6 +39,20 @@ geo.gridFilter(
|
|||
)
|
||||
```
|
||||
|
||||
{{% note %}}
|
||||
#### s2_cell_id must be part of the group key
|
||||
To filter geo-temporal data with `geo.gridFilter()`, `s2_cell_id` must be part
|
||||
of the [group key](/v2.0/reference/glossary/#group-key).
|
||||
To add `s2_cell_id` to the group key, use [`experimental.group`](/v2.0/reference/flux/stdlib/experimental/group):
|
||||
|
||||
```js
|
||||
import "experimental"
|
||||
|
||||
// ...
|
||||
|> experimental.group(columns: ["s2_cell_id"], mode: "extend")
|
||||
```
|
||||
{{% /note %}}
|
||||
|
||||
### Non-strict and strict filtering
|
||||
In most cases, the specified geographic region does not perfectly align with S2 grid cells.
|
||||
|
||||
|
|
|
|||
|
|
@ -0,0 +1,117 @@
|
|||
---
|
||||
title: geo.shapeData() function
|
||||
description: >
|
||||
The `geo.shapeData()` function renames existing latitude and longitude fields to
|
||||
**lat** and **lon** and adds an **s2_cell_id** tag.
|
||||
Use `geo.shapeData()` to ensure geo-temporal data meets the requirements of the Geo package.
|
||||
menu:
|
||||
v2_0_ref:
|
||||
name: geo.shapeData
|
||||
parent: Geo
|
||||
weight: 401
|
||||
v2.0/tags: [functions, geo]
|
||||
---
|
||||
|
||||
The `geo.shapeData()` function renames existing latitude and longitude fields to
|
||||
**lat** and **lon** and adds an **s2_cell_id** tag.
|
||||
Use `geo.shapeData()` to ensure geo-temporal data meets the
|
||||
[requirements of the Geo package](/v2.0/reference/flux/stdlib/experimental/geo/#geo-schema-requirements):
|
||||
|
||||
1. Rename existing latitude and longitude fields to `lat` and `lon`.
|
||||
2. Pivot data into row-wise sets based on the [`correlationKey`](#correlationkey).
|
||||
3. Generate `s2_cell_id` tags using `lat` and `lon` values and a specified
|
||||
[S2 cell level](https://s2geometry.io/resources/s2cell_statistics.html).
|
||||
|
||||
_**Function type:** Transformation_
|
||||
|
||||
```js
|
||||
import "experimental/geo"
|
||||
|
||||
geo.shapeData(
|
||||
latField: "latitude",
|
||||
lonField: "longitude",
|
||||
level: 10,
|
||||
correlationKey: ["_time"]
|
||||
)
|
||||
```
|
||||
|
||||
## Parameters
|
||||
|
||||
### latField
|
||||
Name of the existing field that contains the latitude value in **decimal degrees** (WGS 84).
|
||||
Field is renamed to `lat`.
|
||||
|
||||
_**Data type:** String_
|
||||
|
||||
### lonField
|
||||
Name of the existing field that contains the longitude value in **decimal degrees** (WGS 84).
|
||||
Field is renamed to `lon`.
|
||||
|
||||
_**Data type:** String_
|
||||
|
||||
### level
|
||||
[S2 cell level](https://s2geometry.io/resources/s2cell_statistics.html) to use
|
||||
when generating the S2 cell ID token.
|
||||
|
||||
_**Data type:** Integer_
|
||||
|
||||
### correlationKey
|
||||
List of columns used to uniquely identify a row for output.
|
||||
Default is `["_time"]`.
|
||||
|
||||
_**Data type:** Array of strings_
|
||||
|
||||
## Examples
|
||||
|
||||
##### Shape data to meet the requirements of the Geo package
|
||||
```js
|
||||
import "experimental/geo"
|
||||
|
||||
from(bucket: "example-bucket")
|
||||
|> range(start: -1h)
|
||||
|> filter(fn: (r) => r._measurement == "example-measurement")
|
||||
|> geo.shapeData(
|
||||
latField: "latitude",
|
||||
lonField: "longitude",
|
||||
level: 10
|
||||
)
|
||||
```
|
||||
|
||||
### geo.shapeData input and output
|
||||
|
||||
{{< flex >}}
|
||||
{{% flex-content %}}
|
||||
**Given the following input:**
|
||||
|
||||
| _time | _field | _value |
|
||||
|:----- |:------: | ------:|
|
||||
| 0001 | latitude | 30.0 |
|
||||
| 0002 | latitude | 30.5 |
|
||||
| 0003 | latitude | 30.7 |
|
||||
| 0004 | latitude | 31.1 |
|
||||
| • • • | • • • | • • • |
|
||||
| 0001 | longitude | 20.0 |
|
||||
| 0002 | longitude | 19.8 |
|
||||
| 0003 | longitude | 19.2 |
|
||||
| 0004 | longitude | 19.5 |
|
||||
{{% /flex-content %}}
|
||||
{{% flex-content %}}
|
||||
|
||||
**The following function would output:**
|
||||
|
||||
```js
|
||||
|> geo.shapeData(
|
||||
latField: "latitude",
|
||||
lonField: "longitude",
|
||||
level: 5
|
||||
)
|
||||
```
|
||||
|
||||
| _time | lat | lon | s2_cell_id |
|
||||
|:----- |:--------:|:---------:| ----------:|
|
||||
| 0001 | 30.0 | 20.0 | 138c |
|
||||
| 0002 | 30.5 | 19.8 | 1384 |
|
||||
| 0003 | 30.7 | 19.2 | 139c |
|
||||
| 0004 | 31.1 | 19.5 | 139c |
|
||||
{{% /flex-content %}}
|
||||
{{< /flex >}}
|
||||
|
|
@ -52,7 +52,9 @@ Appends columns defined in the [`columns` parameter](#columns) to all existing
|
|||
|
||||
###### Include the value column in each groups' group key
|
||||
```js
|
||||
import "experimental"
|
||||
|
||||
from(bucket: "example-bucket")
|
||||
|> range(start: -1m)
|
||||
|> group(columns: ["_value"], mode: "extend")
|
||||
|> experimental.group(columns: ["_value"], mode: "extend")
|
||||
```
|
||||
|
|
|
|||
|
|
@ -0,0 +1,160 @@
|
|||
---
|
||||
title: experimental.join() function
|
||||
description: >
|
||||
The `experimental.join()` function joins two streams of tables on the
|
||||
group key with the addition of the `_time` column.
|
||||
menu:
|
||||
v2_0_ref:
|
||||
name: experimental.join
|
||||
parent: Experimental
|
||||
weight: 302
|
||||
---
|
||||
|
||||
The `experimental.join()` function joins two streams of tables on the
|
||||
[group key](/v2.0/reference/glossary/#group-key) and `_time` column.
|
||||
Use the [`fn` parameter](#fn) to map new output tables using values from input tables.
|
||||
|
||||
{{% note %}}
|
||||
To join streams of tables with different fields or measurements, use [`group()`](/v2.0/reference/flux/stdlib/built-in/transformations/group/)
|
||||
or [`drop()`](/v2.0/reference/flux/stdlib/built-in/transformations/drop/) to remove
|
||||
`_field` and `_measurement` from the group key before joining.
|
||||
_See an example [below](#join-two-streams-of-tables-with-different-fields)._
|
||||
{{% /note %}}
|
||||
|
||||
_**Function type:** Transformation_
|
||||
|
||||
```js
|
||||
import "experimental"
|
||||
|
||||
// ...
|
||||
|
||||
experimental.join(
|
||||
left: left,
|
||||
right: right,
|
||||
fn: (left, right) => ({left with lv: left._value, rv: right._value })
|
||||
)
|
||||
```
|
||||
|
||||
{{% note %}}
|
||||
This function will likely replace the [`join` function](/v2.0/reference/flux/stdlib/built-in/transformations/join/)
|
||||
when sufficiently vetted.
|
||||
{{% /note %}}
|
||||
|
||||
## Parameters
|
||||
|
||||
### left
|
||||
First of two streams of tables to join.
|
||||
|
||||
_**Data type:** Stream of tables_
|
||||
|
||||
### right
|
||||
Second of two streams of tables to join.
|
||||
|
||||
_**Data type:** Stream of tables_
|
||||
|
||||
### fn
|
||||
A function with `left` and `right` arguments that maps a new output object
|
||||
using values from the `left` and `right` input objects.
|
||||
The return value must be an object.
|
||||
|
||||
_**Data type:** Function_
|
||||
|
||||
## Examples
|
||||
|
||||
### Input and output tables
|
||||
|
||||
**Given the following input tables:**
|
||||
{{< flex >}}
|
||||
{{% flex-content %}}
|
||||
##### left
|
||||
| _time | _field | _value |
|
||||
|:----- |:------:| ------:|
|
||||
| 0001 | temp | 80.1 |
|
||||
| 0002 | temp | 80.2 |
|
||||
| 0003 | temp | 79.9 |
|
||||
| 0004 | temp | 80.0 |
|
||||
{{% /flex-content %}}
|
||||
{{% flex-content %}}
|
||||
##### right
|
||||
| _time | _field | _value |
|
||||
|:----- |:------:| ------:|
|
||||
| 0001 | temp | 72.1 |
|
||||
| 0002 | temp | 72.2 |
|
||||
| 0003 | temp | 71.9 |
|
||||
| 0004 | temp | 72.0 |
|
||||
{{% /flex-content %}}
|
||||
{{< /flex >}}
|
||||
|
||||
**The following `experimental.join()` function would output:**
|
||||
|
||||
```js
|
||||
import "experimental"
|
||||
|
||||
experimental.join(
|
||||
left: left,
|
||||
right: right,
|
||||
fn: (left, right) => ({
|
||||
left with
|
||||
lv: left._value,
|
||||
rv: right._value,
|
||||
diff: left._value - right._value
|
||||
})
|
||||
)
|
||||
```
|
||||
|
||||
| _time | _field | lv | rv | diff |
|
||||
|:----- |:------:|:--: |:--: | ----:|
|
||||
| 0001 | temp | 80.1 | 72.1 | 8.0 |
|
||||
| 0002 | temp | 80.2 | 72.2 | 8.0 |
|
||||
| 0003 | temp | 79.9 | 71.9 | 8.0 |
|
||||
| 0004 | temp | 80.0 | 72.0 | 8.0 |
|
||||
|
||||
---
|
||||
|
||||
###### Join two streams of tables
|
||||
```js
|
||||
import "experimental"
|
||||
|
||||
s1 = from(bucket: "example-bucket")
|
||||
|> range(start: -1h)
|
||||
|> filter(fn: (r) => r._measurement == "foo")
|
||||
|
||||
s2 = from(bucket: "example-bucket")
|
||||
|> range(start: -1h)
|
||||
|> filter(fn: (r) => r._measurement == "bar")
|
||||
|
||||
experimental.join(
|
||||
left: s1,
|
||||
right: s2,
|
||||
fn: (left, right) => ({
|
||||
left with
|
||||
s1_value: left._value,
|
||||
s2_value: right._value
|
||||
})
|
||||
)
|
||||
```
|
||||
|
||||
###### Join two streams of tables with different fields and measurements
|
||||
```js
|
||||
import "experimental"
|
||||
|
||||
s1 = from(bucket: "example-bucket")
|
||||
|> range(start: -1h)
|
||||
|> filter(fn: (r) => r._measurement == "foo" and r._field == "bar")
|
||||
|> group(columns: ["_time", "_measurement", "_field", "_value"], mode: "except")
|
||||
|
||||
s2 = from(bucket: "example-bucket")
|
||||
|> range(start: -1h)
|
||||
|> filter(fn: (r) => r._measurement == "baz" and r._field == "quz")
|
||||
|> group(columns: ["_time", "_measurement", "_field", "_value"], mode: "except")
|
||||
|
||||
experimental.join(
|
||||
left: s1,
|
||||
right: s2,
|
||||
fn: (left, right) => ({
|
||||
left with
|
||||
bar_value: left._value,
|
||||
quz_value: right._value
|
||||
})
|
||||
)
|
||||
```
|
||||
|
|
@ -1,7 +1,7 @@
|
|||
---
|
||||
title: monitor.stateChanges() function
|
||||
description: >
|
||||
The `monitor.stateChanges()` function detects state changes in a stream of data and
|
||||
The `monitor.stateChanges()` function detects state changes in a stream of tables and
|
||||
outputs records that change from `fromLevel` to `toLevel`.
|
||||
aliases:
|
||||
- /v2.0/reference/flux/functions/monitor/statechanges/
|
||||
|
|
@ -12,12 +12,8 @@ menu:
|
|||
weight: 202
|
||||
---
|
||||
|
||||
The `monitor.stateChanges()` function detects state changes in a stream of data and
|
||||
outputs records that change from `fromLevel` to `toLevel`.
|
||||
|
||||
{{% note %}}
|
||||
`monitor.stateChanges` operates on data in the `statuses` measurement and requires a `_level` column .
|
||||
{{% /note %}}
|
||||
The `monitor.stateChanges()` function detects state changes in a stream of data with
|
||||
a `_level` column and outputs records that change from `fromLevel` to `toLevel`.
|
||||
|
||||
_**Function type:** Transformation_
|
||||
|
||||
|
|
@ -26,7 +22,7 @@ import "influxdata/influxdb/monitor"
|
|||
|
||||
monitor.stateChanges(
|
||||
fromLevel: "any",
|
||||
toLevel: "crit"
|
||||
toLevel: "any"
|
||||
)
|
||||
```
|
||||
|
||||
|
|
@ -41,15 +37,51 @@ _**Data type:** String_
|
|||
### toLevel
|
||||
The level to detect a change to.
|
||||
The function output records that change to this level.
|
||||
Defaults to `"any"`.
|
||||
|
||||
_**Data type:** String_
|
||||
|
||||
## Examples
|
||||
|
||||
### Detect when the state changes to critical
|
||||
##### Detect when the state changes to critical
|
||||
```js
|
||||
import "influxdata/influxdb/monitor"
|
||||
|
||||
monitor.from(start: -1h)
|
||||
|> monitor.stateChanges(toLevel: "crit")
|
||||
```
|
||||
|
||||
{{< flex >}}
|
||||
{{% flex-content %}}
|
||||
**Given the following input:**
|
||||
|
||||
| _time | _level |
|
||||
|:----- |:------:|
|
||||
| 0001 | ok |
|
||||
| 0002 | ok |
|
||||
| 0003 | warn |
|
||||
| 0004 | crit |
|
||||
{{% /flex-content %}}
|
||||
{{% flex-content %}}
|
||||
**The following function outputs:**
|
||||
|
||||
```js
|
||||
monitor.stateChanges(
|
||||
toLevel: "crit"
|
||||
)
|
||||
```
|
||||
|
||||
| _time | _level |
|
||||
|:----- |:------:|
|
||||
| 0004 | crit |
|
||||
{{% /flex-content %}}
|
||||
{{< /flex >}}
|
||||
|
||||
## Function definition
|
||||
```js
|
||||
stateChanges = (fromLevel="any", toLevel="any", tables=<-) => {
|
||||
return
|
||||
if fromLevel == "any" and toLevel == "any" then tables |> stateChangesOnly()
|
||||
else tables |> _stateChanges(fromLevel: fromLevel, toLevel: toLevel)
|
||||
}
|
||||
```
|
||||
|
|
|
|||
|
|
@ -0,0 +1,55 @@
|
|||
---
|
||||
title: monitor.stateChangesOnly() function
|
||||
description: >
|
||||
The `monitor.stateChangesOnly()` function takes a stream of tables that contains a `_level`
|
||||
column and returns a stream of tables where each record represents a state change.
|
||||
menu:
|
||||
v2_0_ref:
|
||||
name: monitor.stateChangesOnly
|
||||
parent: InfluxDB Monitor
|
||||
weight: 202
|
||||
---
|
||||
|
||||
The `monitor.stateChangesOnly()` function takes a stream of tables that contains a `_level`
|
||||
column and returns a stream of tables where each record represents a state change.
|
||||
|
||||
_**Function type:** Transformation_
|
||||
|
||||
```js
|
||||
import "influxdata/influxdb/monitor"
|
||||
|
||||
monitor.stateChangesOnly()
|
||||
```
|
||||
|
||||
## Examples
|
||||
|
||||
##### Return records representing state changes
|
||||
```js
|
||||
import "influxdata/influxdb/monitor"
|
||||
|
||||
|
||||
monitor.from(start: -1h)
|
||||
|> monitor.stateChangesOnly()
|
||||
```
|
||||
|
||||
{{< flex >}}
|
||||
{{% flex-content %}}
|
||||
**Given the following input:**
|
||||
|
||||
| _time | _level |
|
||||
|:----- |:------:|
|
||||
| 0001 | ok |
|
||||
| 0002 | ok |
|
||||
| 0003 | warn |
|
||||
| 0004 | crit |
|
||||
{{% /flex-content %}}
|
||||
{{% flex-content %}}
|
||||
**`monitor.stateChangesOnly()` outputs:**
|
||||
|
||||
| _time | _level |
|
||||
|:----- |:------:|
|
||||
| 0002 | ok |
|
||||
| 0003 | warn |
|
||||
| 0004 | crit |
|
||||
{{% /flex-content %}}
|
||||
{{< /flex >}}
|
||||
|
|
@ -16,6 +16,22 @@ Though newer versions of Flux may be available, they will not be included with
|
|||
InfluxDB until the next InfluxDB v2.0 release._
|
||||
{{% /note %}}
|
||||
|
||||
## v0.65.0 [2020-03-27]
|
||||
|
||||
### Features
|
||||
- Add [`experimental.join()`](/v2.0/reference/flux/stdlib/experimental/join/) function.
|
||||
- Store comments in the AST and preserve on format.
|
||||
- Add [`shapeData()`](/v2.0/reference/flux/stdlib/experimental/geo/shapedata/) function to Geo package.
|
||||
- Expose format to Wasm users.
|
||||
|
||||
### Bug fixes
|
||||
- Reimplement `stateChanges()` function.
|
||||
- Remove the `set -x` in the xcc script.
|
||||
- Publishes Flux as a public npm package.
|
||||
- Pivot message passing.
|
||||
|
||||
---
|
||||
|
||||
## v0.64.0 [2020-03-11]
|
||||
|
||||
### Features
|
||||
|
|
|
|||
Loading…
Reference in New Issue