Merge pull request #1297 from influxdata/flux-0.79

Flux 0.78 & 0.79

content/v2.0/reference/flux/stdlib/experimental/array/_index.md

@@ -0,0 +1,22 @@
+---
+title: Flux Array package
+list_title: Array package
+description: >
+  The Flux Array package provides functions for building tables from Flux arrays.
+  Import the `experimental/array` package.
+menu:
+  v2_0_ref:
+    name: Array
+    parent: Experimental
+weight: 301
+v2.0/tags: [functions, array, package, table]
+---
+
+Flux Array functions provide tools for building tables from Flux arrays.
+Import the `experimental/array` package:
+
+```js
+import "experimental/array"
+```
+
+{{< children type="functions" show="pages" >}}

content/v2.0/reference/flux/stdlib/experimental/array/from.md

@@ -0,0 +1,63 @@
+---
+title: array.from() function
+description: >
+  The experimental `array.from()` function constructs a table from an array of objects.
+menu:
+  v2_0_ref:
+    name: array.from
+    parent: Array
+weight: 401
+---
+
+The experimental `array.from()` function constructs a table from an array of objects.
+Each object in the array is converted into an output row or record.
+All records must have the same keys and data types.
+
+_**Function type:** Input_
+
+{{< keep-url >}}
+```js
+import "experimental/array"
+
+array.from(rows: [
+  {_time: 2020-01-01T00:00:00Z, _field: "exampleField", _value: 3, foo: "bar"},
+  {_time: 2020-01-01T00:01:00Z, _field: "exampleField", _value: 4, foo: "bar"},
+  {_time: 2020-01-01T00:02:00Z, _field: "exampleField", _value: 1, foo: "bar"}
+])
+```
+
+## Parameters
+
+### rows
+Array of records to construct a table with.
+
+_**Data type:** Array of objects_
+
+## Examples
+
+##### Build an arbitrary table
+```js
+import "experimental/array"
+
+rows = [
+  {foo: "bar", baz: 21.2}
+  {foo: "bar", baz: 23.8}
+]
+
+array.from(rows: rows)
+```
+
+##### Union custom rows with query results
+```js
+import "influxdata/influxdb/v1"
+import "experimental/array"
+
+tags = v1.tagValues(
+  bucket: "example-bucket",
+  tag: "host"
+)
+
+wildcard_tag = array.from(rows: [{_value: "*"}])
+
+union(tables: [tags, wildcard_tag])
+```

content/v2.0/reference/flux/stdlib/experimental/geo/_index.md

@@ -74,12 +74,21 @@ to add `s2_cell_id` tags to data that includes fields with latitude and longitud
   )
 ```
 
+## Latitude and longitude values
+Flux supports latitude and longitude values in **decimal degrees** (WGS 84).
+
+| Coordinate | Minimum | Maximum |
+|:---------- | -------:| -------:|
+| Latitude   | -90.0   | 90.0    |
+| Longitude  | -180.0  | 180.0   |
+
 ## Region definitions
 Many functions in the Geo package filter data based on geographic region.
 Define geographic regions using the following shapes:
 
 - [box](#box)
 - [circle](#circle)
+- [point](#point)
 - [polygon](#polygon)
 
 ### box
@@ -116,6 +125,20 @@ Define a circular region by specifying an object containing the following proper
 }
 ```
 
+### point
+Define a point region by specifying an object containing the following properties:
+
+- **lat**: latitude in decimal degrees (WGS 84) _(Float)_
+- **lon**: longitude in decimal degrees (WGS 84) _(Float)_
+
+##### Example point region
+```js
+{
+  lat: 40.671659,
+  lon: -73.936631
+}
+```
+
 ### polygon
 Define a custom polygon region using an object containing the following properties:
 
@@ -136,3 +159,38 @@ Define a custom polygon region using an object containing the following properti
   ]
 }
 ```
+
+## GIS geometry definitions
+Many functions in the Geo package manipulate data based on geographic information system (GIS) data.
+Define GIS geometry using the following:
+
+- Any [region type](#region-definitions) _(typically [point](#point))_
+- [linestring](#linestring)
+
+### linestring
+Define a geographic linestring path using an object containing the following properties:
+
+- **linestring**: string containing comma-separated longitude and latitude
+  coordinate pairs (`lon lat,`):
+
+```js
+{
+  linestring: "39.7515 14.01433, 38.3527 13.9228, 36.9978 15.08433"
+}
+```
+
+## Distance units
+The Geo package supports the following units of measurement for distance:
+
+- `m` - meters
+- `km` - kilometers _(default)_
+- `mile` - miles
+
+### Define distance units
+Use the Geo package `units` option to define custom units of measurement:
+
+```js
+import "experimental/geo"
+
+option geo.units = {distance: "mile"}
+```

content/v2.0/reference/flux/stdlib/experimental/geo/filterrows.md

@@ -196,19 +196,30 @@ filterRows = (
   maxSize=-1,
   level=-1,
   s2cellIDLevel=-1,
-  correlationKey=["_time"],
   strict=true
 ) => {
+  _columns =
+    |> columns(column: "_value")
+    |> tableFind(fn: (key) => true )
+    |> getColumn(column: "_value")
   _rows =
-    tables
+    if contains(value: "lat", set: _columns) then
-      |> gridFilter(
+      tables
-        region,
+        |> gridFilter(
-        minSize: minSize,
+          region: region,
-        maxSize: maxSize,
+          minSize: minSize,
-        level: level,
+          maxSize: maxSize,
-        s2cellIDLevel: s2cellIDLevel
+          level: level,
-      )
+          s2cellIDLevel: s2cellIDLevel)
-      |> toRows(correlationKey)
+    else
+      tables
+        |> gridFilter(
+          region: region,
+          minSize: minSize,
+          maxSize: maxSize,
+          level: level,
+          s2cellIDLevel: s2cellIDLevel)
+        |> toRows()
   _result =
     if strict then
       _rows

content/v2.0/reference/flux/stdlib/experimental/geo/s2celllatlon.md

@@ -0,0 +1,36 @@
+---
+title: geo.s2CellLatLon() function
+description: >
+  The `geo.s2CellLatLon()` function returns the latitude and longitude of the
+  center of an S2 cell.
+menu:
+  v2_0_ref:
+    name: geo.s2CellLatLon
+    parent: Geo
+weight: 401
+v2.0/tags: [functions, geo]
+related:
+  - /v2.0/query-data/flux/geo/
+---
+
+The `geo.s2CellLatLon()` function returns the latitude and longitude of the
+center of an S2 cell.
+
+_**Function type:** Transformation_
+
+```js
+import "experimental/geo"
+
+geo.s2CellLatLon(
+  token: "89c284"
+)
+
+// Returns {lat: 40.812535546624574, lon: -73.55941282728273}
+```
+
+## Parameters
+
+### token
+S2 cell ID token.
+
+_**Data type:** String_

content/v2.0/reference/flux/stdlib/experimental/geo/shapedata.md

@@ -32,8 +32,7 @@ import "experimental/geo"
 geo.shapeData(
   latField: "latitude",
   lonField: "longitude",
-  level: 10,
+  level: 10
-  correlationKey: ["_time"]
 )
 ```
 
@@ -57,12 +56,6 @@ when generating the S2 cell ID token.
 
 _**Data type:** Integer_
 
-### correlationKey
-List of columns used to uniquely identify a row for output.
-Default is `["_time"]`.
-
-_**Data type:** Array of strings_
-
 ## Examples
 
 ##### Shape data to meet the requirements of the Geo package

content/v2.0/reference/flux/stdlib/experimental/geo/st_contains.md

@@ -0,0 +1,86 @@
+---
+title: geo.ST_Contains() function
+description: >
+  The `geo.ST_Contains()` function tests if the specified region contains the specified
+  GIS geometry and returns `true` or `false`.
+menu:
+  v2_0_ref:
+    name: geo.ST_Contains
+    parent: Geo
+weight: 401
+v2.0/tags: [functions, geo, GIS]
+related:
+  - /v2.0/query-data/flux/geo/
+---
+
+The `geo.ST_Contains()` function tests if the specified region contains the specified
+geographic information system (GIS) geometry and returns `true` or `false`.
+
+_**Function type:** Test_
+
+```js
+import "experimental/geo"
+
+geo.ST_Contains(
+  region: {lat: 40.7, lon: -73.3, radius: 20.0},
+  geometry: {lon: 39.7515, lat: 15.08433}
+)
+
+// Returns false
+```
+
+## Parameters
+
+### region
+The region to test.
+Specify object properties for the shape.
+_See [Region definitions](/v2.0/reference/flux/stdlib/experimental/geo/#region-definitions)._
+
+_**Data type:** Object_
+
+### geometry
+The GIS geometry to test.
+Can be either point or linestring geometry.
+_See [GIS geometry definitions](/v2.0/reference/flux/stdlib/experimental/geo/#gis-geometry-definitions)._
+
+_**Data type:** Object_
+
+## Examples
+
+##### Test if geographic points are inside of a region
+```js
+import "experimental/geo"
+
+region = {
+  minLat: 40.51757813,
+  maxLat: 40.86914063,
+  minLon: -73.65234375,
+  maxLon: -72.94921875
+}
+
+data
+  |> geo.toRows()
+  |> map(fn: (r) => ({
+    r with st_contains: geo.ST_Contains(region: region, geometry: {lat: r.lat, lon: r.lon})
+  }))
+```
+
+##### Test if tracks are inside of a region
+```js
+import "experimental/geo"
+
+region = {
+  minLat: 40.51757813,
+  maxLat: 40.86914063,
+  minLon: -73.65234375,
+  maxLon: -72.94921875
+}
+
+data
+  |> geo.toRows()
+  |> geo.asTracks()
+  |> geo.ST_LineString()
+  |> map(fn: (r) => ({
+    r with st_contains: geo.ST_Contains(region: region, geometry: {linestring: r.st_linestring})
+  }))
+```

content/v2.0/reference/flux/stdlib/experimental/geo/st_distance.md

@@ -0,0 +1,85 @@
+---
+title: geo.ST_Distance() function
+description: >
+  The `geo.ST_Distance()` function returns the distance between the specified region
+  and specified GIS geometry.
+menu:
+  v2_0_ref:
+    name: geo.ST_Distance
+    parent: Geo
+weight: 401
+v2.0/tags: [functions, geo, GIS]
+related:
+  - /v2.0/query-data/flux/geo/
+---
+
+The `geo.ST_Distance()` function returns the distance between the specified region
+and specified geographic information system (GIS) geometry.
+Define distance units with the [`geo.units` option](/v2.0/reference/flux/stdlib/experimental/geo/#define-distance-units).
+
+_**Function type:** Transformation_
+
+```js
+import "experimental/geo"
+
+geo.ST_Distance(
+  region: {lat: 40.7, lon: -73.3, radius: 20.0},
+  geometry: {lon: 39.7515, lat: 15.08433}
+)
+
+// Returns 10734.184618677662 (km)
+```
+
+## Parameters
+
+### region
+The region to test.
+Specify object properties for the shape.
+_See [Region definitions](/v2.0/reference/flux/stdlib/experimental/geo/#region-definitions)._
+
+_**Data type:** Object_
+
+### geometry
+The GIS geometry to test.
+Can be either point or linestring geometry.
+_See [GIS geometry definitions](/v2.0/reference/flux/stdlib/experimental/geo/#gis-geometry-definitions)._
+
+_**Data type:** Object_
+
+## Examples
+
+##### Test if geographic points are inside of a region
+```js
+import "experimental/geo"
+
+region = {
+  minLat: 40.51757813,
+  maxLat: 40.86914063,
+  minLon: -73.65234375,
+  maxLon: -72.94921875
+}
+
+data
+  |> geo.toRows()
+  |> map(fn: (r) => ({
+    r with st_contains: ST_Distance(region: region, geometry: {lat: r.lat, lon: r.lon})
+  }))
+```
+
+##### Calculate the distance between geographic points and a region
+```js
+import "experimental/geo"
+
+region = {
+  minLat: 40.51757813,
+  maxLat: 40.86914063,
+  minLon: -73.65234375,
+  maxLon: -72.94921875
+}
+
+data
+  |> geo.toRows()
+  |> map(fn: (r) => ({
+    r with st_distance: ST_Distance(region: region, geometry: {lat: r.lat, lon: r.lon})
+  }))
+```

content/v2.0/reference/flux/stdlib/experimental/geo/st_dwithin.md

@@ -0,0 +1,75 @@
+---
+title: geo.ST_DWithin() function
+description: >
+  The `geo.ST_DWithin()` function tests if the specified region is within a defined
+  distance from the specified geographic information system (GIS) geometry and
+  returns `true` or `false`.
+menu:
+  v2_0_ref:
+    name: geo.ST_DWithin
+    parent: Geo
+weight: 401
+v2.0/tags: [functions, geo, GIS]
+related:
+  - /v2.0/query-data/flux/geo/
+---
+
+The `geo.ST_DWithin()` function tests if the specified region is within a defined
+distance from the specified geographic information system (GIS) geometry and
+returns `true` or `false`.
+
+_**Function type:** Test_
+
+```js
+import "experimental/geo"
+
+geo.ST_DWithin(
+  region: {lat: 40.7, lon: -73.3, radius: 20.0},
+  geometry: {lon: 39.7515, lat: 15.08433},
+  distance: 1000.0
+)
+
+// Returns false
+```
+
+## Parameters
+
+### region
+The region to test.
+Specify object properties for the shape.
+_See [Region definitions](/v2.0/reference/flux/stdlib/experimental/geo/#region-definitions)._
+
+_**Data type:** Object_
+
+### geometry
+The GIS geometry to test.
+Can be either point or linestring geometry.
+_See [GIS geometry definitions](/v2.0/reference/flux/stdlib/experimental/geo/#gis-geometry-definitions)._
+
+_**Data type:** Object_
+
+### distance
+Maximum distance allowed between the region and geometry.
+_Define distance units with the [`geo.units` option](/v2.0/reference/flux/stdlib/experimental/geo/#define-distance-units)._
+
+_**Data type:** Float_
+
+## Examples
+
+##### Test if geographic points are within a distance from a region
+```js
+import "experimental/geo"
+
+region = {
+  minLat: 40.51757813,
+  maxLat: 40.86914063,
+  minLon: -73.65234375,
+  maxLon: -72.94921875
+}
+
+data
+  |> geo.toRows()
+  |> map(fn: (r) => ({
+    r with st_within: geo.ST_DWithin(region: box, geometry: {lat: r.lat, lon: r.lon}, distance: 15.0)
+  }))
+```

content/v2.0/reference/flux/stdlib/experimental/geo/st_intersects.md

@@ -0,0 +1,66 @@
+---
+title: geo.ST_Intersects() function
+description: >
+  The `geo.ST_Intersects()` function tests if the specified geographic information
+  system (GIS) geometry intersects with the specified region and returns `true` or `false`.
+menu:
+  v2_0_ref:
+    name: geo.ST_Intersects
+    parent: Geo
+weight: 401
+v2.0/tags: [functions, geo, GIS]
+related:
+  - /v2.0/query-data/flux/geo/
+---
+
+The `geo.ST_Intersects()` function tests if the specified geographic information
+system (GIS) geometry intersects with the specified region and returns `true` or `false`.
+
+_**Function type:** Test_
+
+```js
+import "experimental/geo"
+
+geo.ST_Intersects(
+  region: {lat: 40.7, lon: -73.3, radius: 20.0},
+  geometry: {linestring: "39.7515 14.01433, 38.3527 13.9228, 36.9978 15.08433"}
+)
+
+// Returns false
+```
+
+## Parameters
+
+### region
+The region to test.
+Specify object properties for the shape.
+_See [Region definitions](/v2.0/reference/flux/stdlib/experimental/geo/#region-definitions)._
+
+_**Data type:** Object_
+
+### geometry
+The GIS geometry to test.
+Can be either point or linestring geometry.
+_See [GIS geometry definitions](/v2.0/reference/flux/stdlib/experimental/geo/#gis-geometry-definitions)._
+
+_**Data type:** Object_
+
+## Examples
+
+##### Test if geographic points intersect with a region
+```js
+import "experimental/geo"
+
+region = {
+  minLat: 40.51757813,
+  maxLat: 40.86914063,
+  minLon: -73.65234375,
+  maxLon: -72.94921875
+}
+
+data
+  |> geo.toRows()
+  |> map(fn: (r) => ({
+    r with st_within: geo.ST_Intersects(region: box, geometry: {lat: r.lat, lon: r.lon})
+  }))
+```

content/v2.0/reference/flux/stdlib/experimental/geo/st_length.md

@@ -0,0 +1,62 @@
+---
+title: geo.ST_Length() function
+description: >
+  The `geo.ST_Length()` function returns the [spherical length or distance](https://mathworld.wolfram.com/SphericalDistance.html)
+  of the specified geographic information system (GIS) geometry.
+menu:
+  v2_0_ref:
+    name: geo.ST_Length
+    parent: Geo
+weight: 401
+v2.0/tags: [functions, geo, GIS]
+related:
+  - /v2.0/query-data/flux/geo/
+---
+
+The `geo.ST_Length()` function returns the [spherical length or distance](https://mathworld.wolfram.com/SphericalDistance.html)
+of the specified geographic information system (GIS) geometry.
+Define distance units with the [`geo.units` option](/v2.0/reference/flux/stdlib/experimental/geo/#define-distance-units).
+
+_**Function type:** Transformation_
+
+```js
+import "experimental/geo"
+
+geo.ST_Length(
+  geometry: {linestring: "39.7515 14.01433, 38.3527 13.9228, 36.9978 15.08433"}
+)
+
+// Returns 346.1023974652474 (km)
+```
+
+## Parameters
+
+### geometry
+The GIS geometry to measure.
+Can be either point or linestring geometry.
+Points will always return `0.0`.
+_See [GIS geometry definitions](/v2.0/reference/flux/stdlib/experimental/geo/#gis-geometry-definitions)._
+
+_**Data type:** Object_
+
+## Examples
+
+##### Calculate the length of geographic paths
+```js
+import "experimental/geo"
+
+region = {
+  minLat: 40.51757813,
+  maxLat: 40.86914063,
+  minLon: -73.65234375,
+  maxLon: -72.94921875
+}
+
+data
+  |> geo.toRows()
+  |> geo.asTracks()
+  |> geo.ST_LineString()
+  |> map(fn: (r) => ({
+    r with st_length: geo.ST_Length(geometry: {linestring: r.st_linestring})
+  }))
+```

content/v2.0/reference/flux/stdlib/experimental/geo/st_linestring.md

@@ -0,0 +1,69 @@
+---
+title: geo.ST_LineString() function
+description: >
+  The `geo.ST_LineString()` function converts a series of geographic points into
+  [linestring](/v2.0/reference/flux/stdlib/experimental/geo/#linestring).
+menu:
+  v2_0_ref:
+    name: geo.ST_LineString
+    parent: Geo
+weight: 401
+v2.0/tags: [functions, geo, GIS]
+related:
+  - /v2.0/query-data/flux/geo/
+---
+
+The `geo.ST_LineString()` function converts a series of geographic points into
+[linestring](/v2.0/reference/flux/stdlib/experimental/geo/#linestring).
+Group data into meaningful, ordered paths to before converting to linestring.
+Rows in each table must have `lat` and `lon` columns.
+Output tables contain a single row with a `st_linestring` column containing the resulting linestring.
+
+_**Function type:** Aggregate_
+
+```js
+import "experimental/geo"
+
+geo.ST_LineString()
+```
+
+## Examples
+
+### Convert a series of geographic points into linestring
+
+##### Input data
+
+| _time                | id    | lon     | lat      |
+|:-----                |:--:   | ---:    | ---:     |
+| 2020-01-01T00:00:00Z | a213b | 39.7515 | 14.01433 |
+| 2020-01-02T00:00:00Z | a213b | 38.3527 | 13.9228  |
+| 2020-01-03T00:00:00Z | a213b | 36.9978 | 15.08433 |
+
+
+```js
+import "experimental/geo"
+
+data
+  |> geo.ST_LineString()
+```
+
+##### Output data
+
+| id    | st_linestring                                       |
+|:--    |:-------------                                       |
+| a213b | 39.7515 14.01433, 38.3527 13.9228, 36.9978 15.08433 |
+
+## Function definition
+```js
+ST_LineString = (tables=<-) =>
+  tables
+    |> reduce(fn: (r, accumulator) => ({
+        __linestring: accumulator.__linestring + (if accumulator.__count > 0 then ", " else "") + string(v: r.lat) + " " + string(v: r.lon),
+        __count: accumulator.__count + 1
+      }), identity: {
+        __linestring: "",
+        __count: 0
+      }
+    )
+    |> rename(columns: {__linestring: "st_linestring"})
+```

content/v2.0/reference/flux/stdlib/experimental/geo/torows.md

@@ -22,19 +22,9 @@ _**Function type:** Transformation_
 ```js
 import "experimental/geo"
 
-geo.toRows(
+geo.toRows()
-  correlationKey: ["_time"]
-)
 ```
 
-## Parameters
-
-### correlationKey
-List of columns used to uniquely identify a row for output.
-Default is `["_time"]`.
-
-_**Data type:** Array of strings_
-
 ## Examples
 ```js
 import "experimental/geo"
@@ -49,9 +39,5 @@ from(bucket: "example-bucket")
 ```js
 toRows = (tables=<-, correlationKey=["_time"]) =>
   tables
-    |> pivot(
+    |> v1.fieldsAsCols()
-      rowKey: correlationKey,
-      columnKey: ["_field"],
-      valueColumn: "_value"
-    )
 ```

content/v2.0/reference/release-notes/flux.md

@@ -16,6 +16,33 @@ Though newer versions of Flux may be available, they will not be included with
 InfluxDB until the next InfluxDB v2.0 release._
 {{% /note %}}
 
+## v0.79.0 [2020-08-11]
+
+### Features
+- Add `array.from()` function to convert Flux values into a table.
+
+### Bug fixes
+- Add bounds to Geo package end-to-end tests.
+
+---
+
+## v0.78.0 [2020-08-10]
+
+### Breaking changes
+- Removed `correlationKey` parameter from `geo.toRows` and `geo.shapeData`.
+
+### Features
+- Add functions to convert semantic monotype to AST type.
+- Add BigQuery support.
+- Rust flatbuffer serialization for `MonoType` and `TypeExpression`.
+- Extend with Geo package with GIS functions and
+  [unit support](/v2.0/reference/flux/stdlib/experimental/geo/#distance-units).
+
+### Bug fixes
+- String interpolation in arrays.
+
+---
+
 ## v0.77.1 [2020-08-03]
 
 ### Bug fixes