docs-v2/content/influxdb/v1/flux/guides/mathematic-operations.md

229 lines
6.3 KiB
Markdown

---
title: Transform data with mathematic operations
seotitle: Transform data with mathematic operations in Flux
list_title: Transform data with math
description: >
Use the `map()` function to remap column values and apply mathematic operations.
menu:
influxdb_v1:
name: Transform data with math
parent: Query with Flux
weight: 5
list_query_example: map_math
canonical: /influxdb/v2/query-data/flux/mathematic-operations/
alt_links:
v2: /influxdb/v2/query-data/flux/mathematic-operations/
---
Flux supports mathematic expressions in data transformations.
This article describes how to use [Flux arithmetic operators](/flux/v0/language/operators/#arithmetic-operators)
to "map" over data and transform values using mathematic operations.
If you're just getting started with Flux queries, check out the following:
- [Get started with Flux](/influxdb/v1/flux/get-started/) for a conceptual overview of Flux and parts of a Flux query.
- [Execute queries](/influxdb/v1/flux/guides/execute-queries/) to discover a variety of ways to run your queries.
##### Basic mathematic operations
```js
// Examples executed using the Flux REPL
> 9 + 9
18
> 22 - 14
8
> 6 * 5
30
> 21 / 7
3
```
<p style="font-size:.85rem;font-style:italic;margin-top:-2rem;">See <a href="/influxdb/v1/flux/guides/execute-queries/#influx-cli">Flux read-eval-print-loop (REPL)</a>.</p>
{{% note %}}
#### Operands must be the same type
Operands in Flux mathematic operations must be the same data type.
For example, integers cannot be used in operations with floats.
Otherwise, you will get an error similar to:
```
Error: type error: float != int
```
To convert operands to the same type, use [type-conversion functions](/flux/v0/function-types#type-conversions)
or manually format operands.
The operand data type determines the output data type.
For example:
```js
100 // Parsed as an integer
100.0 // Parsed as a float
// Example evaluations
> 20 / 8
2
> 20.0 / 8.0
2.5
```
{{% /note %}}
## Custom mathematic functions
Flux lets you [create custom functions](/influxdb/v2/query-data/flux/custom-functions) that use mathematic operations.
View the examples below.
###### Custom multiplication function
```js
multiply = (x, y) => x * y
multiply(x: 10, y: 12)
// Returns 120
```
###### Custom percentage function
```js
percent = (sample, total) => (sample / total) * 100.0
percent(sample: 20.0, total: 80.0)
// Returns 25.0
```
### Transform values in a data stream
To transform multiple values in an input stream, your function needs to:
- [Handle piped-forward data](/influxdb/v2/query-data/flux/custom-functions/#use-piped-forward-data-in-a-custom-function).
- Each operand necessary for the calculation exists in each row _(see [Pivot vs join](#pivot-vs-join) below)_.
- Use the [`map()` function](/flux/v0/stdlib/universe/map) to iterate over each row.
The example `multiplyByX()` function below includes:
- A `tables` parameter that represents the input data stream (`<-`).
- An `x` parameter which is the number by which values in the `_value` column are multiplied.
- A `map()` function that iterates over each row in the input stream.
It uses the `with` operator to preserve existing columns in each row.
It also multiples the `_value` column by `x`.
```js
multiplyByX = (x, tables=<-) =>
tables
|> map(fn: (r) => ({
r with
_value: r._value * x
})
)
data
|> multiplyByX(x: 10)
```
## Examples
### Convert bytes to gigabytes
To convert active memory from bytes to gigabytes (GB), divide the `active` field
in the `mem` measurement by 1,073,741,824.
The `map()` function iterates over each row in the piped-forward data and defines
a new `_value` by dividing the original `_value` by 1073741824.
```js
from(bucket: "db/rp")
|> range(start: -10m)
|> filter(fn: (r) =>
r._measurement == "mem" and
r._field == "active"
)
|> map(fn: (r) => ({
r with
_value: r._value / 1073741824
})
)
```
You could turn that same calculation into a function:
```js
bytesToGB = (tables=<-) =>
tables
|> map(fn: (r) => ({
r with
_value: r._value / 1073741824
})
)
data
|> bytesToGB()
```
#### Include partial gigabytes
Because the original metric (bytes) is an integer, the output of the operation is an integer and does not include partial GBs.
To calculate partial GBs, convert the `_value` column and its values to floats using the
[`float()` function](/flux/v0/stdlib/universe/float)
and format the denominator in the division operation as a float.
```js
bytesToGB = (tables=<-) =>
tables
|> map(fn: (r) => ({
r with
_value: float(v: r._value) / 1073741824.0
})
)
```
### Calculate a percentage
To calculate a percentage, use simple division, then multiply the result by 100.
```js
> 1.0 / 4.0 * 100.0
25.0
```
_For an in-depth look at calculating percentages, see [Calculate percentages](/influxdb/v1/flux/guides/calculate-percentages)._
## Pivot vs join
To query and use values in mathematical operations in Flux, operand values must
exists in a single row.
Both `pivot()` and `join()` will do this, but there are important differences between the two:
#### Pivot is more performant
`pivot()` reads and operates on a single stream of data.
`join()` requires two streams of data and the overhead of reading and combining
both streams can be significant, especially for larger data sets.
#### Use join for multiple data sources
Use `join()` when querying data from different buckets or data sources.
##### Pivot fields into columns for mathematic calculations
```js
data
|> pivot(rowKey:["_time"], columnKey: ["_field"], valueColumn: "_value")
|> map(fn: (r) => ({ r with
_value: (r.field1 + r.field2) / r.field3 * 100.0
}))
```
##### Join multiple data sources for mathematic calculations
```js
import "sql"
import "influxdata/influxdb/secrets"
pgUser = secrets.get(key: "POSTGRES_USER")
pgPass = secrets.get(key: "POSTGRES_PASSWORD")
pgHost = secrets.get(key: "POSTGRES_HOST")
t1 = sql.from(
driverName: "postgres",
dataSourceName: "postgresql://${pgUser}:${pgPass}@${pgHost}",
query:"SELECT id, name, available FROM exampleTable"
)
t2 = from(bucket: "db/rp")
|> range(start: -1h)
|> filter(fn: (r) =>
r._measurement == "example-measurement" and
r._field == "example-field"
)
join(tables: {t1: t1, t2: t2}, on: ["id"])
|> map(fn: (r) => ({ r with _value: r._value_t2 / r.available_t1 * 100.0 }))
```