From 1a89406071e5a71007829fd834579b284ca94775 Mon Sep 17 00:00:00 2001 From: Scott Anderson Date: Mon, 22 Apr 2019 13:41:07 -0600 Subject: [PATCH 1/8] WIP custom aggregate guide --- .../query-data/guides/custom-aggregate.md | 43 +++++++++++++++++++ 1 file changed, 43 insertions(+) create mode 100644 content/v2.0/query-data/guides/custom-aggregate.md diff --git a/content/v2.0/query-data/guides/custom-aggregate.md b/content/v2.0/query-data/guides/custom-aggregate.md new file mode 100644 index 000000000..b74ebf392 --- /dev/null +++ b/content/v2.0/query-data/guides/custom-aggregate.md @@ -0,0 +1,43 @@ +--- +title: Create custom aggregate functions +description: Create your own custom aggregate functions in Flux using the `reduce()` function. transform and manipulate data. +v2.0/tags: [functions, custom, flux] +menu: + v2_0: + name: Create custom aggregates + parent: How-to guides +weight: 208 +--- + +## Characteristics of an aggregate +Before creating a custom aggregate function, you must understand the characteristics of an aggregate function. + +- Takes all records/rows in a table and combines them into a single row. +- Aggregates operator on input tables individually. + +## How reduce() works +- You start with an identity. It's an object. The identity defines the initial values for the `accumulator` parameter in the reduce `fn`. +- The identity/accumulator object is passed through the reduce function and transformed. It outputs a new accumulator object with updated values for the same parameters. +- The new object is passed back into the reduce function. +- This cycle repeats until all records in the table have been read and modified. +- It produces a table with a single record. + +## Examples + +### Average +```js +average(tables=<-, outputField="average") => + tables + |> reduce( + identity: {sum: 0.0, count: 1.0, avg: 0.0} + fn: (r, accumulator) => ({ + sum: accumulator.sum + r._value, + count: accumulator.count + 1.0, + avg: accumulator.sum / accumulator.count + }) + ) + |> set(key: "_field", value: outputField) + |> duplicate(column: "avg", as: "_value") +``` + +**Note:** `accumulator.x` represents the x attribute of the accumulator object before it is passed back into the `fn`. From 4fd2a33516470c70f2f2916b60accc9ee243cd0b Mon Sep 17 00:00:00 2001 From: Scott Anderson Date: Tue, 23 Apr 2019 17:26:53 -0600 Subject: [PATCH 2/8] fleshed out custom aggregate function doc, resolves #149 --- .../query-data/guides/custom-aggregate.md | 43 --- .../_index.md} | 0 .../custom-functions/custom-aggregate.md | 249 ++++++++++++++++++ 3 files changed, 249 insertions(+), 43 deletions(-) delete mode 100644 content/v2.0/query-data/guides/custom-aggregate.md rename content/v2.0/query-data/guides/{custom-functions.md => custom-functions/_index.md} (100%) create mode 100644 content/v2.0/query-data/guides/custom-functions/custom-aggregate.md diff --git a/content/v2.0/query-data/guides/custom-aggregate.md b/content/v2.0/query-data/guides/custom-aggregate.md deleted file mode 100644 index b74ebf392..000000000 --- a/content/v2.0/query-data/guides/custom-aggregate.md +++ /dev/null @@ -1,43 +0,0 @@ ---- -title: Create custom aggregate functions -description: Create your own custom aggregate functions in Flux using the `reduce()` function. transform and manipulate data. -v2.0/tags: [functions, custom, flux] -menu: - v2_0: - name: Create custom aggregates - parent: How-to guides -weight: 208 ---- - -## Characteristics of an aggregate -Before creating a custom aggregate function, you must understand the characteristics of an aggregate function. - -- Takes all records/rows in a table and combines them into a single row. -- Aggregates operator on input tables individually. - -## How reduce() works -- You start with an identity. It's an object. The identity defines the initial values for the `accumulator` parameter in the reduce `fn`. -- The identity/accumulator object is passed through the reduce function and transformed. It outputs a new accumulator object with updated values for the same parameters. -- The new object is passed back into the reduce function. -- This cycle repeats until all records in the table have been read and modified. -- It produces a table with a single record. - -## Examples - -### Average -```js -average(tables=<-, outputField="average") => - tables - |> reduce( - identity: {sum: 0.0, count: 1.0, avg: 0.0} - fn: (r, accumulator) => ({ - sum: accumulator.sum + r._value, - count: accumulator.count + 1.0, - avg: accumulator.sum / accumulator.count - }) - ) - |> set(key: "_field", value: outputField) - |> duplicate(column: "avg", as: "_value") -``` - -**Note:** `accumulator.x` represents the x attribute of the accumulator object before it is passed back into the `fn`. diff --git a/content/v2.0/query-data/guides/custom-functions.md b/content/v2.0/query-data/guides/custom-functions/_index.md similarity index 100% rename from content/v2.0/query-data/guides/custom-functions.md rename to content/v2.0/query-data/guides/custom-functions/_index.md diff --git a/content/v2.0/query-data/guides/custom-functions/custom-aggregate.md b/content/v2.0/query-data/guides/custom-functions/custom-aggregate.md new file mode 100644 index 000000000..e2163f625 --- /dev/null +++ b/content/v2.0/query-data/guides/custom-functions/custom-aggregate.md @@ -0,0 +1,249 @@ +--- +title: Create custom aggregate functions +description: Create your own custom aggregate functions in Flux using the `reduce()` function. +v2.0/tags: [functions, custom, flux, aggregates] +menu: + v2_0: + name: Custom aggregate functions + parent: Create custom functions +weight: 301 +--- + +Flux provides a number of built-in [aggregate functions](/v2.0/reference/flux/functions/built-in/transformations/aggregates/) +that aggregate data in specific ways. +However, these built-in functions may not meet your specific needs. +The [`reduce()` function](/v2.0/reference/flux/functions/built-in/transformations/aggregates/reduce/) +function provides a way to create custom aggregate functions in Flux. + +## Aggregate function characteristics +Aggregate functions all have the same basic characteristics: + +- They operate on individual input tables and aggregate a table's records into a single record. +- The output table has the same group key as the input table. + +## How reduce() works +The `reduce()` function operates on one row at a time using the function defined in +the [`fn` parameter](/v2.0/reference/flux/functions/built-in/transformations/aggregates/reduce/#fn). +The function maps keys to specific values using two objects specified by the +following parameters: + +| Parameter | Description | +|:---------: |:----------- | +| `r` | An object that represents the row or record. | +| `accumulator` | An object that contains values used in each row's aggregate calculation. | + +The `reduce()` function's [`identity` parameter](/v2.0/reference/flux/functions/built-in/transformations/aggregates/reduce/#identity) +defines the initial `accumulator` object. + +##### Example reduce() function +```js +|> reduce(fn: (r, accumulator) => ({ + sum: r._value + accumulator.sum, + product: r._value * accumulator.product + }) + identity: {sum: 0.0, product: 1.0} +) +``` + +After processing a row, `reduce()` produces an output object and uses it as the +`accumulator` object when processing the next row. + +{{% note %}} +Because `reduce()` uses the output object as the accumulator when processing the next row, +keys mapped in the `reduce()` function must match the `identity` object's keys. +{{% /note %}} + +This cycle repeats until `reduce()` processes all records in the table. +When it processes the last record, it outputs a table containing a single record +with columns for each mapped key. + +### Reduce process example +The example `reduce()` function [above](#example-reduce-function), which produces +a sum and product of all values in a table, would work as follows: + +##### Sample table +```txt + _time _value +----------------------- ------- +2019-04-23T16:10:49.00Z 1.6 +2019-04-23T16:10:59.00Z 2.3 +2019-04-23T16:11:09.00Z 0.7 +2019-04-23T16:11:19.00Z 1.2 +2019-04-23T16:11:29.00Z 3.8 +``` + +#### Processing the first row +`reduce()` uses the row data to define `r` and the `identity` object to define `accumulator`. + +``` +Input Objects +------------- +r: { _time: 2019-04-23T16:10:49.00Z, _value: 1.6 } +accumulator: { sum: 0.0, product: 1.0 } + +Mappings +-------- +sum: 1.6 + 0.0 +product: 1.6 * 1.0 + +Output Object +------------- +{ sum: 1.6, product: 1.6 } +``` + +#### Processing the second row +`reduce()` uses the output object from the first row as the `accumulator` object +when processing the second row: + +``` +Input Objects +------------- +r: { _time: 2019-04-23T16:10:59.00Z, _value: 2.3 } +accumulator: { sum: 1.6, product: 1.6 } + +Mappings +-------- +sum: 2.3 + 1.6 +product: 2.3 * 1.6 + +Output Object +------------- +{ sum: 3.9, product: 3.68 } +``` + +#### Processing all other rows +The cycle continues until all other rows are processed. +Using the [sample table](#sample-table), the final output object would be: + +##### Final output object +```txt +{ sum: 9.6, product: 11.74656 } +``` + +And the output table would look like: + +##### Output table +```txt + sum product +---- --------- + 9.6 11.74656 +``` + +{{% note %}} +Because `_time` is not part of the group key and is not mapped in the `reduce()` function, +it is dropped from the output table. +{{% /note %}} + +## Custom aggregate function examples + +### Custom averaging function +This example illustrates how to create a custom aggregate function that averages values in a table. +However, the built-in [`mean()` function](/v2.0/reference/flux/functions/built-in/tranformations/aggregates/mean/) +does the same thing and is much more performant. + +{{< code-tabs-wrapper >}} +{{% code-tabs %}} +[Comments](#) +[No Comments](#) +{{% /code-tabs %}} + +{{% code-tab-content %}} + +```js +average = (tables=<-, outputField="average") => + tables + |> reduce( + // Define the initial accumulator object + identity: { + count: 1.0, + sum: 0.0, + avg: 0.0 + } + fn: (r, accumulator) => ({ + // Increment the counter on each reduce loop + count: accumulator.count + 1.0, + // Add the _value to the existing sum + sum: accumulator.sum + r._value, + // Divide the existing sum by the existing count for a new average + avg: accumulator.sum / accumulator.count + }) + ) + // Drop the sum and count columns since they are no longer needed + |> drop(columns: ["sum", "count"]) + // Set the _field column of the output table to whatever + // is provided in the outputField parameter + |> set(key: "_field", value: outputField) + // Rename to avg column to _value + |> rename(columns: {avg: "_value"}) +``` +{{% /code-tab-content %}} + +{{% code-tab-content %}} +```js +average = (tables=<-, outputField="average") => + tables + |> reduce( + identity: { + count: 1.0, + sum: 0.0, + avg: 0.0 + } + fn: (r, accumulator) => ({ + count: accumulator.count + 1.0, + sum: accumulator.sum + r._value, + avg: accumulator.sum / accumulator.count + }) + ) + |> drop(columns: ["sum", "count"]) + |> set(key: "_field", value: outputField) + |> rename(columns: {avg: "_value"}) +``` +{{% /code-tab-content %}} +{{< /code-tabs-wrapper >}} + +### Aggregate multiple columns +Built-in aggregate functions only operated on one column. +To aggregate multiple columns at once, use the `reduce()` function to create a custom aggregate function. + +The following function expects input tables to have `c1_value` and `c2_value` columns +and generates the average for each. + +```js +multiAvg = (tables=<-) => + tables + |> reduce( + identity: { + count: 1.0, + c1_sum: 0.0, + c1_avg: 0.0, + c2_sum: 0.0, + c2_avg: 0.0 + } + fn: (r, accumulator) => ({ + count: accumulator.count + 1.0, + c1_sum: accumulator.c1_sum + r.c1_value, + c1_avg: accumulator.c1_sum / accumulator.count, + c2_sum: accumulator.c2_sum + r.c2_value, + c2_avg: accumulator.c2_sum / accumulator.count + }) + ) +``` + +### Aggregate gross and net profit +This example aggregates gross and net profit. +It expects `profit` and `expenses` columns in the input tables. + +```js +profitSummary = (tables=<-) => + tables + |> reduce( + identity: { + gross: 0.0, + net: 0.0 + } + fn: (r, accumulator) => ({ + gross: accumulator.gross + r.profit, + net: accumulator.net + r.profit - r.expenses + }) + ) +``` From 3a6f1b9f7bf94986ff89ad5a109eabfe995fd875 Mon Sep 17 00:00:00 2001 From: Scott Anderson Date: Wed, 24 Apr 2019 09:03:47 -0600 Subject: [PATCH 3/8] added link to custom functions doc --- .../query-data/guides/custom-functions/custom-aggregate.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/content/v2.0/query-data/guides/custom-functions/custom-aggregate.md b/content/v2.0/query-data/guides/custom-functions/custom-aggregate.md index e2163f625..68a7697eb 100644 --- a/content/v2.0/query-data/guides/custom-functions/custom-aggregate.md +++ b/content/v2.0/query-data/guides/custom-functions/custom-aggregate.md @@ -135,6 +135,9 @@ it is dropped from the output table. {{% /note %}} ## Custom aggregate function examples +The following examples apply the principles outlined in [Creating custom functions](/v2.0/query-data/guides/custom-functions) +to create custom aggregate functions using the `reduce()` function. + ### Custom averaging function This example illustrates how to create a custom aggregate function that averages values in a table. From 7f2786edc4c36e49d3b4319087d5d83a836d9855 Mon Sep 17 00:00:00 2001 From: Scott Anderson Date: Thu, 25 Apr 2019 09:37:17 -0600 Subject: [PATCH 4/8] addressed PR feedback for custom aggregate doc --- .../custom-functions/custom-aggregate.md | 22 +++++++++---------- 1 file changed, 11 insertions(+), 11 deletions(-) diff --git a/content/v2.0/query-data/guides/custom-functions/custom-aggregate.md b/content/v2.0/query-data/guides/custom-functions/custom-aggregate.md index 68a7697eb..22c3ce73e 100644 --- a/content/v2.0/query-data/guides/custom-functions/custom-aggregate.md +++ b/content/v2.0/query-data/guides/custom-functions/custom-aggregate.md @@ -9,16 +9,15 @@ menu: weight: 301 --- -Flux provides a number of built-in [aggregate functions](/v2.0/reference/flux/functions/built-in/transformations/aggregates/) -that aggregate data in specific ways. -However, these built-in functions may not meet your specific needs. -The [`reduce()` function](/v2.0/reference/flux/functions/built-in/transformations/aggregates/reduce/) -function provides a way to create custom aggregate functions in Flux. +To aggregate your data, use the Flux +[built-in aggregate functions](/v2.0/reference/flux/functions/built-in/transformations/aggregates/) +or create custom aggregate functions using the +[`reduce()`function](/v2.0/reference/flux/functions/built-in/transformations/aggregates/reduce/). ## Aggregate function characteristics Aggregate functions all have the same basic characteristics: -- They operate on individual input tables and aggregate a table's records into a single record. +- They operate on individual input tables and transform all records into a single record. - The output table has the same group key as the input table. ## How reduce() works @@ -135,14 +134,15 @@ it is dropped from the output table. {{% /note %}} ## Custom aggregate function examples -The following examples apply the principles outlined in [Creating custom functions](/v2.0/query-data/guides/custom-functions) -to create custom aggregate functions using the `reduce()` function. - +To create custom aggregate functions, use principles outlined in +[Creating custom functions](/v2.0/query-data/guides/custom-functions) +and the `reduce()` function. ### Custom averaging function This example illustrates how to create a custom aggregate function that averages values in a table. -However, the built-in [`mean()` function](/v2.0/reference/flux/functions/built-in/tranformations/aggregates/mean/) -does the same thing and is much more performant. +_This is meant for demonstration purposes only. +The built-in [`mean()` function](/v2.0/reference/flux/functions/built-in/tranformations/aggregates/mean/) +does the same thing and is much more performant._ {{< code-tabs-wrapper >}} {{% code-tabs %}} From 9fb9d9774a7fb7dc12a8995e9bda37f9a3307021 Mon Sep 17 00:00:00 2001 From: Scott Anderson Date: Thu, 25 Apr 2019 09:45:01 -0600 Subject: [PATCH 5/8] fixed typos in aggregate function doc --- .../guides/custom-functions/custom-aggregate.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/content/v2.0/query-data/guides/custom-functions/custom-aggregate.md b/content/v2.0/query-data/guides/custom-functions/custom-aggregate.md index 22c3ce73e..c2de773dc 100644 --- a/content/v2.0/query-data/guides/custom-functions/custom-aggregate.md +++ b/content/v2.0/query-data/guides/custom-functions/custom-aggregate.md @@ -205,11 +205,11 @@ average = (tables=<-, outputField="average") => {{< /code-tabs-wrapper >}} ### Aggregate multiple columns -Built-in aggregate functions only operated on one column. -To aggregate multiple columns at once, use the `reduce()` function to create a custom aggregate function. +Built-in aggregate functions only operate on one column. +Use the `reduce()` function to create a custom aggregate function that aggregates multiple columns. -The following function expects input tables to have `c1_value` and `c2_value` columns -and generates the average for each. +The following function expects input tables to have `c1_value` and `c2_value` +columns and generates an average for each. ```js multiAvg = (tables=<-) => From 76ab433185c7dbc9423f1eca0765ac64818e6a0c Mon Sep 17 00:00:00 2001 From: Scott Anderson Date: Thu, 25 Apr 2019 11:16:50 -0600 Subject: [PATCH 6/8] more changes to custom aggregate functions to address PR feedback --- .../query-data/guides/custom-functions/custom-aggregate.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/content/v2.0/query-data/guides/custom-functions/custom-aggregate.md b/content/v2.0/query-data/guides/custom-functions/custom-aggregate.md index c2de773dc..10d6214d1 100644 --- a/content/v2.0/query-data/guides/custom-functions/custom-aggregate.md +++ b/content/v2.0/query-data/guides/custom-functions/custom-aggregate.md @@ -136,10 +136,10 @@ it is dropped from the output table. ## Custom aggregate function examples To create custom aggregate functions, use principles outlined in [Creating custom functions](/v2.0/query-data/guides/custom-functions) -and the `reduce()` function. +and the `reduce()` function to aggregate rows in each input table. ### Custom averaging function -This example illustrates how to create a custom aggregate function that averages values in a table. +This example illustrates how to create a function that averages values in a table. _This is meant for demonstration purposes only. The built-in [`mean()` function](/v2.0/reference/flux/functions/built-in/tranformations/aggregates/mean/) does the same thing and is much more performant._ From a900719841363371aefe12dd197964f0f3976842 Mon Sep 17 00:00:00 2001 From: Scott Anderson Date: Thu, 25 Apr 2019 11:17:48 -0600 Subject: [PATCH 7/8] added group key link to aggregate function doc --- .../v2.0/query-data/guides/custom-functions/custom-aggregate.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/v2.0/query-data/guides/custom-functions/custom-aggregate.md b/content/v2.0/query-data/guides/custom-functions/custom-aggregate.md index 10d6214d1..578df7946 100644 --- a/content/v2.0/query-data/guides/custom-functions/custom-aggregate.md +++ b/content/v2.0/query-data/guides/custom-functions/custom-aggregate.md @@ -18,7 +18,7 @@ or create custom aggregate functions using the Aggregate functions all have the same basic characteristics: - They operate on individual input tables and transform all records into a single record. -- The output table has the same group key as the input table. +- The output table has the same [group key](/v2.0/query-data/get-started/#group-keys) as the input table. ## How reduce() works The `reduce()` function operates on one row at a time using the function defined in From c0228077eb2f4be6598f28bed54a2aa0b452e9a0 Mon Sep 17 00:00:00 2001 From: Scott Anderson Date: Fri, 26 Apr 2019 14:43:55 -0600 Subject: [PATCH 8/8] restructured aggregate functions doc to address PR feedback --- .../custom-functions/custom-aggregate.md | 128 +++++++++--------- 1 file changed, 66 insertions(+), 62 deletions(-) diff --git a/content/v2.0/query-data/guides/custom-functions/custom-aggregate.md b/content/v2.0/query-data/guides/custom-functions/custom-aggregate.md index 578df7946..c72f25da2 100644 --- a/content/v2.0/query-data/guides/custom-functions/custom-aggregate.md +++ b/content/v2.0/query-data/guides/custom-functions/custom-aggregate.md @@ -23,18 +23,23 @@ Aggregate functions all have the same basic characteristics: ## How reduce() works The `reduce()` function operates on one row at a time using the function defined in the [`fn` parameter](/v2.0/reference/flux/functions/built-in/transformations/aggregates/reduce/#fn). -The function maps keys to specific values using two objects specified by the -following parameters: +The `fn` function maps keys to specific values using two [objects](/v2.0/query-data/get-started/syntax-basics/#objects) +specified by the following parameters: | Parameter | Description | |:---------: |:----------- | | `r` | An object that represents the row or record. | | `accumulator` | An object that contains values used in each row's aggregate calculation. | +{{% note %}} The `reduce()` function's [`identity` parameter](/v2.0/reference/flux/functions/built-in/transformations/aggregates/reduce/#identity) defines the initial `accumulator` object. +{{% /note %}} + +### Example reduce() function +The following example `reduce()` function produces a sum and product of all values +in an input table. -##### Example reduce() function ```js |> reduce(fn: (r, accumulator) => ({ sum: r._value + accumulator.sum, @@ -44,23 +49,8 @@ defines the initial `accumulator` object. ) ``` -After processing a row, `reduce()` produces an output object and uses it as the -`accumulator` object when processing the next row. +To illustrate how this function works, take this simplified table for example: -{{% note %}} -Because `reduce()` uses the output object as the accumulator when processing the next row, -keys mapped in the `reduce()` function must match the `identity` object's keys. -{{% /note %}} - -This cycle repeats until `reduce()` processes all records in the table. -When it processes the last record, it outputs a table containing a single record -with columns for each mapped key. - -### Reduce process example -The example `reduce()` function [above](#example-reduce-function), which produces -a sum and product of all values in a table, would work as follows: - -##### Sample table ```txt _time _value ----------------------- ------- @@ -71,66 +61,80 @@ a sum and product of all values in a table, would work as follows: 2019-04-23T16:11:29.00Z 3.8 ``` -#### Processing the first row -`reduce()` uses the row data to define `r` and the `identity` object to define `accumulator`. +###### Input objects +The `fn` function uses the data in the first row to define the `r` object. +It defines the `accumulator` object using the `identity` parameter. +```js +r = { _time: 2019-04-23T16:10:49.00Z, _value: 1.6 } +accumulator = { sum : 0.0, product : 1.0 } ``` -Input Objects -------------- -r: { _time: 2019-04-23T16:10:49.00Z, _value: 1.6 } -accumulator: { sum: 0.0, product: 1.0 } -Mappings --------- +###### Key mappings +It then uses the `r` and `accumulator` objects to populate values in the key mappings: +```js +// sum: r._value + accumulator.sum sum: 1.6 + 0.0 -product: 1.6 * 1.0 -Output Object -------------- +// product: r._value * accumulator.product +product: 1.6 * 1.0 +``` + +###### Output object +This produces an output object with the following key value pairs: + +```js { sum: 1.6, product: 1.6 } ``` -#### Processing the second row -`reduce()` uses the output object from the first row as the `accumulator` object -when processing the second row: +The function then processes the next row using this **output object** as the `accumulator`. -``` -Input Objects -------------- -r: { _time: 2019-04-23T16:10:59.00Z, _value: 2.3 } -accumulator: { sum: 1.6, product: 1.6 } +{{% note %}} +Because `reduce()` uses the output object as the `accumulator` when processing the next row, +keys mapped in the `fn` function must match keys in the `identity` and `accumulator` objects. +{{% /note %}} -Mappings --------- +###### Processing the next row +```js +// Input objects for the second row +r = { _time: 2019-04-23T16:10:59.00Z, _value: 2.3 } +accumulator = { sum : 1.6, product : 1.6 } + +// Key mappings for the second row sum: 2.3 + 1.6 product: 2.3 * 1.6 -Output Object -------------- +// Output object of the second row { sum: 3.9, product: 3.68 } ``` -#### Processing all other rows -The cycle continues until all other rows are processed. -Using the [sample table](#sample-table), the final output object would be: +It then uses the new output object as the `accumulator` for the next row. +This cycle continues until all rows in the table are processed. -##### Final output object -```txt +##### Final output object and table +After all records in the table are processed, `reduce()` uses the final output object +to create a transformed table with one row and columns for each mapped key. + +```js +// Final output object { sum: 9.6, product: 11.74656 } -``` -And the output table would look like: - -##### Output table -```txt +// Output table sum product ---- --------- 9.6 11.74656 ``` {{% note %}} -Because `_time` is not part of the group key and is not mapped in the `reduce()` function, -it is dropped from the output table. +#### What happened to the \_time column? +The `reduce()` function only keeps columns that are: + +1. Are part of the input table's [group key](/v2.0/query-data/get-started/#group-keys). +2. Explicitly mapped in the `fn` function. + +It drops all other columns. +Because `_time` is not part of the group key and is not mapped in the `fn` function, +it isn't included in the output table. {{% /note %}} ## Custom aggregate function examples @@ -138,7 +142,7 @@ To create custom aggregate functions, use principles outlined in [Creating custom functions](/v2.0/query-data/guides/custom-functions) and the `reduce()` function to aggregate rows in each input table. -### Custom averaging function +### Create a custom average function This example illustrates how to create a function that averages values in a table. _This is meant for demonstration purposes only. The built-in [`mean()` function](/v2.0/reference/flux/functions/built-in/tranformations/aggregates/mean/) @@ -171,12 +175,12 @@ average = (tables=<-, outputField="average") => avg: accumulator.sum / accumulator.count }) ) - // Drop the sum and count columns since they are no longer needed + // Drop the sum and the count columns since they are no longer needed |> drop(columns: ["sum", "count"]) - // Set the _field column of the output table to whatever - // is provided in the outputField parameter + // Set the _field column of the output table to to the value + // provided in the outputField parameter |> set(key: "_field", value: outputField) - // Rename to avg column to _value + // Rename avg column to _value |> rename(columns: {avg: "_value"}) ``` {{% /code-tab-content %}} @@ -206,7 +210,7 @@ average = (tables=<-, outputField="average") => ### Aggregate multiple columns Built-in aggregate functions only operate on one column. -Use the `reduce()` function to create a custom aggregate function that aggregates multiple columns. +Use `reduce()` to create a custom aggregate function that aggregates multiple columns. The following function expects input tables to have `c1_value` and `c2_value` columns and generates an average for each. @@ -233,8 +237,8 @@ multiAvg = (tables=<-) => ``` ### Aggregate gross and net profit -This example aggregates gross and net profit. -It expects `profit` and `expenses` columns in the input tables. +Use `reduce()` to create a function that aggregates gross and net profit. +This example expects `profit` and `expenses` columns in the input tables. ```js profitSummary = (tables=<-) =>