From d083160b83d94cfed75aab079c9d2c4144eb9011 Mon Sep 17 00:00:00 2001 From: Scott Anderson Date: Mon, 9 Mar 2020 11:15:57 -0600 Subject: [PATCH 1/4] added warning about keeping empty tables on first filter call --- .../flux/stdlib/built-in/transformations/filter.md | 12 +++++++++--- 1 file changed, 9 insertions(+), 3 deletions(-) diff --git a/content/v2.0/reference/flux/stdlib/built-in/transformations/filter.md b/content/v2.0/reference/flux/stdlib/built-in/transformations/filter.md index e2a21a91d..4ba0f6170 100644 --- a/content/v2.0/reference/flux/stdlib/built-in/transformations/filter.md +++ b/content/v2.0/reference/flux/stdlib/built-in/transformations/filter.md @@ -49,13 +49,19 @@ Defines the behavior for empty tables. Potential values are `keep` and `drop`. Defaults to `drop`. +_**Data type:** String_ + ##### drop -Empty tables are dropped. +Tables left without rows are dropped. ##### keep -Empty tables are output to the next transformation. +Tables left without rows are output to the next transformation. -_**Data type:** String_ +{{% warn %}} +Keeping empty tables with your first `filter()` function can have severe performance costs. +For higher performance, use your first `filter()` function to do basic filtering, +then keep empty tables on subsequent `filter()` calls. +{{% /warn %}} ## Examples From f2c539ffdae40704e29506fac1a3d9bb4d407139 Mon Sep 17 00:00:00 2001 From: Scott Anderson Date: Mon, 9 Mar 2020 11:39:37 -0600 Subject: [PATCH 2/4] explained why keeping empty tables can affect performance, added example --- .../stdlib/built-in/transformations/filter.md | 18 ++++++++++++++---- 1 file changed, 14 insertions(+), 4 deletions(-) diff --git a/content/v2.0/reference/flux/stdlib/built-in/transformations/filter.md b/content/v2.0/reference/flux/stdlib/built-in/transformations/filter.md index 4ba0f6170..d92bfe593 100644 --- a/content/v2.0/reference/flux/stdlib/built-in/transformations/filter.md +++ b/content/v2.0/reference/flux/stdlib/built-in/transformations/filter.md @@ -52,15 +52,17 @@ Defaults to `drop`. _**Data type:** String_ ##### drop -Tables left without rows are dropped. +Tables without rows are dropped. ##### keep -Tables left without rows are output to the next transformation. +Tables without rows are output to the next transformation. {{% warn %}} -Keeping empty tables with your first `filter()` function can have severe performance costs. +Keeping empty tables with your first `filter()` function can have severe performance +costs since it retains empty tables from your entire data set. For higher performance, use your first `filter()` function to do basic filtering, -then keep empty tables on subsequent `filter()` calls. +then keep empty tables on subsequent `filter()` calls with a smaller data set. +_[See the example below](#keep-empty-tables-when-filtering)._ {{% /warn %}} ## Examples @@ -90,6 +92,14 @@ from(bucket:"example-bucket") |> filter(fn: (r) => r._value > 50.0 and r._value < 65.0 ) ``` +##### Keep empty tables when filtering +```js +from(bucket: "example-bucket") + |> range(start: -1h) + |> filter(fn: (r) => r._measurement == "events" and r._field == "open") + |> filter(fn: (r) => r.doorId == "201df", onEmpty: "keep") +``` +
##### Related InfluxQL functions and statements: From ab4d0d44b4aab6de90793e11427bf1915acb16c6 Mon Sep 17 00:00:00 2001 From: Scott Anderson Date: Mon, 9 Mar 2020 11:40:20 -0600 Subject: [PATCH 3/4] fixed typos --- .../reference/flux/stdlib/built-in/transformations/filter.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/v2.0/reference/flux/stdlib/built-in/transformations/filter.md b/content/v2.0/reference/flux/stdlib/built-in/transformations/filter.md index d92bfe593..b3014bc88 100644 --- a/content/v2.0/reference/flux/stdlib/built-in/transformations/filter.md +++ b/content/v2.0/reference/flux/stdlib/built-in/transformations/filter.md @@ -61,7 +61,7 @@ Tables without rows are output to the next transformation. Keeping empty tables with your first `filter()` function can have severe performance costs since it retains empty tables from your entire data set. For higher performance, use your first `filter()` function to do basic filtering, -then keep empty tables on subsequent `filter()` calls with a smaller data set. +then keep empty tables on subsequent `filter()` calls with smaller data sets. _[See the example below](#keep-empty-tables-when-filtering)._ {{% /warn %}} From 9a13ffe9c970eea8ad68c3869e2a563fcd636445 Mon Sep 17 00:00:00 2001 From: Scott Anderson Date: Mon, 9 Mar 2020 11:45:18 -0600 Subject: [PATCH 4/4] updated keep empty tables on filter example --- .../reference/flux/stdlib/built-in/transformations/filter.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/v2.0/reference/flux/stdlib/built-in/transformations/filter.md b/content/v2.0/reference/flux/stdlib/built-in/transformations/filter.md index b3014bc88..6f16a5c6e 100644 --- a/content/v2.0/reference/flux/stdlib/built-in/transformations/filter.md +++ b/content/v2.0/reference/flux/stdlib/built-in/transformations/filter.md @@ -97,7 +97,7 @@ from(bucket:"example-bucket") from(bucket: "example-bucket") |> range(start: -1h) |> filter(fn: (r) => r._measurement == "events" and r._field == "open") - |> filter(fn: (r) => r.doorId == "201df", onEmpty: "keep") + |> filter(fn: (r) => r.doorId =~ /^2.*/, onEmpty: "keep") ```