From 0600d450f939a137a612d241c835a4f404979320 Mon Sep 17 00:00:00 2001 From: Scott Anderson Date: Tue, 8 Dec 2020 11:46:03 -0700 Subject: [PATCH 1/4] added the operator profiler, closes #1944 --- .../reference/flux/stdlib/profiler/_index.md | 32 +++++++++++-------- 1 file changed, 19 insertions(+), 13 deletions(-) diff --git a/content/influxdb/v2.0/reference/flux/stdlib/profiler/_index.md b/content/influxdb/v2.0/reference/flux/stdlib/profiler/_index.md index 9d276a981..1aa6f2a34 100644 --- a/content/influxdb/v2.0/reference/flux/stdlib/profiler/_index.md +++ b/content/influxdb/v2.0/reference/flux/stdlib/profiler/_index.md @@ -32,12 +32,14 @@ _**Data type:** Array of strings_ ```js import "profiler" -option profiler.enabledProfilers = [""] +option profiler.enabledProfilers = ["query", "operator"] ``` -#### Available profilers +## Available profilers +- [query](#query) +- [operator](#operator) -##### query +### query The `query` profiler provides statistics about the execution of an entire Flux script. When enabled, results returned by [`yield()`](/influxdb/v2.0/reference/flux/stdlib/built-in/outputs/yield/) include a table with the following columns: @@ -56,14 +58,18 @@ include a table with the following columns: - **influxdb/scanned-values**: value scanned by InfluxDB. - **influxdb/scanned-bytes**: number of bytes scanned by InfluxDB. -#### Use the query profiler +### operator +The `operator` profiler output statistics about each operation in a query. +[Pushdown operations](/influxdb/v2.0/query-data/optimize-queries/#start-queries-with-pushdown-functions) +that execute in the data source storage engine are returned as a single operation. +Operations executed in memory are returned as individual operations. +When enabled, results returned by [`yield()`](/influxdb/v2.0/reference/flux/stdlib/built-in/outputs/yield/) +include a table with a row for each operation and the following columns: -Use the query profiler to output statistics about query execution. - -```js -import "profiler" - -option profiler.enabledProfilers = ["query"] - -// ... Query to profile -``` +- **Type:** operation type +- **Label:** operation name +- **Count:** total number of times the operation executed +- **MinDuration:** minimum duration of the operation in nanoseconds +- **MaxDuration:** maximum duration of the operation in nanoseconds +- **DurationSum:** total duration of all operation executions in nanoseconds +- **MeanDuration:** average duration of all operation executions in nanoseconds From b015efbb68b4e19e1ed15a4f77fe7721e768a9ba Mon Sep 17 00:00:00 2001 From: Scott Anderson Date: Tue, 8 Dec 2020 11:49:31 -0700 Subject: [PATCH 2/4] added duplicate shortcode to cloud profiler pkg --- .../reference/flux/stdlib/profiler/_index.md | 56 +------------------ 1 file changed, 2 insertions(+), 54 deletions(-) diff --git a/content/influxdb/cloud/reference/flux/stdlib/profiler/_index.md b/content/influxdb/cloud/reference/flux/stdlib/profiler/_index.md index b58084e43..223b71137 100644 --- a/content/influxdb/cloud/reference/flux/stdlib/profiler/_index.md +++ b/content/influxdb/cloud/reference/flux/stdlib/profiler/_index.md @@ -9,61 +9,9 @@ menu: name: Profiler parent: Flux standard library weight: 202 -influxdb/v2.0/tags: [functions, optimize, package] +influxdb/cloud/tags: [functions, optimize, package] related: - /influxdb/cloud/query-data/optimize-queries/ --- -The Flux Profiler package provides performance profiling tools for Flux queries and operations. -Import the `profiler` package: - -```js -import "profiler" -``` - -## Options -The Profiler package includes the following options: - -### enabledProfilers -Enable Flux profilers. - -_**Data type:** Array of strings_ - -```js -import "profiler" - -option profiler.enabledProfilers = [""] -``` - -#### Available profilers - -##### query -The `query` profiler provides statistics about the execution of an entire Flux script. -When enabled, results returned by [`yield()`](/influxdb/cloud/reference/flux/stdlib/built-in/outputs/yield/) -include a table with the following columns: - -- **TotalDuration**: total query duration in nanoseconds. -- **CompileDuration**: number of nanoseconds spent compiling the query. -- **QueueDuration**: number of nanoseconds spent queueing. -- **RequeueDuration**: number fo nanoseconds spent requeueing. -- **PlanDuration**: number of nanoseconds spent planning the query. -- **ExecuteDuration**: number of nanoseconds spent executing the query. -- **Concurrency**: number of goroutines allocated to process the query. -- **MaxAllocated**: maximum number of bytes the query allocated. -- **TotalAllocated**: total number of bytes the query allocated (includes memory that was freed and then used again). -- **RuntimeErrors**: error messages returned during query execution. -- **flux/query-plan**: Flux query plan. -- **influxdb/scanned-values**: value scanned by InfluxDB. -- **influxdb/scanned-bytes**: number of bytes scanned by InfluxDB. - -#### Use the query profiler - -Use the query profiler to output statistics about query execution. - -```js -import "profiler" - -option profiler.enabledProfilers = ["query"] - -// ... Query to profile -``` +{{< duplicate-oss >}} \ No newline at end of file From 649004e291bf9cf73dfd8636497767c5353066a8 Mon Sep 17 00:00:00 2001 From: Scott Anderson Date: Tue, 8 Dec 2020 13:36:55 -0700 Subject: [PATCH 3/4] updated language around storage vs in memory operations --- .../influxdb/v2.0/reference/flux/stdlib/profiler/_index.md | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/content/influxdb/v2.0/reference/flux/stdlib/profiler/_index.md b/content/influxdb/v2.0/reference/flux/stdlib/profiler/_index.md index 1aa6f2a34..5e70758b4 100644 --- a/content/influxdb/v2.0/reference/flux/stdlib/profiler/_index.md +++ b/content/influxdb/v2.0/reference/flux/stdlib/profiler/_index.md @@ -60,10 +60,9 @@ include a table with the following columns: ### operator The `operator` profiler output statistics about each operation in a query. -[Pushdown operations](/influxdb/v2.0/query-data/optimize-queries/#start-queries-with-pushdown-functions) -that execute in the data source storage engine are returned as a single operation. -Operations executed in memory are returned as individual operations. -When enabled, results returned by [`yield()`](/influxdb/v2.0/reference/flux/stdlib/built-in/outputs/yield/) +[Operations executed in the storage tier](/influxdb/v2.0/query-data/optimize-queries/#start-queries-with-pushdown-functions) +return as a single operation. +When the `operator` profile is enabled, results returned by [`yield()`](/influxdb/v2.0/reference/flux/stdlib/built-in/outputs/yield/) include a table with a row for each operation and the following columns: - **Type:** operation type From d000fd9746dfe09becc6bb15d0444197d9ebef17 Mon Sep 17 00:00:00 2001 From: Scott Anderson Date: Tue, 8 Dec 2020 14:33:20 -0700 Subject: [PATCH 4/4] added comment to profiler code example --- content/influxdb/v2.0/reference/flux/stdlib/profiler/_index.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/content/influxdb/v2.0/reference/flux/stdlib/profiler/_index.md b/content/influxdb/v2.0/reference/flux/stdlib/profiler/_index.md index 5e70758b4..597327845 100644 --- a/content/influxdb/v2.0/reference/flux/stdlib/profiler/_index.md +++ b/content/influxdb/v2.0/reference/flux/stdlib/profiler/_index.md @@ -33,6 +33,8 @@ _**Data type:** Array of strings_ import "profiler" option profiler.enabledProfilers = ["query", "operator"] + +// Query to profile ``` ## Available profilers