History

Adam a84c86b06d Fix initial transpiler grouping so that aggregates will compute (#460 ) * feat(query/influxql): Fix initial transpiler grouping so that aggregates will compute --update spec --add mean test to demonstrate new feature		2018-07-25 17:56:33 -04:00
..
README.md	chore(ifql): Rename IFQL to Flux	2018-06-11 09:26:10 -06:00
compile.go	flux options interface	2018-07-16 14:24:37 -07:00
execute.go	feat(query): Change Encode interface to return bytes written	2018-06-27 10:31:04 -06:00
operations.go	remove query domain	2018-07-09 13:27:30 -07:00

README.md

Query Tests

This package simulates an end-to-end query test without requiring launching a full service. This is currently achieved by dynamically replacing instances of from()---which access a storage location---with instances of fromCSV() which is a data source that takes a single CSV text argument.

Getting Started

You do not need to use fromCSV in your query. You can write your query as you normally would, or copy it from an application as-is. What you will need to provide, along with the query file, are two csv files that contain the input and output data for the query. The test framework will identify the Flux query by the file extension .flux and then locate the required input/output files using the same root filename.

As an example, consider the test simple_max which is defined by the file simple_max.flux:

simple_max.flux

from(db:"test") |> range(start:-5m)  |> group(by:["_measurement"]) |> max(useRowTime:true) |> map(fn: (r) => {max:r._value})

The test will read in this query, as well as the associated CSV files that are in the flux-spec compatible CSV format:

simple_max_in.csv

#datatype,long,dateTime:RFC3339,dateTime:RFC3339,dateTime:RFC3339,tag,tag,double
,blkid,_start,_stop,_time,_measurement,_field,_value
,0,2018-04-17T00:00:00Z,2018-04-17T00:05:00Z,2018-04-17T00:00:00Z,m1,f1,42.0
,0,2018-04-17T00:00:00Z,2018-04-17T00:05:00Z,2018-04-17T00:00:01Z,m1,f1,43.0

simple_max_out.csv

#datatype,long,dateTime:RFC3339Nano,dateTime:RFC3339Nano,dateTime:RFC3339Nano,string,double
,blkid,_start,_stop,_time,_measurement,max
,0,2018-04-17T00:00:00Z,2018-04-17T00:05:00Z,2018-04-17T00:00:01Z,m1,43

Note that all of the test files are identified by the test name, simple_max. So given a , a new test must provide the following files:

.flux: a Flux query file
_in.csv: input file for the query
_out.csv: output file for the query
.influxql: if present, this query will be transpiled and tested against the input and output csv files, but only if there is a corresponding .flux query.

Notes about text formatting

Compliant to the HTTP spec for CSV data, the output produced by the fluxd process has some specific requirements:

UTF-8 normalized
line endings are \r\n

The go program query/querytest/prepcsvtests/prepcsvtests.go can be run to prepare tests. A valid test case must have .flux and .in.csv files. The prepcsvtests executable will iterate over all such test cases in the user-supplied directory, run the Flux query on the input and prompt the user to approve saving the result as .out.csv. go build ./querytest/prepcsvtests/prepcsvtests.go && ./prepcsvtests query/querytests/test_cases

Optionally, you can give a CASENAME to prep the output for a single case: go build ./querytest/prepcsvtests/prepcsvtests.go && ./prepcsvtests query/querytests/test_cases CASENAME