This refactors everything to generate and use a flux AST when
transpiling an influxql query. This also updates the spectests so they
use flux instead of writing out the AST and compare the resulting
AST's.
We reorganized the functions in flux to have the structure:
/functions
/inputs
/transformations
/outputs
this PR catches up platform to work with the new package layout.
As a separate refactoring issue, we should discuss:
from(bucket: ) should migrate from flux --> platform
to_http and to_kafka should migrate from platform --> flux
* feat(query/influxql): add regex support to transpiler
also added test case to querytest
* Add raw_with_condition test to transpiler unit tests
* add unit tests for regex conditions on raw query
The transpiler compilation tests will now not allow skip to be
specified. Instead, it must return an error message that starts with
`unimplemented` and then the reason will be used as the skip message.
This way, it will be easier to identify the failing tests in the
transpiler. In the previous method, it was possible for a test to be
marked as skip, but for the transpiler to return the wrong error message
because the test did not differentiate between an unimplemented error
message and an incorrect error message.
This extra flexibility makes it easier for the transpiler to generate a
specification since the map step can be focused on only generating the
columns related to fields. In particular, it makes it easier to
implement wildcards for tags because the tags will get passed along with
the partition key.
The spec says to use the `_time` column for the time in the output, but
we were mapping `r._time` to `time` and using the `time` variable. This
modifies the encoder to use the `_time` column and rename it to `time`
for the column name.
There are a few changes to how the transpiler works. The first is that
the streams are now abstracted behind a `cursor` interface. The
interface keeps track of which AST nodes (like variables or function
calls) are represented by the data inside of the stream and the method
of how to access the underlying data. This makes it easier to make a
generic interface for things like the join and map operations. This also
makes it easier to, in the future, use the same code from the map
operation for a filter so we can implement conditions.
This also follows the transpiler readme's methods and takes advantage of
the updates to the ifql language. This means it will group the relevant
cursors into a cursor group, perform any necessary joins, and allow us
to continue building on this as we flesh out more parts of the
transpiler and the language.
The cursor interface makes it so we no longer have to keep a symbol
table mapping the generated names to the locations because that is all
kept within the incoming cursor rather than as a separate data
structure.
It also splits the transpiler into more files so it is easier to find
the relevant code for each stage of the transpiler.
Jonathan A. Sternberg
Nathaniel Cook
Adam
Stuart Carnie