We reorganized the functions in flux to have the structure:
/functions
/inputs
/transformations
/outputs
this PR catches up platform to work with the new package layout.
As a separate refactoring issue, we should discuss:
from(bucket: ) should migrate from flux --> platform
to_http and to_kafka should migrate from platform --> flux
This commit provides a new join implementation that can join
tables with different group keys. It also implements a strategy
for evicting stale data from the join operation's cache when
it can be determined that a table will no longer be joined.
Currently early table eviction from the buffers of the join occurs
when the left-most group key column of both input streams are equal
and that column is part of the join key.
There are a few changes to how the transpiler works. The first is that
the streams are now abstracted behind a `cursor` interface. The
interface keeps track of which AST nodes (like variables or function
calls) are represented by the data inside of the stream and the method
of how to access the underlying data. This makes it easier to make a
generic interface for things like the join and map operations. This also
makes it easier to, in the future, use the same code from the map
operation for a filter so we can implement conditions.
This also follows the transpiler readme's methods and takes advantage of
the updates to the ifql language. This means it will group the relevant
cursors into a cursor group, perform any necessary joins, and allow us
to continue building on this as we flesh out more parts of the
transpiler and the language.
The cursor interface makes it so we no longer have to keep a symbol
table mapping the generated names to the locations because that is all
kept within the incoming cursor rather than as a separate data
structure.
It also splits the transpiler into more files so it is easier to find
the relevant code for each stage of the transpiler.
Adam
Jonathan A. Sternberg
Stuart Carnie
jlapacik