CreateNextRun centralizes StoreTaskMeta modification and therefore
simplifies Store functions a bit.
Store.CreateNextRun isn't called anywhere yet. The next step here is to
use CreateNextRun from the scheduler and remove Store.CreateRun.
The logic used to be only in the scheduler, but now Options exposes it.
This is another step towards making StoreTaskMeta schedule-aware to
enable creating manually requested runs.
The logger is now threaded into the query controller, executor, and the
dispatcher so that we can log panics. They are logged at the info level
because the panics do not result in the system crashing and becoming
unusable.
This logic was repeated in multiple places, so now it can live in a
single place. This also sets a precedent going forward, for adding
methods to the StoreTaskMeta type. We'll likely be adding more methods
to support manual runs of a task.
Durations are changed to be a 3 vector to form a linear basis of
seconds, days, and months.
Interval comprehensions are introduced to be able to define complex
calendar intervals.
Specification is added around time zones.
The window function is update to default to not returning incomplete
windows.
The call to `setErr` would grab a lock that `Pop` used, but `setErr`
requires the controller run loop to be executing for it to work. If we
reverse the order of these calls, it should be fine.
When the controller moves to one of the finished states, it will finish
the parent span so that can be recorded. It presently will do this
multiple times when transitioning between different finished states.
This normally happens within the finishing states, but when compiling or
queueing fails it never enters those finished states and is instead
discarded. We need to signal that the query itself has finished in the
metrics.
This documents the responsibilities of what the Controller does and is
expected to do. It describes some behaviors that aren't implemented, but
acts as a guide for what the Controller should do as we continue
developing the query engine and improving the internal mechanics.
This commit provides a new join implementation that can join
tables with different group keys. It also implements a strategy
for evicting stale data from the join operation's cache when
it can be determined that a table will no longer be joined.
Currently early table eviction from the buffers of the join occurs
when the left-most group key column of both input streams are equal
and that column is part of the join key.
More specifically, introduce a `scheduleAfter` argument to
Store.CreateTask. The previous behavior was leaving a new task's meta
LatestComplete value set to zero, which meant that the first run of a
schedule would start from 1970. Now, it's set to time.Now unless
otherwise specified.
Updates #595.
Adam Perlin
Mark Rushakoff
Nathaniel Cook
Jorge Landivar
Jonathan A. Sternberg
Jade McGough
Adam
jlapacik
Stuart Carnie
Michael Desa