Task ID is now a required value on run and log filters. It was
effectively required by all implementations before anyway, so now those
types reflect that requirement.
Organization ID was removed from those same fields. The TaskService
looks up the organization ID via the task in cases where we need it at a
lower layer.
This was a missed case from #11817.
This case currently occurs when creating a task through the UI, using a
session rather than a full-fledged authorization. It doesn't fix that
case yet, but at least it will log an informative message.
Immediately before the executor calls out to the query service, the
executor loads the authorizer associated with the task, and associates
that authorizer with the context used to execute the query.
Accept token when creating or updating a task, but only report back the
authorization ID.
This means the executor and the platform adapter are now both aware of
an Authorization Service.
This switches run status from a tag to a field. This is likely a
breaking change to existing task logs.
Using a one-off local query, for 250 records, the previous approach took
around 10 seconds and the new approach is about 30 milliseconds. At 1000
records, the previous approach was roughly 110 seconds and the new
approach is around 70 milliseconds.
filter out resources that have mission IDs
fix(influxdb): simplify auth check in PermissionAllowed
review(platform): update as noted in review
fix(influxdb): ensure permission has valid org id
I did this with a dumb editor macro, so some comments changed too.
Also rename root package from platform to influxdb.
In interest of minimizing risk, anyone importing the root package has
now aliased it to "platform" so that no changes beyond imports were
necessary in those files.
Lastly, replace the old platform module to local path /dev/null so that
nobody can accidentally reintroduce a platform dependency while
migrating platform code to influxdb.
Also rename RetryAlreadyQueuedError by running:
gorename -from '"github.com/influxdata/platform/task/backend".RetryAlreadyQueuedError' -to RequestStillQueuedError
and some further manual cleanup for comments.
A standard Makefile is used now in all subdirs that run go generate.
Make will only generate the file if its source files changed.
The checkgenerate target runs clean to ensure all targets a generated
fresh.
Previously, the WithTicker option would call TickScheduler.Tick every
time the underlying time.Ticker sent a time on its channel. This meant
we used a 1s period, which meant that in the worst case, we would see a
tick at about 999ms after the second rollover.
This change increases the underlying time.Ticker frequency, but only
calls TickScheduler.Tick after a second rolls over. Since we now use a
tick frequency of 100ms, during normal operation, TickScheduler.Tick
will be called within 0.1s after the second rolls over.
Previously, using every=1m would run every minute from when the task was
created. This change restores the original intent, which is that "every
1m" happens on the minute, "every 1h" on the hour, etc.
The keys used to be strings, back when platform.ID was a byte slice, not
a uint64.
And rename the field from runners to meta, which is much more accurate.
The flux query controller was updated to include a Shutdown method a
while ago. Explicitly handle query controller creation and shutdown
where applicable.
In influxd, this ensures that outstanding queries are handled before the
process dies. In tests, this ensures that query controller goroutines
aren't leaked, which drastically simplifies reading full stack traces.
This change also registers query controller metrics with the prometheus
registry in influxd.
This is to ensure that Scheduler.Stop blocks until outstanding task runs
finish. There were enterprise tests failing because outstanding runs of
a task were calling (*testing.T).Log after the test finished.
Before, running go test -short -count=1 ./task/... would take about 15
seconds. By skipping these two long tests in short mode, that takes
about 10 seconds instead. In particular, the task package itself went
from 10 seconds to under a second, but there wasn't a realized 10 second
gain due to packages being tested in parallel.
feat(platform): add ToPermissions method to user resource mapping
The ToPermissions method returns a set of permissions that is granted
via a user resource mapping.
feat(bolt): resolve sessions permissions on lookup
feat(http): use authorizer instead of authorization service for write api
feat(bolt): create user resource mappings for org users in bucket create
feat(bolt): create user resource mapping for first org/user
fix(platform): use authorizer for query endpoint instead of authorization
test(http): use cmp instead of reflect for decode test
- Brought over enterprise's QueryLogReader, with small adjustments
- Time filters are for the run's ScheduledFor field, per spec
- Adjusted run log timestamps for consistent formatting:
- ScheduledFor is RFC3339 because it's a whole-second timestamp
- StartedAt, FinishedAt use RFC3339Nano for high precision
- Several test adjustments to use relative time, for easier integration
with storage retention
This also adjusts the TaskService interface's RetryRun method to accept
a task ID rather than an org ID. Internally, we still look up runs by
organization, and maybe that will change later, but this is a more
natural way for clients to look it up.
This also changes the backend.Store API to remove the EnableTask and
DisableTask methods, merging their functionality into ModifyTask, which
has been named to UpdateTask to keep closer to the CRUD acronym.
This renames TaskEnabled and TaskDisabled to TaskActive and
TaskInactive, to keep in line with the swagger and other parts of the
system. But I left the EnableTask and DisableTask methods on the Store
interface for now. I could see eliminating those methods if we adjust
the signature of the UpdateTask method.
The generate commands have been modified to take advantage of the new
functionality in Go 1.11 that allows `go run` to execute a package
instead of individual files.
This functionality combined with Go modules allows us to execute a
package directly out of our pinned dependencies rather than accidentally
picking up another binary outside of the build environment.
This also simplifies the Makefile because they no longer have to be
responsible for installing the correct tooling since the Go command
takes care of that logic. It also makes it so that the Makefiles with
file generation can now be invoked from their appropriate subdirectories
so they are contained within the directory itself rather than relying on
values in the top level Makefile.
It is now possible to generate all files within this project by using:
go generate ./...
Or the Makefile can continue to be used.
This commit also copies over the special copy of `tmpl` that the storage
engine uses within the influxdb repository. It was never copied over so
using `go generate` on these packages did not work.
We are moving the necessary code for 2.0 from the influxdb 1.X
repository to the platform 2.0 repository. The logger is an unnecessary
dependency on the old influxdb that is making life more complicated.
Lyon Hill
Mark Rushakoff
Jacob Marble
docmerlin (j. Emrys Landivar)
Jonathan A. Sternberg
Michael Desa
Lorenzo Affetti
Chris Goller
Mark Rushakoff
Nathaniel Cook
Andrew Watkins
Christopher M. Wolff
Mark Rushakoff
Edd Robinson
zhulongcheng
Ben Johnson
jlapacik
Adam
Leonardo Di Donato
Brandon Farmer