Commit Graph

175 Commits (02cc100b4f56eaefabc521eba96fb1c12e5b75f1)

Author SHA1 Message Date
Chris Goller 49782c3be4 feat(bolt): add metrics for resources stored in boltdb 2019-01-18 17:45:45 -06:00
Lyon Hill 523e52d662
Show organization names in task api actions (#11223)
* Show organization names on task actions.

closes #10876
2019-01-18 09:10:14 -07:00
Mark Rushakoff f93ecaa137 refactor(task): look up runs more efficiently
This switches run status from a tag to a field. This is likely a
breaking change to existing task logs.

Using a one-off local query, for 250 records, the previous approach took
around 10 seconds and the new approach is about 30 milliseconds. At 1000
records, the previous approach was roughly 110 seconds and the new
approach is around 70 milliseconds.
2019-01-16 13:25:28 -08:00
Lyon Hill 382de51683
Make task validation errors more clear (#11093)
* fix task validation errors to be more clear
fixes #10987
2019-01-15 13:35:04 -07:00
Michael Desa c60031a6b7 fix(influxdb): change resource to resource type 2019-01-15 12:57:57 -05:00
Nathaniel Cook d0603457b7 refactor(flux): make packages mirror Flux namespaces 2019-01-14 18:00:45 -07:00
Nathaniel Cook 622106caaa Merge branch 'master' into flux-staging 2019-01-14 17:03:57 -07:00
Lyon Hill dfbbfc53cd
Merge pull request #11015 from influxdata/feat/task-validation
feat(task): Add validations for all task actions
2019-01-14 08:16:27 -07:00
Lyon Hill bc9755ae75 fix a naming typo 2019-01-12 10:02:57 -07:00
Jorge Landivar f8f1c1e7b6
Merge pull request #10954 from influxdata/feature/update_task_options
feat(tasks): update task options from api
2019-01-11 14:58:10 -06:00
Nathaniel Cook 3a2231be98 Merge branch 'master' into flux-staging 2019-01-11 13:47:49 -07:00
Lyon Hill d1fc5dc58c vet code 2019-01-11 13:34:30 -07:00
Nathaniel Cook 1708a41fa7 refactor: update query functions for Flux builtins 2019-01-11 13:11:57 -07:00
Lyon Hill 6967734cda feat(task): Add validations for all task actions 2019-01-11 13:09:31 -07:00
j. Emrys Landivar (docmerlin) a2674fa03d feat(tasks): update task options from api 2019-01-11 11:55:46 -06:00
Michael Desa 944de4157a fix(influxdb): add org id to permissions
filter out resources that have mission IDs

fix(influxdb): simplify auth check in PermissionAllowed

review(platform): update as noted in review

fix(influxdb): ensure permission has valid org id
2019-01-11 12:23:28 -05:00
jlapacik 5266cba346
deps: updates for packages and imports (#10962) 2019-01-10 14:33:25 -08:00
Jonathan A. Sternberg a59e6b8d25 refactor: rename DoArrow to Do (#2372)
See influxdata/flux#783 for details.
2019-01-10 10:30:25 -07:00
Jonathan A. Sternberg 6ccb8e8c75 refactor: convert usages of ColReader to ArrowColReader 2019-01-10 10:25:10 -07:00
Mark Rushakoff d73d73c0d4 chore: rename imports from platform to influxdb
I did this with a dumb editor macro, so some comments changed too.

Also rename root package from platform to influxdb.

In interest of minimizing risk, anyone importing the root package has
now aliased it to "platform" so that no changes beyond imports were
necessary in those files.

Lastly, replace the old platform module to local path /dev/null so that
nobody can accidentally reintroduce a platform dependency while
migrating platform code to influxdb.
2019-01-09 20:51:47 -08:00
Michael Desa ad75185629 fix(task): update the permission checked in CreateTask
CreateTasks now check that the user has the write permission to the
tasks resource belonging to an organization. This change comes after
https://github.com/influxdata/platform/pull/2157 modified the structure
of authorization.
2019-01-09 10:36:25 -05:00
Andrew Watkins 3429e8d0c6
feat(auth): rewrite authorization service (#2157)
* feat(view-token-overlay): add view token overlay

* test(tokens): update tests

* chore(auths): rename mock data file

* feat(token-view): clicking on description opens token view modal

* feat(token-view): add ability to close overlay

* feat(token-view): display token permissions with updated permissions shape

* feat(token-view): wip update authorization and permission shapes

* feat(auth): wip refactor auth permissions

* fix(auth): generate permissions via functions

* fix(auth): make Id ID

* chore(types): update generated client

* feat(auth): wip add user and org names to auth

* fix(user): didnt save rebase

* feat(auth): WIP refactor auth

* feat(auth): check for user existence during auth creation

* feat(auth): org must exist during auth creation

* fix(auth): pluralize telegrafs resource type

* docs(http): update swagger definition for the Authorization

* test(auth): fix broken tests

* docs(swagger): update cur_swagger Authrorizations

* fix(api): remove trace from cur_swag

* test(ui/token): update components with new generated type definitions

* feat(http): add lookup service adding names for permissions

* fix(http): remove debugging panics

* chore: go tidy

* fix: unsaved rebase

* test(idpe): add ids to Authorizations for log tests
2018-12-28 15:02:19 -08:00
Mark Rushakoff 0e41f240f2 feat(task): support forcing a run with arbitrary timestamp
Also rename RetryAlreadyQueuedError by running:

gorename -from '"github.com/influxdata/platform/task/backend".RetryAlreadyQueuedError' -to RequestStillQueuedError

and some further manual cleanup for comments.
2018-12-21 14:37:53 -05:00
Chris Goller 31518b7022
Merge pull request #1562 from bednar/task_service
Fixed the task id path, Added RunController instance to pAdapter
2018-12-21 08:48:01 -06:00
Nathaniel Cook 61e36cbee6 chore(Makefile): add target to check generated files are accurate
A standard Makefile is used now in all subdirs that run go generate.
Make will only generate the file if its source files changed.
The checkgenerate target runs clean to ensure all targets a generated
fresh.
2018-12-18 12:54:17 -07:00
Lyon Hill f7ae8c3524
feat(api/task): Add validation for both tasks and buckets on task create (#1900)
fixes #1622
2018-12-12 16:07:40 -07:00
Mark Rushakoff 2e81dd31b4 refactor(task): enable high-frequency WithTicker option
Previously, the WithTicker option would call TickScheduler.Tick every
time the underlying time.Ticker sent a time on its channel. This meant
we used a 1s period, which meant that in the worst case, we would see a
tick at about 999ms after the second rollover.

This change increases the underlying time.Ticker frequency, but only
calls TickScheduler.Tick after a second rolls over. Since we now use a
tick frequency of 100ms, during normal operation, TickScheduler.Tick
will be called within 0.1s after the second rolls over.
2018-12-07 08:38:59 -08:00
Lyon Hill 678acd5307
feat(task): Allow the most recent run time to be shown in the api. (#1765)
* feat(task): Allow the most recent run time to be shown in the api.

When showing tasks in the api latest_completed will now show in the api if it has been run.

fixes #1705
2018-12-06 16:46:16 -07:00
j. Emrys Landivar (docmerlin) 0cb888afc0 Change delay to offset in tasks to match the flux spec 2018-12-06 11:00:07 -06:00
Christopher M. Wolff f1d21b8132
refactor(query): update transpiler to reflect signature change to `group()` (#1689)
update other modules that used old syntax as well
2018-12-04 09:32:37 -08:00
Jakub Bednar 3d99af0c78 Fixed the task id path, Added RunController instance to pAdapter 2018-12-03 08:23:10 +01:00
Mark Rushakoff 47d04f198d fix(task): align task every with whole units
Previously, using every=1m would run every minute from when the task was
created. This change restores the original intent, which is that "every
1m" happens on the minute, "every 1h" on the hour, etc.
2018-11-30 16:26:27 -08:00
j. Emrys Landivar (docmerlin) 7ffd18c7b8 remove requestedAt param from retry call and refactor TaskService interface 2018-11-30 13:14:21 -06:00
Mark Rushakoff dc5616e3f9 refactor(task): store meta objects by raw ID, not ID string
The keys used to be strings, back when platform.ID was a byte slice, not
a uint64.

And rename the field from runners to meta, which is much more accurate.
2018-11-30 08:47:17 -08:00
Mark Rushakoff b1c80c8991
Merge pull request #1627 from zhulongcheng/task-page-size
fix(task): add limit field to TaskFilter
2018-11-30 08:17:12 -08:00
Edd Robinson 186e0392ed Address PR feedback 2018-11-30 10:54:24 +00:00
Edd Robinson 9403c1ec8e Ensure error strings not capitalised ST1005 2018-11-30 10:54:24 +00:00
zhulongcheng 535716044f unify task page size 2018-11-30 10:02:41 +08:00
zhulongcheng d8c80fa50c add limit param to task service 2018-11-30 10:02:41 +08:00
Ben Johnson 868fdbbe03
megacheck 2018-11-28 15:52:11 -07:00
Ben Johnson 1862b4421d
Integrate scanned values statistics tracking. 2018-11-28 15:32:06 -07:00
Christopher M. Wolff 367ee7965c
Remove last reference to flux.control.Config.Verbose (#1588) 2018-11-27 15:36:48 -07:00
docmerlin (j. Emrys Landivar) 57fe285e56 add run return to rerun 2018-11-27 11:48:32 -06:00
Mark Rushakoff a06d4dea16 refactor(query): explicitly shut down query controller
The flux query controller was updated to include a Shutdown method a
while ago. Explicitly handle query controller creation and shutdown
where applicable.

In influxd, this ensures that outstanding queries are handled before the
process dies. In tests, this ensures that query controller goroutines
aren't leaked, which drastically simplifies reading full stack traces.

This change also registers query controller metrics with the prometheus
registry in influxd.
2018-11-26 16:13:19 -08:00
jlapacik 2b52224e84 update flux dependency - add missing Statistics() methods 2018-11-26 13:19:14 -08:00
Mark Rushakoff 232fb90edb feat(task): add Delay field to platform.Task
This brings Delay in line with Every and Cron.
2018-11-21 14:37:35 -08:00
Mark Rushakoff ed15dd2305 refactor(task): add Executor.Wait method
This is to ensure that Scheduler.Stop blocks until outstanding task runs
finish. There were enterprise tests failing because outstanding runs of
a task were calling (*testing.T).Log after the test finished.
2018-11-21 14:27:56 -08:00
Mark Rushakoff 5b3f51a7b2 test(task): ensure that deleting a task, deletes its meta 2018-11-21 14:11:37 -08:00
Mark Rushakoff 558d10f28b test(task): skip long tests in -short mode
Before, running go test -short -count=1 ./task/... would take about 15
seconds. By skipping these two long tests in short mode, that takes
about 10 seconds instead. In particular, the task package itself went
from 10 seconds to under a second, but there wasn't a realized 10 second
gain due to packages being tested in parallel.
2018-11-21 14:11:11 -08:00
Michael Desa 81d1520dc6 fix(platform): keep authorization struct on proxy query request
As a result of pr https://github.com/influxdata/platform/pull/1494, idpe
broke. This PR undoes some of the work done from that PR, but fixes the
underlying issue with #1494.
2018-11-20 15:20:51 -08:00