Commit Graph

54 Commits (32d41229315a2eb30272b7191697e50e55e10c7a)

Author SHA1 Message Date
Chris Goller 510f7bb057 fix(task): lookup fully hydrated task after creation (#13709)
* fix(task): lookup fully hydrated task after creation

* ci(circle): save go race and litmus test results artifact
2019-05-01 14:54:27 -06:00
Lyon Hill 5b8c4ea0e8
fix task limit in kv and add tests Also fix a minor race condition in task scheduler (#13538)
* fix task limit in kv and add tests Also fix a minor race condition in task scheduler

* update comment
2019-04-22 13:09:19 -06:00
Lyon Hill 726fbefe9a
Integrate the new changes to tasks (#13473)
* Integrat the new changes to tasks
2019-04-19 08:37:59 -06:00
Todd Persen cd64ec8718 Fix typos in miscellaneous packages 2019-04-17 13:30:22 -07:00
Lyon Hill 9a41cc46e4
Create an analytical storage middleware layer (#13418)
* Create an analytical storage middleware layer
2019-04-17 08:41:05 -06:00
Lyon Hill c29e77e6e4
build kv based TaskService (#13228)
* chore(tasks): task kv

* WIP

* get the kv task service complete and passing test
2019-04-09 16:52:54 -06:00
kelwang be674622c6
Revert "fix(inmem): remove the old inmem implementation" 2019-04-09 14:24:40 -04:00
zhulongcheng cacd6a8223 fix(inmem): replace inmem.Service with kv.Service 2019-04-08 15:18:38 +08:00
docmerlin (j. Emrys Landivar) cc408e9e78 chore(tasks): more task tests 2019-04-05 13:27:48 -05:00
docmerlin (j. Emrys Landivar) 492d8406aa fix(tasks): make durations visibly show up nicely 2019-03-29 13:50:26 -05:00
Lyon Hill c78344c441
update task schedler to use the new task control service (#12949) 2019-03-28 16:52:25 -06:00
Lyon Hill 291319483d
Update task servicetest to move dependency to the new TaskControlService (#12817)
* Update task servicetest to move dependency to the new TaskControlService

closes #12724

We will now have the capability to write new task services that dont have to implement the backend.Store or LogReader or LogWriters
2019-03-21 15:11:22 -06:00
docmerlin (j. Emrys Landivar) ea82893abd feat(tasks): remove removable options 2019-03-19 10:24:42 -05:00
Mark Rushakoff ceaed4c31e fix(task): only update authorization when requested
The previous code was mistakenly updating the task's authorization even
when it wasn't part of the update request.
2019-03-06 08:27:05 -08:00
Lyon Hill 17026c2053
change task logs from a string to a struct (#12363)
* change task logs from a string to a struct
2019-03-05 15:58:33 -07:00
docmerlin (j. Emrys Landivar) 80242b3446 fix(tasks): fix task update validation 2019-03-01 09:00:53 -06:00
Lyon Hill b3f450eca9
Allow the CLI to find tasks by org name (#12202)
* Allow the CLI to find tasks by org name
2019-02-27 08:56:32 -07:00
Lyon Hill 3665feef38
Populate organization name on task create action (#12004)
* Populate organization name on task create action

* Move the responsiblity of organization lookup to the platform adaptor
2019-02-20 13:36:35 -07:00
Mark Rushakoff cbef811c63 refactor(task): remove explicit org from run/log lookup
Task ID is now a required value on run and log filters. It was
effectively required by all implementations before anyway, so now those
types reflect that requirement.

Organization ID was removed from those same fields. The TaskService
looks up the organization ID via the task in cases where we need it at a
lower layer.
2019-02-16 14:34:49 -08:00
Lyon Hill 48cc643992
Clean out user from the task store's (#11882)
* Clean out user from the task store's
2019-02-15 15:50:09 -07:00
Mark Rushakoff caf08b5078 feat(task): store authorization ID with task
Accept token when creating or updating a task, but only report back the
authorization ID.

This means the executor and the platform adapter are now both aware of
an Authorization Service.
2019-02-14 08:53:40 -08:00
Mark Rushakoff 007f5059c7 refactor(task): extract TaskCreate struct
With the ongoing authorization work, creation arguments will differ from
what's returned on reads. More specifically, creation will accept a
token, but reads will report back a token ID.

This refactor facilitates that authorization work, and also brings the
code closer to the swagger definition, for the TaskCreateRequest type in
particular.
2019-02-12 15:10:03 -08:00
Mark Rushakoff 4b80fc43c4 test(task): deflake test for CreatedAt, UpdatedAt
The earliest time needed to be captured before creating the task, not
after; otherwise a test running close to a second boundary would fail.

Also make the failure message slightly more readable.
2019-02-12 11:35:55 -08:00
Lyon Hill 512af6e4b3
Add createdAt and updatedAt to tasks api (#11794)
fixes #11061
2019-02-12 11:06:23 -07:00
Mark Rushakoff a0ff60c075 fix(task): respect limit when finding runs 2019-01-24 15:32:16 -08:00
Lyon Hill bdf65f5dfe
Remove owner from task json responses (#11504)
* Remove owner from task json responses

* allow for crud to run in parallel with other tests

To be parallel we just cant assume we only have 1 task.
2019-01-24 16:05:24 -07:00
Mark Rushakoff 882750c751 fix(task): populate created task response, the same as finding
The response from a created task was missing the name field; now a test
ensures that response matches what is returned when listing tasks.
2019-01-22 11:32:17 -08:00
Lyon Hill 523e52d662
Show organization names in task api actions (#11223)
* Show organization names on task actions.

closes #10876
2019-01-18 09:10:14 -07:00
Nathaniel Cook d0603457b7 refactor(flux): make packages mirror Flux namespaces 2019-01-14 18:00:45 -07:00
Nathaniel Cook 622106caaa Merge branch 'master' into flux-staging 2019-01-14 17:03:57 -07:00
Nathaniel Cook 1708a41fa7 refactor: update query functions for Flux builtins 2019-01-11 13:11:57 -07:00
j. Emrys Landivar (docmerlin) a2674fa03d feat(tasks): update task options from api 2019-01-11 11:55:46 -06:00
Mark Rushakoff d73d73c0d4 chore: rename imports from platform to influxdb
I did this with a dumb editor macro, so some comments changed too.

Also rename root package from platform to influxdb.

In interest of minimizing risk, anyone importing the root package has
now aliased it to "platform" so that no changes beyond imports were
necessary in those files.

Lastly, replace the old platform module to local path /dev/null so that
nobody can accidentally reintroduce a platform dependency while
migrating platform code to influxdb.
2019-01-09 20:51:47 -08:00
Mark Rushakoff 0e41f240f2 feat(task): support forcing a run with arbitrary timestamp
Also rename RetryAlreadyQueuedError by running:

gorename -from '"github.com/influxdata/platform/task/backend".RetryAlreadyQueuedError' -to RequestStillQueuedError

and some further manual cleanup for comments.
2018-12-21 14:37:53 -05:00
Lyon Hill 678acd5307
feat(task): Allow the most recent run time to be shown in the api. (#1765)
* feat(task): Allow the most recent run time to be shown in the api.

When showing tasks in the api latest_completed will now show in the api if it has been run.

fixes #1705
2018-12-06 16:46:16 -07:00
j. Emrys Landivar (docmerlin) 0cb888afc0 Change delay to offset in tasks to match the flux spec 2018-12-06 11:00:07 -06:00
j. Emrys Landivar (docmerlin) 7ffd18c7b8 remove requestedAt param from retry call and refactor TaskService interface 2018-11-30 13:14:21 -06:00
Ben Johnson 868fdbbe03
megacheck 2018-11-28 15:52:11 -07:00
docmerlin (j. Emrys Landivar) 57fe285e56 add run return to rerun 2018-11-27 11:48:32 -06:00
Mark Rushakoff 232fb90edb feat(task): add Delay field to platform.Task
This brings Delay in line with Every and Cron.
2018-11-21 14:37:35 -08:00
Mark Rushakoff 558d10f28b test(task): skip long tests in -short mode
Before, running go test -short -count=1 ./task/... would take about 15
seconds. By skipping these two long tests in short mode, that takes
about 10 seconds instead. In particular, the task package itself went
from 10 seconds to under a second, but there wasn't a realized 10 second
gain due to packages being tested in parallel.
2018-11-21 14:11:11 -08:00
Mark Rushakoff b0b0d965ff feat(task): integrate task logs with system buckets
- Brought over enterprise's QueryLogReader, with small adjustments
  - Time filters are for the run's ScheduledFor field, per spec
- Adjusted run log timestamps for consistent formatting:
  - ScheduledFor is RFC3339 because it's a whole-second timestamp
  - StartedAt, FinishedAt use RFC3339Nano for high precision
- Several test adjustments to use relative time, for easier integration
  with storage retention
2018-11-15 08:59:15 -08:00
Mark Rushakoff a7e6a6634e test(task): deflake TaskService shared test
What happened here is kind of a long story.

The TaskService tests exercise different parts of the TaskService
concurrently, in hopes of exercising code that may be racy. This worked
fine so long as we were able to use arbitrary user and organization IDs
picked during test.

Then we added tests for the http-client-backed TaskService. The instance
of http.TaskService takes a single token, so the TaskService tests were
updated to use an optional override to get the user and org ID from the
TaskService implementation.

That meant our parallel tests were using one user and org ID in some
cases. One particular test to exercise concurrency would create tasks
and then retrieve them and delete one. Because this test was filtering
by user and org, sometimes it would delete a task created in a different
test.

The fix was to maintain a whitelist of tasks created in that particular
test, so that it wouldn't delete tasks created in a different test.
2018-10-30 18:52:21 -07:00
Mark Rushakoff cd80e41220 feat(task): add ability to retry runs
This also adjusts the TaskService interface's RetryRun method to accept
a task ID rather than an org ID. Internally, we still look up runs by
organization, and maybe that will change later, but this is a more
natural way for clients to look it up.
2018-10-30 09:20:40 -07:00
docmerlin (j. Emrys Landivar) 6e2620418c Now you can cancel runs 2018-10-29 16:08:50 -05:00
Mark Rushakoff 3fb86665eb test(task): fix check for setting task RequestedAt 2018-10-26 09:49:45 -07:00
Mark Rushakoff 43b7617316 test(task): check task status in more methods
This TODO was meant to be removed as part #1179, but was mistakenly
overlooked.
2018-10-24 11:04:07 -07:00
Lyon Hill 19b7709bad
Add tasks to the influx cli (#1163) 2018-10-23 11:51:13 -06:00
Mark Rushakoff 8c0aec7975 fix(task): propagate status updates when modifying tasks
This also changes the backend.Store API to remove the EnableTask and
DisableTask methods, merging their functionality into ModifyTask, which
has been named to UpdateTask to keep closer to the CRUD acronym.
2018-10-19 16:49:18 -07:00
Mark Rushakoff 3cf88009c4 test(task): fix incorrect test assertion
We changed the semantics of finding a task that doesn't exist, from
returning `nil, nil`, to `nil, ErrTaskNotFound`; this test didn't
reflect that change until this commit.
2018-10-15 10:24:23 -07:00