Commit Graph

250 Commits (78063a747d914fa730a552ea7272bcf320be7a7e)

Author SHA1 Message Date
Jacob Marble f56c42794b
chore(tracing): Cleanup (#13296)
* chore(tracing): Cleanup

* broken test

* fix unused var

* fix test
2019-04-10 19:28:21 -07:00
Lyon Hill c29e77e6e4
build kv based TaskService (#13228)
* chore(tasks): task kv

* WIP

* get the kv task service complete and passing test
2019-04-09 16:52:54 -06:00
Lyon Hill dad731ceb2
Allow coordinator to skip task claiming in startup (#13264)
fixes #13261
2019-04-09 14:35:32 -06:00
kelwang be674622c6
Revert "fix(inmem): remove the old inmem implementation" 2019-04-09 14:24:40 -04:00
zhulongcheng cacd6a8223 fix(inmem): replace inmem.Service with kv.Service 2019-04-08 15:18:38 +08:00
Jorge Landivar fa470ef926
Merge pull request #13165 from influxdata/fix/rpc-options-panic
chore(tasks): more task tests
2019-04-05 14:24:38 -05:00
docmerlin (j. Emrys Landivar) cc408e9e78 chore(tasks): more task tests 2019-04-05 13:27:48 -05:00
Jeff Wendling 5dc3e52fd9 chore: bump gogo/protobuf and regenerate
It had been bumped previously from v1.1.1 to v1.2.0 and nothing was
regenerated. This bumps it to v1.2.1 and regenerates.
2019-04-03 15:42:33 -06:00
Lyon Hill 986a1acdbe
Update task executor to use a TaskService (#13099) 2019-04-02 15:55:31 -06:00
Lyon Hill 83000361c3
Update task coordinator to act as a TaskService middleware (#13068)
* Update task coordinator to act as a TaskService middleware
2019-04-01 16:45:44 -06:00
docmerlin (j. Emrys Landivar) 492d8406aa fix(tasks): make durations visibly show up nicely 2019-03-29 13:50:26 -05:00
Lyon Hill 1d405f90b1
Ensure requested at is only set when a task was manually requested (#13013) 2019-03-29 09:58:38 -06:00
Lyon Hill c78344c441
update task schedler to use the new task control service (#12949) 2019-03-28 16:52:25 -06:00
Mark Rushakoff 7ecb7a23a4 chore: update staticcheck to latest
And fix newly discovered errors.
2019-03-25 10:20:59 -07:00
Lyon Hill 291319483d
Update task servicetest to move dependency to the new TaskControlService (#12817)
* Update task servicetest to move dependency to the new TaskControlService

closes #12724

We will now have the capability to write new task services that dont have to implement the backend.Store or LogReader or LogWriters
2019-03-21 15:11:22 -06:00
Lyon Hill 49d025ba64
add mutex around modifying runner count (#12778) 2019-03-20 10:40:13 -06:00
Lyon Hill 2ae969b31d
fix(task): Fix flakey tests introduced by slower task updates (#12753)
Task updates now attempt to keep the existing runners working.
This causes the system to be slightly slower after a task update and caused a flakey test.
2019-03-20 09:39:44 -06:00
Mark Rushakoff d04b9e516a feat(task): reject unknown task options 2019-03-20 08:11:32 -07:00
Mark Rushakoff 9541fbb859 fix(task): report query statistics correctly
The synchronous executor was missing a call to ResultIterator.Release.
The asynchronous executor wasn't even calling Query.Statistics.

Also add a test that the scheduler records the statistics to the run
log, and that the statistics are visible from the launcher test. The
launcher test is the most likely place to catch if something goes wrong
in the full stack.
2019-03-20 08:11:14 -07:00
Jorge Landivar e86a7d0a79
Merge pull request #12370 from influxdata/feat/task-delete-options
Feat/task delete options
2019-03-19 16:18:39 -05:00
docmerlin (j. Emrys Landivar) ea82893abd feat(tasks): remove removable options 2019-03-19 10:24:42 -05:00
Lyon Hill b796033287
Add a task control service to facilitate a new structure for tasks (#12657)
* Add a task control service to facilitate a new structure for tasks
2019-03-19 08:36:54 -06:00
Mark Rushakoff 0168d3107e fix(task): only claim active tasks at startup 2019-03-18 20:40:02 -07:00
Mark Rushakoff c8e2339d61 fix(task): filter by measurement early in query logreader
The late measurement filter, after a pivot, had the potential to result
in empty groups without a runID, which would cause a runtime error,
which would cause the whole query to fail.

Experimentation has shown that those empty tables will no longer arrive
by filtering early on measurement.
2019-03-15 22:28:13 -07:00
Mark Rushakoff 50b5b25f71 feat(task): include run errors in task log
This should considerably simplify debugging when things go wrong with
the tasks, as this error can be displayed from the UI or CLI. Prior to
this change, you would have to view the console output from influxd.

Fixes #12548.
2019-03-15 08:12:39 -07:00
Mark Rushakoff ce7b53ef37 feat(task): log authorization errors
This should simplify debugging when a user isn't allowed to do something
with a task, that they expect to be able to do.
2019-03-14 14:04:37 -07:00
Lyon Hill fca6a9c412
Ensure when a manual run is called we run it as soon as possible (#12593)
* Ensure when a manual run is called we run it as soon as possible
2019-03-14 10:25:45 -06:00
Lyon Hill 69db8099d4
Fix requested at for task runs (#12559)
* Fix requested at for task runs

* add short delay to allow storage propigation
2019-03-13 15:01:05 -06:00
kelwang 020c4bc093
Merge pull request #12465 from influxdata/preauthorizer_confusing_bucket_name
fix(query): fix same bucket name with different orgs
2019-03-08 21:31:26 -05:00
Kelvin Wang dcc052b1c2 fix(query): fix same bucket name with different orgs 2019-03-08 20:37:05 -05:00
Chris Goller 639fb5d768 fix(task): set query request authorization
Co-authored-by: Mark Rushakoff <mark@influxdata.com>
2019-03-08 17:48:48 -06:00
Lyon Hill a1ed6e3cec
Return empty list of runs and logs when none are found (#12432)
* Return empty list of runs and logs when none are found
2019-03-07 14:55:19 -07:00
Jacob Marble 603a1f26e0 use tracing.StartSpanFromContext 2019-03-07 12:12:31 -07:00
Jacob Marble 9541e861a3 goimports -w -local github.com/influxdata/influxdb 2019-03-07 12:12:31 -07:00
Jacob Marble 92fa813c45 add spans to multiple services 2019-03-07 12:12:31 -07:00
Lyon Hill bea47b1a7e
Respect the incoming flux script (#12390) 2019-03-06 09:30:11 -07:00
Mark Rushakoff ceaed4c31e fix(task): only update authorization when requested
The previous code was mistakenly updating the task's authorization even
when it wasn't part of the update request.
2019-03-06 08:27:05 -08:00
Lyon Hill 17026c2053
change task logs from a string to a struct (#12363)
* change task logs from a string to a struct
2019-03-05 15:58:33 -07:00
Mark Rushakoff 6591191359 test(task): allow log reader/writer test harness to provide authz
This is particularly relevant when the backing systems (like a real
query service) check authorizations.
2019-03-05 09:02:22 -08:00
Jacob Marble b9c7ec439e
feat(influxd): Tracing refactor (#12318)
* feat(launcher): Tracing to log disabled by default

* remove traceLogger and use opentracing directly

* add Jaeger tracing

* go vet && go fmt
2019-03-04 11:48:11 -08:00
Mark Rushakoff 3307276be1 fix(task): don't error when URM task disappears
In the platform adapter, we ask the URM for a list of tasks the user
owns, and then we look up each task individually.

The task service tests uncovered a legitimate bug where FindTasks would
return a "task not found" error, originating from looking up a task that
was present when we interrogated the URM but was deleted before we could
find it in the task store.

This change also removes duplicated URM logic from the HTTP handler
which has since been pushed down into the platform adapter.
2019-03-04 08:35:04 -08:00
docmerlin (j. Emrys Landivar) 46cb3a2fef chore(task): Clean up comments. 2019-03-01 09:42:13 -06:00
docmerlin (j. Emrys Landivar) 80242b3446 fix(tasks): fix task update validation 2019-03-01 09:00:53 -06:00
Lyon Hill b3f450eca9
Allow the CLI to find tasks by org name (#12202)
* Allow the CLI to find tasks by org name
2019-02-27 08:56:32 -07:00
Jonathan A. Sternberg 78430df2bb
refactor(task/backend): replace a removed interface with the method name (#12203) 2019-02-26 14:20:37 -06:00
Jorge Landivar 8b7aac2bba
Merge pull request #12152 from influxdata/bug/cron-and-every
fix(tasks): allow switching between cron and every options
2019-02-26 12:00:24 -06:00
docmerlin (j. Emrys Landivar) df75f9b9a0 fix(tasks): allow switching between cron and every options 2019-02-26 11:12:56 -06:00
Lyon Hill 049e8f8c90
Add stats to the run log (#12157) 2019-02-26 08:53:36 -07:00
Michael Desa 10dc061dc0
Merge pull request #12143 from influxdata/chore/pw-context
Add context to storage.PointsWriter
2019-02-25 12:31:46 -05:00
Michael Desa 0d3d0d4d78
chore(influxdb): add context to storage.PointsWriter 2019-02-25 11:11:20 -05:00