Commit Graph

218 Commits (69542b1752c3c7dce5aeff5b177c80173f719044)

Author SHA1 Message Date
Lyon Hill 435c8bbed5
fix(task): add logging into the task analytical store (#14124)
* fix(task): add logging into the task analytical store

* add in more detail around log structure
2019-06-12 20:01:41 -06:00
Alirie Gray 0ae2d318b1 chore(tasks): update run limits in swagger and FindRuns 2019-06-12 14:48:32 -07:00
Lyon Hill 68e9b30423
fix(task): Apply grease to the analtic system (#14112) 2019-06-11 13:43:33 -06:00
Lyon Hill ab79b68840
fix(task): Remove older unused task store code. (#14068)
* fix(task): Remove older unused task store code.
2019-06-06 10:05:27 -06:00
Lyon Hill 27fd7ece2d
fix(tasks): move debug task logs to debug (#14053) 2019-06-03 14:11:11 -06:00
Jonathan A. Sternberg 21c80f3e93
refactor(query/control): move the controller from flux to influxdb (#13991)
The controller implementation is primarily used by influxdb so it
shouldn't be part of the flux repository. This copies the code from flux
to influxdb so it can be removed from the next flux release.
2019-05-29 09:04:34 -05:00
Lyon Hill 6c11be0304
fix(task): Remove timeout on task run updates (#13965)
Now that the run status updates are transactional actions
We no longer have to add a timer to keep things on track.

This is causing a problem where some runs are showing up without a start or stop time if the system is busy.
I would rather have the scheduler hang on the update then leave a run action without required fields.
2019-05-20 09:47:20 -06:00
zhulongcheng 91b0741698 fix(task): filter tasks by after param 2019-05-09 20:05:42 +08:00
Lyon Hill 69b26c3df6
fix(task): fix metrics for tasks currently running tasks (#13832) 2019-05-07 13:42:49 -06:00
Lyon Hill e4b69914c9
feat(task): Track queue duration (#13805)
* feat(task): Track queue duration
Track the time it takes for a task to go from it's dueAt time until it actually starts
2019-05-07 10:03:30 -06:00
Chris Goller 510f7bb057 fix(task): lookup fully hydrated task after creation (#13709)
* fix(task): lookup fully hydrated task after creation

* ci(circle): save go race and litmus test results artifact
2019-05-01 14:54:27 -06:00
Jacob Marble 8c269e0153
chore(log): Put trace_id back in logs (#13712)
* chore(log): Put trace_id back in logs

* fix tests
2019-04-30 18:51:22 -07:00
Lyon Hill 166e50f594
Fix inactive task scheduling (#13721) 2019-04-30 16:30:02 -06:00
Alirie Gray cf2c9668fc test(tasks): added coverage for ListTasks with OrgID 2019-04-30 14:20:46 -07:00
Lyon Hill 8fa86488c3
Newly activated tasks should not attempt to 'catch up' (#13700)
* Newly activated tasks should not attempt to 'catch up'
2019-04-29 15:28:34 -06:00
Lyon Hill 7796af7160
task(fix): Tasks should no longer have inaccurate response data (#13641)
* task(fix): Tasks should no longer have inaccurate response data
tasks should be able to pull from a table with both success and failed results

Co-authored-by: AlirieGray <alirie@influxdata.com>
Co-authored-by: docmerlin <emrys@influxdata.com>
2019-04-26 00:40:04 -06:00
Nathaniel Cook a49d1696dc Merge branch 'master' into flux-staging 2019-04-22 14:25:33 -06:00
Christopher M. Wolff c9f94d4ee8
refactor(query): add parameters to query controller config (#13554)
Recently added to controller config:
  ConcurrencyQuota
  MemoryBytesQuotaPerQuery
  QueueSize
2019-04-22 12:11:32 -07:00
Lyon Hill 5b8c4ea0e8
fix task limit in kv and add tests Also fix a minor race condition in task scheduler (#13538)
* fix task limit in kv and add tests Also fix a minor race condition in task scheduler

* update comment
2019-04-22 13:09:19 -06:00
Delmer 429baf5ca7
test(task/scheduler): skip flakey parallel tests (#13516) 2019-04-19 16:40:28 -04:00
Lyon Hill 726fbefe9a
Integrate the new changes to tasks (#13473)
* Integrat the new changes to tasks
2019-04-19 08:37:59 -06:00
Nathaniel Cook 4c77e3598b Merge branch 'master' into flux-staging 2019-04-18 11:01:22 -06:00
Todd Persen cd64ec8718 Fix typos in miscellaneous packages 2019-04-17 13:30:22 -07:00
Nathaniel Cook 8b918539db Merge branch 'master' into flux-staging 2019-04-17 12:26:10 -06:00
Lyon Hill 9a41cc46e4
Create an analytical storage middleware layer (#13418)
* Create an analytical storage middleware layer
2019-04-17 08:41:05 -06:00
Jonathan A. Sternberg b68b5053db refactor: replace usages of the spec compiler with the ast compiler (#13222)
This replaces usages of the spec compiler with the ast compiler and it
removes the error message referencing the spec compiler as an available
input.

It does not remove any of the code using the spec compiler that is
involved for proxying requests and it does not remove it from the API.
2019-04-12 07:15:27 -07:00
Christopher M. Wolff c3d0122a75 fix(task): fix go fmt failure (#13224) 2019-04-12 07:15:27 -07:00
Christopher M. Wolff 16b9158b39 refactor(task): make tasks use new Query interface (#13219)
Fixes #12883.
2019-04-12 07:15:27 -07:00
Jacob Marble f56c42794b
chore(tracing): Cleanup (#13296)
* chore(tracing): Cleanup

* broken test

* fix unused var

* fix test
2019-04-10 19:28:21 -07:00
Lyon Hill dad731ceb2
Allow coordinator to skip task claiming in startup (#13264)
fixes #13261
2019-04-09 14:35:32 -06:00
kelwang be674622c6
Revert "fix(inmem): remove the old inmem implementation" 2019-04-09 14:24:40 -04:00
zhulongcheng cacd6a8223 fix(inmem): replace inmem.Service with kv.Service 2019-04-08 15:18:38 +08:00
Jeff Wendling 5dc3e52fd9 chore: bump gogo/protobuf and regenerate
It had been bumped previously from v1.1.1 to v1.2.0 and nothing was
regenerated. This bumps it to v1.2.1 and regenerates.
2019-04-03 15:42:33 -06:00
Lyon Hill 986a1acdbe
Update task executor to use a TaskService (#13099) 2019-04-02 15:55:31 -06:00
Lyon Hill 83000361c3
Update task coordinator to act as a TaskService middleware (#13068)
* Update task coordinator to act as a TaskService middleware
2019-04-01 16:45:44 -06:00
docmerlin (j. Emrys Landivar) 492d8406aa fix(tasks): make durations visibly show up nicely 2019-03-29 13:50:26 -05:00
Lyon Hill 1d405f90b1
Ensure requested at is only set when a task was manually requested (#13013) 2019-03-29 09:58:38 -06:00
Lyon Hill c78344c441
update task schedler to use the new task control service (#12949) 2019-03-28 16:52:25 -06:00
Mark Rushakoff 7ecb7a23a4 chore: update staticcheck to latest
And fix newly discovered errors.
2019-03-25 10:20:59 -07:00
Lyon Hill 291319483d
Update task servicetest to move dependency to the new TaskControlService (#12817)
* Update task servicetest to move dependency to the new TaskControlService

closes #12724

We will now have the capability to write new task services that dont have to implement the backend.Store or LogReader or LogWriters
2019-03-21 15:11:22 -06:00
Lyon Hill 49d025ba64
add mutex around modifying runner count (#12778) 2019-03-20 10:40:13 -06:00
Lyon Hill 2ae969b31d
fix(task): Fix flakey tests introduced by slower task updates (#12753)
Task updates now attempt to keep the existing runners working.
This causes the system to be slightly slower after a task update and caused a flakey test.
2019-03-20 09:39:44 -06:00
Mark Rushakoff 9541fbb859 fix(task): report query statistics correctly
The synchronous executor was missing a call to ResultIterator.Release.
The asynchronous executor wasn't even calling Query.Statistics.

Also add a test that the scheduler records the statistics to the run
log, and that the statistics are visible from the launcher test. The
launcher test is the most likely place to catch if something goes wrong
in the full stack.
2019-03-20 08:11:14 -07:00
Jorge Landivar e86a7d0a79
Merge pull request #12370 from influxdata/feat/task-delete-options
Feat/task delete options
2019-03-19 16:18:39 -05:00
docmerlin (j. Emrys Landivar) ea82893abd feat(tasks): remove removable options 2019-03-19 10:24:42 -05:00
Lyon Hill b796033287
Add a task control service to facilitate a new structure for tasks (#12657)
* Add a task control service to facilitate a new structure for tasks
2019-03-19 08:36:54 -06:00
Mark Rushakoff 0168d3107e fix(task): only claim active tasks at startup 2019-03-18 20:40:02 -07:00
Mark Rushakoff c8e2339d61 fix(task): filter by measurement early in query logreader
The late measurement filter, after a pivot, had the potential to result
in empty groups without a runID, which would cause a runtime error,
which would cause the whole query to fail.

Experimentation has shown that those empty tables will no longer arrive
by filtering early on measurement.
2019-03-15 22:28:13 -07:00
Mark Rushakoff 50b5b25f71 feat(task): include run errors in task log
This should considerably simplify debugging when things go wrong with
the tasks, as this error can be displayed from the UI or CLI. Prior to
this change, you would have to view the console output from influxd.

Fixes #12548.
2019-03-15 08:12:39 -07:00
Lyon Hill fca6a9c412
Ensure when a manual run is called we run it as soon as possible (#12593)
* Ensure when a manual run is called we run it as soon as possible
2019-03-14 10:25:45 -06:00