Commit Graph

406 Commits (4d377b2fb099eb00a5bb99e80bccc97484dd996d)

Author SHA1 Message Date
Sebastian Borza 42856e1394
chore(storageflux): fix linter issue 2020-03-10 17:54:06 -05:00
Jonathan A. Sternberg 834a8740e0
refactor(task/backend): move the task/backend constants to the global package (#17133)
This moves a few types and constants to the global package so it can be
used without importing the `task/backend` package. These constants are
referenced in non tasks-specific code.

This is needed to break a dependency chain where the task backend will
call into the flux runtime to perform parsing or evaluation of a script
and to prevent the http package from inheriting that dependency.
2020-03-06 16:19:32 -06:00
George bc64968b0b
chore(task): add test around paging using after (#16774)
* chore(task): add test around paging using after

* fix(kv): support filter.After in FindTasksByUser
2020-02-07 15:48:55 +00:00
docmerlin (j. Emrys Landivar) 06102ba3d4 fix(tasks): reduce memory allocations in scheduler
This will reduce memory allocations in the scheduler by removing
unneccessary delete/replace actions on the btree that is used as an
internal priority queue.
2020-02-07 09:30:50 -06:00
docmerlin (j. Emrys Landivar) faa87485bd fix(tasks): use LatestCompleted if LatestScheduled is behind 2020-01-17 13:15:13 -06:00
Lorenzo Affetti 0e0b36be0c feat(task): make task query without returning content 2020-01-17 16:51:01 +01:00
j. Emrys Landivar (docmerlin) fb2080f223 feat(tasks): enable full flux durations for every and offset options 2020-01-06 15:42:01 -06:00
Alirie Gray 6c6bd39ffc
refactor(tasks): remove dead scheduler code (#16252) 2020-01-02 11:41:21 -08:00
George ff38420558
feat(tasks): remove flux stats from run log and replace with trace id (#16263)
* feat(tasks): trace id in task run logs

* refactor(tracing): move trace info utility into kit/tracing package
2019-12-18 16:30:38 +01:00
Alirie Gray fc9df92d52
fix(tasks): fix flaky run latency metric test (#16236) 2019-12-16 18:59:54 -08:00
docmerlin (j. Emrys Landivar) bf88cd7502 chore(tasks): add test for task alignment 2019-12-12 13:26:59 -06:00
Bucky Schwarz bc3bfbc61e test: skip flaky cypress test 2019-12-12 13:26:59 -06:00
docmerlin (j. Emrys Landivar) 07345a58f4 fix(tasks): align @every schedule firings 2019-12-12 13:26:59 -06:00
Alirie Gray b5ccad3c07
feat(metrics): add run latency to executor metrics (#16190) 2019-12-11 14:50:32 -08:00
Stuart Carnie 2640f87538
fix(task): LatestScheduled is set when task is updated to active
This change ensures the following behavior:

* task LatestScheduled is always set when a task is updated and
  transitions from a status of inactive to active
* task LatestScheduled is non-zero when created, to set the initial
  schedule time as some point after it was created

In addition, the kv.Service introduces clock.Clock that is used for
task create and task updates only. This change permits testing
in a deterministic fashion.
2019-12-06 11:52:54 -07:00
Jacob Marble 5f19c6cace
chore: Remove several instances of WithLogger (#15996)
* chore: Remove several instances of WithLogger

* chore: unexport Logger fields

* chore: unexport some more Logger fields

* chore: go fmt

chore: fix test

chore: s/logger/log

chore: fix test

chore: revert http.Handler.Handler constructor initialization

* refactor: integrate review feedback, fix all test nop loggers

* refactor: capitalize all log messages

* refactor: rename two logger to log
2019-12-04 15:10:23 -08:00
Alirie Gray 9417936069
feat(tasks): add ability to filter by active tasks (#15893) 2019-12-02 14:16:10 -08:00
George 3dbb9c0fd2
feat(tracing): dont trace spans with full request paths as operation name in ExtractFromHTTPRequest (#15971)
* feat(tracing): dont trace spans with full URL path names in ExtractFromHTTPRequest

* chore(multiple): replace all occurrences of julienschmidt/httprouter with influxdata/httprouter
2019-11-25 15:22:19 +01:00
Edd Robinson c493713d90 fix(tasks): ensure field is 64-bit aligned 2019-11-22 13:25:48 +00:00
George dcfd0688af
revert(tasks): construct permissions for finish run to write to system buckets (#15986) (#16008) 2019-11-22 11:29:20 +01:00
George da647589c4
fix(tasks): construct permissions for finish run to write to system buckets (#15986) 2019-11-21 10:38:20 +01:00
Lyon Hill 5c7787743e
fix(task): add hinting to allow us to skip task auth lookup (#15973) 2019-11-19 11:06:04 -07:00
Jade McGough 6b19ea7569
fix(kv): don't stub system buckets when searching by name (#15928)
* Removed an unused function as cleanup.
* Users without true system buckets won't have fake buckets returned if they don't specify their org in the request, but this shouldn't break tasks.
2019-11-18 23:33:15 -08:00
Lyon Hill 698200b188
fix(task): make calls to update runs force UTC time (#15945)
It is best to standardize the data we store in etcd and influxdb
2019-11-15 19:36:07 -07:00
Alirie Gray f0ecc0e89d
refactor(tasks): use go Time for Task CreatedAt, UpdatedAt, LatestCompleted, Offset (#15672) 2019-11-12 17:13:56 -08:00
docmerlin (j. Emrys Landivar) 56d406afe8 fix(tasks): fix every parsing 2019-11-12 14:46:19 -06:00
Lyon Hill 7421836a1e
fix(task): add error handling for when a `FinishRun` fails (#15845) 2019-11-11 13:52:04 -07:00
Gavin Cabbage 3e29464d4b fix(tasks): fake flux deps 2019-11-08 09:35:20 -05:00
Lyon Hill bb6aa1df3b
feat(task): Add a latestScheduled field for tasks (#15785)
Previously we overwrote the tasks existing latestCompleted to be used for latestCompleted as well as latestScheduled.
For obvious reasons this is confusing and missleading. I believe by seperating the two fields we can have a clear seperation
of concerns.
2019-11-06 15:10:52 -07:00
Lyon Hill d4494273e1
fix(task): Use existing scheduler metrics (#15765) 2019-11-05 10:36:45 -07:00
Lyon Hill 3eacf77bea
fix(task): Set a valid start time for run duration (#15750)
When using a run duration we need a start time that is valid to
obtain a accurate run duration delta
2019-11-04 17:27:44 -07:00
Stuart Carnie b1875d2def
feat(kv): Cursor API accepts hints for improving performance
Implementations of the `kv.Bucket#Cursor` API may use
the hints to instruct the access or read behavior to
the underlying key/value store.

The `findAllTasks` function was also fixed to ensure
that paging works as expected when using a name filter.
Tests were added to verify this behavior.

Redundant error checks were also removed.
2019-11-01 14:48:50 -07:00
docmerlin (j. Emrys Landivar) 71a2590fba fix(tasks): scheduler btree panic 2019-11-01 10:49:47 -05:00
George bb40ca35b8
fix(tasks): thread auth on context when calling FinishRun (#15685) 2019-11-01 10:52:35 +00:00
Gavin Cabbage c60f7dbe6f Revert "fix(flux): use flux.EvalOptions over problematic flux.EvalAST"
This reverts commit d1a0ab85f7.
2019-10-31 16:04:41 -04:00
Gavin Cabbage d1a0ab85f7 fix(flux): use flux.EvalOptions over problematic flux.EvalAST 2019-10-31 12:58:41 -04:00
George 0dde41f895
fix(tasks): wire auth context down to scheduler runner (#15675) 2019-10-31 15:47:12 +00:00
Gavin Cabbage 75147d014c
feat(tasks): expose lastRunStatus and lastRunError in task API 2019-10-30 14:00:22 -04:00
docmerlin (j. Emrys Landivar) 6feae8f3ce feat(tasks): fixes for changeover 2019-10-29 10:36:47 -05:00
docmerlin (j. Emrys Landivar) aa5b094c8d refactor(task): feature flag for nw scheduler 2019-10-29 10:36:47 -05:00
Brandon Farmer bad4751709 fix(influxdb): authorizing system buckets 2019-10-21 15:49:10 -07:00
Brandon Farmer ea82dc3470 fix(tasks): tasks look up system bucket id 2019-10-21 14:48:47 -07:00
Alirie Gray 552168d3ae
refactor(tasks): use Go time objects for timestamps on task Runs (#15406) 2019-10-17 17:23:45 -07:00
George 975289fba1
refactor(tasks): separate run recording behaviour out from analytical storage (#15412) 2019-10-17 10:37:03 +01:00
Alirie Gray f096605327
fix(tasks): replace deactivation of unrecoverable errors with metric (#15430) 2019-10-16 16:00:58 -07:00
Jonathan A. Sternberg b73870d3ed
test(tasks): skip flaky test in the scheduler 2019-10-15 09:18:41 -05:00
docmerlin (j. Emrys Landivar) 0958c26382 feat(tasks): add scheduler release test 2019-10-14 14:02:25 -05:00
docmerlin (j. Emrys Landivar) b8b8422384 feat(tasks): update new scheduler in response to pr comments 2019-10-14 14:02:25 -05:00
docmerlin (j. Emrys Landivar) 4b732acb3b feat(tasks): switch the new scheduler to use clock instead of custom time mocker 2019-10-14 14:02:25 -05:00
j. Emrys Landivar (docmerlin) 3fd94cbb69 feat(task): new scheduler now with more tests 2019-10-14 14:02:25 -05:00