Commit Graph

64 Commits (db/update-flux-version)

Author SHA1 Message Date
Jeffrey Smith II a0f1184bb3
fix: manually scheduled task runs now run when expected (#23664)
* fix: run manually scheduled tasks at their scheduled time

* fix: actually use it

* fix: get tests building

* fix: fix tests

* fix: lint
2022-10-13 14:57:57 -04:00
Nathaniel Cook a321e72d15
build(flux): update flux to v0.185.0 (#23775) 2022-10-05 08:25:58 -06:00
Sam Arnold dece95d1dd
feat: tsm compaction metrics via prometheus (#22904)
* feat: tsm compaction metrics via prometheus

* chore: fix formatting

* chore: make activeCompactions a pointer
2021-11-19 14:51:22 -05:00
William Baker 84776d7428
fix: manual task runs are scheduled asyncronously (#22702) 2021-10-20 09:12:57 -06:00
Sam Arnold 7b2e122869
refactor: refactor for tasks backport (#21117)
* chore: remove dead code

* refactor: move FluxLanguageService interface to fluxlang

* chore: run fmt

* refactor: move task.go from top level to task/taskmodel

* chore: run formatter

* chore: fix up import ordering with gci
2021-04-07 14:42:55 -04:00
Daniel Moran 00afd95cb7
refactor: automated move of errors and id from root to kit (#21101)
Co-authored-by: Sam Arnold <sarnold@influxdata.com>
2021-03-30 14:10:02 -04:00
Daniel Moran 15b9531273
fix: correct various typos (#19987)
Co-authored-by: kumakichi <xyesan@gmail.com>
2020-11-11 13:54:21 -05:00
Christopher M. Wolff e7cbbaa722
feat: upgrade Flux to v0.83.2 (#19569) 2020-09-16 17:59:15 -07:00
Stuart Carnie 0e0b2a3d87
Merge remote-tracking branch 'origin/master' into sgc/tsm1 2020-08-25 09:03:07 -07:00
Lyon Hill f0e382adc2
chore: remove feature flag for task executor (#19422)
We now default task execution to pull from the user service a permission
set.
2020-08-25 08:39:03 -06:00
Stuart Carnie a97ec1f34e
feat: Initial commit with working reads 2020-08-03 09:18:34 -07:00
Lyon Hill a31b651057
feat(task): Add new permission lookup pattern for executor (#18920)
* feat(task): Add new permission lookup pattern for executor

We can now use the user service to populate task owners permissions.
This should improve the task lookup time and decouple the task system
from the URM system. In the future we will have the ability to better isolate
tenant pieces from the rest of the service.

* feat: add feature flagging
2020-07-13 14:47:58 -06:00
George b8f5604613
revert(tasks): Add new permission lookup pattern for executor (#18869) (#18915)
This reverts commit e9e4d794cf.
2020-07-10 11:05:07 +01:00
Lyon Hill e9e4d794cf
feat(task): Add new permission lookup pattern for executor (#18869)
We can now use the user service to populate task owners permissions.
This should improve the task lookup time and decouple the task system
from the URM system. In the future we will have the ability to better isolate
tenant pieces from the rest of the service.
2020-07-09 14:06:46 -06:00
Brett Buddin 0f4b0fbe68
fix(task/executor): Don't send Now for Flux compiler. See TODO for details. 2020-06-18 15:27:28 -04:00
Lyon Hill 22c573df68
fix: clean up old runs when we fail to enqueue it (#18482)
* fix: clean up old runs when we fail to enqueue it

We need to make sure the runs are removed if we fail to enqueue the run to be worked.
for this we will now add an error message stating why we failed and move the run out of our kv store.

* fix: update fail message
2020-06-12 13:02:31 -06:00
Brett Buddin 9b53315e33
feat(task): Separate System from Non-System query compiler feature flagging. 2020-06-08 15:04:13 -04:00
Jonathan A. Sternberg 0c8d19e896
Merge branch 'master' into feat/use-algo-w 2020-05-18 13:25:26 -05:00
Brett Buddin f59c2ce009
feat(task): Allow switching of compiler from outside Executor. 2020-05-14 14:22:43 -04:00
jlapacik 7541af8414 chore: merge master into algow 2020-04-15 14:42:03 -07:00
Brett Buddin c08486ba5a
feat(task): Add option to Executor to set maximum worker count. 2020-04-09 09:37:17 -04:00
Jonathan A. Sternberg d3fa60991f
Merge branch 'master' into chore/merge-master 2020-04-06 12:17:47 -05:00
Jonathan A. Sternberg 0ae8bebd75
refactor: rewrite imports to include the /v2 suffix for version 2 2020-04-03 12:39:20 -05:00
Jonathan A. Sternberg 469c5849c4
Merge branch 'master' into feat/use-algo-w 2020-04-03 09:30:04 -05:00
Gavin Cabbage b5c24f3c9d
refactor(tasks): auth user id (#17562) 2020-04-02 09:40:00 -04:00
Christopher M. Wolff 20c2439230
refactor(http): change *ast.Package to json.RawMessage in query requests (#17316) 2020-03-23 12:39:24 -07:00
Jonathan A. Sternberg 0d6e4e310b
Merge branch 'master' into feat/use-algo-w 2020-03-06 16:46:49 -06:00
Jonathan A. Sternberg 834a8740e0
refactor(task/backend): move the task/backend constants to the global package (#17133)
This moves a few types and constants to the global package so it can be
used without importing the `task/backend` package. These constants are
referenced in non tasks-specific code.

This is needed to break a dependency chain where the task backend will
call into the flux runtime to perform parsing or evaluation of a script
and to prevent the http package from inheriting that dependency.
2020-03-06 16:19:32 -06:00
Jonathan A. Sternberg 935180e9bd
refactor: update flux for the runtime package (#17002) 2020-02-27 11:31:45 -06:00
Lorenzo Affetti 0e0b36be0c feat(task): make task query without returning content 2020-01-17 16:51:01 +01:00
Alirie Gray 6c6bd39ffc
refactor(tasks): remove dead scheduler code (#16252) 2020-01-02 11:41:21 -08:00
George ff38420558
feat(tasks): remove flux stats from run log and replace with trace id (#16263)
* feat(tasks): trace id in task run logs

* refactor(tracing): move trace info utility into kit/tracing package
2019-12-18 16:30:38 +01:00
Jacob Marble 5f19c6cace
chore: Remove several instances of WithLogger (#15996)
* chore: Remove several instances of WithLogger

* chore: unexport Logger fields

* chore: unexport some more Logger fields

* chore: go fmt

chore: fix test

chore: s/logger/log

chore: fix test

chore: revert http.Handler.Handler constructor initialization

* refactor: integrate review feedback, fix all test nop loggers

* refactor: capitalize all log messages

* refactor: rename two logger to log
2019-12-04 15:10:23 -08:00
Stuart Carnie 7240d21e20
fix(task): PR feedback to fix docs 2019-09-17 12:02:04 -07:00
Stuart Carnie 57a710bb9c
fix(task): Improve Executor#Execute error consistency
Implementations of the backend.Executor produce errors limited to
querying the KV store. The remainder of the errors will be processed
in the implementation of a `RunPromise`.

Fixes #15161
2019-09-16 17:10:02 -07:00
Lyon Hill a8d7870689
feat(task): impersonate user on task execution (#14675)
* feat(task): impersonate user on task execution

Passing tokens to tasks is cumbersome and we needed a way to more easily create tasks. With this change we no longer need a token on task create. We take the user that created the task and pass that in as the "owner". As far as the task is concerned the owner is the source of permissions.

This is done by adding an additional field on task create that is OwnerID. We will no longer respect the token passed in and it will be deprecated soon.

Things to do still:
Task updates need to allow for owners to be set.
2019-08-15 18:31:52 -06:00
Sebastian Borza 95b834ca2a
push task backend logging messages back to debug 2019-07-25 17:17:19 -05:00
Sebastian Borza 0653043353
move logger facility back to Info from Debug 2019-07-23 07:35:57 -05:00
Alirie Gray c7f09d6a56
chore(tasks): use pointers for task errors (#14343) 2019-07-15 13:57:51 -07:00
George 61e35a8554
fix(tasks) report errors found when iterating over flux query (#14287)
* Report errors found when iterating over flux query in task

* Add failing test for tasks executor result iterator exhaust failure

* Ensure errors exhausting tasks query result iterator are surfaced as task failure

* Update CHANGELOG with task result iteration error surfacing fix
2019-07-11 17:45:45 +01:00
Alirie Gray 66157c9d44 chore(tasks): consolidate task errors into task_errors.go 2019-06-17 16:03:18 -07:00
Lyon Hill 27fd7ece2d
fix(tasks): move debug task logs to debug (#14053) 2019-06-03 14:11:11 -06:00
Jacob Marble 8c269e0153
chore(log): Put trace_id back in logs (#13712)
* chore(log): Put trace_id back in logs

* fix tests
2019-04-30 18:51:22 -07:00
Jonathan A. Sternberg b68b5053db refactor: replace usages of the spec compiler with the ast compiler (#13222)
This replaces usages of the spec compiler with the ast compiler and it
removes the error message referencing the spec compiler as an available
input.

It does not remove any of the code using the spec compiler that is
involved for proxying requests and it does not remove it from the API.
2019-04-12 07:15:27 -07:00
Christopher M. Wolff c3d0122a75 fix(task): fix go fmt failure (#13224) 2019-04-12 07:15:27 -07:00
Christopher M. Wolff 16b9158b39 refactor(task): make tasks use new Query interface (#13219)
Fixes #12883.
2019-04-12 07:15:27 -07:00
Lyon Hill 986a1acdbe
Update task executor to use a TaskService (#13099) 2019-04-02 15:55:31 -06:00
Mark Rushakoff 9541fbb859 fix(task): report query statistics correctly
The synchronous executor was missing a call to ResultIterator.Release.
The asynchronous executor wasn't even calling Query.Statistics.

Also add a test that the scheduler records the statistics to the run
log, and that the statistics are visible from the launcher test. The
launcher test is the most likely place to catch if something goes wrong
in the full stack.
2019-03-20 08:11:14 -07:00
Chris Goller 639fb5d768 fix(task): set query request authorization
Co-authored-by: Mark Rushakoff <mark@influxdata.com>
2019-03-08 17:48:48 -06:00
Lyon Hill 049e8f8c90
Add stats to the run log (#12157) 2019-02-26 08:53:36 -07:00