Commit Graph

406 Commits (4d377b2fb099eb00a5bb99e80bccc97484dd996d)

Author SHA1 Message Date
Mark Rushakoff 479f3dbb3d test(task): add platform adapter tests
Also fix a handful of segfaults caused by improperly retained byte slices
from bolt, and combine two view transactions into one to avoid  data
race when a delete happened between them.
2018-09-05 10:06:55 -07:00
Lyon Hill 5be44d7fac
fix(taskd): Make WithTicker not block. (#765)
A mistake in the WithTicker method caused it to block.
This was not evident because until recently we used a cron styler timer.
Removing the block should allow the task deamon to start up properly.
2018-09-04 10:16:40 -06:00
Nathaniel Cook 46a32e874c chore: Update 'from' to match spec
Removes the `db` parameter and used only bucket and bucketID.
2018-08-30 14:51:51 -06:00
Mark Rushakoff d4753e2ed6 chore(platform): remove Last Run field from Task
Closes #626.
2018-08-30 11:02:17 -07:00
Chris Goller 8fa72fa35b feat(auth): allow authorizations to be enabled/disabled 2018-08-29 13:07:38 -05:00
Mark Rushakoff a8018532b2 test(task): fix two flaky tests
In TestScheduler_RunLog, the run log gets updated asynchronously so we
need to poll to ensure we don't read a stale value.

In TestScheduler_CreateNextRunOnTick, we were previously reading the
one entry out of two from a map, so it was often the entry we weren't
interested in.
2018-08-29 10:02:45 -07:00
Mark Rushakoff caa4539d0e fix(task): fix data race in scheduler 2018-08-28 08:11:16 -07:00
Mark Rushakoff f72ed76a47 feat(task): handle manual runs in scheduler 2018-08-26 20:34:05 -07:00
Mark Rushakoff ac03771c62 feat(task): handle manual runs in StoreTaskMeta.CreateNextRun 2018-08-26 20:34:05 -07:00
Mark Rushakoff 4573c16367 feat(task): add ManuallyRunTimeRange to Store 2018-08-26 20:34:05 -07:00
Lyon Hill 816a11c9a0
refactor(task): add org id to log reader calls (#684)
* refactor(task): add org id to log reader calls
2018-08-23 12:30:59 -06:00
Mark Rushakoff 813ba2f221 chore(task): rename StoreTaskMeta.LastCompleted to LatestCompleted
LastCompleted implies they could be out of order, so LatestCompleted is
more accurate.
2018-08-22 10:36:03 -07:00
Lyon Hill f718b9cdca
feat(task): create validation layer for TaskService (#591)
* feat(task): create validation layer for TaskService
2018-08-20 15:15:04 -06:00
Mark Rushakoff 9149c6508d hotfix(task): skip long test on Jenkins or with -short 2018-08-17 14:34:13 -07:00
Mark Rushakoff 66e97c938d pr: address feedback 2018-08-17 13:45:19 -07:00
Mark Rushakoff de577125a8 refactor(task): remove CreateRun method from Store
This method has been superseded by CreateNextRun.
2018-08-17 11:49:09 -07:00
Mark Rushakoff 11fe3acf05 refactor(task): use CreateNextRun in scheduler 2018-08-17 11:08:23 -07:00
Mark Rushakoff 64ad4bd1c2 refactor(task): add CreateNextRun method to Store interface
CreateNextRun centralizes StoreTaskMeta modification and therefore
simplifies Store functions a bit.

Store.CreateNextRun isn't called anywhere yet. The next step here is to
use CreateNextRun from the scheduler and remove Store.CreateRun.
2018-08-17 11:08:23 -07:00
Mark Rushakoff 05a930ecb3 refactor(task): add CreateNextRun method to StoreTaskMeta 2018-08-17 11:08:23 -07:00
Mark Rushakoff cd9b9d65a4 refactor(task): set Delay on StoreTaskMeta 2018-08-17 11:08:23 -07:00
Mark Rushakoff c2c4de05b0 chore(task): update StoreTaskMeta for manually requested runs 2018-08-17 11:08:23 -07:00
Mark Rushakoff d0cd454a55 refactor(task): add EffectiveCronString method to Options
The logic used to be only in the scheduler, but now Options exposes it.
This is another step towards making StoreTaskMeta schedule-aware to
enable creating manually requested runs.
2018-08-15 16:15:36 -07:00
Mark Rushakoff 769791d366 refactor(task): add FinishRun method to StoreTaskMeta
This logic was repeated in multiple places, so now it can live in a
single place. This also sets a precedent going forward, for adding
methods to the StoreTaskMeta type. We'll likely be adding more methods
to support manual runs of a task.
2018-08-15 09:45:25 -07:00
Mark Rushakoff a5371de0d0 fix(task): undo accidental type rename 2018-08-14 14:34:20 -07:00
Jade McGough 70ccb78c5d chore(task): update run schema 2018-08-14 13:33:05 -07:00
Mark Rushakoff 0759cd28e6 feat(task): allow specifying when task scheduling should start
More specifically, introduce a `scheduleAfter` argument to
Store.CreateTask. The previous behavior was leaving a new task's meta
LatestComplete value set to zero, which meant that the first run of a
schedule would start from 1970. Now, it's set to time.Now unless
otherwise specified.

Updates #595.
2018-08-13 10:47:08 -07:00
docmerlin (j. Emrys Landivar) 5b52f75f7e task name modification and uniqueness 2018-08-10 18:24:41 -05:00
Mark Rushakoff 42ce671724 chore: only set one run_id field on runner logs
We were previously doing `r.logger = r.logger.With(...)`, which ended up
duplicating the provided field in the log output. Now ensure that we
only set the run_id once in a logger.
2018-08-09 13:43:00 -07:00
Nathaniel Cook 5bde0b5be6 fix: Update query services to use Request type
Moves idpe.QueryService into platform/query.ProxyQueryService
Splits the Request into ProxyRequest and Request.

Changes query.QueryService and query.AsyncQueryService to use a Request
type. This means that the Compiler interface is consumed by the service
to abstract out transpilation vs Flux compilation vs raw spec.

The transpiler handler is removed.

There are separate http handlers and service implementations for each of
the three query services.

Query logging types are moved into platform.

The ResultIterator now expects Cancel to always be called.

The fluxd binary exposes the query endpoint specified in the swagger
file.
2018-08-08 15:31:35 -06:00
Lyon Hill e1ecceaca0 refactor(task): make setting now on every execution
By switching to passing a spec it allows us more control
2018-08-08 12:50:24 -06:00
Lyon Hill 3873ac0c65
Merge pull request #578 from influxdata/task/connect
feat(task): connect the api to tasks
2018-08-07 16:57:38 -06:00
Lyon Hill 212210eaec make necessary changes 2018-08-07 16:51:33 -06:00
Lyon Hill 02d1b6507b feat(task): connect the api to tasks
We need a coordination layer to facilitate the communication in the api to the task service.
The api was also not connected in main to the http service.
2018-08-07 15:53:17 -06:00
Mark Rushakoff 24ea772ee4 test(task): cache parsed options in test
This is to work around slow parsing (#484).

This change saves about 2 seconds on tests for task/backend and
task/backend/bolt. It saves around 45 seconds on enterprise tests.
2018-08-07 12:46:50 -07:00
Lyon Hill a647f10c14
Merge pull request #560 from influxdata/task/enable
feat(task): allow tasks to be enabled and disabled.
2018-08-07 09:26:45 -06:00
Lyon Hill 63b9adf86c feat(task): allow tasks to be enabled and disabled. 2018-08-06 15:24:34 -06:00
Mark Rushakoff 4745f27bef fix(task/backend): return useful error when validating empty script
After we started parsing options inside of the task validation methods,
we accidentally shadowed the "missing script" error when the script was
empty. This restores the behavior of telling the user "missing script"
rather than a cryptic syntax error.
2018-08-03 17:13:15 -07:00
Lyon Hill 7f6d51f45d add in comments 2018-08-03 14:27:07 -06:00
Lyon Hill 80fe53f2b6 refactor(task): Move task meta into the backend
Task meta belongs with the rest of the task backend content.
2018-08-03 14:27:07 -06:00
Lyon Hill 0661d05b1f update mock scheduler 2018-08-01 13:36:47 -06:00
Lyon Hill 3ba8784fed fix bad commit with merge issue 2018-08-01 13:36:47 -06:00
Lyon Hill 6bba50a2a1 minor cleanup 2018-08-01 13:32:54 -06:00
Lyon Hill 69761a98f7 feat(task): update the scheduler and logwriter interface
Preperatory change that should enable us to build more complex log writers.
2018-08-01 13:32:54 -06:00
Michael Desa 8c87c9d132 revert #442 2018-08-01 14:54:32 -04:00
Leonardo Di Donato bdaee706d6 Fixing last changes obtained from master to make them use uint64 IDs
Signed-off-by: Leonardo Di Donato <leodidonato@gmail.com>
2018-08-01 18:20:59 +02:00
Leonardo Di Donato 5addb88eb5 MustIDFromString no more requires test instance
It panics now.
2018-08-01 18:20:59 +02:00
Leonardo Di Donato 02a80a0665 Refactoring bolt usage of IDs (tasks)
It should be no more necessary to perform operations directly on ID bytes (pad vs unpad).
2018-08-01 18:20:59 +02:00
Leonardo Di Donato b67d3123e2 Refactoring executor tests' usage of IDs (tasks) 2018-08-01 18:20:59 +02:00
Leonardo Di Donato d2e764200e Refactoring task backend usage of IDs (tasks) 2018-08-01 18:20:59 +02:00
Leonardo Di Donato 14da8b1319 Refactoring storetest package-s usage of IDs (tasks) 2018-08-01 18:20:59 +02:00
Leonardo Di Donato 3651350cc5 Refactoring mock usage of IDs (task package) 2018-08-01 18:20:59 +02:00
Leonardo Di Donato a688eb84b3 Starting refactoring task package usage of IDs 2018-08-01 18:20:59 +02:00
docmerlin (j. Emrys Landivar) d61abc0721 Delete users and orgs 2018-07-31 11:57:09 -05:00
Lyon Hill 67c47aba33 minor fixes 2018-07-31 10:23:35 -06:00
Lyon Hill 84d36e6987 feat(tasks): add get meta to the store interface. 2018-07-31 09:47:12 -06:00
Mark Rushakoff 090d337bf0 feat(task): migrate task code from enterprise
Co-authored-by: j. Emrys Landivar <landivar@gmail.com>
Co-authored-by: Lyon Hill <lyondhill@gmail.com>
2018-07-26 15:16:56 -07:00