Commit Graph

455 Commits (05c8a00b8df6e66c82fef1a7b141d829caa895c2)

Author SHA1 Message Date
Stuart Carnie ab31e7763b chore: Closing in goroutine causes race with logging framework 2020-09-09 10:32:06 -07:00
Stuart Carnie a607ac30b9
chore: Re-enable `TemporaryEngine` via `--e2e-testing` 2020-08-31 13:50:20 -07:00
Stuart Carnie 9163edb187
Merge remote-tracking branch 'origin/master' into sgc/tsm1 2020-08-26 10:14:42 -07:00
Stuart Carnie 8104677639
revert(tasks): Revert incompatible commits
Revert 1ae2541
Revert fde2129
2020-08-26 09:44:21 -07:00
Stuart Carnie 39ab4a10c5
Merge remote-tracking branch 'origin/master' into sgc/tsm1
# Conflicts:
#	cmd/influxd/launcher/query_test.go
#	go.mod
#	go.sum
#	query/promql/internal/promqltests/go.mod
#	task/backend/executor/executor.go
#	task/backend/executor/support_test.go
2020-08-26 09:26:43 -07:00
Lyon Hill a8fe07bfc7
chore(task): clean up old maxPermission code (#19432)
Tasks no longer uses the max permission system to pupulate an
authorization. We can safely remove this dead code.
2020-08-26 10:26:32 -06:00
Brett Buddin fde2129002
fix(task/executor): Use correct name for last success time extern (#19435)
* fix(task/executor): Use correct name for last success time extern.

* chore(tasks/executor): Don't reuse this constant in the tests. Test the name.
2020-08-25 17:32:36 -04:00
Brett Buddin 1ae2541bf3
feat(task): Inject Task's LatestSuccess Timestamp In Flux Extern (#19402)
* feat(task): Inject latest success/failure into extern.

* chore(task/backend): Don't specify an extern if there are no statements.

* chore(task/executor): Don't apply the latest failure for now.

* chore(changelog): Add 19402 to changelog.

* chore(kit): Introduce feature flag for time injection.

* chore(task/executor): Guard injection into extern by feature flag.

* chore(task/executor): No need for this subtest pattern.

* chore(task/executor): Add tests for extern injection.
2020-08-25 13:06:26 -04:00
Paul Hummer 342a29897e
Merge pull request #19427 from influxdata/flux-staging/v0.82.2
build(flux): update Flux to v0.82.2
2020-08-25 11:03:52 -06:00
Stuart Carnie 0e0b2a3d87
Merge remote-tracking branch 'origin/master' into sgc/tsm1 2020-08-25 09:03:07 -07:00
Lyon Hill f0e382adc2
chore: remove feature flag for task executor (#19422)
We now default task execution to pull from the user service a permission
set.
2020-08-25 08:39:03 -06:00
Paul Hummer df41d42457 build(flux): update Flux to v0.82.2 2020-08-25 08:04:28 -06:00
Stuart Carnie 7c972387e2
chore(task): Flux compatibility fixes
This commit temporarily pulls in code from the flux/ast/edit
package ensure builds continue. Once Flux is upgraded to the
latest, the pkg/flux package will be removed.
2020-08-20 11:25:51 -07:00
Stuart Carnie cc656e5221
Merge remote-tracking branch 'origin/master' into sgc/tsm1
# Conflicts:
#	storage/reads/aggregate_resultset.go
#	storage/reads/aggregate_resultset_test.go
#	storage/reads/array_cursor.gen.go
#	storage/reads/array_cursor.gen.go.tmpl
#	storage/reads/array_cursor.go
#	storage/reads/array_cursor_gen_test.go
#	storage/reads/array_cursor_test.gen.go.tmpl
#	storage/reads/group_resultset.go
#	storage/reads/group_resultset_test.go
#	storage/reads/resultset.go
2020-08-20 10:52:34 -07:00
Brett Buddin 0780232b83
feat(task): Parse Task Options using Flux AST Helpers (#19326)
* feat(task): Extract options using AST-based method.

* feat(task): Use AST-based option APIs for updating task option.

* chore(task): Use the old way of parsing durations.

* fix(task): Ordering changed on us. Fixing tests to reflect the new order.

* fix(task): There's no way for us to know if there are multiples with the current APIs.

* chore(task): Guard against duplicate options. Minor cleanup.

* fix(kit/feature): Break cyclical dependency between influxdb and pkgs that use feature.

* chore(task): Feature flag updating Flux options.

* chore(task): Ensure we are testing both paths of feature flag.

* chore: Remove dead code.

* chore(task/options): Remove unnecessary conditional.

* chore(task/options): Unexport some error helpers.
2020-08-20 11:57:23 -04:00
Stuart Carnie f47bab1f0d
fix(task): Configure meta.Client to allow tests to pass 2020-08-03 15:13:47 -07:00
Stuart Carnie 533b5c4ed6
fix: Build error 2020-08-03 09:20:51 -07:00
Stuart Carnie 26c18ac0a5
chore: Converging on 1.x tsdb 2020-08-03 09:20:48 -07:00
Edd Robinson 2b175291be
refactor: WIP removing tsbd 2020-08-03 09:18:34 -07:00
Stuart Carnie a97ec1f34e
feat: Initial commit with working reads 2020-08-03 09:18:34 -07:00
Stuart Carnie e3060c291c
refactor: tsdb store builds an runs 2020-08-03 09:18:32 -07:00
Lyon Hill 0a9e8fdb4a
chore: update task tests to use the tenant service (#19169)
* chore: update task tests to use the tenant service

After the introduction of the tenant system we need to switch the testing frameworks
to use it instead of the old kv system

* chore: update onboarding to allow injected middleware
2020-08-03 09:33:46 -06:00
Lyon Hill a31b651057
feat(task): Add new permission lookup pattern for executor (#18920)
* feat(task): Add new permission lookup pattern for executor

We can now use the user service to populate task owners permissions.
This should improve the task lookup time and decouple the task system
from the URM system. In the future we will have the ability to better isolate
tenant pieces from the rest of the service.

* feat: add feature flagging
2020-07-13 14:47:58 -06:00
George b8f5604613
revert(tasks): Add new permission lookup pattern for executor (#18869) (#18915)
This reverts commit e9e4d794cf.
2020-07-10 11:05:07 +01:00
Lyon Hill e9e4d794cf
feat(task): Add new permission lookup pattern for executor (#18869)
We can now use the user service to populate task owners permissions.
This should improve the task lookup time and decouple the task system
from the URM system. In the future we will have the ability to better isolate
tenant pieces from the rest of the service.
2020-07-09 14:06:46 -06:00
George 96d84b9126
refactor: migrator and introduce Store.(Create|Delete)Bucket (#18570)
* refactor: migrator and introduce Store.(Create|Delete)Bucket

feat: kvmigration internal utility to create / managing kv store migrations

fix: ensure migrations applied in all test cases

* chore: update kv and migration documentation
2020-07-01 12:08:20 +01:00
Jonathan A. Sternberg 5aeca082c8
chore: update staticcheck and fix newly identified lint checks (#18737) 2020-06-26 18:54:09 -05:00
Johnny Steenbergen 3c14db1c77 fix(influx): support days as duration type
closes: #18639
2020-06-22 20:07:53 -07:00
Brett Buddin 0f4b0fbe68
fix(task/executor): Don't send Now for Flux compiler. See TODO for details. 2020-06-18 15:27:28 -04:00
Lyon Hill 22c573df68
fix: clean up old runs when we fail to enqueue it (#18482)
* fix: clean up old runs when we fail to enqueue it

We need to make sure the runs are removed if we fail to enqueue the run to be worked.
for this we will now add an error message stating why we failed and move the run out of our kv store.

* fix: update fail message
2020-06-12 13:02:31 -06:00
Brett Buddin 9b53315e33
feat(task): Separate System from Non-System query compiler feature flagging. 2020-06-08 15:04:13 -04:00
Jonathan A. Sternberg 0c8d19e896
Merge branch 'master' into feat/use-algo-w 2020-05-18 13:25:26 -05:00
Brett Buddin f59c2ce009
feat(task): Allow switching of compiler from outside Executor. 2020-05-14 14:22:43 -04:00
jlapacik 34e581ca0a Merge branch 'master' into chore/merge-master 2020-05-12 11:18:25 -07:00
Lyon Hill bdc882f6ce
feat(session): Build out a new session service (#17950)
This new session service has the ability to work independant of other systems
it relies on having its own store type which should allow us to be more flexible
then using the built in kv system.

I have included an in mem session store.
2020-05-11 15:04:11 -06:00
jlapacik 7541af8414 chore: merge master into algow 2020-04-15 14:42:03 -07:00
Brett Buddin c08486ba5a
feat(task): Add option to Executor to set maximum worker count. 2020-04-09 09:37:17 -04:00
Jonathan A. Sternberg d3fa60991f
Merge branch 'master' into chore/merge-master 2020-04-06 12:17:47 -05:00
Jonathan A. Sternberg 0ae8bebd75
refactor: rewrite imports to include the /v2 suffix for version 2 2020-04-03 12:39:20 -05:00
George 2840e6f26e
feat(kv): add ability to configure a max permissions using auth filter function (#17598) 2020-04-03 17:02:51 +01:00
Jonathan A. Sternberg 469c5849c4
Merge branch 'master' into feat/use-algo-w 2020-04-03 09:30:04 -05:00
Gavin Cabbage b5c24f3c9d
refactor(tasks): auth user id (#17562) 2020-04-02 09:40:00 -04:00
Brett Buddin 3d826d96cf
feat(influxd): flag to disable scheduling of tasks 2020-03-27 12:52:07 -04:00
Brett Buddin 79cade52a6
fix(task/scheduler): Reuse slices built by iterator to reduce allocations. 2020-03-24 10:14:02 -04:00
Christopher M. Wolff 20c2439230
refactor(http): change *ast.Package to json.RawMessage in query requests (#17316) 2020-03-23 12:39:24 -07:00
Sebastian Borza 42856e1394
chore(storageflux): fix linter issue 2020-03-10 17:54:06 -05:00
Jonathan A. Sternberg 0d6e4e310b
Merge branch 'master' into feat/use-algo-w 2020-03-06 16:46:49 -06:00
Jonathan A. Sternberg 834a8740e0
refactor(task/backend): move the task/backend constants to the global package (#17133)
This moves a few types and constants to the global package so it can be
used without importing the `task/backend` package. These constants are
referenced in non tasks-specific code.

This is needed to break a dependency chain where the task backend will
call into the flux runtime to perform parsing or evaluation of a script
and to prevent the http package from inheriting that dependency.
2020-03-06 16:19:32 -06:00
Jonathan A. Sternberg bcbb9df72e
refactor(task): tasks will now use the flux language service (#17104)
The tasks subsystem will now use the flux language service to parse and
evaluate flux instead of directly interacting with the parser or
runtime. This helps break the dependency on the libflux parser for the
base influxdb package.

This includes the task notification packages which were changed at the
same time.
2020-03-05 14:36:58 -06:00
Jonathan A. Sternberg 935180e9bd
refactor: update flux for the runtime package (#17002) 2020-02-27 11:31:45 -06:00