Commit Graph

282 Commits (d2bd4b76d99028c8e68928bafb6488e19547cd62)

Author SHA1 Message Date
Kelvin Wang 9d3b5932ba fix(influx): return the error when token file already exists 2019-03-07 20:10:48 -05:00
Jacob Marble 603a1f26e0 use tracing.StartSpanFromContext 2019-03-07 12:12:31 -07:00
Jacob Marble 9541e861a3 goimports -w -local github.com/influxdata/influxdb 2019-03-07 12:12:31 -07:00
Jacob Marble 92fa813c45 add spans to multiple services 2019-03-07 12:12:31 -07:00
Chris Goller 0d74ea8d55 feat(influxd/launcher): add test mimicing chronograf setup 2019-03-07 11:07:02 -06:00
Edd Robinson 8bdf857ddb Fix expected flux cases 2019-03-07 09:56:07 +00:00
Edd Robinson 3f1bec0836 Update emitted keys and tests 2019-03-07 09:56:07 +00:00
Michael Desa 2687b7fd61
feat(influxdb): add support for templates
feat(influxdb): add generic store for documents

feat(influxdb): support authorizations in document store

feat(influxdb): support orgs in user resource mapping

feat(influxdb): add read-only included field on documents

feat(influxdb): add labels support to documents service

fix(influxdb): rename data field to content on documents

feat(influxdb): add with org id options for document store

feat(http): add templates swagger

feat(influxdb): add documentation to document options

doc(kv): add documentation for kv document store

test(kv): pull document tests in to the testing package

fix(http): fix swagger specification of templates endpoints
2019-03-06 13:18:31 -08:00
Nathaniel Cook b129026334
Merge pull request #12371 from influxdata/flux-staging
upgrade Flux to v0.21.3
2019-03-06 12:17:12 -07:00
Kelvin Wang e93ea03d38 fix(setup): create dir when influxdb ran as a different user 2019-03-06 09:43:36 -05:00
Nathaniel Cook a190ddb44c fix(http): fix tests for encoding changes 2019-03-05 16:56:35 -07:00
Lyon Hill 17026c2053
change task logs from a string to a struct (#12363)
* change task logs from a string to a struct
2019-03-05 15:58:33 -07:00
Jacob Marble b9c7ec439e
feat(influxd): Tracing refactor (#12318)
* feat(launcher): Tracing to log disabled by default

* remove traceLogger and use opentracing directly

* add Jaeger tracing

* go vet && go fmt
2019-03-04 11:48:11 -08:00
Jeff Wendling 2544a90cac buildtsi: support specifying a specific index path 2019-02-28 15:20:33 -07:00
Lyon Hill b3f450eca9
Allow the CLI to find tasks by org name (#12202)
* Allow the CLI to find tasks by org name
2019-02-27 08:56:32 -07:00
Nathaniel Cook a75adf6c4b fix(http): separate InfluxQL and Flux services
Previously the APIBackend understood only a ProxyQueryService,
but it needs to understand that there are two implementations of the
ProxyQueryService one for handling InfluxQL queries and one for handling
Flux queries. The names of the fields have been updated to make this
clear. As well as the FluxBackend is now initialized using the
FluxService explicitly.
2019-02-22 10:31:20 -07:00
Lyon Hill c78477314a
Allow tasks to skip catchup (#12068) 2019-02-22 09:47:04 -07:00
Julius Volz e7a27243be Check error when writing token file during setup
Signed-off-by: Julius Volz <julius.volz@gmail.com>
2019-02-22 09:08:41 -07:00
Chris Goller c0266ba5cf refactor(cmd/influxd): use simple query rather than the flux service 2019-02-21 11:18:08 -08:00
Chris Goller 1ec7b69d85 fix(kv): error org search if neither name nor id is set 2019-02-21 11:18:08 -08:00
Stuart Carnie 2872233252 feedback: Add end-to-end write and read tests for multiple organizations 2019-02-21 11:18:08 -08:00
Chris Goller df95564c07
Merge pull request #12052 from zhulongcheng/show-auth-in-task-cmd
feat(cmd/influx): show AuthorizationID of tasks
2019-02-21 10:28:25 -06:00
zhulongcheng 126d3edef7 feat(cmd/influx): show AuthorizationID of task 2019-02-22 00:07:45 +08:00
Chris Goller bfcc753836 fix(cmd/influx): remove unsigned for equality check 2019-02-21 09:13:05 -06:00
Chris Goller 79e0a2ae90 feat(cmd/influx): allow infinite retention in non-interactive setup 2019-02-21 08:51:37 -06:00
Lyon Hill 3665feef38
Populate organization name on task create action (#12004)
* Populate organization name on task create action

* Move the responsiblity of organization lookup to the platform adaptor
2019-02-20 13:36:35 -07:00
Chris Goller 40525fcfe8
Merge pull request #11934 from Zyqsempai/check-setup-password-length
Added password length check
2019-02-19 18:47:12 -06:00
Michael Desa 6a8a1fce32 feat(kv): implemented key/value store with end-to-end integration tests
* feat(kv:inmem:bolt): implement user service in a kv

* refactor(kv): use consistent func receiver name

* feat(kv): add initial basic auth service

* refactor(passwords): move auth interface into own file

* refactor(passwords): rename basic auth files to passwords

* refactor(passwords): rename from BasicAuth to Passwords

* refactor(kv): copy bolt user test into kv

Co-authored-by: Michael Desa <mjdesa@gmail.com>

* feat(kv): add inmem testing to kv store

* fix(kv): remove extra user index initialization

* feat(kv): attempt at making errors nice

* fix(http): return not found error if filter is invalid

* fix(http): s/platform/influxdb/ for user service

* fix(http): s/platform/influxdb/ for user service

* feat(kv): initial port of telegraf configs to kv

* feat(kv): first pass at migrating bolt org service to kv

* feat(kv): first pass at bucket service

* feat(kv): first pass at migrating kvlog to kv package

* feat(kv): add resource op logs

* feat(kv): first pass at user resource mapping migration

* feat(kv): add urm usage to bucket and org services

* feat(kv): first pass at kv authz service

* feat(kv): add cascading auth delete for users

* feat(kv): first pass d authorizer.OrganizationService in kv

* feat(cmd/influxd/launcher): user kv services where appropriate

* fix(kv): initialize authorizations

* fix(influxdb): use same buckets while slowly migrating stuff

* fix(kv): make staticcheck pass

* feat(kv): add dashboards to kv

review: make suggestions from pr review

fix: use common bucket names for bolt/kv stores

* test(kv): add complete password test coverage

* chore(kv): fixes for staticcheck

* feat(kv): implement labels generically on kv

* feat(kv): implement macro service

* feat(kv): add source service

* feat(kv): add session service

* feat(kv): add kv secret service

* refactor(kv): update telegraf and urm with error messages

* feat(kv): add lookup service

* feat(kv): add kv onboarding service

* refactor(kv): update telegraf to avoid repetition

* feat(cmd/influxd): use kv lookup service

* feat(kv): add telegraf to lookup service

* feat(cmd/influxd): use kv telegraf service

* feat(kv): initial port of scrapers in bolt to kv

* feat(kv): update scraper error messaging

* feat(cmd/influxd): add kv scraper

* feat(kv): add inmem backend tests

* refactor(kv): copy paste errors

* refactor(kv): add code to password errors

* fix(testing): update error messages for incorrect passwords

* feat(kv:inmem:bolt): implement user service in a kv

* refactor(kv): use consistent func receiver name

* refactor(kv): copy bolt user test into kv

Co-authored-by: Michael Desa <mjdesa@gmail.com>

* feat(kv): add inmem testing to kv store

* fix(kv): remove extra user index initialization

* feat(kv): attempt at making errors nice

* fix(http): return not found error if filter is invalid

* fix(http): s/platform/influxdb/ for user service

* feat(kv): first pass at migrating bolt org service to kv

* feat(kv): first pass at bucket service

* feat(kv): first pass at migrating kvlog to kv package

* feat(kv): add resource op logs

* feat(kv): first pass at user resource mapping migration

* feat(kv): add urm usage to bucket and org services

* feat(kv): first pass at kv authz service

* feat(kv): add cascading auth delete for users

* feat(kv): first pass d authorizer.OrganizationService in kv

* feat(cmd/influxd/launcher): user kv services where appropriate

* feat(kv): add initial basic auth service

* refactor(passwords): move auth interface into own file

* refactor(passwords): rename basic auth files to passwords

* fix(http): s/platform/influxdb/ for user service

* fix(kv): initialize authorizations

* fix(influxdb): use same buckets while slowly migrating stuff

* fix(kv): make staticcheck pass

* feat(kv): add dashboards to kv

review: make suggestions from pr review

fix: use common bucket names for bolt/kv stores

* feat(kv): implement labels generically on kv

* refactor(passwords): rename from BasicAuth to Passwords

* test(kv): add complete password test coverage

* chore(kv): fixes for staticcheck

* feat(kv): implement macro service

* feat(kv): add source service

* feat(kv): add session service

* feat(kv): initial port of telegraf configs to kv

* feat(kv): initial port of scrapers in bolt to kv

* feat(kv): add kv secret service

* refactor(kv): update telegraf and urm with error messages

* feat(kv): add lookup service

* feat(kv): add kv onboarding service

* refactor(kv): update telegraf to avoid repetition

* feat(cmd/influxd): use kv lookup service

* feat(kv): add telegraf to lookup service

* feat(cmd/influxd): use kv telegraf service

* feat(kv): update scraper error messaging

* feat(cmd/influxd): add kv scraper

* feat(kv): add inmem backend tests

* refactor(kv): copy paste errors

* refactor(kv): add code to password errors

* fix(testing): update error messages for incorrect passwords

* feat(http): initial support for flushing all key/values from kv store

* feat(kv): rename macro to variable

* feat(cmd/influxd/launcher): user kv services where appropriate

* refactor(passwords): rename from BasicAuth to Passwords

* feat(kv): implement macro service

* test(ui): introduce cypress

* test(ui): introduce first typescript test

* test(ui/e2e): add ci job

* chore: update gitignore to ignore test outputs

* feat(inmem): in memory influxdb

* test(e2e): adding pinger that checks if influxdb is alive

* hackathon

* hack

* hack

* hack

* hack

* Revert "feat(inmem): in memory influxdb"

This reverts commit 30ddf032003e704643b07ce80df61c3299ea7295.

* hack

* hack

* hack

* hack

* hack

* hack

* hack

* hack

* hack

* hack

* hack

* hack

* hack

* chore: lint ignore node_modules

* hack

* hack

* hack

* add user and flush

* hack

* remove unused vars

* hack

* hack

* ci(circle): prefix e2e artifacts

* change test to testid

* update cypress

* moar testid

* fix npm warnings

* remove absolte path

* chore(ci): remove /home/circleci proto mkdir hack

* wip: crud resources e2e

* fix(inmem): use inmem kv store services

* test(dashboard): add first dashboard crud tests

* hack

* undo hack

* fix: use response from setup for orgID

* chore: wip

* add convenience getByTitle function

* test(e2e): ui can create orgs

* test(e2e): add test for org deletion and update

* test(e2e): introduce task creation test

* test(e2e): create and update of buckets on org view

* chore: move types to declaration file

* chore: use route fixture in dashboard tests

* chore(ci): hack back

* test(ui): update snapshots

* chore: package-lock

* chore: remove macros

* fix: launcher rebase issues

* fix: compile errors

* fix: compile errors

* feat(cmd/influxdb): add explicit testing, asset-path, and store flags

Co-authored-by: Andrew Watkins <watts@influxdb.com>

* fix(cmd/influxd): set default HTTP handler and flags

Co-authored-by: Andrew Watkins <watts@influxdb.com>

* build(Makefile): add run-e2e and PHONY

* feat(kv:inmem:bolt): implement user service in a kv

* refactor(kv): use consistent func receiver name

* feat(kv): add initial basic auth service

* refactor(passwords): move auth interface into own file

* refactor(passwords): rename basic auth files to passwords

* refactor(passwords): rename from BasicAuth to Passwords

* refactor(kv): copy bolt user test into kv

Co-authored-by: Michael Desa <mjdesa@gmail.com>

* feat(kv): add inmem testing to kv store

* fix(kv): remove extra user index initialization

* feat(kv): attempt at making errors nice

* fix(http): return not found error if filter is invalid

* fix(http): s/platform/influxdb/ for user service

* fix(http): s/platform/influxdb/ for user service

* feat(kv): initial port of telegraf configs to kv

* feat(kv): initial port of scrapers in bolt to kv

* feat(kv): first pass at migrating bolt org service to kv

* feat(kv): first pass at bucket service

* feat(kv): first pass at migrating kvlog to kv package

* feat(kv): add resource op logs

* feat(kv): first pass at user resource mapping migration

* feat(kv): add urm usage to bucket and org services

* feat(kv): first pass at kv authz service

* feat(kv): add cascading auth delete for users

* feat(kv): first pass d authorizer.OrganizationService in kv

* feat(cmd/influxd/launcher): user kv services where appropriate

* fix(kv): initialize authorizations

* fix(influxdb): use same buckets while slowly migrating stuff

* fix(kv): make staticcheck pass

* feat(kv): add dashboards to kv

review: make suggestions from pr review

fix: use common bucket names for bolt/kv stores

* test(kv): add complete password test coverage

* chore(kv): fixes for staticcheck

* feat(kv): implement labels generically on kv

* feat(kv): implement macro service

* feat(kv): add source service

* feat(kv): add session service

* feat(kv): add kv secret service

* refactor(kv): update telegraf and urm with error messages

* feat(kv): add lookup service

* feat(kv): add kv onboarding service

* refactor(kv): update telegraf to avoid repetition

* feat(cmd/influxd): use kv lookup service

* feat(kv): add telegraf to lookup service

* feat(cmd/influxd): use kv telegraf service

* feat(kv): update scraper error messaging

* feat(cmd/influxd): add kv scraper

* feat(kv): add inmem backend tests

* refactor(kv): copy paste errors

* refactor(kv): add code to password errors

* fix(testing): update error messages for incorrect passwords

* feat(kv): rename macro to variable

* refactor(kv): auth/bucket/org/user unique checks return errors now

* feat(inmem): add way to get all bucket names from store

* feat(inmem): Buckets to return slice of bytes rather than strings

* feat(inmem): add locks around Buckets to avoid races

* feat(cmd/influx): check for unauthorized error in wrapCheckSetup

* chore(e2e): add video and screenshot artifcats to gitignore

* docs(ci): add build instructions for e2e tests

* feat(kv): add id lookup for authorized resources
2019-02-19 17:47:19 -06:00
Mark Rushakoff cbef811c63 refactor(task): remove explicit org from run/log lookup
Task ID is now a required value on run and log filters. It was
effectively required by all implementations before anyway, so now those
types reflect that requirement.

Organization ID was removed from those same fields. The TaskService
looks up the organization ID via the task in cases where we need it at a
lower layer.
2019-02-16 14:34:49 -08:00
Lyon Hill 48cc643992
Clean out user from the task store's (#11882)
* Clean out user from the task store's
2019-02-15 15:50:09 -07:00
bpopovschi bd0db5d0f9 Added password length check 2019-02-15 18:07:18 +02:00
Mark Rushakoff 954cea93bc fix(cmd/influx): don't set authorizer context when creating task
Setting the authorizer from the HTTP client wasn't necessary, and
looking up the token to do so didn't work as written.

Fixes #11903.
2019-02-14 17:27:31 -08:00
Alirie Gray 5f524eb92d Rename all occurences of Macro to Variable 2019-02-14 13:21:57 -08:00
Mark Rushakoff d562d6bdde feat(task): pass authorizer to query service
Immediately before the executor calls out to the query service, the
executor loads the authorizer associated with the task, and associates
that authorizer with the context used to execute the query.
2019-02-14 08:53:40 -08:00
Mark Rushakoff caf08b5078 feat(task): store authorization ID with task
Accept token when creating or updating a task, but only report back the
authorization ID.

This means the executor and the platform adapter are now both aware of
an Authorization Service.
2019-02-14 08:53:40 -08:00
Mark Rushakoff 007f5059c7 refactor(task): extract TaskCreate struct
With the ongoing authorization work, creation arguments will differ from
what's returned on reads. More specifically, creation will accept a
token, but reads will report back a token ID.

This refactor facilitates that authorization work, and also brings the
code closer to the swagger definition, for the TaskCreateRequest type in
particular.
2019-02-12 15:10:03 -08:00
Michael Desa 386c70ea0d fix(influxdb): revert functionality added in pr#10947 2019-02-06 15:54:09 -05:00
Michael Desa f79075deda fix(cmd/influxd/launcher): use http clients for testing launcher 2019-02-06 14:24:21 -05:00
Michael Desa d96787dd67 fix(cmd/influxd/launcher): remove platform alias for influxdb 2019-02-06 14:24:21 -05:00
Jeff Wendling c9bb55b889 refactor: move the tsm1/wal into the storage/wal package
Because the WAL relies on the tsm1.Value type, we move that into its own
tsm1/value package and set up some aliases forwarding them into tsm1. This
also required adding some methods and changing consumers to avoid the
unexported fields. I imagine this step will be useful one day when we make
the write path more efficient with respect to consuming points.

This commit additionally fixes some issues with generation. The iterator.tmpldata
and generation for array_cursor_* were removed accidentally when removing
iterators, making those generated files stale. Restore that and regenerate.

No change in functionality.
2019-02-04 10:32:52 -07:00
docmerlin (j. Emrys Landivar) 0fec7aa6c4 feat(tasks): end-to-end tests for the task system 2019-01-25 13:28:25 -06:00
Jacob Marble 296e119783
feat: Add influx ping subcommand (#11603)
* feat: Add influx ping subcommand
2019-01-25 10:01:55 -08:00
Mark Rushakoff 84f8f8d88c chore(task): move 'task retry' command to 'task run retry'
Runs are retried; tasks aren't.

Closes #11170.
2019-01-23 16:34:10 -08:00
Chris Goller bb09a74732 fix(gather): add all subscribers to a single nats queue group
Previously, scrapers would scrape the target 10 times.  This was
because each scraper subscriber was not put into a queue group.

I've added the queue group "metrics" and now we the subcribers
will only scrape the target once.

Additionally, I moved to using nats memory instead of nats file
store.  We don't need durability for scraper runs across restarts.
2019-01-22 23:05:24 -06:00
Chris Goller e2a1e25707 feat(cmd/influxd): log build information 2019-01-22 21:39:40 -06:00
Michael Desa d2a2338c8d fix(cmd/influx): add context to error messages in bucket commands 2019-01-22 17:10:43 -08:00
Michael Desa 057e99befc fix(cmd/influx): add contextual information to org command errors 2019-01-22 15:40:54 -08:00
Michael Desa b68fff97f5 fix(cmd/influx): remove bolt op prefix from http org/bucket service 2019-01-22 15:40:27 -08:00
Jacob Marble 62ebb2c9ad feat: Onboarding without interaction 2019-01-22 14:36:34 -08:00
Jade McGough fefea5566c fix(main): require either org or org-id when creating buckets 2019-01-22 14:30:27 -08:00