* Update task servicetest to move dependency to the new TaskControlService
closes#12724
We will now have the capability to write new task services that dont have to implement the backend.Store or LogReader or LogWriters
In a development build (i.e. does not have the assets build tag), the
first request to GET /api/v2/swagger.json will produce log output like:
INFLUXDB_VALID_SWAGGER_PATH not set; falling back to checking relative paths {"log_id": "0E4I7Dkl000", "service": "swagger-loader"}
Successfully loaded swagger.yml {"log_id": "0E4I7Dkl000", "service": "swagger-loader", "path": "/.../influxdb/http/swagger.yml"}
There is no such log line in production builds.
But now both builds correctly serve swagger.json, instead of just
production builds.
* feat: add telegraf configs to the config page
* feat: add tokens to telegraf configs
* feat(perms): display proper permissions to user
* feat: add tokens to redux
* wip: add token to auths
* hack: make server return labels and links
* wip: create a label for telegraf config
* fix(http/telegraf): JSON marshaling using pointer receiver
* chore: add back whitespace
* chore: add back whitespace
* add telegraf token to popup
* feat(token/tele): remove token when config gets deleted
* test: sadness
* change to streaming
* unskip test
Co-Authored-By: Kelvin Wang <sherkrainwang@gmail.com>
Co-Authored-By: Michael Desa <mjdesa@gmail.com>
Co-Authored-By: Christopher M. Wolff <chris.wolff@influxdata.com>
feat(influxdb): add generic store for documents
feat(influxdb): support authorizations in document store
feat(influxdb): support orgs in user resource mapping
feat(influxdb): add read-only included field on documents
feat(influxdb): add labels support to documents service
fix(influxdb): rename data field to content on documents
feat(influxdb): add with org id options for document store
feat(http): add templates swagger
feat(influxdb): add documentation to document options
doc(kv): add documentation for kv document store
test(kv): pull document tests in to the testing package
fix(http): fix swagger specification of templates endpoints
Co-authored-by: Mark Rushakoff <mark@influxdata.com>
Tasks instantiates the query service but does not know the token
it will use ahead of time. This allows the token to be optionally
set on contex.
The previous "." argument with "-ignore go" accidentally matched
README.md and Makefile too, neither of which we care to actually include
as assets.
Now we just explicitly include swagger.yml. It's simple enough to add
another file to the go:generate line if and when we need it.
In the platform adapter, we ask the URM for a list of tasks the user
owns, and then we look up each task individually.
The task service tests uncovered a legitimate bug where FindTasks would
return a "task not found" error, originating from looking up a task that
was present when we interrogated the URM but was deleted before we could
find it in the task store.
This change also removes duplicated URM logic from the HTTP handler
which has since been pushed down into the platform adapter.
The query system specifically expects an Authorization. When a request
comes in using a Session, use the target task's Authorization, if we are
allowed to read it, when executing a query against the system bucket.
Previously the APIBackend understood only a ProxyQueryService,
but it needs to understand that there are two implementations of the
ProxyQueryService one for handling InfluxQL queries and one for handling
Flux queries. The names of the fields have been updated to make this
clear. As well as the FluxBackend is now initialized using the
FluxService explicitly.
Also set the generated token's description while we're here.
This enables us to use task's Authorization when we need to query the
system bucket to get run logs, etc. but we only have a Session.
The YAML parser used by the go-openapi libraries treats an unquoted y as
a boolean key, which will lead to a difficult-to-understand parser
error:
types don't match expect map key string or int got: bool
See also https://yaml.org/type/bool.html.
* feat(kv:inmem:bolt): implement user service in a kv
* refactor(kv): use consistent func receiver name
* feat(kv): add initial basic auth service
* refactor(passwords): move auth interface into own file
* refactor(passwords): rename basic auth files to passwords
* refactor(passwords): rename from BasicAuth to Passwords
* refactor(kv): copy bolt user test into kv
Co-authored-by: Michael Desa <mjdesa@gmail.com>
* feat(kv): add inmem testing to kv store
* fix(kv): remove extra user index initialization
* feat(kv): attempt at making errors nice
* fix(http): return not found error if filter is invalid
* fix(http): s/platform/influxdb/ for user service
* fix(http): s/platform/influxdb/ for user service
* feat(kv): initial port of telegraf configs to kv
* feat(kv): first pass at migrating bolt org service to kv
* feat(kv): first pass at bucket service
* feat(kv): first pass at migrating kvlog to kv package
* feat(kv): add resource op logs
* feat(kv): first pass at user resource mapping migration
* feat(kv): add urm usage to bucket and org services
* feat(kv): first pass at kv authz service
* feat(kv): add cascading auth delete for users
* feat(kv): first pass d authorizer.OrganizationService in kv
* feat(cmd/influxd/launcher): user kv services where appropriate
* fix(kv): initialize authorizations
* fix(influxdb): use same buckets while slowly migrating stuff
* fix(kv): make staticcheck pass
* feat(kv): add dashboards to kv
review: make suggestions from pr review
fix: use common bucket names for bolt/kv stores
* test(kv): add complete password test coverage
* chore(kv): fixes for staticcheck
* feat(kv): implement labels generically on kv
* feat(kv): implement macro service
* feat(kv): add source service
* feat(kv): add session service
* feat(kv): add kv secret service
* refactor(kv): update telegraf and urm with error messages
* feat(kv): add lookup service
* feat(kv): add kv onboarding service
* refactor(kv): update telegraf to avoid repetition
* feat(cmd/influxd): use kv lookup service
* feat(kv): add telegraf to lookup service
* feat(cmd/influxd): use kv telegraf service
* feat(kv): initial port of scrapers in bolt to kv
* feat(kv): update scraper error messaging
* feat(cmd/influxd): add kv scraper
* feat(kv): add inmem backend tests
* refactor(kv): copy paste errors
* refactor(kv): add code to password errors
* fix(testing): update error messages for incorrect passwords
* feat(kv:inmem:bolt): implement user service in a kv
* refactor(kv): use consistent func receiver name
* refactor(kv): copy bolt user test into kv
Co-authored-by: Michael Desa <mjdesa@gmail.com>
* feat(kv): add inmem testing to kv store
* fix(kv): remove extra user index initialization
* feat(kv): attempt at making errors nice
* fix(http): return not found error if filter is invalid
* fix(http): s/platform/influxdb/ for user service
* feat(kv): first pass at migrating bolt org service to kv
* feat(kv): first pass at bucket service
* feat(kv): first pass at migrating kvlog to kv package
* feat(kv): add resource op logs
* feat(kv): first pass at user resource mapping migration
* feat(kv): add urm usage to bucket and org services
* feat(kv): first pass at kv authz service
* feat(kv): add cascading auth delete for users
* feat(kv): first pass d authorizer.OrganizationService in kv
* feat(cmd/influxd/launcher): user kv services where appropriate
* feat(kv): add initial basic auth service
* refactor(passwords): move auth interface into own file
* refactor(passwords): rename basic auth files to passwords
* fix(http): s/platform/influxdb/ for user service
* fix(kv): initialize authorizations
* fix(influxdb): use same buckets while slowly migrating stuff
* fix(kv): make staticcheck pass
* feat(kv): add dashboards to kv
review: make suggestions from pr review
fix: use common bucket names for bolt/kv stores
* feat(kv): implement labels generically on kv
* refactor(passwords): rename from BasicAuth to Passwords
* test(kv): add complete password test coverage
* chore(kv): fixes for staticcheck
* feat(kv): implement macro service
* feat(kv): add source service
* feat(kv): add session service
* feat(kv): initial port of telegraf configs to kv
* feat(kv): initial port of scrapers in bolt to kv
* feat(kv): add kv secret service
* refactor(kv): update telegraf and urm with error messages
* feat(kv): add lookup service
* feat(kv): add kv onboarding service
* refactor(kv): update telegraf to avoid repetition
* feat(cmd/influxd): use kv lookup service
* feat(kv): add telegraf to lookup service
* feat(cmd/influxd): use kv telegraf service
* feat(kv): update scraper error messaging
* feat(cmd/influxd): add kv scraper
* feat(kv): add inmem backend tests
* refactor(kv): copy paste errors
* refactor(kv): add code to password errors
* fix(testing): update error messages for incorrect passwords
* feat(http): initial support for flushing all key/values from kv store
* feat(kv): rename macro to variable
* feat(cmd/influxd/launcher): user kv services where appropriate
* refactor(passwords): rename from BasicAuth to Passwords
* feat(kv): implement macro service
* test(ui): introduce cypress
* test(ui): introduce first typescript test
* test(ui/e2e): add ci job
* chore: update gitignore to ignore test outputs
* feat(inmem): in memory influxdb
* test(e2e): adding pinger that checks if influxdb is alive
* hackathon
* hack
* hack
* hack
* hack
* Revert "feat(inmem): in memory influxdb"
This reverts commit 30ddf032003e704643b07ce80df61c3299ea7295.
* hack
* hack
* hack
* hack
* hack
* hack
* hack
* hack
* hack
* hack
* hack
* hack
* hack
* chore: lint ignore node_modules
* hack
* hack
* hack
* add user and flush
* hack
* remove unused vars
* hack
* hack
* ci(circle): prefix e2e artifacts
* change test to testid
* update cypress
* moar testid
* fix npm warnings
* remove absolte path
* chore(ci): remove /home/circleci proto mkdir hack
* wip: crud resources e2e
* fix(inmem): use inmem kv store services
* test(dashboard): add first dashboard crud tests
* hack
* undo hack
* fix: use response from setup for orgID
* chore: wip
* add convenience getByTitle function
* test(e2e): ui can create orgs
* test(e2e): add test for org deletion and update
* test(e2e): introduce task creation test
* test(e2e): create and update of buckets on org view
* chore: move types to declaration file
* chore: use route fixture in dashboard tests
* chore(ci): hack back
* test(ui): update snapshots
* chore: package-lock
* chore: remove macros
* fix: launcher rebase issues
* fix: compile errors
* fix: compile errors
* feat(cmd/influxdb): add explicit testing, asset-path, and store flags
Co-authored-by: Andrew Watkins <watts@influxdb.com>
* fix(cmd/influxd): set default HTTP handler and flags
Co-authored-by: Andrew Watkins <watts@influxdb.com>
* build(Makefile): add run-e2e and PHONY
* feat(kv:inmem:bolt): implement user service in a kv
* refactor(kv): use consistent func receiver name
* feat(kv): add initial basic auth service
* refactor(passwords): move auth interface into own file
* refactor(passwords): rename basic auth files to passwords
* refactor(passwords): rename from BasicAuth to Passwords
* refactor(kv): copy bolt user test into kv
Co-authored-by: Michael Desa <mjdesa@gmail.com>
* feat(kv): add inmem testing to kv store
* fix(kv): remove extra user index initialization
* feat(kv): attempt at making errors nice
* fix(http): return not found error if filter is invalid
* fix(http): s/platform/influxdb/ for user service
* fix(http): s/platform/influxdb/ for user service
* feat(kv): initial port of telegraf configs to kv
* feat(kv): initial port of scrapers in bolt to kv
* feat(kv): first pass at migrating bolt org service to kv
* feat(kv): first pass at bucket service
* feat(kv): first pass at migrating kvlog to kv package
* feat(kv): add resource op logs
* feat(kv): first pass at user resource mapping migration
* feat(kv): add urm usage to bucket and org services
* feat(kv): first pass at kv authz service
* feat(kv): add cascading auth delete for users
* feat(kv): first pass d authorizer.OrganizationService in kv
* feat(cmd/influxd/launcher): user kv services where appropriate
* fix(kv): initialize authorizations
* fix(influxdb): use same buckets while slowly migrating stuff
* fix(kv): make staticcheck pass
* feat(kv): add dashboards to kv
review: make suggestions from pr review
fix: use common bucket names for bolt/kv stores
* test(kv): add complete password test coverage
* chore(kv): fixes for staticcheck
* feat(kv): implement labels generically on kv
* feat(kv): implement macro service
* feat(kv): add source service
* feat(kv): add session service
* feat(kv): add kv secret service
* refactor(kv): update telegraf and urm with error messages
* feat(kv): add lookup service
* feat(kv): add kv onboarding service
* refactor(kv): update telegraf to avoid repetition
* feat(cmd/influxd): use kv lookup service
* feat(kv): add telegraf to lookup service
* feat(cmd/influxd): use kv telegraf service
* feat(kv): update scraper error messaging
* feat(cmd/influxd): add kv scraper
* feat(kv): add inmem backend tests
* refactor(kv): copy paste errors
* refactor(kv): add code to password errors
* fix(testing): update error messages for incorrect passwords
* feat(kv): rename macro to variable
* refactor(kv): auth/bucket/org/user unique checks return errors now
* feat(inmem): add way to get all bucket names from store
* feat(inmem): Buckets to return slice of bytes rather than strings
* feat(inmem): add locks around Buckets to avoid races
* feat(cmd/influx): check for unauthorized error in wrapCheckSetup
* chore(e2e): add video and screenshot artifcats to gitignore
* docs(ci): add build instructions for e2e tests
* feat(kv): add id lookup for authorized resources
Task ID is now a required value on run and log filters. It was
effectively required by all implementations before anyway, so now those
types reflect that requirement.
Organization ID was removed from those same fields. The TaskService
looks up the organization ID via the task in cases where we need it at a
lower layer.
Accept token when creating or updating a task, but only report back the
authorization ID.
This means the executor and the platform adapter are now both aware of
an Authorization Service.
With the ongoing authorization work, creation arguments will differ from
what's returned on reads. More specifically, creation will accept a
token, but reads will report back a token ID.
This refactor facilitates that authorization work, and also brings the
code closer to the swagger definition, for the TaskCreateRequest type in
particular.
For an operation that looks up a task or a run, when that operation
fails, only set the status to 404 if that operation explicitly returns
ErrTaskNotFound or ErrRunNotFound.
It's possible that the operation could fail for a reason other than the
ID being invalid: for example, if there was an IO error preventing the
lookup from succeeding.
Harden that behavior with tests for the task handler.
Closes#11589.
And ensure that the task HTTP client consistently expects influxdb
errors, so that they're properly extracted and returned to the caller.
Closes#11426.
fix(cmd/influx): format error for bare influx command
feat(cmd/influx): add org flag to query command
fix(cmd/influx): check correct error in wrapCheckSetup
fix(influx/cmd): use correct shorthand flag for org flag
feat(cmd/influx): wrapCheckSetup with wrapErrorFmt
feat(cmd/influx): cleanup language and errors during setup command
fix(http): use id query param in get buckets request
feat(cmd/influx): add helpful error messages for write command
feat(cmd/influx): add context to query command
Jade McGough
kelwang
Kelvin Wang
zhulongcheng
Lyon Hill
Nathaniel Cook
Michael Desa
Lorenzo Affetti
Christopher M. Wolff
Jonathan A. Sternberg
Jorge Landivar
Mark Rushakoff
Deniz Kusefoglu
Iris Scholten
Chris Goller
Andrew Watkins
Jacob Marble
Palak Bhojani
Alirie Gray
Jacob Marble
Ben Johnson
Brandon Farmer
Delmer Reed
jlapacik
Adam
Christopher Henn
Jakub Bednar
Leonardo Di Donato