* feat(task): Allow the most recent run time to be shown in the api.
When showing tasks in the api latest_completed will now show in the api if it has been run.
fixes#1705
* feat(platform): add label struct
* feat(platform): add label service interface
* WIP label service
* chore(http): simplify URMapping validation
* WIP label service skeleton
* add new GET labels handler
* feat(http): add POST handler for label service
* feat(http): add label delete handler
* fix json tag
* add happy path testing
* feat(http): add label endpoints to dashboard service
* make fmt
* add label delete handler
* make fmt again :(
* feat(bolt): add label implementation
* wire boltdb implementation of labels
* feat(mock): add mock for label service
* fix tests, add delete test
* add remaining tests, flesh out GET params
* unused code
* remove comments
* add labels endpoints
* clean up labels after deleting a dashboard
* fix issues from code review
* WIP remove type from labels
* feat(http): add label cli functions
* fixes
* make fmt
* feat(http): add labels to tasks and buckets
* feat(bolt): remove labels if their resources are deleted
* remove bolt cleanup for now
* make fmt
* type conversion magic
* feat(inmem): implement label service
* prevent race conditions
* remove labels from inmem if their resources are deleted
* fix race condition
* fix formatting
test(platform): run testcontainer integration tests for nightly release
Integration tests for the vault secret service using testcontiners
should not run along with unit tests, however, they should run on some
regular schedule. This commit introduces `make test-integration` which
runs integration tests for vault using testcontainers. The command introduced
relies on docker being available on the host it is executed on.
chore(platform): make go modules tidy
chore: try to fix go mod
chore(platform): remove explicit logrus dependency
chore(platform): run go mod tidy
chore(platform): replace github.com/Sirupsen/logrus with github.com/sirupsen/logrus
chore(platform): update docker dependency
feat(vault): add vault implementation of secret service
test(platform): run testcontainer integration tests for nightly release
Integration tests for the vault secret service using testcontiners
should not run along with unit tests, however, they should run on some
regular schedule. This commit introduces `make test-integration` which
runs integration tests for vault using testcontainers. The command introduced
relies on docker being available on the host it is executed on.
chore(platform): make go modules tidy
chore: try to fix go mod
chore(platform): run go mod tidy
feat(vault): add vault implementation of secret service
chore(platform): make go modules tidy
feat(platform): add Put/Patch/Delete methods on secret service
feat(vault): add Put/Patch/Delete methods on vault secret service
feat(http): add http handler methods for secret service
feat(bolt): add Put/Delete/Patch methods to bolt secret service
feat(testing): add tests for Put/Patch/Delete methods in secret service
feat(mock): add mock secret service
feat(http): add tests for secrets endpoints
feat(http): update swagger for secrets endpoints
chore: run go mod tidy
feat(platform): add ToPermissions method to user resource mapping
The ToPermissions method returns a set of permissions that is granted
via a user resource mapping.
feat(bolt): resolve sessions permissions on lookup
feat(http): use authorizer instead of authorization service for write api
feat(bolt): create user resource mappings for org users in bucket create
feat(bolt): create user resource mapping for first org/user
fix(platform): use authorizer for query endpoint instead of authorization
test(http): use cmp instead of reflect for decode test
- Brought over enterprise's QueryLogReader, with small adjustments
- Time filters are for the run's ScheduledFor field, per spec
- Adjusted run log timestamps for consistent formatting:
- ScheduledFor is RFC3339 because it's a whole-second timestamp
- StartedAt, FinishedAt use RFC3339Nano for high precision
- Several test adjustments to use relative time, for easier integration
with storage retention
Previously, a v1 source was configured with a `URL` and `fluxURL`. The
`URL` was used for querying InfluxQL data, while the `fluxURL` pointed
to a `fluxd` instance and was used for querying Flux data. Since then,
`fluxd` has been subsumed by the InfluxDB 1.7 release, which supports
both InfluxQL and Flux.
This commit updates the source proxy query service to query Flux data
from a V1 source directly.
feat(platform): add dashboard operation log interface
feat(bolt): add dashboard operation log to bolt client
feat(platform): add links to dashboard load response
fix(http): update links in json response of dashboards
feat(bolt): add operation logs for each resource
feat(http): add routes for operation logs
feat(cmd/influxd): set operation logs on http api backend
This also adjusts the TaskService interface's RetryRun method to accept
a task ID rather than an org ID. Internally, we still look up runs by
organization, and maybe that will change later, but this is a more
natural way for clients to look it up.
Hello
As suggested in https://github.com/influxdata/platform/issues/983 I
changed `auths` to `authorizations` to have the key and the link in
line.
I went for this fix in order to avoid BC. I don't know if that's
the fix you had in mind. Or you were speaking more about short and long
terms.
Fixed#983
Signed-off-by: Gianluca Arbezzano <gianarb92@gmail.com>
* Add error handling to spinner
* Fix broken test
* Update retentionPeriod to retentionRules
* fix(http): bucket retention period now empty
Co-authored-by: Chris Goller <goller@gmail.com>
* fix(bucket): remove bucket key from createBucket API
* feat(bucket): update organization buckets
* fix: coerce Bucket type
* chore: create separate components for bucket create / update
feat(testing): test for meta field on dashboard conformance tests
feat(http): ensure that dashboard json is encoded correctly
feat(bolt): update dashboard meta appropriately
fix(testing): use equate empty in dashboard test for empty cells list
feat(inmem): add meta to dashboards for conformance tests
feat(http): add support for dashboard conformance tests
feat(platform): add find options to FindDashboards method on dashboard svc
feat(testing): use find options in dashboard conformance tests
feat(mock): add find options to dashboard service
feat(bolt): add find options to dashboard service
feat(inmem): add find options to dashboard service
feat(http): add find options to dashboard service
feat(http): update dashboard swagger specification
fix(platform): fix mistakes after rebase
review(http): use default find options for dashboards
We are moving the necessary code for 2.0 from the influxdb 1.X
repository to the platform 2.0 repository. The logger is an unnecessary
dependency on the old influxdb that is making life more complicated.
It may become a uint64 in the future, for example. This does mean
that we have to call Decode on some data that we just Encoded, but
we can fix that later.
This iterface is supposed to be something that both sessions and
authorizations can share so that other components can authorize requests
as they see fit.
Per #905, we will eventually support inferring organization from token.
In the meantime, allow the user to specify the org for their task.
No tests for this change; there is some test coverage in enterprise.
Since all paths for the various query services were the same even though
the API was different it was confusing to work with and debug.
Now all internal paths have a unique name.
* add types to bucket service tests
* add type to bucket cmd interface
* bucket type needs to be defined in json for POST creations
* rip out bucket type stuff
* remove type from bucket tests
* add InternalBucketID helper fn
* remove more code
* remove org from internal bucket ID
Using query request struct to query resources
Signed-off-by: Lorenzo Fontana <lo@linux.com>
Use query.ProxyRequest instead query.Request
Signed-off-by: Lorenzo Fontana <lo@linux.com>
Proxy request from idpd
Signed-off-by: Lorenzo Fontana <lo@linux.com>
Comments about the desired results
Signed-off-by: Lorenzo Fontana <lo@linux.com>
V1 endpoints working with flux
Signed-off-by: Lorenzo Fontana <lo@linux.com>
Influxql working for v1
Signed-off-by: Lorenzo Fontana <lo@linux.com>
Co-authored-by: Michael De Sa <mjdesa@gmail.com>
V2 influxql query endpoint working
Signed-off-by: Lorenzo Fontana <lo@linux.com>
Co-authored-by: Michael De Sa <mjdesa@gmail.com>
Signed-off-by: Lorenzo Fontana <lo@linux.com>
V2 Flux compiler support
Co-authored-by: Michael De Sa <mjdesa@gmail.com>
Signed-off-by: Lorenzo Fontana <lo@linux.com>
Improve comments in bolt sources and give error on self
Signed-off-by: Lorenzo Fontana <lo@linux.com>
Co-authored-by: Michael De Sa <mjdesa@gmail.com>
Review tests failing
Signed-off-by: Lorenzo Fontana <lo@linux.com>
Co-authored-by: Michael De Sa <mjdesa@gmail.com>
Avoid type casts for compiler types
Signed-off-by: Lorenzo Fontana <lo@linux.com>
Co-authored-by: Michael De Sa <mjdesa@gmail.com>
Using nil instead of dbrp mapping service for influxql v1
Signed-off-by: Lorenzo Fontana <lo@linux.com>
Check if compiler types are valid for influxql
Signed-off-by: Lorenzo Fontana <lo@linux.com>
Organization as query param in the flux external handler
Signed-off-by: Lorenzo Fontana <lo@linux.com>
feat(http): update swagger documentation for flux query endpoint
feat(http): document query endpoint design
The code documented does not currently work. It is indended that this
will be implemented in follow up PRs.
feat(platform): move source to platform package
The source Query endpoint implements what's in the query swagger docs
Signed-off-by: Lorenzo Fontana <lo@linux.com>
Co-authored-by: Michael De Sa <mjdesa@gmail.com>
feat(platform): allow for encoding and decoding of csv dialects
feat(platform): specify dialect in flux page
Co-authored-by: Andrew Watkins <andrew.watkinz@gmail.com>
Co-authored-by: Michael Desa <mjdesa@gmail.com>
test(testing): add tests for dashboards and cells
test(bolt): all conformance tests for dashbaords and cells
fix(bolt): rename dashboardV2Bucket to dashboardBucket
feat(chronograf): introduce v2 dashboards
Co-authored-by: Andrew Watkins <andrew.watkinz@gmail.com>
Co-authored-by: Michael Desa <mjdesa@gmail.com>
chore(chronograf): add tests for v2 dashboard reducer
Co-authored-by: Andrew Watkins <andrew.watkinz@gmail.com>
Co-authored-by: Michael Desa <mjdesa@gmail.com>
chore(chronograf): create dashboard from dashboard index
Co-authored-by: Andrew Watkins <andrew.watkinz@gmail.com>
Co-authored-by: Michael Desa <mjdesa@gmail.com>
chore(chronograf): move tests to same level of heirarchy as file tested
chronograf(chore): implement import dashboard with v2 api
chore(chronograf): delete dashboards from v2 api
chore(chronograf): add source health handler
chore(chronograf): move sources reducer to sources dir
chore(chronograf): remove stutter in notify
WIP stop namespacing by sourceID
chore(chronograf): no longer namespace routes under sources/:sourceID
WIP move dashtimeV1 to ranges
WIP remove CEO
chrore(chronograf): WIP Remove CEO and QueryMaker
chore(chronograf): introduce updateDashboard v2
chore(chronograf): Change cell to view
A dashboard's cell object now only contains positional information
for the cell in it's grid and a reference to the cell's view data.
A cell's view contains all necessary information for the cell's
visualization.
Add react-grid-layout types
chore(chronograf): introduce add cell
chore(chronograf): fix type errors
Not on DashboardPage
feat(platform): add cell methods to dashboard service interface
feat(mock): update dashboard service with cell methods
feat(testing): add cell methods to testing package
feat(bolt): add dashboard cell methods to bolt dashbaord service
feat(http): add cell routes to dashbaord handler
feat(platform): add dependent create/destroy of views from cells
chore(chronograf): introduce update dashboard cells endpoint
WIP update cells FE
fix(http): rename Cells to cells on dashboard cells response
chore(chronograf): re-introduce dashboard cell dragging
feat(platform): add copy dashboard cell to dashboard service
chore(platform): rename cell to view across codebase
feat(bolt): add replace dashboard tests
Move Layouts to Cells
Introduce delete cell
Fix broken test
fix(platform): update route for copying a a dashboard cell
UI for delete cell
Introduce copy cell
feat(platform): add copy view options to AddDashboardCell
feat(bolt): delete views when dashboard is removed.
Cleanup
Fix type errors
Fix links not updating
Remove annotations from RefreshingGraph
Sources and types work
fix(platform): add TODO.go files back
fix(view): rename visualizationType to type in view JSON
Co-authored-by: Deniz Kusefoglu <deniz@influxdata.com>
Co-authored-by: Michael Desa <mjdesa@gmail.com>
fix(dashboardTime): change dashboardID to string
Co-authored-by: Deniz Kusefoglu <deniz@influxdata.com>
Co-authored-by: Michael Desa <mjdesa@gmail.com>
feat(http): add dashboard api to swagger documentation
review(http): fix comments and function naming
feat(http): update sources swagger documentation
review(http): update the swagger to reflect the implementation
feat(platform): add usingView options to POST /dashboard/:id/cells
Moves idpe.QueryService into platform/query.ProxyQueryService
Splits the Request into ProxyRequest and Request.
Changes query.QueryService and query.AsyncQueryService to use a Request
type. This means that the Compiler interface is consumed by the service
to abstract out transpilation vs Flux compilation vs raw spec.
The transpiler handler is removed.
There are separate http handlers and service implementations for each of
the three query services.
Query logging types are moved into platform.
The ResultIterator now expects Cancel to always be called.
The fluxd binary exposes the query endpoint specified in the swagger
file.
Currently all that is supported is v1 sources. This code will likely
need revisiting.
Co-authored-by: Andrew Watkins <andrew.watkinz@gmail.com>
Co-authored-by: Michael Desa <mjdesa@gmail.com>
We need a coordination layer to facilitate the communication in the api to the task service.
The api was also not connected in main to the http service.
The limit was introduced in 510325ea5c but
it didn't specify why 64 was chosen.
According to https://stackoverflow.com/a/6160643, we should be able to
assume at least 8kb of space for HTTP headers, so I'm assuming 256 bytes
should be sufficient to avoid truncating most error messages, without
being likely to hit the total HTTP header limit.
Fixes#530.
This makes it so we can encode errors in HTTP headers so long as the
encoder did not write anything.
This also changes the behavior to encode the stats trailer even in the
event of an error.
Introduces the Statisticser interface which ResultIterators may
implement.
The HTTP implementation uses HTTP trailers to preserve the statistics.
This way we do not need to have all encoders and decoders support
statistics.
This will match the /ping endpoint from the influxdb 1.x httpd service.
This endpoint is used by many of the clients that use the query service
and it will also be helpful for testing that the transpiler service is
up and working.
In the future, we will add the appropriate health endpoint that is
standard for 2.0. But, for the best compatibility, we have to return
something viable for 1.x clients.
feat(platform): add dashboard service interface and associated types
feat(bolt): add bolt implementation ofplatform dashboard service
feat(http): add dashboard http handler and http implementation dashboard service
feat(cmd/idpd): use bolt dashboard service in idpd
test(testing): add conformance test for dashboard service
test(bolt): ensure conforms dashboard service conforms to tests
fix(platform): rename DashboardContents to DashboardCellContents in test
fix(bolt): correct mispelled comments
The /query endpoint didn't have an orgID parameter so we can't have one
in the transpiler endpoint. This removes the parameter and replaces it
with a command line parameter that sets which organization we are using
for querying ifql.
This modifies the `MultiResultDecoder` interface to accept an
`io.ReadCloser` so the `ResultIterator` can close the `io.Reader`
instead of doing it through a defer call. It then makes it so the
`Cancel()` method will close the reader or the reader will be
automatically closed when `More()` returns false.
The query service now handles some errors (not all) from the ifqld
service. This way, the transpiler, which uses the query service, can
report when it encounters an error from ifqld.
The transpiler will now correctly return errors in the JSON response.
The http package now contains a server that handles signals and proper
shutdown procedure. It has now been updated to use it.
The http package has also added a `ListenAndServe` convenience function
that is similar to the `net/http` one, but also takes in a logger and
will automatically use the most common signals when running an http
server.
It handles graceful termination by listening to signals which will be
particularly useful for commands where the sole purpose is to run an
http server.
feat(platform): add id to authorization
feat(platform): add user arg to CreateAuthorization method on auth svc
migrate(platform): move idp command to platform directory
This comit did not move the ifql command as it depends on the query
service which has yet to be migrated.
feat(platform): add optional user name to authorization struct
feat(platform): add organization name to bucket struct
Additionally allow filtering buckets by organization name.
feat(prom): ensure that prom auth svc implement base interface
feat(prometheus): add user to create authorization method
feat(prom): drop user string from create authorization
feat(zap): ensure that zap auth svc implements base service interface
feat(zap): add user to create authorization method
feat(zap): drop user string from create authorization
feat(http): add ids to authorization service
feat(http): ensure that http authoriztaion service implements auth svc interface
feat(http): use authorization ids in authorization handler
squash(http): add check for http status accepted in authorization service
feat(http): clean up authorization service and handlers
feat(http): drop user string from create authorization
fix(http): normalize the http authorization service
feat(http): normalize bucket service and handler methods
Additonally, we added support for DELETE bucket
feat(http): add delete user handler
Additionally, there was a bit of general cleanup
feat(http): add delete route for organization handler and service
Did a bit of additional cleanup of the http code.
test(testing): add service conformance tests
test(testing): add organization service conformance tests
test(testing): add conformance test for orgs service
Additionally, there was a bit of cleanup in the users service tests
test(testing): add conformance test for authorizations service
test(testing): update auth tests to validate that user exists
test(testing): update authorization conformance tests with user name
test(testing): update bucket conformance tests to include organizations
feat(bolt): add bolt implementation services
feat(bolt): add bolt implementation of organization service
feat(bolt): add bolt implementation of users service
feat(bolt): add bolt implementation of authorization service
feat(bolt): add user to create authorization method
feat(bolt): drop user string from create authorization
fix(bolt): set user name on authorization after put
feat(bolt): update bucket servie to include organizations
feat(bolt): add dependent destroy of resources
feat(cmd/idpd): use bolt services in platform server
feat(cmd/idpd): use bolt organization service in platform server
feat(cmd/idpd): use bolt users service in plaform server
feat(cmd/idpd): use bolt client as authorization service
feat(cmd/idp): show user name in output of auth sub command
feat(cmd/idp): clean up bucket subcommand of idp command
fix(cmd/idp): normalize idp command output for users
fix(cmd/idp): normalize auth subcommand output
feat(cmd/idp): add support for delete organiztion command
migrate(idp): move ifql subcommand of idp to platform
Lyon Hill
Deniz Kusefoglu
Palak Bhojani
Jorge Landivar
Kelvin Wang
Iris Scholten
Jade McGough
zhulongcheng
Christopher M. Wolff
Jakub Bednar
Michael Desa
kelwang
Mark Rushakoff
Edd Robinson
Christopher Henn
Delmer
Mark Rushakoff
Ben Johnson
Brandon Farmer
Alirie Gray
Chris Goller
Jonathan A. Sternberg
Nathaniel Cook
Gianluca Arbezzano
Andrew Watkins
Adam
Leonardo Di Donato
Nathaniel Cook
Jeff Wendling
Jared Scheib
lukevmorris
Lorenzo Fontana
Gianluca Arbezzano
Stuart Carnie
Adam