This pulls in the code that allows doing reads with flux into the
platform repo, and removes extra.go.
The reusable portion is under storage/reads, where the concrete
implementation for one of the platform's engines is in
storage/readservice.
In order to make this more reusable, the cursors had to move into
their own package, decoupling it from all of the other code in the
tsdb package. tsdb/cursors is this new package, and type/function
aliases have been added to the tsdb package to point at it.
The models package already is very light on transitive dependencies
and so it was allowed to be depended on in a concrete way in the
cursors package.
Finally, the protobuf definitions for issuing GRPC reads has been
moved into its own package for two reasons:
1. It's a clean separation, and helps keep it that way.
2. Many/most consumers will not be using GRPC. We just
use the datatypes to express the API which helps making
a GRPC server easier.
It is left up to future refactorings (specifically ones that involve
GPRC) to determine if these types should remain, or if there is a
cleaner way.
There's still some dependencies on both github.com/influxdata/influxql
and github.com/influxdata/influxdb/logger that we can hopefully remove
in future refactorings.
We reorganized the functions in flux to have the structure:
/functions
/inputs
/transformations
/outputs
this PR catches up platform to work with the new package layout.
As a separate refactoring issue, we should discuss:
from(bucket: ) should migrate from flux --> platform
to_http and to_kafka should migrate from platform --> flux
The race detector was picking up a data race because of the
unsynchronized reassignment of ctx:
```
$ go run -race ./cmd/influxd
...
^C
==================
WARNING: DATA RACE
Write at 0x00c00053d220 by main goroutine:
main.platformF()
/Users/mr/gomod/platform/cmd/influxd/main.go:381 +0x2cf7
github.com/spf13/cobra.(*Command).execute()
/Users/mr/go/pkg/mod/github.com/spf13/cobra@v0.0.3/command.go:766 +0x8b2
github.com/spf13/cobra.(*Command).ExecuteC()
/Users/mr/go/pkg/mod/github.com/spf13/cobra@v0.0.3/command.go:852 +0x432
github.com/spf13/cobra.(*Command).Execute()
/Users/mr/go/pkg/mod/github.com/spf13/cobra@v0.0.3/command.go:800 +0x38
main.Execute()
/Users/mr/gomod/platform/cmd/influxd/main.go:388 +0x4e
main.main()
/Users/mr/gomod/platform/cmd/influxd/main.go:46 +0x2f
Previous read at 0x00c00053d220 by goroutine 147:
main.platformF.func1()
/Users/mr/gomod/platform/cmd/influxd/main.go:330 +0x3c
Goroutine 147 (running) created at:
main.platformF()
/Users/mr/gomod/platform/cmd/influxd/main.go:329 +0x2554
github.com/spf13/cobra.(*Command).execute()
/Users/mr/go/pkg/mod/github.com/spf13/cobra@v0.0.3/command.go:766 +0x8b2
github.com/spf13/cobra.(*Command).ExecuteC()
/Users/mr/go/pkg/mod/github.com/spf13/cobra@v0.0.3/command.go:852 +0x432
github.com/spf13/cobra.(*Command).Execute()
/Users/mr/go/pkg/mod/github.com/spf13/cobra@v0.0.3/command.go:800 +0x38
main.Execute()
/Users/mr/gomod/platform/cmd/influxd/main.go:388 +0x4e
main.main()
/Users/mr/gomod/platform/cmd/influxd/main.go:46 +0x2f
==================
```
So just assign context.WithCancel to a new variable instead.
* add types to bucket service tests
* add type to bucket cmd interface
* bucket type needs to be defined in json for POST creations
* rip out bucket type stuff
* remove type from bucket tests
* add InternalBucketID helper fn
* remove more code
* remove org from internal bucket ID
Using query request struct to query resources
Signed-off-by: Lorenzo Fontana <lo@linux.com>
Use query.ProxyRequest instead query.Request
Signed-off-by: Lorenzo Fontana <lo@linux.com>
Proxy request from idpd
Signed-off-by: Lorenzo Fontana <lo@linux.com>
Comments about the desired results
Signed-off-by: Lorenzo Fontana <lo@linux.com>
V1 endpoints working with flux
Signed-off-by: Lorenzo Fontana <lo@linux.com>
Influxql working for v1
Signed-off-by: Lorenzo Fontana <lo@linux.com>
Co-authored-by: Michael De Sa <mjdesa@gmail.com>
V2 influxql query endpoint working
Signed-off-by: Lorenzo Fontana <lo@linux.com>
Co-authored-by: Michael De Sa <mjdesa@gmail.com>
Signed-off-by: Lorenzo Fontana <lo@linux.com>
V2 Flux compiler support
Co-authored-by: Michael De Sa <mjdesa@gmail.com>
Signed-off-by: Lorenzo Fontana <lo@linux.com>
Improve comments in bolt sources and give error on self
Signed-off-by: Lorenzo Fontana <lo@linux.com>
Co-authored-by: Michael De Sa <mjdesa@gmail.com>
Review tests failing
Signed-off-by: Lorenzo Fontana <lo@linux.com>
Co-authored-by: Michael De Sa <mjdesa@gmail.com>
Avoid type casts for compiler types
Signed-off-by: Lorenzo Fontana <lo@linux.com>
Co-authored-by: Michael De Sa <mjdesa@gmail.com>
Using nil instead of dbrp mapping service for influxql v1
Signed-off-by: Lorenzo Fontana <lo@linux.com>
Check if compiler types are valid for influxql
Signed-off-by: Lorenzo Fontana <lo@linux.com>
Organization as query param in the flux external handler
Signed-off-by: Lorenzo Fontana <lo@linux.com>
feat(http): update swagger documentation for flux query endpoint
feat(http): document query endpoint design
The code documented does not currently work. It is indended that this
will be implemented in follow up PRs.
feat(platform): move source to platform package
The source Query endpoint implements what's in the query swagger docs
Signed-off-by: Lorenzo Fontana <lo@linux.com>
Co-authored-by: Michael De Sa <mjdesa@gmail.com>
feat(platform): allow for encoding and decoding of csv dialects
feat(platform): specify dialect in flux page
Co-authored-by: Andrew Watkins <andrew.watkinz@gmail.com>
Co-authored-by: Michael Desa <mjdesa@gmail.com>
test(testing): add tests for dashboards and cells
test(bolt): all conformance tests for dashbaords and cells
fix(bolt): rename dashboardV2Bucket to dashboardBucket
feat(chronograf): introduce v2 dashboards
Co-authored-by: Andrew Watkins <andrew.watkinz@gmail.com>
Co-authored-by: Michael Desa <mjdesa@gmail.com>
chore(chronograf): add tests for v2 dashboard reducer
Co-authored-by: Andrew Watkins <andrew.watkinz@gmail.com>
Co-authored-by: Michael Desa <mjdesa@gmail.com>
chore(chronograf): create dashboard from dashboard index
Co-authored-by: Andrew Watkins <andrew.watkinz@gmail.com>
Co-authored-by: Michael Desa <mjdesa@gmail.com>
chore(chronograf): move tests to same level of heirarchy as file tested
chronograf(chore): implement import dashboard with v2 api
chore(chronograf): delete dashboards from v2 api
chore(chronograf): add source health handler
chore(chronograf): move sources reducer to sources dir
chore(chronograf): remove stutter in notify
WIP stop namespacing by sourceID
chore(chronograf): no longer namespace routes under sources/:sourceID
WIP move dashtimeV1 to ranges
WIP remove CEO
chrore(chronograf): WIP Remove CEO and QueryMaker
chore(chronograf): introduce updateDashboard v2
chore(chronograf): Change cell to view
A dashboard's cell object now only contains positional information
for the cell in it's grid and a reference to the cell's view data.
A cell's view contains all necessary information for the cell's
visualization.
Add react-grid-layout types
chore(chronograf): introduce add cell
chore(chronograf): fix type errors
Not on DashboardPage
feat(platform): add cell methods to dashboard service interface
feat(mock): update dashboard service with cell methods
feat(testing): add cell methods to testing package
feat(bolt): add dashboard cell methods to bolt dashbaord service
feat(http): add cell routes to dashbaord handler
feat(platform): add dependent create/destroy of views from cells
chore(chronograf): introduce update dashboard cells endpoint
WIP update cells FE
fix(http): rename Cells to cells on dashboard cells response
chore(chronograf): re-introduce dashboard cell dragging
feat(platform): add copy dashboard cell to dashboard service
chore(platform): rename cell to view across codebase
feat(bolt): add replace dashboard tests
Move Layouts to Cells
Introduce delete cell
Fix broken test
fix(platform): update route for copying a a dashboard cell
UI for delete cell
Introduce copy cell
feat(platform): add copy view options to AddDashboardCell
feat(bolt): delete views when dashboard is removed.
Cleanup
Fix type errors
Fix links not updating
Remove annotations from RefreshingGraph
Sources and types work
fix(platform): add TODO.go files back
fix(view): rename visualizationType to type in view JSON
Co-authored-by: Deniz Kusefoglu <deniz@influxdata.com>
Co-authored-by: Michael Desa <mjdesa@gmail.com>
fix(dashboardTime): change dashboardID to string
Co-authored-by: Deniz Kusefoglu <deniz@influxdata.com>
Co-authored-by: Michael Desa <mjdesa@gmail.com>
feat(http): add dashboard api to swagger documentation
review(http): fix comments and function naming
feat(http): update sources swagger documentation
review(http): update the swagger to reflect the implementation
feat(platform): add usingView options to POST /dashboard/:id/cells
Moves idpe.QueryService into platform/query.ProxyQueryService
Splits the Request into ProxyRequest and Request.
Changes query.QueryService and query.AsyncQueryService to use a Request
type. This means that the Compiler interface is consumed by the service
to abstract out transpilation vs Flux compilation vs raw spec.
The transpiler handler is removed.
There are separate http handlers and service implementations for each of
the three query services.
Query logging types are moved into platform.
The ResultIterator now expects Cancel to always be called.
The fluxd binary exposes the query endpoint specified in the swagger
file.
We need a coordination layer to facilitate the communication in the api to the task service.
The api was also not connected in main to the http service.
It was effectively a copied and pasted platform.ID, so change it to a
type alias. Once our known references to the query/id package are
updated to platform.ID, we'll delete the package.
feat(platform): add dashboard service interface and associated types
feat(bolt): add bolt implementation ofplatform dashboard service
feat(http): add dashboard http handler and http implementation dashboard service
feat(cmd/idpd): use bolt dashboard service in idpd
test(testing): add conformance test for dashboard service
test(bolt): ensure conforms dashboard service conforms to tests
fix(platform): rename DashboardContents to DashboardCellContents in test
fix(bolt): correct mispelled comments
This removes the `--org` parameter because we can't properly decode the
name into an organization id without access to the idp daemon. We will
either have to open access to that or not have that feature.
The /query endpoint didn't have an orgID parameter so we can't have one
in the transpiler endpoint. This removes the parameter and replaces it
with a command line parameter that sets which organization we are using
for querying ifql.
The query service now handles some errors (not all) from the ifqld
service. This way, the transpiler, which uses the query service, can
report when it encounters an error from ifqld.
The transpiler will now correctly return errors in the JSON response.
Edd Robinson
Adam
Jeff Wendling
Chris Goller
Kelvin Wang
kelwang
Mark Rushakoff
Jade McGough
Michael Desa
Nathaniel Cook
Lyon Hill
Delmer Reed
Christopher Henn
Stuart Carnie
Jonathan A. Sternberg
Leonardo Di Donato