Commit Graph

60 Commits (8d146db9ad6767295af4da619981b504e6d10301)

Author SHA1 Message Date
Mark Rushakoff ec1019b930 chore: add .editorconfig and enable in CI 2019-02-19 16:58:07 -08:00
Michael Desa 6a8a1fce32 feat(kv): implemented key/value store with end-to-end integration tests
* feat(kv:inmem:bolt): implement user service in a kv

* refactor(kv): use consistent func receiver name

* feat(kv): add initial basic auth service

* refactor(passwords): move auth interface into own file

* refactor(passwords): rename basic auth files to passwords

* refactor(passwords): rename from BasicAuth to Passwords

* refactor(kv): copy bolt user test into kv

Co-authored-by: Michael Desa <mjdesa@gmail.com>

* feat(kv): add inmem testing to kv store

* fix(kv): remove extra user index initialization

* feat(kv): attempt at making errors nice

* fix(http): return not found error if filter is invalid

* fix(http): s/platform/influxdb/ for user service

* fix(http): s/platform/influxdb/ for user service

* feat(kv): initial port of telegraf configs to kv

* feat(kv): first pass at migrating bolt org service to kv

* feat(kv): first pass at bucket service

* feat(kv): first pass at migrating kvlog to kv package

* feat(kv): add resource op logs

* feat(kv): first pass at user resource mapping migration

* feat(kv): add urm usage to bucket and org services

* feat(kv): first pass at kv authz service

* feat(kv): add cascading auth delete for users

* feat(kv): first pass d authorizer.OrganizationService in kv

* feat(cmd/influxd/launcher): user kv services where appropriate

* fix(kv): initialize authorizations

* fix(influxdb): use same buckets while slowly migrating stuff

* fix(kv): make staticcheck pass

* feat(kv): add dashboards to kv

review: make suggestions from pr review

fix: use common bucket names for bolt/kv stores

* test(kv): add complete password test coverage

* chore(kv): fixes for staticcheck

* feat(kv): implement labels generically on kv

* feat(kv): implement macro service

* feat(kv): add source service

* feat(kv): add session service

* feat(kv): add kv secret service

* refactor(kv): update telegraf and urm with error messages

* feat(kv): add lookup service

* feat(kv): add kv onboarding service

* refactor(kv): update telegraf to avoid repetition

* feat(cmd/influxd): use kv lookup service

* feat(kv): add telegraf to lookup service

* feat(cmd/influxd): use kv telegraf service

* feat(kv): initial port of scrapers in bolt to kv

* feat(kv): update scraper error messaging

* feat(cmd/influxd): add kv scraper

* feat(kv): add inmem backend tests

* refactor(kv): copy paste errors

* refactor(kv): add code to password errors

* fix(testing): update error messages for incorrect passwords

* feat(kv:inmem:bolt): implement user service in a kv

* refactor(kv): use consistent func receiver name

* refactor(kv): copy bolt user test into kv

Co-authored-by: Michael Desa <mjdesa@gmail.com>

* feat(kv): add inmem testing to kv store

* fix(kv): remove extra user index initialization

* feat(kv): attempt at making errors nice

* fix(http): return not found error if filter is invalid

* fix(http): s/platform/influxdb/ for user service

* feat(kv): first pass at migrating bolt org service to kv

* feat(kv): first pass at bucket service

* feat(kv): first pass at migrating kvlog to kv package

* feat(kv): add resource op logs

* feat(kv): first pass at user resource mapping migration

* feat(kv): add urm usage to bucket and org services

* feat(kv): first pass at kv authz service

* feat(kv): add cascading auth delete for users

* feat(kv): first pass d authorizer.OrganizationService in kv

* feat(cmd/influxd/launcher): user kv services where appropriate

* feat(kv): add initial basic auth service

* refactor(passwords): move auth interface into own file

* refactor(passwords): rename basic auth files to passwords

* fix(http): s/platform/influxdb/ for user service

* fix(kv): initialize authorizations

* fix(influxdb): use same buckets while slowly migrating stuff

* fix(kv): make staticcheck pass

* feat(kv): add dashboards to kv

review: make suggestions from pr review

fix: use common bucket names for bolt/kv stores

* feat(kv): implement labels generically on kv

* refactor(passwords): rename from BasicAuth to Passwords

* test(kv): add complete password test coverage

* chore(kv): fixes for staticcheck

* feat(kv): implement macro service

* feat(kv): add source service

* feat(kv): add session service

* feat(kv): initial port of telegraf configs to kv

* feat(kv): initial port of scrapers in bolt to kv

* feat(kv): add kv secret service

* refactor(kv): update telegraf and urm with error messages

* feat(kv): add lookup service

* feat(kv): add kv onboarding service

* refactor(kv): update telegraf to avoid repetition

* feat(cmd/influxd): use kv lookup service

* feat(kv): add telegraf to lookup service

* feat(cmd/influxd): use kv telegraf service

* feat(kv): update scraper error messaging

* feat(cmd/influxd): add kv scraper

* feat(kv): add inmem backend tests

* refactor(kv): copy paste errors

* refactor(kv): add code to password errors

* fix(testing): update error messages for incorrect passwords

* feat(http): initial support for flushing all key/values from kv store

* feat(kv): rename macro to variable

* feat(cmd/influxd/launcher): user kv services where appropriate

* refactor(passwords): rename from BasicAuth to Passwords

* feat(kv): implement macro service

* test(ui): introduce cypress

* test(ui): introduce first typescript test

* test(ui/e2e): add ci job

* chore: update gitignore to ignore test outputs

* feat(inmem): in memory influxdb

* test(e2e): adding pinger that checks if influxdb is alive

* hackathon

* hack

* hack

* hack

* hack

* Revert "feat(inmem): in memory influxdb"

This reverts commit 30ddf032003e704643b07ce80df61c3299ea7295.

* hack

* hack

* hack

* hack

* hack

* hack

* hack

* hack

* hack

* hack

* hack

* hack

* hack

* chore: lint ignore node_modules

* hack

* hack

* hack

* add user and flush

* hack

* remove unused vars

* hack

* hack

* ci(circle): prefix e2e artifacts

* change test to testid

* update cypress

* moar testid

* fix npm warnings

* remove absolte path

* chore(ci): remove /home/circleci proto mkdir hack

* wip: crud resources e2e

* fix(inmem): use inmem kv store services

* test(dashboard): add first dashboard crud tests

* hack

* undo hack

* fix: use response from setup for orgID

* chore: wip

* add convenience getByTitle function

* test(e2e): ui can create orgs

* test(e2e): add test for org deletion and update

* test(e2e): introduce task creation test

* test(e2e): create and update of buckets on org view

* chore: move types to declaration file

* chore: use route fixture in dashboard tests

* chore(ci): hack back

* test(ui): update snapshots

* chore: package-lock

* chore: remove macros

* fix: launcher rebase issues

* fix: compile errors

* fix: compile errors

* feat(cmd/influxdb): add explicit testing, asset-path, and store flags

Co-authored-by: Andrew Watkins <watts@influxdb.com>

* fix(cmd/influxd): set default HTTP handler and flags

Co-authored-by: Andrew Watkins <watts@influxdb.com>

* build(Makefile): add run-e2e and PHONY

* feat(kv:inmem:bolt): implement user service in a kv

* refactor(kv): use consistent func receiver name

* feat(kv): add initial basic auth service

* refactor(passwords): move auth interface into own file

* refactor(passwords): rename basic auth files to passwords

* refactor(passwords): rename from BasicAuth to Passwords

* refactor(kv): copy bolt user test into kv

Co-authored-by: Michael Desa <mjdesa@gmail.com>

* feat(kv): add inmem testing to kv store

* fix(kv): remove extra user index initialization

* feat(kv): attempt at making errors nice

* fix(http): return not found error if filter is invalid

* fix(http): s/platform/influxdb/ for user service

* fix(http): s/platform/influxdb/ for user service

* feat(kv): initial port of telegraf configs to kv

* feat(kv): initial port of scrapers in bolt to kv

* feat(kv): first pass at migrating bolt org service to kv

* feat(kv): first pass at bucket service

* feat(kv): first pass at migrating kvlog to kv package

* feat(kv): add resource op logs

* feat(kv): first pass at user resource mapping migration

* feat(kv): add urm usage to bucket and org services

* feat(kv): first pass at kv authz service

* feat(kv): add cascading auth delete for users

* feat(kv): first pass d authorizer.OrganizationService in kv

* feat(cmd/influxd/launcher): user kv services where appropriate

* fix(kv): initialize authorizations

* fix(influxdb): use same buckets while slowly migrating stuff

* fix(kv): make staticcheck pass

* feat(kv): add dashboards to kv

review: make suggestions from pr review

fix: use common bucket names for bolt/kv stores

* test(kv): add complete password test coverage

* chore(kv): fixes for staticcheck

* feat(kv): implement labels generically on kv

* feat(kv): implement macro service

* feat(kv): add source service

* feat(kv): add session service

* feat(kv): add kv secret service

* refactor(kv): update telegraf and urm with error messages

* feat(kv): add lookup service

* feat(kv): add kv onboarding service

* refactor(kv): update telegraf to avoid repetition

* feat(cmd/influxd): use kv lookup service

* feat(kv): add telegraf to lookup service

* feat(cmd/influxd): use kv telegraf service

* feat(kv): update scraper error messaging

* feat(cmd/influxd): add kv scraper

* feat(kv): add inmem backend tests

* refactor(kv): copy paste errors

* refactor(kv): add code to password errors

* fix(testing): update error messages for incorrect passwords

* feat(kv): rename macro to variable

* refactor(kv): auth/bucket/org/user unique checks return errors now

* feat(inmem): add way to get all bucket names from store

* feat(inmem): Buckets to return slice of bytes rather than strings

* feat(inmem): add locks around Buckets to avoid races

* feat(cmd/influx): check for unauthorized error in wrapCheckSetup

* chore(e2e): add video and screenshot artifcats to gitignore

* docs(ci): add build instructions for e2e tests

* feat(kv): add id lookup for authorized resources
2019-02-19 17:47:19 -06:00
Brandon Farmer 3992fa9d54 Remove typescript client generation
- All api calls are now through the client or raw XMLHTTPRequest
2019-02-05 17:13:57 -08:00
Chris Goller a0121c8819 ci(goreleaser): create a release version of goreleaser
Co-authored-by: Luke Morris <lukevmorris@gmail.com>
Co-authored-by: Mark Rushakoff <mark@influxdata.com>
2019-01-23 14:19:05 -06:00
Chris Goller dea64a3685 feat(proto): read JSON files in proto
We used go-bindata to embed all JSON files in proto to be shipped
with our released influxdb.

We are committing the bin_gen.go so that it simplifies the building
via make and go build.

If you would like to add another proto JSON file, run make
to generate a new bin_gen.go.

Co-Authored-By: Deniz Kusefoglu <deniz@influxdata.com>
2019-01-18 15:13:19 -06:00
Nathaniel Cook 073b034c24 chore: add go.mod and go.sum to list of Go sources
This enables workflows that use `go mod edit` commands to rely on the
Makefile correctly rebuilding the Go binaries.
2019-01-16 15:33:14 -07:00
AJ Bond e54b71f6e4 chore(image):Set image tag to version
By default goreleaser will set the image tag to latest. Include the .Version template variable to allow for nightly tags
2019-01-16 12:23:22 -05:00
AJ Bond cd11b348ed fix(ui): Fix UI generation issues
This issue allows for the ui to be generated during the build proccess
2019-01-15 20:45:38 -05:00
Brandon Farmer 547a147d6b Make client generation optional via ENV variable 2019-01-03 11:33:23 -08:00
Brandon Farmer ff0d2110ba
Merge pull request #2091 from influxdata/chore/generation-add-typescript-task
Add back typescript client generation task
2018-12-20 15:09:35 -08:00
AJ Bond 3e81678520 chore(ci): Run make generate
gorelease must run make generate before compiling
2018-12-20 17:17:32 -05:00
Brandon Farmer 4d1a7e3a28 Add back typescript client generation task 2018-12-20 13:51:08 -08:00
Nathaniel Cook b99ede0b76 chore(Makefile): fix issue with go run not having modules enabled 2018-12-20 12:47:00 -07:00
ABond 0f8f169558
Merge pull request #2074 from influxdata/chore/true-nightlies
chore(CircleCI): turn off deploy on commits to master
2018-12-20 13:11:23 -05:00
AJ Bond da67b392da chore(ci): Update make to build for each PR 2018-12-20 12:13:41 -05:00
Nathaniel Cook f8af55c9c0 chore(Makefile): fix build order of node_modules and subdirs 2018-12-19 17:41:33 -07:00
Nathaniel Cook 89f4525841 build(Makefile): fix various bug with makefiles
Fixes subdir ordering.
Works around issue with stringer not working with Go modules.
Fixes issues with generated code being ignored.

Fixes #2044
2018-12-19 17:02:19 -07:00
Nathaniel Cook 5d31665ae8 chore(Makefile): fix broken nightly builds after Makefile updates
There were two issues.First the `chronograf` subdir was not listed.
Second the tmpl generate dependency was not listed in the tools.go list
of dependencies.

Fixes #2044
2018-12-19 13:52:33 -07:00
Nathaniel Cook 61e36cbee6 chore(Makefile): add target to check generated files are accurate
A standard Makefile is used now in all subdirs that run go generate.
Make will only generate the file if its source files changed.
The checkgenerate target runs clean to ensure all targets a generated
fresh.
2018-12-18 12:54:17 -07:00
Nathaniel Cook e3eff83aaa chore(Makefile): remove yq dependency to generate ts client 2018-12-18 11:19:56 -07:00
Edd Robinson 170aaafa46 Add more verbosity to Go tests in CI 2018-12-07 14:32:34 +00:00
Michael Desa 56de056085 feat(vault): add vault implementation of secret service
test(platform): run testcontainer integration tests for nightly release

Integration tests for the vault secret service using testcontiners
should not run along with unit tests, however, they should run on some
regular schedule. This commit introduces `make test-integration` which
runs integration tests for vault using testcontainers. The command introduced
relies on docker being available on the host it is executed on.

chore(platform): make go modules tidy

chore: try to fix go mod

chore(platform): remove explicit logrus dependency

chore(platform): run go mod tidy

chore(platform): replace github.com/Sirupsen/logrus with github.com/sirupsen/logrus

chore(platform): update docker dependency

feat(vault): add vault implementation of secret service

test(platform): run testcontainer integration tests for nightly release

Integration tests for the vault secret service using testcontiners
should not run along with unit tests, however, they should run on some
regular schedule. This commit introduces `make test-integration` which
runs integration tests for vault using testcontainers. The command introduced
relies on docker being available on the host it is executed on.

chore(platform): make go modules tidy

chore: try to fix go mod

chore(platform): run go mod tidy

feat(vault): add vault implementation of secret service

chore(platform): make go modules tidy

feat(platform): add Put/Patch/Delete methods on secret service

feat(vault): add Put/Patch/Delete methods on vault secret service

feat(http): add http handler methods for secret service

feat(bolt): add Put/Delete/Patch methods to bolt secret service

feat(testing): add tests for Put/Patch/Delete methods in secret service

feat(mock): add mock secret service

feat(http): add tests for secrets endpoints

feat(http): update swagger for secrets endpoints

chore: run go mod tidy
2018-11-30 13:07:32 -08:00
Brandon Farmer b4aa8ff1d6 Add generated API code and use for task creation
Co-authored-by: Deniz Kusefoglu <deniz@influxdata.com>
2018-11-20 09:49:39 -08:00
Chris Goller a8c1155b21 fix(Makefile): remove old clean targets 2018-11-15 20:21:30 -06:00
Michael Desa fdbec9303d chore(ui): update misc build files related to moving chronograf/ui to ui 2018-10-29 14:00:45 -04:00
Jonathan A. Sternberg e9600b1f0b
refactor: modify tooling to take advantage of go run
The generate commands have been modified to take advantage of the new
functionality in Go 1.11 that allows `go run` to execute a package
instead of individual files.

This functionality combined with Go modules allows us to execute a
package directly out of our pinned dependencies rather than accidentally
picking up another binary outside of the build environment.

This also simplifies the Makefile because they no longer have to be
responsible for installing the correct tooling since the Go command
takes care of that logic. It also makes it so that the Makefiles with
file generation can now be invoked from their appropriate subdirectories
so they are contained within the directory itself rather than relying on
values in the top level Makefile.

It is now possible to generate all files within this project by using:

    go generate ./...

Or the Makefile can continue to be used.

This commit also copies over the special copy of `tmpl` that the storage
engine uses within the influxdb repository. It was never copied over so
using `go generate` on these packages did not work.
2018-10-15 11:01:27 -05:00
Jonathan A. Sternberg 0e14b63425
feat: checktidy will check the go.mod and go.sum files in circleci 2018-10-10 15:35:02 -05:00
Chris Goller 2f2334a3f3 ci(gofmt): check go formatting with each PR 2018-10-08 17:23:54 -05:00
Andrew Watkins e34d2e76ea
chronograf(v2-views): Introduce new view patterns, linting, and TypeScript 3.x (#901)
* Simplifiy color type

* Fix type

* Introduce V2 data shape for views

* WIP Split style and parsing to separate Dygraph components

* Add basic dygraph view types

* Add Gauge to v2 view shapes

* Upgrade TypeScript to ^3.0

* Add tsc to Circle build

* Fix Dygraph component paths

* Add testURL to jest config

* Upgrade lodash types

* Remove redundant test linter stetp

* Upgrade to TypeScript ^3

* Remove TableGraph (temporarily)
2018-09-27 10:46:48 -07:00
Nathaniel Cook 58c994fde0 chore: Remove fluxd
Fluxd is no longer necessary as the influxd and InfluxDB OSS both
implement the /v2/query endpoint.
2018-09-26 11:08:37 -06:00
Chris Goller a5a95ad213 ci(goreleaser): override goreleaser build flags 2018-09-26 00:32:44 -05:00
Chris Goller 41c9c595a5 ci(goreleaser): add new go env vars to goreleaser 2018-09-25 20:50:47 -05:00
Delmer Reed ce05a40e45 ci(Makefile): Add run target to Makefile
Adds run target to Makefile and removes older Chronograf files.

Co-authored-by: Chris Goller <goller@gmail.com>
2018-09-25 18:51:56 -04:00
Jonathan A. Sternberg 2ee5308a27
fix: the master build no longer uses -i when building the influx binary
The `-i` flag causes a problem when a cross compilation happens because
it attempts to build files in GOROOT which is located in a directory
that is not writable by the build user. This same problem exists for
when the `-tags` are changed like adding `-tags assets` in the `all`
build that runs on the master branch.

This removes `-i` because, in the latest go release, the `-i` flag
doesn't actually do anything useful since there is another build cache
that the results are placed in regardless of whether `-i` is used.

This fixes the build on master so that `make all` and `make nightly`
will function properly.
2018-09-25 11:33:07 -05:00
Jonathan A. Sternberg 97de3469a5
fix: restore the generate task in the Makefile
This was mistakenly deleted when I deleted vendor. I have no idea why it
was deleted so it was probably a mistake or something left over from
testing.
2018-09-24 15:02:45 -05:00
Jonathan A. Sternberg 46ed9bc4fb feat: switch to using go modules for building
This updates the dependency manager to use go modules instead of dep so
that we can remove dep as a dependency and we can begin using and giving
feedback on modules within this project. It should simplify dependency
management and make it faster to run the builds too.

The big change is updating the various Makefiles to stop relying on the
vendor directory because it no longer exists. This change creates a
`tools.go` file with the revisions of the tools we use that are currently
in the `required` section of `Gopkg.toml`. We are currently in other
discussions to modify how we handle tool dependency management, but this
change does not change that.
2018-09-20 08:33:15 -07:00
Nathaniel Cook e3fc3ff1b6 chore: Update GoReleaser to publish snapshots 2018-09-17 17:29:20 -07:00
Chris Goller 55491eae5b ci(Makefile): add asset generation to nightly 2018-09-11 16:11:16 -05:00
Chris Goller 7d61b90bca ci(Makefile): update PATH to have go-bindata 2018-09-11 10:22:06 -05:00
Stuart Carnie 7a05dfb61e refactor: Update Makefiles 2018-09-06 13:56:59 -07:00
Chris Goller ef7bfdd6d2 build(chronograf): add conditional build tags for asset generation 2018-09-04 20:46:22 -05:00
Chris Goller b37db6ca57 ci(goreleaser): update to latest goreleaser to support building 2 containers 2018-08-31 13:28:03 -05:00
Chris Goller 0ecb5aaefc chore(cmd/influxd): rename idpd to influxd 2018-08-31 12:52:40 -05:00
Mark Rushakoff d26233f47e ci: enable GOCACHE for Circle
With a clean GOCACHE, `make test-go` took about 30 seconds. It's hard to
tell the exact time since `make vendor` was implicitly executed in the
same block.

Now, `make test-go` uses GOCACHE and takes about 5 seconds to restore
the cache, then about 8 seconds to run through a fully cached set of
tests. If any test is actually broken or doesn't compile, this will give
us fast feedback.

But this change also runs `go test -race -count=1`, so that we fully
exercise any possible data races. That currently takes about 19 seconds.

Then finally we save GOCACHE, which has our `go test` results, and a
populated build cache for our test files, including built with the race
detector. Saving that takes about 14 seconds.

That means we took about 30 seconds originally to just run `go test
-count=1`, and now we take (5+8+19+14)=46 seconds to run plain go test
and go test with the race detector. There's probably an argument to be
made for just running with the race detector, but running both gives us
more coverage IMO, and it does allow us to run tests that aren't enabled
on race builds.

See https://circleci.com/gh/influxdata/platform/1840 for a run with full
caching.
2018-08-28 13:59:25 -07:00
lukevmorris 7e9f90dbce
chore(ci): parallelize js and go tests (#670)
This PR teases apart the golang and js build steps so that they can be run in parallel in circleci, or locally with make test -j2.

Closes #599
2018-08-27 08:34:17 -07:00
Nathaniel Cook 5bde0b5be6 fix: Update query services to use Request type
Moves idpe.QueryService into platform/query.ProxyQueryService
Splits the Request into ProxyRequest and Request.

Changes query.QueryService and query.AsyncQueryService to use a Request
type. This means that the Compiler interface is consumed by the service
to abstract out transpilation vs Flux compilation vs raw spec.

The transpiler handler is removed.

There are separate http handlers and service implementations for each of
the three query services.

Query logging types are moved into platform.

The ResultIterator now expects Cancel to always be called.

The fluxd binary exposes the query endpoint specified in the swagger
file.
2018-08-08 15:31:35 -06:00
Lyon Hill 80fe53f2b6 refactor(task): Move task meta into the backend
Task meta belongs with the rest of the task backend content.
2018-08-03 14:27:07 -06:00
Mark Rushakoff 0f52899677 hotfix(chronograf): add stub Asset functions to fix build
We need a long-term decision, but in the meantime we need a not-broken
build.
2018-07-26 14:58:20 -07:00
Michael Desa 346f19783a ci: update js tests to make test 2018-07-24 16:18:21 -04:00
Michael Desa dc7418f7fe makefile: fix make file to run go generate 2018-07-23 17:42:03 -04:00