Commit Graph

118 Commits (f385d197131d76201b08d80169c0482fd7360bd4)

Author SHA1 Message Date
Jacob Marble b836ab9c17
feat(storage): implement backup and restore (#16504)
* feat(backup): `influx backup` creates data backup

* feat(backup): initial restore work

* feat(restore): initial restore impl

Adds a restore tool which does offline restore of data and metadata.

* fix(restore): pr cleanup

* fix(restore): fix data dir creation

* fix(restore): pr cleanup

* chore: amend CHANGELOG

* fix: restore to empty dir fails differently

* feat(backup): backup and restore credentials

Saves the credentials file to backups and restores it from backups.

Additionally adds some logging for errors when fetching backup files.

* fix(restore): add missed commit

* fix(restore): pr cleanup

* fix(restore): fix default credentials restore path

* fix(backup): actually copy the credentials file for the backup

* fix: dirs get 0777, files get 0666

* fix: small review feedback

Co-authored-by: tmgordeeva <tanya@influxdata.com>
2020-01-21 14:22:45 -08:00
Stuart Carnie 4122a5dfec
feat(http): Enforce limits in write handler and expose config options 2020-01-14 09:13:40 -07:00
George 560cd3faee
fix(http): expose max batch size bytes limit open on *APIBackend (#16497) 2020-01-10 16:00:37 +00:00
Greg 41e771a464
feat(http): expose list of available telegraf plugins (#16233) 2019-12-17 16:53:19 -07:00
Johnny Steenbergen 8729cd0959 chore(http): refactor api handler take advantage of route tree instead of switch statement
is very much WIP at this point
2019-12-11 09:53:29 -08:00
Jacob Marble 5f19c6cace
chore: Remove several instances of WithLogger (#15996)
* chore: Remove several instances of WithLogger

* chore: unexport Logger fields

* chore: unexport some more Logger fields

* chore: go fmt

chore: fix test

chore: s/logger/log

chore: fix test

chore: revert http.Handler.Handler constructor initialization

* refactor: integrate review feedback, fix all test nop loggers

* refactor: capitalize all log messages

* refactor: rename two logger to log
2019-12-04 15:10:23 -08:00
Johnny Steenbergen dc2d931f00 feat: add support for setting password and org for a new user in the cli
one thing to note here is that new endpoint was created. there was no
endpoint for setting an initial password that worked. The existin endpoint
was a bit messy and coupled across multiple routes. Having multiple auth
schemes proved incredibly taxing to write against.
2019-11-20 09:16:31 -08:00
Johnny Steenbergen bc083e169c chore: refactor password service to provide userID instead of name 2019-11-20 09:16:31 -08:00
Johnny Steenbergen b4b337e414 chore(http): provide mountable router for registering routes on APIHandler 2019-11-07 09:44:24 -08:00
Johnny Steenbergen 2902638c8a feat(pkger): add pgker http server to the api handler 2019-11-07 09:44:24 -08:00
Jade McGough fcc06d4291 feat(http): add predicate endpoint 2019-10-18 12:02:53 -04:00
Johnny Steenbergen 9d5392cfd9 fix(logging): add debug logging for all http requests 2019-10-07 14:19:21 -07:00
Kelvin Wang 7a677e9532 fix(http): fix notfication endpoint secret store 2019-08-28 16:02:17 -04:00
Michael Desa add4bcb873
fix(http): remove non-txn task Create/Update/Delete for nofitication rules 2019-08-26 07:55:43 -04:00
Michael Desa df628f21de
fix(http): remove non-txn task Create/Update/Delete for checks 2019-08-26 07:55:43 -04:00
Kelvin Wang 03f6214d0e chore(http): split taskservice from check and notification 2019-08-19 18:56:17 -04:00
Kelvin Wang c5100f3eae feat(http): add notification endpoint 2019-08-15 21:56:21 -04:00
Jade McGough 5d826961db feat(http): add checks http handler 2019-07-31 19:00:26 -04:00
Kelvin Wang dcda49d1d0 feat(http): add notification rule handler 2019-07-25 15:15:35 -04:00
Kelvin Wang 9ecada4dce feat(http): convert errorEncode to use interface 2019-06-26 21:41:01 -04:00
Iris Scholten 2e26f23f9e
feat(authentication): Add cli args for specifying session length and renewal (#13924)
Co-authored-by: Jade McGough <jade@influxdata.com>

* Add session renew option to launcher and use in middlewhere
* pass session options to services
* Update SessionAutoRenew to SessionRenewDisabled
* Add test for service constructor defaults
* Update changelog
2019-05-15 10:16:47 -07:00
Kelvin Wang 717fcc44b3 feat(influxdb): add scraper filter 2019-04-25 11:33:34 -04:00
Nathaniel Cook 8b918539db Merge branch 'master' into flux-staging 2019-04-17 12:26:10 -06:00
Michael Desa e00c071c2c
feat(influxdb): add query/write http event recorders
feat(http): add prometheus counters for tracking write/query usage

feat(http/metric): add metric recoder for recording http metrics

feat(prometheus): implement metric.Recorder for prometheus metrics

fix(prometheus): remove erroneous fmt.Printlns

feat(http): add prometheus registry to API backend

This was done as exposing prometheus metrics to a higher level was quite
difficult. It was much simple to simply pass the registry down to
anything that needs it.

feat(cmd/influxd/launcher): pass prom registry in on api backend

feat(http): collect metrics for write and query endpoints

This was much messier than I would have preferred. Future work is
outlined in TODOs.

review(influxdb): rename metric.Metric to metric.Event
2019-04-17 08:33:31 -04:00
jlapacik 5cf5806880 refactor(http): remove api/v2/query/spec endpoint and handler 2019-04-12 07:15:27 -07:00
Kelvin Wang 1cf600aca9 remove proto 2019-04-03 15:42:25 -04:00
kelwang a64c4fd138
Merge pull request #12246 from zhulongcheng/rm-view
fix(http): remove view service
2019-03-27 20:13:10 -04:00
kelwang b879d8ae59
Merge pull request #12677 from influxdata/swagger_resource_buckets
fix(http): change source buckets from slice to obj
2019-03-18 18:29:23 -04:00
Kelvin Wang 6a64e0818f fix(http): change source buckets from slice to obj 2019-03-18 17:53:53 -04:00
Jade McGough 5500419357 fix(http): wrap label service with authorizer 2019-03-17 20:47:21 -07:00
Mark Rushakoff 9bbe321d23 chore: serve swagger.json in development builds
In a development build (i.e. does not have the assets build tag), the
first request to GET /api/v2/swagger.json will produce log output like:

INFLUXDB_VALID_SWAGGER_PATH not set; falling back to checking relative paths	{"log_id": "0E4I7Dkl000", "service": "swagger-loader"}
Successfully loaded swagger.yml	{"log_id": "0E4I7Dkl000", "service": "swagger-loader", "path": "/.../influxdb/http/swagger.yml"}

There is no such log line in production builds.

But now both builds correctly serve swagger.json, instead of just
production builds.
2019-03-09 12:42:44 -08:00
zhulongcheng 1a46cc8f80 fix(http): remove view handler 2019-03-07 10:18:55 +08:00
Michael Desa 2687b7fd61
feat(influxdb): add support for templates
feat(influxdb): add generic store for documents

feat(influxdb): support authorizations in document store

feat(influxdb): support orgs in user resource mapping

feat(influxdb): add read-only included field on documents

feat(influxdb): add labels support to documents service

fix(influxdb): rename data field to content on documents

feat(influxdb): add with org id options for document store

feat(http): add templates swagger

feat(influxdb): add documentation to document options

doc(kv): add documentation for kv document store

test(kv): pull document tests in to the testing package

fix(http): fix swagger specification of templates endpoints
2019-03-06 13:18:31 -08:00
Nathaniel Cook a75adf6c4b fix(http): separate InfluxQL and Flux services
Previously the APIBackend understood only a ProxyQueryService,
but it needs to understand that there are two implementations of the
ProxyQueryService one for handling InfluxQL queries and one for handling
Flux queries. The names of the fields have been updated to make this
clear. As well as the FluxBackend is now initialized using the
FluxService explicitly.
2019-02-22 10:31:20 -07:00
Michael Desa 6a8a1fce32 feat(kv): implemented key/value store with end-to-end integration tests
* feat(kv:inmem:bolt): implement user service in a kv

* refactor(kv): use consistent func receiver name

* feat(kv): add initial basic auth service

* refactor(passwords): move auth interface into own file

* refactor(passwords): rename basic auth files to passwords

* refactor(passwords): rename from BasicAuth to Passwords

* refactor(kv): copy bolt user test into kv

Co-authored-by: Michael Desa <mjdesa@gmail.com>

* feat(kv): add inmem testing to kv store

* fix(kv): remove extra user index initialization

* feat(kv): attempt at making errors nice

* fix(http): return not found error if filter is invalid

* fix(http): s/platform/influxdb/ for user service

* fix(http): s/platform/influxdb/ for user service

* feat(kv): initial port of telegraf configs to kv

* feat(kv): first pass at migrating bolt org service to kv

* feat(kv): first pass at bucket service

* feat(kv): first pass at migrating kvlog to kv package

* feat(kv): add resource op logs

* feat(kv): first pass at user resource mapping migration

* feat(kv): add urm usage to bucket and org services

* feat(kv): first pass at kv authz service

* feat(kv): add cascading auth delete for users

* feat(kv): first pass d authorizer.OrganizationService in kv

* feat(cmd/influxd/launcher): user kv services where appropriate

* fix(kv): initialize authorizations

* fix(influxdb): use same buckets while slowly migrating stuff

* fix(kv): make staticcheck pass

* feat(kv): add dashboards to kv

review: make suggestions from pr review

fix: use common bucket names for bolt/kv stores

* test(kv): add complete password test coverage

* chore(kv): fixes for staticcheck

* feat(kv): implement labels generically on kv

* feat(kv): implement macro service

* feat(kv): add source service

* feat(kv): add session service

* feat(kv): add kv secret service

* refactor(kv): update telegraf and urm with error messages

* feat(kv): add lookup service

* feat(kv): add kv onboarding service

* refactor(kv): update telegraf to avoid repetition

* feat(cmd/influxd): use kv lookup service

* feat(kv): add telegraf to lookup service

* feat(cmd/influxd): use kv telegraf service

* feat(kv): initial port of scrapers in bolt to kv

* feat(kv): update scraper error messaging

* feat(cmd/influxd): add kv scraper

* feat(kv): add inmem backend tests

* refactor(kv): copy paste errors

* refactor(kv): add code to password errors

* fix(testing): update error messages for incorrect passwords

* feat(kv:inmem:bolt): implement user service in a kv

* refactor(kv): use consistent func receiver name

* refactor(kv): copy bolt user test into kv

Co-authored-by: Michael Desa <mjdesa@gmail.com>

* feat(kv): add inmem testing to kv store

* fix(kv): remove extra user index initialization

* feat(kv): attempt at making errors nice

* fix(http): return not found error if filter is invalid

* fix(http): s/platform/influxdb/ for user service

* feat(kv): first pass at migrating bolt org service to kv

* feat(kv): first pass at bucket service

* feat(kv): first pass at migrating kvlog to kv package

* feat(kv): add resource op logs

* feat(kv): first pass at user resource mapping migration

* feat(kv): add urm usage to bucket and org services

* feat(kv): first pass at kv authz service

* feat(kv): add cascading auth delete for users

* feat(kv): first pass d authorizer.OrganizationService in kv

* feat(cmd/influxd/launcher): user kv services where appropriate

* feat(kv): add initial basic auth service

* refactor(passwords): move auth interface into own file

* refactor(passwords): rename basic auth files to passwords

* fix(http): s/platform/influxdb/ for user service

* fix(kv): initialize authorizations

* fix(influxdb): use same buckets while slowly migrating stuff

* fix(kv): make staticcheck pass

* feat(kv): add dashboards to kv

review: make suggestions from pr review

fix: use common bucket names for bolt/kv stores

* feat(kv): implement labels generically on kv

* refactor(passwords): rename from BasicAuth to Passwords

* test(kv): add complete password test coverage

* chore(kv): fixes for staticcheck

* feat(kv): implement macro service

* feat(kv): add source service

* feat(kv): add session service

* feat(kv): initial port of telegraf configs to kv

* feat(kv): initial port of scrapers in bolt to kv

* feat(kv): add kv secret service

* refactor(kv): update telegraf and urm with error messages

* feat(kv): add lookup service

* feat(kv): add kv onboarding service

* refactor(kv): update telegraf to avoid repetition

* feat(cmd/influxd): use kv lookup service

* feat(kv): add telegraf to lookup service

* feat(cmd/influxd): use kv telegraf service

* feat(kv): update scraper error messaging

* feat(cmd/influxd): add kv scraper

* feat(kv): add inmem backend tests

* refactor(kv): copy paste errors

* refactor(kv): add code to password errors

* fix(testing): update error messages for incorrect passwords

* feat(http): initial support for flushing all key/values from kv store

* feat(kv): rename macro to variable

* feat(cmd/influxd/launcher): user kv services where appropriate

* refactor(passwords): rename from BasicAuth to Passwords

* feat(kv): implement macro service

* test(ui): introduce cypress

* test(ui): introduce first typescript test

* test(ui/e2e): add ci job

* chore: update gitignore to ignore test outputs

* feat(inmem): in memory influxdb

* test(e2e): adding pinger that checks if influxdb is alive

* hackathon

* hack

* hack

* hack

* hack

* Revert "feat(inmem): in memory influxdb"

This reverts commit 30ddf032003e704643b07ce80df61c3299ea7295.

* hack

* hack

* hack

* hack

* hack

* hack

* hack

* hack

* hack

* hack

* hack

* hack

* hack

* chore: lint ignore node_modules

* hack

* hack

* hack

* add user and flush

* hack

* remove unused vars

* hack

* hack

* ci(circle): prefix e2e artifacts

* change test to testid

* update cypress

* moar testid

* fix npm warnings

* remove absolte path

* chore(ci): remove /home/circleci proto mkdir hack

* wip: crud resources e2e

* fix(inmem): use inmem kv store services

* test(dashboard): add first dashboard crud tests

* hack

* undo hack

* fix: use response from setup for orgID

* chore: wip

* add convenience getByTitle function

* test(e2e): ui can create orgs

* test(e2e): add test for org deletion and update

* test(e2e): introduce task creation test

* test(e2e): create and update of buckets on org view

* chore: move types to declaration file

* chore: use route fixture in dashboard tests

* chore(ci): hack back

* test(ui): update snapshots

* chore: package-lock

* chore: remove macros

* fix: launcher rebase issues

* fix: compile errors

* fix: compile errors

* feat(cmd/influxdb): add explicit testing, asset-path, and store flags

Co-authored-by: Andrew Watkins <watts@influxdb.com>

* fix(cmd/influxd): set default HTTP handler and flags

Co-authored-by: Andrew Watkins <watts@influxdb.com>

* build(Makefile): add run-e2e and PHONY

* feat(kv:inmem:bolt): implement user service in a kv

* refactor(kv): use consistent func receiver name

* feat(kv): add initial basic auth service

* refactor(passwords): move auth interface into own file

* refactor(passwords): rename basic auth files to passwords

* refactor(passwords): rename from BasicAuth to Passwords

* refactor(kv): copy bolt user test into kv

Co-authored-by: Michael Desa <mjdesa@gmail.com>

* feat(kv): add inmem testing to kv store

* fix(kv): remove extra user index initialization

* feat(kv): attempt at making errors nice

* fix(http): return not found error if filter is invalid

* fix(http): s/platform/influxdb/ for user service

* fix(http): s/platform/influxdb/ for user service

* feat(kv): initial port of telegraf configs to kv

* feat(kv): initial port of scrapers in bolt to kv

* feat(kv): first pass at migrating bolt org service to kv

* feat(kv): first pass at bucket service

* feat(kv): first pass at migrating kvlog to kv package

* feat(kv): add resource op logs

* feat(kv): first pass at user resource mapping migration

* feat(kv): add urm usage to bucket and org services

* feat(kv): first pass at kv authz service

* feat(kv): add cascading auth delete for users

* feat(kv): first pass d authorizer.OrganizationService in kv

* feat(cmd/influxd/launcher): user kv services where appropriate

* fix(kv): initialize authorizations

* fix(influxdb): use same buckets while slowly migrating stuff

* fix(kv): make staticcheck pass

* feat(kv): add dashboards to kv

review: make suggestions from pr review

fix: use common bucket names for bolt/kv stores

* test(kv): add complete password test coverage

* chore(kv): fixes for staticcheck

* feat(kv): implement labels generically on kv

* feat(kv): implement macro service

* feat(kv): add source service

* feat(kv): add session service

* feat(kv): add kv secret service

* refactor(kv): update telegraf and urm with error messages

* feat(kv): add lookup service

* feat(kv): add kv onboarding service

* refactor(kv): update telegraf to avoid repetition

* feat(cmd/influxd): use kv lookup service

* feat(kv): add telegraf to lookup service

* feat(cmd/influxd): use kv telegraf service

* feat(kv): update scraper error messaging

* feat(cmd/influxd): add kv scraper

* feat(kv): add inmem backend tests

* refactor(kv): copy paste errors

* refactor(kv): add code to password errors

* fix(testing): update error messages for incorrect passwords

* feat(kv): rename macro to variable

* refactor(kv): auth/bucket/org/user unique checks return errors now

* feat(inmem): add way to get all bucket names from store

* feat(inmem): Buckets to return slice of bytes rather than strings

* feat(inmem): add locks around Buckets to avoid races

* feat(cmd/influx): check for unauthorized error in wrapCheckSetup

* chore(e2e): add video and screenshot artifcats to gitignore

* docs(ci): add build instructions for e2e tests

* feat(kv): add id lookup for authorized resources
2019-02-19 17:47:19 -06:00
Alirie Gray 5f524eb92d Rename all occurences of Macro to Variable 2019-02-14 13:21:57 -08:00
Iris Scholten aad4193c3f fix root labels endpoint
Co-authored-by: Brandon Farmer <bthesorceror@gmail.com>
Co-authored-by: Delmer Reed <del@influxdata.com>
Co-authored-by: Jade McGough <jade@influxdb.com>
2019-02-07 14:21:49 -08:00
Michael Desa dbc241b728 fix(http): use urm without authz where appropriate 2019-02-06 14:24:21 -05:00
Leonardo Di Donato e6bb9e6b97 feat(http): view backend for view http handler
Signed-off-by: Leonardo Di Donato <leodidonato@gmail.com>
2019-02-06 19:18:23 +01:00
zhulongcheng 57dc6664f5 feat(http): construct flux handler by FluxBackend 2019-02-06 19:18:23 +01:00
zhulongcheng c283b068ed feat(http): construct write handler by WriteBackend 2019-02-06 19:18:23 +01:00
zhulongcheng df93adce9a feat(http): construct telegraf handler by TelegrafBackend 2019-02-06 19:18:23 +01:00
zhulongcheng 18f4825e4f feat(http): construct task handler by TaskBackend 2019-02-06 19:18:23 +01:00
zhulongcheng c95e8b14da feat(http): construct setup handler by SetupBackend 2019-02-06 19:18:23 +01:00
zhulongcheng 2c89ee715b feat(http): construct source handler by SourceBackend 2019-02-06 19:18:23 +01:00
zhulongcheng 5d8b1507f5 feat(http): construct scraper handler by ScraperBackend 2019-02-06 19:18:23 +01:00
zhulongcheng 385cca43b7 feat(http): construct authorization handler by AuthorizationBackend 2019-02-06 19:18:23 +01:00
zhulongcheng 639753ab45 feat(http): construct macro handler by MacroBackend 2019-02-06 19:18:23 +01:00
zhulongcheng ac43477d68 feat(http): construct dashboard handler by DashboardBackend 2019-02-06 19:18:23 +01:00
zhulongcheng ef308ab9bd feat(http): construct org handler by OrgBackend 2019-02-06 19:18:23 +01:00