Commit Graph

547 Commits (db/update-protos)

Author SHA1 Message Date
Sam Arnold 1542d2404f
feat: new recovery subcommand allows creating recovery user/token (#22590)
* feat: influxd recovery-cli allows creating recovery user/token

Closes #12051

* test: do not assert order of users or orgs in list

* fix: code review suggestions from dan

Co-authored-by: Daniel Moran <danxmoran@gmail.com>

* fix: more fixes from dan's review

* chore: depend on tabwriter from influx-cli

* chore: revert onboarding refactor

Co-authored-by: Daniel Moran <danxmoran@gmail.com>
2021-10-06 12:37:36 -04:00
Daniel Moran 7c19225bed
feat: implement replication validation (#22581) 2021-10-05 14:34:38 -04:00
Daniel Moran 2795c600c1
feat: add `--storage-write-timeout` flag to set write request timeouts (#22617) 2021-10-04 15:38:09 -04:00
Daniel Moran e2948d875c
test: skip pkger URI test when running on Windows (#22475) 2021-09-14 10:58:35 -04:00
William Baker ec7841b355
feat: support for flux cardinality query (#22441)
* feat: works with custom iterator

* feat: works with existing iterators

* chore: cleanup

* test: consistent assertions for tests

* fix: better log message if trying to filter on the value of a field key

* fix: comment for handling boolean literal; handle false boolean as well

* fix: make time range checking inclusive
2021-09-13 13:20:56 -06:00
Sam Arnold 5015297d40
fix: more expressive errors (#22448)
* fix: more expressive errors

Closes #22446

* fix: server only logging for untyped errors

* chore: fix formatting
2021-09-13 15:12:35 -04:00
Daniel Moran 153a89dba0
feat: deleting a bucket also deletes all associated replications (#22424) 2021-09-09 15:22:36 -04:00
Daniel Moran 1fa0ccf24a
refactor: move interfaces for remotes & replication services out of root package (#22417) 2021-09-07 16:21:29 -04:00
Daniel Moran 12c8fd28d2
feat: implement metadata management for replications (#22302) 2021-09-01 12:01:41 -04:00
Daniel Moran dc3b501298
refactor: rename Lock/Unlock on KV stores to RLock/RUnlock (#22357) 2021-08-31 17:03:54 -04:00
Daniel Moran cc6accf106
chore: delete old-style DBRP mapping (#22339) 2021-08-30 18:27:11 -04:00
William Baker 74eacf63ba
feat: add `--flux-log-enabled` flag for detailed flux logs (#22072)
* feat: add `--flux-log-enabled` flag for detailed flux logs

* feat: set query to unknown if not available
2021-08-30 14:42:05 -06:00
Daniel Moran 6bb95ae6fe
refactor: clean up launcher code (#22330)
* Modify launcher shutdown to reduce field count
* Move tracing setup into its own method
* Move metastore setup into its own method
* Delete unused getter methods
* Fix imports
2021-08-30 14:46:25 -04:00
Daniel Moran 4dd2d7cc7f
fix: hard limit on field size while parsing line protocol (#22311)
Per https://docs.influxdata.com/influxdb/v2.0/reference/syntax/line-protocol/#string
we only support 64KB, but 1MB is a more realistic practical limit. Before this commit there was
no enforcement of field value size.


Co-authored-by: Sam Arnold <sarnold@influxdata.com>
2021-08-27 10:01:17 -04:00
Daniel Moran b37ad79e20
feat: add logging and metrics middlewares to replications API (#22291) 2021-08-24 14:56:56 -04:00
Daniel Moran 641c02f9a8
feat: add APIs for management of replication streams (#22287) 2021-08-24 14:19:03 -04:00
William Baker 467040a072
feat(annotations): add logging and metrics to annotations service (#22271) 2021-08-20 14:28:37 -06:00
William Baker e6c82b455e
feat(notebooks): add logging and metrics to notebooks service (#22266) 2021-08-19 13:22:52 -06:00
Daniel Moran 69988ae68d
feat: add logging and metrics middlewares to remotes API (#22258) 2021-08-18 17:51:40 -04:00
Daniel Moran 4af5c7bf61
feat: implement metadata management for remote connections (#22254) 2021-08-18 15:25:40 -04:00
Daniel Moran 5c7c6c2702
feat: add APIs for management of remote InfluxDB connections (#22219) 2021-08-17 15:15:05 -04:00
Daniel Moran 07d897d2f2
fix: switch flux formatter to one that preserves comments (#22186) 2021-08-12 14:17:18 -04:00
William Baker 3f7b996a10
chore: delete the rest of chronograf (#21998)
* chore: deleted the rest of chronograf

* chore: tidy
2021-08-02 09:02:54 -06:00
Daniel Moran 9d81f4a096
fix: prevent silently dropped writes with overlapping shards (#21946)
Co-authored-by: Geoffrey Wossum <gwossum@influxdata.com>
2021-07-27 12:15:08 -04:00
William Baker e7685fff01
feat: add `ui-disabled` flag to run server with UI disabled (#21910)
* feat: add `ui-disabled` flag to run server with UI disabled

* chore: update CHANGELOG
2021-07-21 19:52:57 -06:00
Faith Chikwekwe 9ff953c400
build(flux): update flux to v0.123.0 (#21879)
* build(flux): update flux to v0.123.0

* fix(cmd/influxd/launcher): modify selector aggregate tests to use create empty false

Co-authored-by: Jonathan A. Sternberg <jonathan@influxdata.com>
2021-07-19 13:20:32 -07:00
Daniel Moran fa1c352132
fix: don't assume bash when displaying interactive CLI prompts (#21839) 2021-07-14 18:12:56 -04:00
William Baker 0cb0da2060
chore: modify/delete some chronograf refs (#21802)
* chore: modify/delete some chronograf refs

* chore: update CHANGELOG
2021-07-08 09:14:03 -05:00
Daniel Moran b1b076f600
refactor: delete old backup/restore CLI commands, replace with dependency on new CLI repo (#21703) 2021-06-17 09:18:55 -04:00
William Baker 235366a603
feat: enable annotations backend (#21702)
* feat: enable annotations backend
2021-06-16 12:41:23 -04:00
William Baker 40c7d69415
feat: use go embed for sqlite migrations (#21653) 2021-06-10 18:40:57 -04:00
Daniel Moran d747e7ec4e
feat: add config parameters to toggle WAL concurrency and timeouts (#21621)
* feat: add context parameter to Take() method on fixed limiter
* refactor: plumb context through to uses of Take()
* test: update tests to pass context as needed
* feat: add config toggles for setting WAL write concurrency & timeout
2021-06-09 11:03:53 -04:00
William Baker 34a4a8e60e
feat: metadata backup returns bucket manifests (#21601)
* fix: properly omit empty time.Time that are optional

* fix: optimize slice creation in manifest fns
2021-06-04 09:26:17 -04:00
William Baker b0ea3b6675
feat: backup and restore sqlite database (#21584)
* feat: new metadata backup endpoint

* feat: added restore/sql API endpoint

* fix: content-type is multipart/mixed, part names are kv and sql

* fix: changed multipart manifest to buckets and made it .json

* feat: added lock for backing up sqlite and bolt dbs

* fix: use read lock instead of write lock on kv during backup

* fix: use filepath.Join for temp dirs
2021-06-02 19:07:53 -04:00
William Baker ed629bfebe
feat(notebooks): notebooks database implementation (#21573) 2021-05-28 09:29:00 -04:00
William Baker c267b31232
chore: move notebook service interface to root (#21570) 2021-05-26 12:27:56 -04:00
William Baker 19b0470ed7
feat: integrated sqlite db for metadata (#21543) 2021-05-25 12:12:07 -04:00
Daniel Moran 00420fb54c
fix(influxql): make meta queries respect query timeout (#21545)
Co-authored-by: davidby-influx <dbyrne@influxdata.com>
2021-05-24 21:10:53 -04:00
William Baker 40897b9a98
feat(notebooks): authorization for notebooks (#21490)
* feat(notebooks): auth changes to top-level authz

* feat(notebooks): added authorization

* chore(notebooks): added tests for authz
2021-05-17 19:01:27 -04:00
William Baker cc4ebe231d
feat: notebooks api handlers & fake store (#21480)
* feat(notebooks): demo store functionality

* feat: demo store with context and middleware

* refactor: without context

* fix: update endpoint and status code for delete

* chore: changed name

* chore: added some tests

* fix: address review comments
2021-05-17 12:07:48 -04:00
Daniel Moran 4343d5ce07
fix: add logging to NATS streaming server (#21375) 2021-05-05 14:25:39 -04:00
Daniel Moran 942f7095e7
fix(query): make config validation for query controller less strict (#21324)
* fix(query): accept queue-size > 0 when concurrency = 0
* fix(influxd): revert defaults for query settings to avoid validation err
* test: lower the default query concurrency used by test launchers
2021-04-28 17:27:37 -04:00
Daniel Moran 91d59d9588
fix(upgrade): ensure query config written by `influxd upgrade` is valid (#21321)
* test: refactor upgrade test to cover the config upgrade
* fix: ensure upgraded query config is valid
2021-04-28 15:41:44 -04:00
William Baker 0106de9fa7
feat: notebooks HTTP handlers (#21316)
* feat: notebooks HTTP handlers
2021-04-28 11:06:13 -04:00
Timmy Luong 0c88854486
feat: add StaticLegend (#21218) 2021-04-15 15:06:01 -07:00
William Baker 1d431e8bc3
feat: structured logging for HTTP server ErrorLog (#21235)
* feat: structured logging for HTTP server ErrorLog

* chore: update CHANGELOG
2021-04-15 17:08:12 -04:00
Sam Arnold 7b2e122869
refactor: refactor for tasks backport (#21117)
* chore: remove dead code

* refactor: move FluxLanguageService interface to fluxlang

* chore: run fmt

* refactor: move task.go from top level to task/taskmodel

* chore: run formatter

* chore: fix up import ordering with gci
2021-04-07 14:42:55 -04:00
Daniel Moran ff6b55aca4
feat(flux): allow values of 0 to disable controller limits (#21127)
Co-authored-by: Sam Arnold <sarnold@influxdata.com>
2021-04-05 16:58:27 -04:00
Daniel Moran 00afd95cb7
refactor: automated move of errors and id from root to kit (#21101)
Co-authored-by: Sam Arnold <sarnold@influxdata.com>
2021-03-30 14:10:02 -04:00
Daniel Moran 3420ed53d3
refactor: modularize backup/restore behavior so it's possible to test using `go test` (#20991) 2021-03-23 07:28:38 -04:00
Daniel Moran c5edd9092f
feat(http): add config parameters for HTTP timeouts (#20971) 2021-03-16 16:54:46 -04:00
Daniel Moran 9922863185
fix(tls): Update TLS strict cipher suite to actually work (#20921) 2021-03-16 10:45:27 -04:00
Daniel Moran 316eaa24b4
feat(http): add config option to disable metrics endpoint in influxd (#20963) 2021-03-15 15:11:23 -04:00
Daniel Moran ba4099ddb9
feat(cmd/influx): allow setting shard-group durations for buckets via API and CLI (#20911)
Co-authored-by: Alexander Savinykh <asavinykh@nbrx.de>
2021-03-11 14:51:03 -05:00
Daniel Moran 49b83b58ee
refactor: remove panics from CLI kit code and influxd CLI setup (#20863) 2021-03-04 17:18:21 -05:00
Daniel Moran 25738db42c
feat(http): allow for disabling pprof (#20827)
And translate pprof-enabled config during upgrade
2021-03-04 09:28:03 -05:00
Jonathan A. Sternberg 719c8495f2
feat(fluxtest): add support for testing/expect planner rules (#20662) 2021-02-01 15:53:05 -06:00
Jonathan A. Sternberg f6669f7512
feat(internal/cmd): add internal command for influxdb flux test harness (#20508)
The internal command can be used to execute flux tests using the same
internals as the `flux test` command, but it will invoke these tests
against a test influxdb instance that is launched in-process.
2021-01-25 10:31:23 -06:00
Daniel Moran e77c4670d6
feat(cmd/influxd): add `nats-max-payload-bytes` config option to influxd (#20564) 2021-01-20 12:44:43 -08:00
Daniel Moran d8a4b4d6cb
feat(cmd/influxd): add `nats-port` config option to influxd server (#20561) 2021-01-20 10:53:56 -08:00
Daniel Moran e970aae778
feat(cmd/influxd): add `print-config` subcommand to support automation (#20524) 2021-01-19 12:34:12 -08:00
Daniel Moran d05a79d13d
fix: ignore a harmless error to avoid a confusing log message (#20448) 2021-01-06 08:00:51 -08:00
Daniel Moran afdcb18655
refactor: simplify how we set the top-level influxd logger (#20374) 2020-12-21 11:15:08 -08:00
Daniel Moran 6ed1922681
fix: don't use a finished context for graceful shutdown (#20355) 2020-12-16 18:56:21 -08:00
Daniel Moran b5bbc503a0
fix(onboarding): don't ignore failures to set initial password (#20317) 2020-12-16 06:43:43 -08:00
Ben Johnson 7dafc2cf34 feat(tsdb): Implement delete with predicate. 2020-12-02 14:55:02 -07:00
Daniel Moran 4235c12d64
refactor(cmd/influxd): parse log-level CLI opts directly to correct type (#20196) 2020-11-30 13:59:06 -05:00
Daniel Moran 81a671894d
refactor(cmd/influxd): separate CLI parsing from core launcher (#20109) 2020-11-30 09:38:27 -05:00
Dan Moran bbb008acd3 refactor: use explicit int64s for remaining query CLI args 2020-11-24 13:36:27 -05:00
Dan Moran 295d89ba22 fix: limit query queue size to an int32 to avoid panic 2020-11-24 13:36:27 -05:00
Dan Moran 4540645276 fix: limit query concurrency quota to an int32 to avoid panic 2020-11-24 13:36:27 -05:00
Stuart Carnie 1cd9d0b04a feat: Port pipeline tests package from Cloud
This package provides essentially the same API as the Cloud tests
package, utilising the `TestLauncher` type.

Additional With functional options were added to the `Launcher`
in order to expose type-safe configuration of InfluxQL configuration.
Additional With options may be added in the future as the need arises.
2020-11-17 08:14:17 +11:00
George MacRorie 5954db8c4d chore(documents): reinstate minimal read-only document store for dashboard templates 2020-11-16 12:41:26 +00:00
George MacRorie 16d916a952 refactor(kv): delete deprecated kv service code
This includes removal of a lot of kv.Service responsibilities. However,
it does not finish the re-wiring. It removes documents, telegrafs,
notification rules + endpoints, checks, orgs, users, buckets, passwords,
urms, labels and authorizations. There are some oustanding pieces that
are needed to get kv service compiling (dashboard service urm
dependency). Then all the call sites for kv service need updating and
the new implementations of telegraf and notification rules + endpoints
needed installing (along with any necessary migrations).
2020-11-16 12:41:26 +00:00
Stuart Carnie fab99c9512 chore: Remove fragile nats port assignment loop
This resolves observed race conditions when running test suites that
utilize the launcher. It also reduces test times considerably, by
eliminating a slow loop to find a port.
2020-11-13 09:13:48 +11:00
Stuart Carnie 42aabb28fc chore: Do not use global viper APIs, which breaks testing 2020-11-13 08:47:52 +11:00
Daniel Moran 518098e4f3
fix(cmd/influxd): suggest running with -h on error instead of printing usage (#19995) 2020-11-11 16:37:36 -05:00
Daniel Moran 15b9531273
fix: correct various typos (#19987)
Co-authored-by: kumakichi <xyesan@gmail.com>
2020-11-11 13:54:21 -05:00
Daniel Moran d8e7de93cc
fix: check write permission in legacy write path (#19980) 2020-11-10 18:41:50 -05:00
Daniel Moran 3317ea0644
fix: enable write-only users to pass auth checks in the V1 API (#19945) 2020-11-09 18:10:35 -05:00
Daniel Moran ca547cfb29
fix: make influxd help more specific (#19939) 2020-11-09 11:45:15 -05:00
Adrian Thurston 30ad7281aa
refactor: replace builtin package with fluxinit/static (#3304) (#19909)
A static initialization is not desirable in the main binaries, as it forces all
paths of code to init, but it is still useful in tests. It allows static
intialization to be performed once for all tests and eliminates the need to
always add the FluxInit call. Added a fluxinit/static package that calls
fluxinit.FluxInit() to replace the builtin package. This hides the nature of
the initialization and makes it clear that it is mandatory initialization code
getting called.
2020-11-05 16:54:28 -08:00
Ben Johnson 419b0cf76b feat: Implement full restore 2020-11-05 10:05:01 -07:00
Ben Johnson 6e1097c625 feat: Minor updates to backup/restore 2020-11-05 10:05:01 -07:00
Ben Johnson 23679c2375 feat: Implement backup/restore CLI subcommands. 2020-11-05 10:05:01 -07:00
George 8ca2989717 feat(dashboards): isolate service in own package (#19852)
feat(dashboard): add owner ID to dashboard model

This adds the explicit OwnerID field to Dashboard and also adds a
migration which populates dashboard owners IDs based on dashboard owner
URMs.

feat(dashboards): isolate service in own package

This change isolates the dashboards service into its own package. It
also updates the API to no longer interface with user resource mappings.
Instead it defines new handlers which rely on the newly populated owner
ID field.

chore(dashboards): port tests from http package into new service transport package

chore(launcher): use dashboard transport package client in launcher tests

chore(kv): remove now defunkt dashboard service implementations
2020-11-04 15:53:02 +00:00
George MacRorie 5feea36e16 refactor(telegraf): remove references to kv service telegraf service implementation 2020-11-04 15:53:02 +00:00
Adrian Thurston 37ef9d2791
fix: use fluxinit package to init flux library instead of builtin (#19887)
* fix: use fluxinit package to init flux library instead of builtin

The builtin package performs costly initialization work in the go init()
function, which causes the work to be performed for all paths of the main
executables that need it, including help screens. This patch uses the fluxinit
package, which requires a call to do the costly work. It allows help screens
and other code paths to execute quickly.

* fix: need to draw in the influxdb-specific standard library in fluxinit
2020-11-03 13:07:41 -08:00
Stuart Carnie a0958e6603 fix: Add Logger to constructor function to ensure it is initialized 2020-11-02 18:00:22 -07:00
jl fbe85ef548 feat: enable window pushdowns 2020-10-30 18:09:38 -07:00
Stuart Carnie 7e1226b298 feat: Use AuthorizerV1 for authorizing 1.x API requests
Setup the AuthorizerV1 in the launcher and pass this to the V1
authentication handler for authorizing 1.x HTTP requests.
2020-10-30 11:56:09 -07:00
Stuart Carnie 6bc4158a46 feat(authorization): Add bcrypt password support to v1 authorizations
This commit extends the `v1/authorization` package to support
passwords associated with a token.

The summary of changes include:

* authorization.Service implements influxdb.PasswordsService
* Setting passwords for authorizations
* Verifying (comparing) passwords for a given authorization
* A service to cache comparing passwords, using a weaker hash
  that will live in memory only. This implementation is copied
  from InfluxDB 1.x
* Extended HTTP service to set a password using
  /private/legacy/authorizations/{id}/password

Closes #
2020-10-28 13:03:42 -07:00
George 78cafa861b
refactor(notifications): isolate endpoint service (#19818)
* refactor(notifications): isolate endpoint service

Following the ongoing effort to isolate behaviours into their own
packages and off of kv.Service, this change move the notification
endpoints service implementation into its own package. It removes the
endpoint behaviors from the kv service completely.

* chore(influxd): wire up the isolated check service in place of kv service
2020-10-28 15:22:14 +00:00
Faith Chikwekwe 5e1cdda597 feat(storage): enable window agg mean pushdown 2020-10-27 12:53:54 -07:00
Stuart Carnie 5c63c2163d feat(authorization): Create a v1 authorization service
This service is a private API for managing authorization tokens
for v1 API requests.

Note that this commit does not hook up the service to the v1
`/query` and `/write`, which will occur in a subsequent PR.

Closes #19812
2020-10-27 08:26:10 -07:00
Paul Hummer a2dbb572fe
Merge pull request #19826 from influxdata/flux-staging/v0.91.0
build(flux): update Flux to v0.91.0
2020-10-27 09:05:33 -06:00
George 3d643e0681
refactor(notification): move rule service into own package (#19804)
* refactor(notification): move rule service into own package

* chore(launcher): fix tests to use clients as opposed to direct kv service

* chore(influx): update task cli to consume core domain model task from client

* chore(kv): remove rule service behaviours from kv

This also introduces the org id resolver type. Which is transplanted
from the kv service. As this one function coupled all resource
capabilities onto the kv service. Making removing these capabilities
impossible. Moving this type out into its own package which depends on
each service explicitly ensures we don't have one type which has to
implement all the service contracts.

* fix(launcher): remove double reference to influxdb package
2020-10-27 11:45:05 +00:00
Adrian Thurston 7a4b81bf3d fix: profiler testing: stub GetSortedResult from execute.Profile
This function is used by the end-to-end test harness to stabilize query profile
results before diff and is needed when implementing the Profile interface.
2020-10-26 14:39:30 -07:00
Stuart Carnie 2705c57554 fix(launcher): Switch to AuthorizationService from authorization package
This commit ensures OSS is using the new implementation of the
AuthorizationService from the authorization package.

It also removes the associated feature flag.
2020-10-21 07:44:15 -07:00
Paul Hummer 4da32637ed build(flux): update Flux to v0.89.0 2020-10-14 10:51:01 -06:00