Commit Graph

296 Commits (db/update-protos)

Author SHA1 Message Date
William Baker 19b0470ed7
feat: integrated sqlite db for metadata (#21543) 2021-05-25 12:12:07 -04:00
William Baker 40897b9a98
feat(notebooks): authorization for notebooks (#21490)
* feat(notebooks): auth changes to top-level authz

* feat(notebooks): added authorization

* chore(notebooks): added tests for authz
2021-05-17 19:01:27 -04:00
William Baker cc4ebe231d
feat: notebooks api handlers & fake store (#21480)
* feat(notebooks): demo store functionality

* feat: demo store with context and middleware

* refactor: without context

* fix: update endpoint and status code for delete

* chore: changed name

* chore: added some tests

* fix: address review comments
2021-05-17 12:07:48 -04:00
Daniel Moran 4343d5ce07
fix: add logging to NATS streaming server (#21375) 2021-05-05 14:25:39 -04:00
Daniel Moran 91d59d9588
fix(upgrade): ensure query config written by `influxd upgrade` is valid (#21321)
* test: refactor upgrade test to cover the config upgrade
* fix: ensure upgraded query config is valid
2021-04-28 15:41:44 -04:00
William Baker 0106de9fa7
feat: notebooks HTTP handlers (#21316)
* feat: notebooks HTTP handlers
2021-04-28 11:06:13 -04:00
William Baker 1d431e8bc3
feat: structured logging for HTTP server ErrorLog (#21235)
* feat: structured logging for HTTP server ErrorLog

* chore: update CHANGELOG
2021-04-15 17:08:12 -04:00
Sam Arnold 7b2e122869
refactor: refactor for tasks backport (#21117)
* chore: remove dead code

* refactor: move FluxLanguageService interface to fluxlang

* chore: run fmt

* refactor: move task.go from top level to task/taskmodel

* chore: run formatter

* chore: fix up import ordering with gci
2021-04-07 14:42:55 -04:00
Daniel Moran ff6b55aca4
feat(flux): allow values of 0 to disable controller limits (#21127)
Co-authored-by: Sam Arnold <sarnold@influxdata.com>
2021-04-05 16:58:27 -04:00
Daniel Moran 00afd95cb7
refactor: automated move of errors and id from root to kit (#21101)
Co-authored-by: Sam Arnold <sarnold@influxdata.com>
2021-03-30 14:10:02 -04:00
Daniel Moran c5edd9092f
feat(http): add config parameters for HTTP timeouts (#20971) 2021-03-16 16:54:46 -04:00
Daniel Moran 9922863185
fix(tls): Update TLS strict cipher suite to actually work (#20921) 2021-03-16 10:45:27 -04:00
Daniel Moran 316eaa24b4
feat(http): add config option to disable metrics endpoint in influxd (#20963) 2021-03-15 15:11:23 -04:00
Daniel Moran 25738db42c
feat(http): allow for disabling pprof (#20827)
And translate pprof-enabled config during upgrade
2021-03-04 09:28:03 -05:00
Jonathan A. Sternberg 719c8495f2
feat(fluxtest): add support for testing/expect planner rules (#20662) 2021-02-01 15:53:05 -06:00
Daniel Moran e77c4670d6
feat(cmd/influxd): add `nats-max-payload-bytes` config option to influxd (#20564) 2021-01-20 12:44:43 -08:00
Daniel Moran d8a4b4d6cb
feat(cmd/influxd): add `nats-port` config option to influxd server (#20561) 2021-01-20 10:53:56 -08:00
Daniel Moran d05a79d13d
fix: ignore a harmless error to avoid a confusing log message (#20448) 2021-01-06 08:00:51 -08:00
Daniel Moran afdcb18655
refactor: simplify how we set the top-level influxd logger (#20374) 2020-12-21 11:15:08 -08:00
Daniel Moran 6ed1922681
fix: don't use a finished context for graceful shutdown (#20355) 2020-12-16 18:56:21 -08:00
Daniel Moran b5bbc503a0
fix(onboarding): don't ignore failures to set initial password (#20317) 2020-12-16 06:43:43 -08:00
Daniel Moran 4235c12d64
refactor(cmd/influxd): parse log-level CLI opts directly to correct type (#20196) 2020-11-30 13:59:06 -05:00
Daniel Moran 81a671894d
refactor(cmd/influxd): separate CLI parsing from core launcher (#20109) 2020-11-30 09:38:27 -05:00
Dan Moran bbb008acd3 refactor: use explicit int64s for remaining query CLI args 2020-11-24 13:36:27 -05:00
Dan Moran 295d89ba22 fix: limit query queue size to an int32 to avoid panic 2020-11-24 13:36:27 -05:00
Dan Moran 4540645276 fix: limit query concurrency quota to an int32 to avoid panic 2020-11-24 13:36:27 -05:00
Stuart Carnie 1cd9d0b04a feat: Port pipeline tests package from Cloud
This package provides essentially the same API as the Cloud tests
package, utilising the `TestLauncher` type.

Additional With functional options were added to the `Launcher`
in order to expose type-safe configuration of InfluxQL configuration.
Additional With options may be added in the future as the need arises.
2020-11-17 08:14:17 +11:00
George MacRorie 5954db8c4d chore(documents): reinstate minimal read-only document store for dashboard templates 2020-11-16 12:41:26 +00:00
George MacRorie 16d916a952 refactor(kv): delete deprecated kv service code
This includes removal of a lot of kv.Service responsibilities. However,
it does not finish the re-wiring. It removes documents, telegrafs,
notification rules + endpoints, checks, orgs, users, buckets, passwords,
urms, labels and authorizations. There are some oustanding pieces that
are needed to get kv service compiling (dashboard service urm
dependency). Then all the call sites for kv service need updating and
the new implementations of telegraf and notification rules + endpoints
needed installing (along with any necessary migrations).
2020-11-16 12:41:26 +00:00
Stuart Carnie fab99c9512 chore: Remove fragile nats port assignment loop
This resolves observed race conditions when running test suites that
utilize the launcher. It also reduces test times considerably, by
eliminating a slow loop to find a port.
2020-11-13 09:13:48 +11:00
Stuart Carnie 42aabb28fc chore: Do not use global viper APIs, which breaks testing 2020-11-13 08:47:52 +11:00
Daniel Moran 518098e4f3
fix(cmd/influxd): suggest running with -h on error instead of printing usage (#19995) 2020-11-11 16:37:36 -05:00
Daniel Moran d8e7de93cc
fix: check write permission in legacy write path (#19980) 2020-11-10 18:41:50 -05:00
Daniel Moran 3317ea0644
fix: enable write-only users to pass auth checks in the V1 API (#19945) 2020-11-09 18:10:35 -05:00
Daniel Moran ca547cfb29
fix: make influxd help more specific (#19939) 2020-11-09 11:45:15 -05:00
Ben Johnson 23679c2375 feat: Implement backup/restore CLI subcommands. 2020-11-05 10:05:01 -07:00
George 8ca2989717 feat(dashboards): isolate service in own package (#19852)
feat(dashboard): add owner ID to dashboard model

This adds the explicit OwnerID field to Dashboard and also adds a
migration which populates dashboard owners IDs based on dashboard owner
URMs.

feat(dashboards): isolate service in own package

This change isolates the dashboards service into its own package. It
also updates the API to no longer interface with user resource mappings.
Instead it defines new handlers which rely on the newly populated owner
ID field.

chore(dashboards): port tests from http package into new service transport package

chore(launcher): use dashboard transport package client in launcher tests

chore(kv): remove now defunkt dashboard service implementations
2020-11-04 15:53:02 +00:00
George MacRorie 5feea36e16 refactor(telegraf): remove references to kv service telegraf service implementation 2020-11-04 15:53:02 +00:00
Adrian Thurston 37ef9d2791
fix: use fluxinit package to init flux library instead of builtin (#19887)
* fix: use fluxinit package to init flux library instead of builtin

The builtin package performs costly initialization work in the go init()
function, which causes the work to be performed for all paths of the main
executables that need it, including help screens. This patch uses the fluxinit
package, which requires a call to do the costly work. It allows help screens
and other code paths to execute quickly.

* fix: need to draw in the influxdb-specific standard library in fluxinit
2020-11-03 13:07:41 -08:00
Stuart Carnie a0958e6603 fix: Add Logger to constructor function to ensure it is initialized 2020-11-02 18:00:22 -07:00
Stuart Carnie 7e1226b298 feat: Use AuthorizerV1 for authorizing 1.x API requests
Setup the AuthorizerV1 in the launcher and pass this to the V1
authentication handler for authorizing 1.x HTTP requests.
2020-10-30 11:56:09 -07:00
Stuart Carnie 6bc4158a46 feat(authorization): Add bcrypt password support to v1 authorizations
This commit extends the `v1/authorization` package to support
passwords associated with a token.

The summary of changes include:

* authorization.Service implements influxdb.PasswordsService
* Setting passwords for authorizations
* Verifying (comparing) passwords for a given authorization
* A service to cache comparing passwords, using a weaker hash
  that will live in memory only. This implementation is copied
  from InfluxDB 1.x
* Extended HTTP service to set a password using
  /private/legacy/authorizations/{id}/password

Closes #
2020-10-28 13:03:42 -07:00
George 78cafa861b
refactor(notifications): isolate endpoint service (#19818)
* refactor(notifications): isolate endpoint service

Following the ongoing effort to isolate behaviours into their own
packages and off of kv.Service, this change move the notification
endpoints service implementation into its own package. It removes the
endpoint behaviors from the kv service completely.

* chore(influxd): wire up the isolated check service in place of kv service
2020-10-28 15:22:14 +00:00
Stuart Carnie 5c63c2163d feat(authorization): Create a v1 authorization service
This service is a private API for managing authorization tokens
for v1 API requests.

Note that this commit does not hook up the service to the v1
`/query` and `/write`, which will occur in a subsequent PR.

Closes #19812
2020-10-27 08:26:10 -07:00
George 3d643e0681
refactor(notification): move rule service into own package (#19804)
* refactor(notification): move rule service into own package

* chore(launcher): fix tests to use clients as opposed to direct kv service

* chore(influx): update task cli to consume core domain model task from client

* chore(kv): remove rule service behaviours from kv

This also introduces the org id resolver type. Which is transplanted
from the kv service. As this one function coupled all resource
capabilities onto the kv service. Making removing these capabilities
impossible. Moving this type out into its own package which depends on
each service explicitly ensures we don't have one type which has to
implement all the service contracts.

* fix(launcher): remove double reference to influxdb package
2020-10-27 11:45:05 +00:00
Stuart Carnie 2705c57554 fix(launcher): Switch to AuthorizationService from authorization package
This commit ensures OSS is using the new implementation of the
AuthorizationService from the authorization package.

It also removes the associated feature flag.
2020-10-21 07:44:15 -07:00
Christopher M. Wolff e7cbbaa722
feat: upgrade Flux to v0.83.2 (#19569) 2020-09-16 17:59:15 -07:00
Gershon Shif af0c328095
chore(packaging): build rpm and deb packages (#19567)
- Update CIrcleCI configuration to start release process on an RC build
- Update .goreleaser.yml:
   - Start building armel and armhf binaries and rpm and debian packages.
   - Generate sha256 checksum file.
- launcher.go: do not use `max` module to escape integeroverflow problem for armel and armhf builds
- Start using `v0.142.0` of goreleaser
- Added pre and post install/uninstall scripts for rpm amd deb packages
2020-09-16 12:36:44 -07:00
Stuart Carnie 6f0cf049ca feat: Port shard precreation service from InfluxDB 1.x
Provides new configuration parameters:

```
--storage-shard-precreator-advance-period
--storage-shard-precreator-check-interval
```

Closes #19520
2020-09-09 13:34:22 -07:00
Stuart Carnie e265f60b55 fix: Improvements in response to PR feedback
* Pass context.Context to Service.Open
* Remove redundant comments
* Bind to retention.Config configuration to be consistent with 1.x
2020-09-09 10:32:06 -07:00
Stuart Carnie 7c2be6e780 feat: Port 1.x retention policy enforcement service
Configuration of the check interval is available via

```
--storage-retention-check-interval
```

Closes #19309
2020-09-09 10:32:06 -07:00
Stuart Carnie feb2596f3a feat: Expose a subset of InfluxQL coordinator tuning options
Closes #19507
2020-09-05 09:44:34 -07:00
Stuart Carnie 111f39c8b1 feat: Add TSM 1.x storage options as flags
Closes #19505
2020-09-04 15:56:57 -07:00
Stuart Carnie 4266eb5935 chore: Remove page-fault-rate flag, which is not available 2020-09-04 15:56:57 -07:00
Stuart Carnie 7964f86ffc
fix: switch port to 8086
fix(readme): update the port and setup
2020-09-03 14:40:29 -07:00
Stuart Carnie a7239c007e
fix: separate flag to always permit initial user setup
This is not required for e2e tests
2020-08-31 16:38:54 -07:00
Stuart Carnie a607ac30b9
chore: Re-enable `TemporaryEngine` via `--e2e-testing` 2020-08-31 13:50:20 -07:00
Stuart Carnie dee8977d2c
chore: move v2/v1/tsdb → v2/tsdb 2020-08-26 10:46:47 -07:00
Stuart Carnie 7ec178f54c
feat(influxd): Perform checks to determine if prior version exists
Closes #19408
2020-08-24 15:40:25 -07:00
Stuart Carnie 8a7dcc21fb
feat(influxql): Initial implementation of InfluxQL query engine
* Some tests are broken or do not pass; follow up PRs will resolve that
2020-08-14 12:37:30 -07:00
Stuart Carnie 057285e23a
Merge remote-tracking branch 'origin/master' into sgc/tsm1
# Conflicts:
#	query/logging.go
#	query/logging_test.go
#	query/stdlib/influxdata/influxdb/rules.go
#	query/stdlib/influxdata/influxdb/rules_test.go
#	query/stdlib/testing/testing.go
#	storage/flux/reader.go
#	storage/flux/table.gen.go
#	storage/flux/table.gen.go.tmpl
#	storage/flux/table.go
#	storage/flux/table_test.go
2020-08-11 10:25:01 -07:00
George 5e22e0101d
chore(auth): remove unnecessary URM service from Bucket authorizer (#19060) 2020-08-04 14:37:14 +01:00
Stuart Carnie 49067de49f
fix(launcher): Fix launcher tests 2020-08-03 15:17:37 -07:00
Stuart Carnie 262023ae4a
fix(launcher): influxd inspect commands now start instantly
Lazily initialize flux, which increases startup time considerably
2020-08-03 15:16:36 -07:00
Stuart Carnie 21c3733464
fix(storage): meta.Client persists to kv and updates via BucketService
Includes initial migration to add metadata bucket
2020-08-03 09:21:02 -07:00
Stuart Carnie 7c225cb3ac
chore: More cleanup 2020-08-03 09:20:50 -07:00
Stuart Carnie 26c18ac0a5
chore: Converging on 1.x tsdb 2020-08-03 09:20:48 -07:00
Edd Robinson 2b175291be
refactor: WIP removing tsbd 2020-08-03 09:18:34 -07:00
Stuart Carnie a97ec1f34e
feat: Initial commit with working reads 2020-08-03 09:18:34 -07:00
Stuart Carnie e3060c291c
refactor: tsdb store builds an runs 2020-08-03 09:18:32 -07:00
Lyon Hill 0a9e8fdb4a
chore: update task tests to use the tenant service (#19169)
* chore: update task tests to use the tenant service

After the introduction of the tenant system we need to switch the testing frameworks
to use it instead of the old kv system

* chore: update onboarding to allow injected middleware
2020-08-03 09:33:46 -06:00
Alirie Gray d1b8e98798
refactor: remove nested orgs labels route (#19104) 2020-07-30 10:53:14 -07:00
Johnny Steenbergen a652cdf552 chore(pkger): axe deprecated /packges route tree 2020-07-24 09:33:30 -07:00
Ben Johnson c28eb70856 feat(mincore): Add page fault limiter
This commit adds `mincore.Limiter` which throttles page faults caused
by mmap() data. It works by periodically calling `mincore()` to determine
which pages are not resident in memory and using `rate.Limiter` to
throttle accessing using a token bucket algorithm.
2020-07-17 09:37:31 -06:00
Lyon Hill a31b651057
feat(task): Add new permission lookup pattern for executor (#18920)
* feat(task): Add new permission lookup pattern for executor

We can now use the user service to populate task owners permissions.
This should improve the task lookup time and decouple the task system
from the URM system. In the future we will have the ability to better isolate
tenant pieces from the rest of the service.

* feat: add feature flagging
2020-07-13 14:47:58 -06:00
George b8f5604613
revert(tasks): Add new permission lookup pattern for executor (#18869) (#18915)
This reverts commit e9e4d794cf.
2020-07-10 11:05:07 +01:00
Lyon Hill e9e4d794cf
feat(task): Add new permission lookup pattern for executor (#18869)
We can now use the user service to populate task owners permissions.
This should improve the task lookup time and decouple the task system
from the URM system. In the future we will have the ability to better isolate
tenant pieces from the rest of the service.
2020-07-09 14:06:46 -06:00
Alirie Gray e02e950ed0
refactor: add new tenant system struct to consolidate services (#18833) 2020-07-06 10:49:51 -07:00
Ben Johnson 6b0b4d7c4e fix(storage): Refactor LoggingPointsWriter to use BucketFinder
This commit changes the writer to use `storage.BucketFinder` which
is a subset of the `influxdb.BucketService` interface.
2020-07-02 12:46:43 -06:00
George 4a6bc1d74f
chore: remove kv Session service and sessionsv1 bucket (#18823)
* chore: remove sessionsv1 bucket

* chore: add extra configuration to sessions.Service
2020-07-02 15:15:08 +01:00
Ben Johnson 482442ecc1 feat(storage): Add logging points writer
This commit adds a LoggingPointsWriter which wraps a PointsWriter
and provides logging when a write fails.
2020-07-01 11:41:29 -06:00
George 96d84b9126
refactor: migrator and introduce Store.(Create|Delete)Bucket (#18570)
* refactor: migrator and introduce Store.(Create|Delete)Bucket

feat: kvmigration internal utility to create / managing kv store migrations

fix: ensure migrations applied in all test cases

* chore: update kv and migration documentation
2020-07-01 12:08:20 +01:00
Lyon Hill 5776350a53
feat: switch to use the new tenant bucket service (#18738) 2020-06-30 14:48:42 -06:00
Johnny Steenbergen 2927f0e718 feat(pkger): add stable HTTP APIs for stacks and templates
references: #18580
2020-06-29 13:45:32 -07:00
Johnny Steenbergen 2825427cf5 chore(pkger): convert packages endpoints into new naming convention
note: going to make this evolution in two steps so that we have a simple
rollback to get back to a working state. We'll be maintaining both packages
and the new templates and stacks endpoints for a while as users start to
move onto a newer CLI version. Sunsetting by end of July.

references: #18580
2020-06-29 09:45:58 -07:00
Alirie Gray b35479640a
refactor: move checks service to new package (#18671) 2020-06-26 12:52:15 -07:00
Lyon Hill 8522e19b6e
feat: add a embedded handler for label service api (#18745)
We will now be able to embed the label mappings apis into other services
2020-06-25 17:10:06 -06:00
David McKay 67fad644f2 fix: remove redundant break statements 2020-06-25 18:17:24 +01:00
David McKay 79431d1c46 feat: allow restricting TLS version and ciphers 2020-06-25 18:17:24 +01:00
Brett Buddin 1f422b9b94
fix(dbrp): Delete DBRP mappings when their target Bucket is deleted. (#18699) 2020-06-24 13:40:54 -04:00
Alirie Gray 5d3b9e2001
refactor: move name lookup to auth http (#18643) 2020-06-22 16:14:48 -07:00
Johnny Steenbergen 81490d6604 feat(influx): fixup config file env var naming and add support for config directory 2020-06-18 17:09:46 -07:00
Johnny Steenbergen 84723e15d2 feat(influxd): refactor run command to use cli.Program to respect config file 2020-06-18 15:59:53 -07:00
Lyon Hill 3987d3692b
chore: Clean out old session store feature flag (#18596) 2020-06-18 12:18:34 -06:00
Adrian Thurston 5d1a759170
feat: annotate context with feature flags when handling flux queries (#18506)
Annotate the context with feature flags when handling flux queries in influxdb.
Taking advantage of this in flux end-to-end tests. Using a custom flagger that
can set overrides based on the test case that is about to be run, allowing us
to enable features in the end-to-end tests.
2020-06-17 16:55:29 +03:00
Lyon Hill 009b8ec98f
feat: start using the new org handler from the tenant service. (#18371)
* feat: start using the new org handler from the tenant service.

The rest of the tenant system is in place except the org http api handler and the
user api handler.

* fix: update the label service in org handler and add links
2020-06-16 12:56:31 -06:00
Jonathan A. Sternberg 36b0a90b2c
tests(launcher): add a launcher test for read window aggregate push down (#18412)
This adds a launcher test for the read window aggregate push down to
verify that it is done when a query is sent with the appropriate
pattern, the output is correct, and that the metric is incremented that
signals the push down happened.
2020-06-09 15:20:54 -05:00
Alirie Gray 03ba8f250a
refactor: remove urms from new label service (#18301) 2020-06-01 09:28:03 -07:00
Alirie Gray 876238d688
feat: add feature flag for new labels service (#18215) 2020-05-28 08:26:08 -07:00
jlapacik d22380dc8b
Merge pull request #17900 from influxdata/feat/use-algo-w
chore: promote algorithm W to master
2020-05-27 10:20:03 -07:00