Commit Graph

648 Commits (22f4f5bba80420e578389b3c82f9619530671015)

Author SHA1 Message Date
Daniel Moran 1058a8e3f1
fix(upgrade): add DB URL to log message at end up upgrade (#20274) 2020-12-08 11:12:31 -05:00
Ben Johnson 7dafc2cf34 feat(tsdb): Implement delete with predicate. 2020-12-02 14:55:02 -07:00
Daniel Moran 4235c12d64
refactor(cmd/influxd): parse log-level CLI opts directly to correct type (#20196) 2020-11-30 13:59:06 -05:00
Daniel Moran 81a671894d
refactor(cmd/influxd): separate CLI parsing from core launcher (#20109) 2020-11-30 09:38:27 -05:00
Dan Moran bbb008acd3 refactor: use explicit int64s for remaining query CLI args 2020-11-24 13:36:27 -05:00
Dan Moran 295d89ba22 fix: limit query queue size to an int32 to avoid panic 2020-11-24 13:36:27 -05:00
Dan Moran 4540645276 fix: limit query concurrency quota to an int32 to avoid panic 2020-11-24 13:36:27 -05:00
Daniel Moran 89764adabd
feat(cmd/influxd): use a better default for upgraded V2 config, and allow users to override it (#20110) 2020-11-23 13:05:48 -05:00
Daniel Moran 1e324b6a63
fix(cmd/influxd): remove internal subcommands of `upgrade` from help text (#20076) 2020-11-17 15:37:43 -05:00
Daniel Moran 632e8f11ed
fix(cmd/influxd): don't validate unused paths in `upgrade` (#20052) 2020-11-17 00:38:53 -05:00
Daniel Moran b732f3663c
feat(cmd/influxd): export 1.x continuous queries as part of upgrade (#20039) 2020-11-16 16:37:47 -05:00
Stuart Carnie 1cd9d0b04a feat: Port pipeline tests package from Cloud
This package provides essentially the same API as the Cloud tests
package, utilising the `TestLauncher` type.

Additional With functional options were added to the `Launcher`
in order to expose type-safe configuration of InfluxQL configuration.
Additional With options may be added in the future as the need arises.
2020-11-17 08:14:17 +11:00
Daniel Moran 693d153195
feat(cmd/influxd): warn if auth wasn't enabled and users were upgraded (#20036) 2020-11-16 09:05:40 -05:00
George MacRorie 5954db8c4d chore(documents): reinstate minimal read-only document store for dashboard templates 2020-11-16 12:41:26 +00:00
George MacRorie 16d916a952 refactor(kv): delete deprecated kv service code
This includes removal of a lot of kv.Service responsibilities. However,
it does not finish the re-wiring. It removes documents, telegrafs,
notification rules + endpoints, checks, orgs, users, buckets, passwords,
urms, labels and authorizations. There are some oustanding pieces that
are needed to get kv service compiling (dashboard service urm
dependency). Then all the call sites for kv service need updating and
the new implementations of telegraf and notification rules + endpoints
needed installing (along with any necessary migrations).
2020-11-16 12:41:26 +00:00
Daniel Moran 4efb8515b8
fix(cmd/influxd): validate all input paths to `upgrade` up-front (#20012) 2020-11-13 09:25:57 -05:00
Stuart Carnie fab99c9512 chore: Remove fragile nats port assignment loop
This resolves observed race conditions when running test suites that
utilize the launcher. It also reduces test times considerably, by
eliminating a slow loop to find a port.
2020-11-13 09:13:48 +11:00
Stuart Carnie aa8064a43a fix: PR Feedback 2020-11-13 08:47:52 +11:00
Stuart Carnie 42aabb28fc chore: Do not use global viper APIs, which breaks testing 2020-11-13 08:47:52 +11:00
Daniel Moran 518098e4f3
fix(cmd/influxd): suggest running with -h on error instead of printing usage (#19995) 2020-11-11 16:37:36 -05:00
Daniel Moran 15b9531273
fix: correct various typos (#19987)
Co-authored-by: kumakichi <xyesan@gmail.com>
2020-11-11 13:54:21 -05:00
Daniel Moran d8e7de93cc
fix: check write permission in legacy write path (#19980) 2020-11-10 18:41:50 -05:00
alespour 2982701d01
fix(upgrade): add log-level CLI option (#19967)
Co-authored-by: Daniel Moran <danxmoran@gmail.com>
2020-11-10 17:23:29 -05:00
alespour c4eb6290f0
fix(upgrade): check for existing 2.x CLI configs file during upgrade (#19969)
Co-authored-by: Daniel Moran <danxmoran@gmail.com>
2020-11-10 16:42:51 -05:00
alespour 63974e8fe7
fix(upgrade): remove obsolete unused option (influx-command-path) (#19972) 2020-11-10 15:37:38 -05:00
Daniel Moran b643482c45
fix(upgrade): use 10 instead of MaxInt when rewriting query-concurrency (#19959) 2020-11-09 19:00:30 -05:00
Russ Savage cae539055f
fix: updating v1 auth description (#19947)
* fix: updating v1 auth description

* fix: fixing tests

* chore: renaming to legacy
2020-11-09 15:10:46 -08:00
Daniel Moran 3317ea0644
fix: enable write-only users to pass auth checks in the V1 API (#19945) 2020-11-09 18:10:35 -05:00
alespour 07e009c616
fix(upgrade): create CLI configs (#19937)
Co-authored-by: docmerlin (j. Emrys Landivar) <landivar@gmail.com>
Co-authored-by: Daniel Moran <danxmoran@gmail.com>
2020-11-09 17:43:28 -05:00
Daniel Moran b4aa582750
fix(upgrade): use db/rp naming convention when upgrading DBs (#19952) 2020-11-09 17:19:04 -05:00
Daniel Moran 040f5b208a
fix: replace 0 with MaxInt when upgrading query-concurrency (#19930) 2020-11-09 12:01:09 -05:00
Daniel Moran ca547cfb29
fix: make influxd help more specific (#19939) 2020-11-09 11:45:15 -05:00
Daniel Moran ee390ddfd3
fix: parsing of retention policy CLI args in `influx setup` and `influxd upgrade` (#19928) 2020-11-06 17:50:32 -05:00
Daniel Moran a89041b182
fix(cmd/influxd/upgrade): delete references to 'security script' in upgrade command (#19924) 2020-11-06 15:32:38 -05:00
Adrian Thurston 30ad7281aa
refactor: replace builtin package with fluxinit/static (#3304) (#19909)
A static initialization is not desirable in the main binaries, as it forces all
paths of code to init, but it is still useful in tests. It allows static
intialization to be performed once for all tests and eliminates the need to
always add the FluxInit call. Added a fluxinit/static package that calls
fluxinit.FluxInit() to replace the builtin package. This hides the nature of
the initialization and makes it clear that it is mandatory initialization code
getting called.
2020-11-05 16:54:28 -08:00
Ben Johnson 419b0cf76b feat: Implement full restore 2020-11-05 10:05:01 -07:00
Ben Johnson 6e1097c625 feat: Minor updates to backup/restore 2020-11-05 10:05:01 -07:00
Ben Johnson 23679c2375 feat: Implement backup/restore CLI subcommands. 2020-11-05 10:05:01 -07:00
George 8ca2989717 feat(dashboards): isolate service in own package (#19852)
feat(dashboard): add owner ID to dashboard model

This adds the explicit OwnerID field to Dashboard and also adds a
migration which populates dashboard owners IDs based on dashboard owner
URMs.

feat(dashboards): isolate service in own package

This change isolates the dashboards service into its own package. It
also updates the API to no longer interface with user resource mappings.
Instead it defines new handlers which rely on the newly populated owner
ID field.

chore(dashboards): port tests from http package into new service transport package

chore(launcher): use dashboard transport package client in launcher tests

chore(kv): remove now defunkt dashboard service implementations
2020-11-04 15:53:02 +00:00
George MacRorie 5feea36e16 refactor(telegraf): remove references to kv service telegraf service implementation 2020-11-04 15:53:02 +00:00
Adrian Thurston 37ef9d2791
fix: use fluxinit package to init flux library instead of builtin (#19887)
* fix: use fluxinit package to init flux library instead of builtin

The builtin package performs costly initialization work in the go init()
function, which causes the work to be performed for all paths of the main
executables that need it, including help screens. This patch uses the fluxinit
package, which requires a call to do the costly work. It allows help screens
and other code paths to execute quickly.

* fix: need to draw in the influxdb-specific standard library in fluxinit
2020-11-03 13:07:41 -08:00
Dan Moran 428a46b75e fix: remove division of RP in upgrade command
DB-side services no longer multiply requested RPs by time.Hour, so
the upgrade command shouldn't divide by that factor.
2020-11-03 12:37:50 -05:00
Stuart Carnie a0958e6603 fix: Add Logger to constructor function to ensure it is initialized 2020-11-02 18:00:22 -07:00
Ales Pour 0853370cda refactor: use v1 auth service to create tokens 2020-11-02 09:06:34 -07:00
jl fbe85ef548 feat: enable window pushdowns 2020-10-30 18:09:38 -07:00
Stuart Carnie 7e1226b298 feat: Use AuthorizerV1 for authorizing 1.x API requests
Setup the AuthorizerV1 in the launcher and pass this to the V1
authentication handler for authorizing 1.x HTTP requests.
2020-10-30 11:56:09 -07:00
Christopher M. Wolff 5a60488729
fix: make tagKeys and tagValues work for edge cases involving _field (#19856) 2020-10-30 10:37:08 -07:00
Stuart Carnie 6bc4158a46 feat(authorization): Add bcrypt password support to v1 authorizations
This commit extends the `v1/authorization` package to support
passwords associated with a token.

The summary of changes include:

* authorization.Service implements influxdb.PasswordsService
* Setting passwords for authorizations
* Verifying (comparing) passwords for a given authorization
* A service to cache comparing passwords, using a weaker hash
  that will live in memory only. This implementation is copied
  from InfluxDB 1.x
* Extended HTTP service to set a password using
  /private/legacy/authorizations/{id}/password

Closes #
2020-10-28 13:03:42 -07:00
George 78cafa861b
refactor(notifications): isolate endpoint service (#19818)
* refactor(notifications): isolate endpoint service

Following the ongoing effort to isolate behaviours into their own
packages and off of kv.Service, this change move the notification
endpoints service implementation into its own package. It removes the
endpoint behaviors from the kv service completely.

* chore(influxd): wire up the isolated check service in place of kv service
2020-10-28 15:22:14 +00:00
Faith Chikwekwe 5e1cdda597 feat(storage): enable window agg mean pushdown 2020-10-27 12:53:54 -07:00
Stuart Carnie 5c63c2163d feat(authorization): Create a v1 authorization service
This service is a private API for managing authorization tokens
for v1 API requests.

Note that this commit does not hook up the service to the v1
`/query` and `/write`, which will occur in a subsequent PR.

Closes #19812
2020-10-27 08:26:10 -07:00
Paul Hummer a2dbb572fe
Merge pull request #19826 from influxdata/flux-staging/v0.91.0
build(flux): update Flux to v0.91.0
2020-10-27 09:05:33 -06:00
George 3d643e0681
refactor(notification): move rule service into own package (#19804)
* refactor(notification): move rule service into own package

* chore(launcher): fix tests to use clients as opposed to direct kv service

* chore(influx): update task cli to consume core domain model task from client

* chore(kv): remove rule service behaviours from kv

This also introduces the org id resolver type. Which is transplanted
from the kv service. As this one function coupled all resource
capabilities onto the kv service. Making removing these capabilities
impossible. Moving this type out into its own package which depends on
each service explicitly ensures we don't have one type which has to
implement all the service contracts.

* fix(launcher): remove double reference to influxdb package
2020-10-27 11:45:05 +00:00
Adrian Thurston 7a4b81bf3d fix: profiler testing: stub GetSortedResult from execute.Profile
This function is used by the end-to-end test harness to stabilize query profile
results before diff and is needed when implementing the Profile interface.
2020-10-26 14:39:30 -07:00
Gershon Shif ce00f0d226
fix(build): building arm 6 (#19756) 2020-10-21 16:29:48 -07:00
vlastahajek 7947a7fffd fix(upgrade): improving upgrade help 2020-10-21 07:58:32 -07:00
Stuart Carnie 2705c57554 fix(launcher): Switch to AuthorizationService from authorization package
This commit ensures OSS is using the new implementation of the
AuthorizationService from the authorization package.

It also removes the associated feature flag.
2020-10-21 07:44:15 -07:00
Ales Pour 6a43939546 fix: proper default 2.x config filename 2020-10-20 08:03:34 -07:00
Ales Pour 2b828c5b8c fix: fallback to user's home 2020-10-19 12:35:58 -07:00
Paul Hummer 155fa72b34
Merge pull request #19731 from rockstar/flux-staging/v0.89.0
build(flux): update Flux to v0.89.0
2020-10-14 11:26:56 -06:00
Paul Hummer 4da32637ed build(flux): update Flux to v0.89.0 2020-10-14 10:51:01 -06:00
Ales Pour 8a0068546f style: better option descrition 2020-10-14 08:02:30 -07:00
Ales Pour 548eec197d refactor: influx 2.x path is a commamnd line option 2020-10-14 08:02:30 -07:00
Ales Pour 2308e6c50e feat(upgrade): upgrade security 2020-10-14 08:02:30 -07:00
vlastahajek 8763bb1af9 feat(upgrade): upgrade databases 2020-10-08 09:39:30 -07:00
Ales Pour 1e4650913d feat(upgrade): upgrade config file 2020-10-02 12:01:35 -07:00
vlastahajek 999660ae2e feat(upgrade): upgrade command basic skeleton 2020-10-01 08:06:42 -07:00
Stuart Carnie 78d0262217 fix: Remove incorrect implementation of delete APIs and disable HTTP API
This commit removes incorrect implementations of the `DeleteBucket`
and `DeleteBucketRangePredicate` APIs from the storage package,
which remained after the transition to the tsdb 1.x storage engine.

Secondly, this PR utilizes the `ENotImplemented` error code to inform
users which call the `/api/v2/delete`
2020-09-24 11:11:46 -07:00
Stuart Carnie 46db75d9a9 fix: ShardGroupDuration is updated for retention period updates
Fixes #19518
2020-09-18 07:31:48 -07:00
Christopher M. Wolff e7cbbaa722
feat: upgrade Flux to v0.83.2 (#19569) 2020-09-16 17:59:15 -07:00
Gershon Shif af0c328095
chore(packaging): build rpm and deb packages (#19567)
- Update CIrcleCI configuration to start release process on an RC build
- Update .goreleaser.yml:
   - Start building armel and armhf binaries and rpm and debian packages.
   - Generate sha256 checksum file.
- launcher.go: do not use `max` module to escape integeroverflow problem for armel and armhf builds
- Start using `v0.142.0` of goreleaser
- Added pre and post install/uninstall scripts for rpm amd deb packages
2020-09-16 12:36:44 -07:00
Stuart Carnie 6f0cf049ca feat: Port shard precreation service from InfluxDB 1.x
Provides new configuration parameters:

```
--storage-shard-precreator-advance-period
--storage-shard-precreator-check-interval
```

Closes #19520
2020-09-09 13:34:22 -07:00
Stuart Carnie e265f60b55 fix: Improvements in response to PR feedback
* Pass context.Context to Service.Open
* Remove redundant comments
* Bind to retention.Config configuration to be consistent with 1.x
2020-09-09 10:32:06 -07:00
Stuart Carnie 7c2be6e780 feat: Port 1.x retention policy enforcement service
Configuration of the check interval is available via

```
--storage-retention-check-interval
```

Closes #19309
2020-09-09 10:32:06 -07:00
Stuart Carnie feb2596f3a feat: Expose a subset of InfluxQL coordinator tuning options
Closes #19507
2020-09-05 09:44:34 -07:00
Stuart Carnie 111f39c8b1 feat: Add TSM 1.x storage options as flags
Closes #19505
2020-09-04 15:56:57 -07:00
Stuart Carnie 4266eb5935 chore: Remove page-fault-rate flag, which is not available 2020-09-04 15:56:57 -07:00
Stuart Carnie 4b5ac0871d
Merge pull request #19494 from influxdata/port-change
feat: Changing the default port from 9999 to 8086
2020-09-04 11:58:41 -07:00
Stuart Carnie 7964f86ffc
fix: switch port to 8086
fix(readme): update the port and setup
2020-09-03 14:40:29 -07:00
Stuart Carnie 0f1b7fdafb
feat: Create initial stubs for influxd upgrade command
Includes some example sub commands for dumping v1 and v2 metadata

* `influxd upgrade v1-dump-meta`
* `influxd upgrade v2-dump-meta`

Closes #19495
2020-09-03 14:23:51 -07:00
Stuart Carnie b3138d2ead
fix: Ensure TemporaryEngine returns new inner TSDB store 2020-09-01 10:50:44 -07:00
Stuart Carnie a7239c007e
fix: separate flag to always permit initial user setup
This is not required for e2e tests
2020-08-31 16:38:54 -07:00
Stuart Carnie a607ac30b9
chore: Re-enable `TemporaryEngine` via `--e2e-testing` 2020-08-31 13:50:20 -07:00
Stuart Carnie dee8977d2c
chore: move v2/v1/tsdb → v2/tsdb 2020-08-26 10:46:47 -07:00
Stuart Carnie 06fac044e5
fix(storage): Implement SeriesCardinality, which is per-bucket
Fixed unit tests
2020-08-25 14:52:00 -07:00
Stuart Carnie 7ec178f54c
feat(influxd): Perform checks to determine if prior version exists
Closes #19408
2020-08-24 15:40:25 -07:00
Stuart Carnie cfd089b77a
revert(storage): Revert changes from 1501351
Reverted 1501351623
2020-08-24 10:08:46 -07:00
Stuart Carnie 6474c8c9fa
Merge remote-tracking branch 'origin/master' into sgc/tsm1
# Conflicts:
#	storage/engine.go
#	storage/engine_test.go
#	storage/wal/wal_test.go
#	tsdb/tsm1/engine_delete_prefix.go
#	tsdb/tsm1/engine_delete_prefix_test.go
#	tsdb/tsm1/engine_test.go
#	v1/tsdb/engine/tsm1/wal.go
2020-08-24 09:56:18 -07:00
Ben Johnson 1501351623 feat(influxql): Implement DELETE & DROP MEASUREMENT 2020-08-21 13:32:47 -06:00
Stuart Carnie c01a62d5d4
chore(task): Remove unused code 2020-08-20 16:59:26 -07:00
Stuart Carnie 8a7dcc21fb
feat(influxql): Initial implementation of InfluxQL query engine
* Some tests are broken or do not pass; follow up PRs will resolve that
2020-08-14 12:37:30 -07:00
Stuart Carnie 057285e23a
Merge remote-tracking branch 'origin/master' into sgc/tsm1
# Conflicts:
#	query/logging.go
#	query/logging_test.go
#	query/stdlib/influxdata/influxdb/rules.go
#	query/stdlib/influxdata/influxdb/rules_test.go
#	query/stdlib/testing/testing.go
#	storage/flux/reader.go
#	storage/flux/table.gen.go
#	storage/flux/table.gen.go.tmpl
#	storage/flux/table.go
#	storage/flux/table_test.go
2020-08-11 10:25:01 -07:00
Johnny Steenbergen 536e2c9811 feat(pkger): parameterize task queries
references: #18237
2020-08-05 16:14:06 -07:00
jlapacik 46c7345dae
feat: enable pushdowns for sum, count, min, and max by default (#19227) 2020-08-05 12:06:35 -07:00
Johnny Steenbergen efd82a2483 feat(influx): add support for dashboards 2020-08-05 09:14:01 -07:00
Faith Chikwekwe d48dc690a6
feat(query/stdlib): add min and max to ReadGroup (#19158)
Enables the mix and max aggregates for the ReadGroupAggregte pushdown behind a feature flag.

Co-authored-by: Jonathan A. Sternberg <jonathan@influxdata.com>
2020-08-05 09:40:26 -05:00
Johnny Steenbergen 41cb12aeec feat(pkger): add parameterization to dashboard queries
references: #18237
2020-08-04 17:02:34 -07:00
George 5e22e0101d
chore(auth): remove unnecessary URM service from Bucket authorizer (#19060) 2020-08-04 14:37:14 +01:00
Stuart Carnie 49067de49f
fix(launcher): Fix launcher tests 2020-08-03 15:17:37 -07:00
Stuart Carnie 840d0a994f
fix(inspect): Add export-index command for tsdb 1.x 2020-08-03 15:17:17 -07:00