Commit Graph

85 Commits (05840ce2a6e7570b06fbcf078da4780a114afd93)

Author SHA1 Message Date
Dane Strandboge 82d1123e78
build: upgrade to Go 1.18.1 (#23252) 2022-04-13 15:24:27 -05:00
Jonathan A. Sternberg 72c4c55609
build(flux): update flux to v0.162.0 (#23255) 2022-04-05 16:10:41 -05:00
Jamie Strandboge e304ef9764
fix: add write permissions check for DELETE and DROP MEASUREMENT (#23219)
We previously allowed read tokens access to all of v1 query, including
InfluxQL queries that made state changes to the DB, specifically,
'DELETE' and 'DROP MEASUREMENT'. This allowed tokens with only read
permissions to delete points via the legacy /query endpoint.
/api/v2/query was unaffected.

This adjusts the behavior to verify that the token has write permissions
when specifying 'DELETE' and 'DROP MEASUREMENT' InfluxQL queries. We
follow the same pattern as other existing v1 failure scenarios and
instead of failing hard with 401, we use ectx.Send() to send an error to
the user (with 200 status):

{"results":[{"statement_id":0,"error":"insufficient permissions"}]}

Returning in this manner is consistent with Cloud 2, which also returns
200 with "insufficient permissions" for these two InfluxQL queries.

To facilitate authorization unit tests, we add MustNewPermission() to
testing/util.go.

Closes: #22799
2022-03-24 07:28:38 -05:00
Sam Arnold e20b5e99a6
fix: remove nats for scraper processing (#23107)
* fix: remove nats for scraper processing

Scrapers now use go channels instead of NATS and interprocess communication.
This should fix #23085 .

Additionally, found and fixed #23106 .

* chore: fix formatting

* chore: fix static check and go.mod

* test: fix some flaky tests

* fix: mark NATS arguments as deprecated
2022-02-10 11:23:18 -05:00
Sam Arnold b970e359dc
feat: remaining storage metrics from OSS engine (#22938)
* fix: simplify disk size tracking

* refactor: EngineTags in tsdb package

* fix: fewer compaction buckets and dead code removal

* feat: shard metrics

* chore: formatting

* feat: tsdb store metrics

* feat: retention check metrics

* chore: fix go vet

* fix: review comments
2021-12-02 09:01:46 -05:00
Sam Arnold 0a740856c7
feat: point write requests have metrics (#22910)
* fix: remove dead subscriptions code

* feat: storage_writer metrics
2021-11-22 12:58:17 -05:00
Dane Strandboge f4e9ae94ca
build: upgrade protobuf library (#22654) 2021-11-02 16:00:54 -05:00
Sam Arnold 725794ef28
feat: show measurement database and retention policy wildcards (#22396)
* feat: show measurement database and retention policy wildcards

Closes #22390

* chore: formatting

* test: this commit fails tests with empty database

* fix: show measurements with one empty database
2021-10-05 13:20:15 -04:00
Daniel Moran 2795c600c1
feat: add `--storage-write-timeout` flag to set write request timeouts (#22617) 2021-10-04 15:38:09 -04:00
Sam Arnold 0a6f562a44
fix: do not allow shard creation to create overlapping shards (#22604) 2021-10-01 08:07:34 -04:00
William Baker ec7841b355
feat: support for flux cardinality query (#22441)
* feat: works with custom iterator

* feat: works with existing iterators

* chore: cleanup

* test: consistent assertions for tests

* fix: better log message if trying to filter on the value of a field key

* fix: comment for handling boolean literal; handle false boolean as well

* fix: make time range checking inclusive
2021-09-13 13:20:56 -06:00
Sam Arnold 5015297d40
fix: more expressive errors (#22448)
* fix: more expressive errors

Closes #22446

* fix: server only logging for untyped errors

* chore: fix formatting
2021-09-13 15:12:35 -04:00
William Baker 3e275a123d
feat: multi-measurement query optimization (#22301)
* feat: multi-measurement query optimization
2021-09-13 13:00:08 -06:00
William Baker 1482bdfa38
feat: partial support for series cardinality flux query (#22378) 2021-09-09 12:37:11 -06:00
Daniel Moran dc3b501298
refactor: rename Lock/Unlock on KV stores to RLock/RUnlock (#22357) 2021-08-31 17:03:54 -04:00
Daniel Moran 01355a068c
refactor: move helper functions for setting up test stores into testing package (#22347)
* Move tenant.Service unit tests into its package
* Delete the top-level TenantService interface now that it's not used.
* Move helper funcs for setting up test stores into testing pkg
* Delete duplicate implementations scattered through the codebase
* Move error assertions into store-creation helpers
2021-08-31 16:43:45 -04:00
Daniel Moran cc6accf106
chore: delete old-style DBRP mapping (#22339) 2021-08-30 18:27:11 -04:00
Daniel Moran 22c6b78c62
build: fix generation when vendoring dependencies (#22089) 2021-08-06 15:06:36 -04:00
William Baker 8e80798c33
fix: field metaqueries take fast path if predicate is only on `_measurement` (#21962)
* fix: field metaqueries take fast path if predicate is only on `_measurement`

* chore: update CHANGELOG

* test: add test for fields with measurement predicate
2021-07-29 13:57:06 -06:00
Daniel Moran 9d81f4a096
fix: prevent silently dropped writes with overlapping shards (#21946)
Co-authored-by: Geoffrey Wossum <gwossum@influxdata.com>
2021-07-27 12:15:08 -04:00
Daniel Moran 858be90139
fix: regenerate protos using gogo 1.3.2 (#21880)
* Regenerate protos using gogo 1.3.2
* Add protos to generate, add checkgenerate to CI
* Address proto warning
* Add generator tooling to Makefile
* Delete recursive Makefiles, simplify generation run by goreleaser
* Use env bash for fetch-ui-assets
* Add static-data to clean target
2021-07-20 15:11:46 -04:00
Daniel Moran d747e7ec4e
feat: add config parameters to toggle WAL concurrency and timeouts (#21621)
* feat: add context parameter to Take() method on fixed limiter
* refactor: plumb context through to uses of Take()
* test: update tests to pass context as needed
* feat: add config toggles for setting WAL write concurrency & timeout
2021-06-09 11:03:53 -04:00
Yun Zhao c22fe2f0d7
fix(coordinator): avoid unnecessary copy of FieldDimensions result data (#21390) 2021-06-04 13:09:42 -04:00
William Baker b0ea3b6675
feat: backup and restore sqlite database (#21584)
* feat: new metadata backup endpoint

* feat: added restore/sql API endpoint

* fix: content-type is multipart/mixed, part names are kv and sql

* fix: changed multipart manifest to buckets and made it .json

* feat: added lock for backing up sqlite and bolt dbs

* fix: use read lock instead of write lock on kv during backup

* fix: use filepath.Join for temp dirs
2021-06-02 19:07:53 -04:00
Daniel Moran 00420fb54c
fix(influxql): make meta queries respect query timeout (#21545)
Co-authored-by: davidby-influx <dbyrne@influxdata.com>
2021-05-24 21:10:53 -04:00
Sam Arnold d89b9cbaca
refactor: last cursor optimization function name (#21353)
Closes: #21330
2021-05-13 14:44:00 -04:00
Faith Chikwekwe 7bde3413b3
feat(query): enable min/max pushdown (#20994)
* feat(query): enable min/max pushdown

* fix(query): fix the group last pushdown to use descending cursors

* test(storage): add read group test with no agg

Co-authored-by: Jonathan A. Sternberg <jonathan@influxdata.com>
2021-04-20 12:56:43 -07:00
Sean Brickley f862f8cb1f
fix(storage): Detect need for descending cursor in WindowAggregate (#21140)
* fix(storage): Detect need for descending cursor in WindowAggregate

* chore: Format + comments

* chore: PR cosmetic feedback (#21141)

Co-authored-by: Phil Bracikowski <pbracikowski+git@influxdata.com>

* chore: rename testcase and fix comments

Co-authored-by: Phil Bracikowski <pbracikowski+git@influxdata.com>
2021-04-05 18:33:07 -04:00
Daniel Moran f6b073348b
build: delete vendored copy of gogo protos (#21098) 2021-03-30 14:26:05 -04:00
Daniel Moran 00afd95cb7
refactor: automated move of errors and id from root to kit (#21101)
Co-authored-by: Sam Arnold <sarnold@influxdata.com>
2021-03-30 14:10:02 -04:00
Daniel Moran b1283bc998
build: fix 'go generate' and regenerate the universe (#21089) 2021-03-30 10:32:14 -04:00
Daniel Moran ba4099ddb9
feat(cmd/influx): allow setting shard-group durations for buckets via API and CLI (#20911)
Co-authored-by: Alexander Savinykh <asavinykh@nbrx.de>
2021-03-11 14:51:03 -05:00
Daniel Moran 5fc9240dc5
refactor: delete unused v1 monitoring service (#20818) 2021-03-02 12:39:38 -05:00
Daniel Moran 727a7b58c1
test: replace influxlogger with zaptest logger (#20589) 2021-02-11 10:12:39 -05:00
Daniel Moran b5bbc503a0
fix(onboarding): don't ignore failures to set initial password (#20317) 2020-12-16 06:43:43 -08:00
Daniel Moran 5b29ef7060
fix(v1): optimize shard lookup in groups containing only one shard (#20201)
Co-authored-by: Yun Zhao <zhaoyun2316@gmail.com>
2020-11-30 15:16:03 -05:00
Stuart Carnie 3ac530edd2 fix: Track seen databases in map and skip duplicates 2020-11-17 09:17:28 +11:00
Daniel Moran 15b9531273
fix: correct various typos (#19987)
Co-authored-by: kumakichi <xyesan@gmail.com>
2020-11-11 13:54:21 -05:00
Ben Johnson 419b0cf76b feat: Implement full restore 2020-11-05 10:05:01 -07:00
Ben Johnson 23679c2375 feat: Implement backup/restore CLI subcommands. 2020-11-05 10:05:01 -07:00
Christopher M. Wolff 14443a5c39
fix: return empty iterator instead of null in tagValues (#19894)
Fixes influxdata/flux#3300.
2020-11-04 08:09:27 -08:00
Yiqun (Ethan) Zhang 17fcd2dce2
feat: make sure the query plan nodes have unique ids (#19879) 2020-11-02 11:15:39 -06:00
Stuart Carnie 13b3bb81cd fix: PR feedback to move defer logic to its own function for clarity 2020-10-30 11:56:09 -07:00
Stuart Carnie 3fbbd6afcf feat: Implementation of AuthorizerV1
The `AuthorizerV1` defines the behavior for authorizing an InfluxDB
1.x API using `CredentialsV1`. These credentials are extracted from
an API, such as the Authorization header of a HTTP request.
2020-10-30 11:56:09 -07:00
Christopher M. Wolff 5a60488729
fix: make tagKeys and tagValues work for edge cases involving _field (#19856) 2020-10-30 10:37:08 -07:00
Stuart Carnie ecbb5987a6 feat: add v1 auth set-password command to set token passwords
* Added capability to HTTP API to search for tokens by ID or Token
2020-10-29 08:21:38 -07:00
Stuart Carnie 4e96f30f8d fix: V1 user authentication tokens no longer require SHA256
Remove remnants of previous token implementation

* As of #19840, V1 API tokens will be authorized using bcrypt passwords
2020-10-29 08:21:38 -07:00
Stuart Carnie 6bc4158a46 feat(authorization): Add bcrypt password support to v1 authorizations
This commit extends the `v1/authorization` package to support
passwords associated with a token.

The summary of changes include:

* authorization.Service implements influxdb.PasswordsService
* Setting passwords for authorizations
* Verifying (comparing) passwords for a given authorization
* A service to cache comparing passwords, using a weaker hash
  that will live in memory only. This implementation is copied
  from InfluxDB 1.x
* Extended HTTP service to set a password using
  /private/legacy/authorizations/{id}/password

Closes #
2020-10-28 13:03:42 -07:00
Faith Chikwekwe 5e1cdda597 feat(storage): enable window agg mean pushdown 2020-10-27 12:53:54 -07:00
Christopher M. Wolff 790165b05a
refactor: allow newIndexSeriesCursor() to accept an influxql.Expr (#19833)
* refactor: allow newIndexSeriesCursor() to accept an influxql.Expr

In order to let TagKeys and TagValues get the right answer,
sometimes they will need to examine their predicate and
see if using the index is possible, or if a block scan is needed.
For those cases we want to examine the predicate before creating
the index series cursor. This change allows us to create an
index series cursor with an already-deserialized influxql.Expr.
2020-10-27 09:31:46 -07:00