Commit Graph

64 Commits (22f4f5bba80420e578389b3c82f9619530671015)

Author SHA1 Message Date
Daniel Moran d747e7ec4e
feat: add config parameters to toggle WAL concurrency and timeouts ()
* feat: add context parameter to Take() method on fixed limiter
* refactor: plumb context through to uses of Take()
* test: update tests to pass context as needed
* feat: add config toggles for setting WAL write concurrency & timeout
2021-06-09 11:03:53 -04:00
Yun Zhao c22fe2f0d7
fix(coordinator): avoid unnecessary copy of FieldDimensions result data () 2021-06-04 13:09:42 -04:00
William Baker b0ea3b6675
feat: backup and restore sqlite database ()
* feat: new metadata backup endpoint

* feat: added restore/sql API endpoint

* fix: content-type is multipart/mixed, part names are kv and sql

* fix: changed multipart manifest to buckets and made it .json

* feat: added lock for backing up sqlite and bolt dbs

* fix: use read lock instead of write lock on kv during backup

* fix: use filepath.Join for temp dirs
2021-06-02 19:07:53 -04:00
Daniel Moran 00420fb54c
fix(influxql): make meta queries respect query timeout ()
Co-authored-by: davidby-influx <dbyrne@influxdata.com>
2021-05-24 21:10:53 -04:00
Sam Arnold d89b9cbaca
refactor: last cursor optimization function name ()
Closes: 
2021-05-13 14:44:00 -04:00
Faith Chikwekwe 7bde3413b3
feat(query): enable min/max pushdown ()
* feat(query): enable min/max pushdown

* fix(query): fix the group last pushdown to use descending cursors

* test(storage): add read group test with no agg

Co-authored-by: Jonathan A. Sternberg <jonathan@influxdata.com>
2021-04-20 12:56:43 -07:00
Sean Brickley f862f8cb1f
fix(storage): Detect need for descending cursor in WindowAggregate ()
* fix(storage): Detect need for descending cursor in WindowAggregate

* chore: Format + comments

* chore: PR cosmetic feedback ()

Co-authored-by: Phil Bracikowski <pbracikowski+git@influxdata.com>

* chore: rename testcase and fix comments

Co-authored-by: Phil Bracikowski <pbracikowski+git@influxdata.com>
2021-04-05 18:33:07 -04:00
Daniel Moran f6b073348b
build: delete vendored copy of gogo protos () 2021-03-30 14:26:05 -04:00
Daniel Moran 00afd95cb7
refactor: automated move of errors and id from root to kit ()
Co-authored-by: Sam Arnold <sarnold@influxdata.com>
2021-03-30 14:10:02 -04:00
Daniel Moran b1283bc998
build: fix 'go generate' and regenerate the universe () 2021-03-30 10:32:14 -04:00
Daniel Moran ba4099ddb9
feat(cmd/influx): allow setting shard-group durations for buckets via API and CLI ()
Co-authored-by: Alexander Savinykh <asavinykh@nbrx.de>
2021-03-11 14:51:03 -05:00
Daniel Moran 5fc9240dc5
refactor: delete unused v1 monitoring service () 2021-03-02 12:39:38 -05:00
Daniel Moran 727a7b58c1
test: replace influxlogger with zaptest logger () 2021-02-11 10:12:39 -05:00
Daniel Moran b5bbc503a0
fix(onboarding): don't ignore failures to set initial password () 2020-12-16 06:43:43 -08:00
Daniel Moran 5b29ef7060
fix(v1): optimize shard lookup in groups containing only one shard ()
Co-authored-by: Yun Zhao <zhaoyun2316@gmail.com>
2020-11-30 15:16:03 -05:00
Stuart Carnie 3ac530edd2 fix: Track seen databases in map and skip duplicates 2020-11-17 09:17:28 +11:00
Daniel Moran 15b9531273
fix: correct various typos ()
Co-authored-by: kumakichi <xyesan@gmail.com>
2020-11-11 13:54:21 -05:00
Ben Johnson 419b0cf76b feat: Implement full restore 2020-11-05 10:05:01 -07:00
Ben Johnson 23679c2375 feat: Implement backup/restore CLI subcommands. 2020-11-05 10:05:01 -07:00
Christopher M. Wolff 14443a5c39
fix: return empty iterator instead of null in tagValues ()
Fixes .
2020-11-04 08:09:27 -08:00
Yiqun (Ethan) Zhang 17fcd2dce2
feat: make sure the query plan nodes have unique ids () 2020-11-02 11:15:39 -06:00
Stuart Carnie 13b3bb81cd fix: PR feedback to move defer logic to its own function for clarity 2020-10-30 11:56:09 -07:00
Stuart Carnie 3fbbd6afcf feat: Implementation of AuthorizerV1
The `AuthorizerV1` defines the behavior for authorizing an InfluxDB
1.x API using `CredentialsV1`. These credentials are extracted from
an API, such as the Authorization header of a HTTP request.
2020-10-30 11:56:09 -07:00
Christopher M. Wolff 5a60488729
fix: make tagKeys and tagValues work for edge cases involving _field () 2020-10-30 10:37:08 -07:00
Stuart Carnie ecbb5987a6 feat: add v1 auth set-password command to set token passwords
* Added capability to HTTP API to search for tokens by ID or Token
2020-10-29 08:21:38 -07:00
Stuart Carnie 4e96f30f8d fix: V1 user authentication tokens no longer require SHA256
Remove remnants of previous token implementation

* As of , V1 API tokens will be authorized using bcrypt passwords
2020-10-29 08:21:38 -07:00
Stuart Carnie 6bc4158a46 feat(authorization): Add bcrypt password support to v1 authorizations
This commit extends the `v1/authorization` package to support
passwords associated with a token.

The summary of changes include:

* authorization.Service implements influxdb.PasswordsService
* Setting passwords for authorizations
* Verifying (comparing) passwords for a given authorization
* A service to cache comparing passwords, using a weaker hash
  that will live in memory only. This implementation is copied
  from InfluxDB 1.x
* Extended HTTP service to set a password using
  /private/legacy/authorizations/{id}/password

Closes #
2020-10-28 13:03:42 -07:00
Faith Chikwekwe 5e1cdda597 feat(storage): enable window agg mean pushdown 2020-10-27 12:53:54 -07:00
Christopher M. Wolff 790165b05a
refactor: allow newIndexSeriesCursor() to accept an influxql.Expr ()
* refactor: allow newIndexSeriesCursor() to accept an influxql.Expr

In order to let TagKeys and TagValues get the right answer,
sometimes they will need to examine their predicate and
see if using the index is possible, or if a block scan is needed.
For those cases we want to examine the predicate before creating
the index series cursor. This change allows us to create an
index series cursor with an already-deserialized influxql.Expr.
2020-10-27 09:31:46 -07:00
Stuart Carnie 5c63c2163d feat(authorization): Create a v1 authorization service
This service is a private API for managing authorization tokens
for v1 API requests.

Note that this commit does not hook up the service to the v1
`/query` and `/write`, which will occur in a subsequent PR.

Closes 
2020-10-27 08:26:10 -07:00
vlastahajek 8763bb1af9 feat(upgrade): upgrade databases 2020-10-08 09:39:30 -07:00
Jonathan A. Sternberg 025319c387
fix(services/storage): multi measurement queries return all applicable series ()
This fixes multi measurement queries that go through the storage service
to correctly pick up all series that apply with the filter. Previously,
negative queries such as `!=`, `!~`, and predicates attempting to match
empty tags did not work correctly with the storage service when multiple
measurements or `OR` conditions were included.

This was because these predicates would be categorized as "multiple
measurements" and then it would attempt to use the field keys iterator
to find the fields for each measurement. The meta queries for these did
not correctly account for negative equality operators or empty tags when
finding appropriate measurements and those could not be changed because
it would cause a breaking change to influxql too.

This modifies the storage service to use new methods that correctly
account for the above situations rather than the field keys iterator.

Some queries that appeared to be single measurement queries also get
considered as multiple measurement queries. Any query with an `OR`
condition will be considered a multiple measurement query.

This bug did not apply to single measurement queries where one
measurement was selected and all of the logical operators were `AND`
values. This is because it used a different code path that correctly
handled these situations.
2020-09-17 14:28:24 -05:00
Ayan George ca2055c16c
refactor: Replace ctx.Done() with ctx.Err() ()
* refactor: Replace ctx.Done() with ctx.Err()

Prior to this commit we checked for context cancellation with a select
block and context.Context.Done() without multiplexing over any other
channel like:

  select {
    case <-ctx.Done():
      // handle cancellation
    default:
      // fallthrough
  }

This commit replaces those type of blocks with a simple check of
ctx.Err().  This has the following benefits:

* Calling ctx.Err() is much faster than entering a select block.

* ctx.Done() allocates a channel when called for the first time.

* Testing the result of ctx.Err() is a reliable way of determininging if
  a context.Context value has been canceled.

* fix: Fix data race in execDeleteTagValueEntry()
2020-09-16 12:20:09 -04:00
Stuart Carnie 8576d1df99 feat: Implement DELETE FROM and DROP MEASUREMENT
Closes 
2020-09-11 11:37:43 -07:00
Stuart Carnie 7e16da8f74 chore: Consolidate `v1/internal` to `internal`
Take the opportunity to combines duplicated types in both packages
under the root `internal` package
2020-09-11 11:37:43 -07:00
Stuart Carnie 6f0cf049ca feat: Port shard precreation service from InfluxDB 1.x
Provides new configuration parameters:

```
--storage-shard-precreator-advance-period
--storage-shard-precreator-check-interval
```

Closes 
2020-09-09 13:34:22 -07:00
Stuart Carnie e265f60b55 fix: Improvements in response to PR feedback
* Pass context.Context to Service.Open
* Remove redundant comments
* Bind to retention.Config configuration to be consistent with 1.x
2020-09-09 10:32:06 -07:00
Stuart Carnie 7c2be6e780 feat: Port 1.x retention policy enforcement service
Configuration of the check interval is available via

```
--storage-retention-check-interval
```

Closes 
2020-09-09 10:32:06 -07:00
Stuart Carnie 7f94b0d5db
fix: Rewrite predicate to remove _field from schema physical operations
Fixes 
2020-09-02 11:10:40 -07:00
Stuart Carnie b3138d2ead
fix: Ensure TemporaryEngine returns new inner TSDB store 2020-09-01 10:50:44 -07:00
Brett Buddin b917d8d9b0
chore(influxdb): Placate the linter. 2020-08-27 15:46:32 -04:00
Stuart Carnie dee8977d2c
chore: move v2/v1/tsdb → v2/tsdb 2020-08-26 10:46:47 -07:00
Stuart Carnie 8bebf9cbeb
fix(coordinator): Return correct data types for stand-in RP values
Future work is required to ensure the retention policy information
is appropriately retrieved from the bucket service and / or meta client
2020-08-25 16:26:16 -07:00
Stuart Carnie afd5120221
fix(storage): Ensure _field tag key has special handling
Fixes 
2020-08-21 13:06:00 -07:00
Stuart Carnie 2b8d6ca14a
fix(influxql): Fix broken unit tests 2020-08-14 15:09:40 -07:00
Stuart Carnie 8a7dcc21fb
feat(influxql): Initial implementation of InfluxQL query engine
* Some tests are broken or do not pass; follow up PRs will resolve that
2020-08-14 12:37:30 -07:00
Stuart Carnie a29ec693df
fix(tsdb): MaxSeriesPerDatabase is only supported by the inmem index 2020-08-12 09:12:16 -07:00
Stuart Carnie 9b76352f74
fix(tsdb): MaxValuesPerTag is only supported by the inmem index type 2020-08-11 15:02:22 -07:00
Stuart Carnie 381ea6ec85
fix(tsi1): Update SQLIndexExported expected output 2020-08-03 17:10:51 -07:00
Stuart Carnie 840d0a994f
fix(inspect): Add export-index command for tsdb 1.x 2020-08-03 15:17:17 -07:00