Commit Graph

92 Commits (master)

Author SHA1 Message Date
cui fliter 46ec649b9c
chore: fix function name in comment (#24281) 2023-06-14 11:18:13 -04:00
Jeffrey Smith II f74c69c5e4
chore: update to go 1.20 (#24088)
* build: upgrade to go 1.19

* chore: bump go.mod

* chore: `gofmt` changes for doc comments

https://tip.golang.org/doc/comment

* test: update tests for new sort order

* chore: make generate-sources

* chore: make generate-sources

* chore: go 1.20

* chore: handle rand.Seed deprecation

* chore: handle rand.Seed deprecation in tests

---------

Co-authored-by: DStrand1 <dstrandboge@influxdata.com>
2023-02-09 14:14:35 -05:00
Jeffrey Smith II 9582826b3f
fix: handle a potential nil iterator leading to a panic (#23520)
* fix: handle a potential nil iterator leading to a panic

* chore: cleanup
2022-10-31 08:54:57 -04:00
Jeffrey Smith II 4ed184dd82
fix: fixes an error querying virtual dbrps (#23731)
* fix: fixes an error querying virtual dbrps

When the virtual pointer was set to false, the mappings were being ignored.

* fix: missed part in a rebase

* test: add test for shard mapping virtual dbrps

* fix: do not create virtual mappings for equivalent physical mappings
2022-10-13 14:58:07 -04:00
Andrew Lee afbbfaca87
fix: fix virtual DBRP FindMany, make virtual bucket default if not overridden (#23623)
* fix: fix find dbrps, make bucket default if not overridden

* fix: allow filtering of virtual DBRPs, filter in shard mapper

* fix: update tests to mock for virtual filter for shards and update server test
2022-08-12 11:16:37 -06:00
davidby-influx 53580ead1d
fix: remember shards that fail Open(), avoid repeated attempts (#23437) (#23455)
If a shard cannot be opened, store its ID and last error.
Prevent future attempts to open during this invocation of
influxDB. This information is not persisted.

closes https://github.com/influxdata/influxdb/issues/23428
closes https://github.com/influxdata/influxdb/issues/23426

(cherry picked from commit 54ac7e54ed)

closes https://github.com/influxdata/influxdb/issues/23434
closes https://github.com/influxdata/influxdb/issues/23436
2022-06-14 13:01:11 -07:00
Jeffrey Smith II d705841a00
feat: error when creating v1 auth with a nonexistent bucket id (#23422)
* feat: error when creating v1 auth with a nonexistent bucket id

* fix: only check for buckets

* refactor: test cleanup for clarity
2022-06-13 15:52:28 -04:00
Dane Strandboge 82d1123e78
build: upgrade to Go 1.18.1 (#23252) 2022-04-13 15:24:27 -05:00
Jonathan A. Sternberg 72c4c55609
build(flux): update flux to v0.162.0 (#23255) 2022-04-05 16:10:41 -05:00
Jamie Strandboge e304ef9764
fix: add write permissions check for DELETE and DROP MEASUREMENT (#23219)
We previously allowed read tokens access to all of v1 query, including
InfluxQL queries that made state changes to the DB, specifically,
'DELETE' and 'DROP MEASUREMENT'. This allowed tokens with only read
permissions to delete points via the legacy /query endpoint.
/api/v2/query was unaffected.

This adjusts the behavior to verify that the token has write permissions
when specifying 'DELETE' and 'DROP MEASUREMENT' InfluxQL queries. We
follow the same pattern as other existing v1 failure scenarios and
instead of failing hard with 401, we use ectx.Send() to send an error to
the user (with 200 status):

{"results":[{"statement_id":0,"error":"insufficient permissions"}]}

Returning in this manner is consistent with Cloud 2, which also returns
200 with "insufficient permissions" for these two InfluxQL queries.

To facilitate authorization unit tests, we add MustNewPermission() to
testing/util.go.

Closes: #22799
2022-03-24 07:28:38 -05:00
Sam Arnold e20b5e99a6
fix: remove nats for scraper processing (#23107)
* fix: remove nats for scraper processing

Scrapers now use go channels instead of NATS and interprocess communication.
This should fix #23085 .

Additionally, found and fixed #23106 .

* chore: fix formatting

* chore: fix static check and go.mod

* test: fix some flaky tests

* fix: mark NATS arguments as deprecated
2022-02-10 11:23:18 -05:00
Sam Arnold b970e359dc
feat: remaining storage metrics from OSS engine (#22938)
* fix: simplify disk size tracking

* refactor: EngineTags in tsdb package

* fix: fewer compaction buckets and dead code removal

* feat: shard metrics

* chore: formatting

* feat: tsdb store metrics

* feat: retention check metrics

* chore: fix go vet

* fix: review comments
2021-12-02 09:01:46 -05:00
Sam Arnold 0a740856c7
feat: point write requests have metrics (#22910)
* fix: remove dead subscriptions code

* feat: storage_writer metrics
2021-11-22 12:58:17 -05:00
Dane Strandboge f4e9ae94ca
build: upgrade protobuf library (#22654) 2021-11-02 16:00:54 -05:00
Sam Arnold 725794ef28
feat: show measurement database and retention policy wildcards (#22396)
* feat: show measurement database and retention policy wildcards

Closes #22390

* chore: formatting

* test: this commit fails tests with empty database

* fix: show measurements with one empty database
2021-10-05 13:20:15 -04:00
Daniel Moran 2795c600c1
feat: add `--storage-write-timeout` flag to set write request timeouts (#22617) 2021-10-04 15:38:09 -04:00
Sam Arnold 0a6f562a44
fix: do not allow shard creation to create overlapping shards (#22604) 2021-10-01 08:07:34 -04:00
William Baker ec7841b355
feat: support for flux cardinality query (#22441)
* feat: works with custom iterator

* feat: works with existing iterators

* chore: cleanup

* test: consistent assertions for tests

* fix: better log message if trying to filter on the value of a field key

* fix: comment for handling boolean literal; handle false boolean as well

* fix: make time range checking inclusive
2021-09-13 13:20:56 -06:00
Sam Arnold 5015297d40
fix: more expressive errors (#22448)
* fix: more expressive errors

Closes #22446

* fix: server only logging for untyped errors

* chore: fix formatting
2021-09-13 15:12:35 -04:00
William Baker 3e275a123d
feat: multi-measurement query optimization (#22301)
* feat: multi-measurement query optimization
2021-09-13 13:00:08 -06:00
William Baker 1482bdfa38
feat: partial support for series cardinality flux query (#22378) 2021-09-09 12:37:11 -06:00
Daniel Moran dc3b501298
refactor: rename Lock/Unlock on KV stores to RLock/RUnlock (#22357) 2021-08-31 17:03:54 -04:00
Daniel Moran 01355a068c
refactor: move helper functions for setting up test stores into testing package (#22347)
* Move tenant.Service unit tests into its package
* Delete the top-level TenantService interface now that it's not used.
* Move helper funcs for setting up test stores into testing pkg
* Delete duplicate implementations scattered through the codebase
* Move error assertions into store-creation helpers
2021-08-31 16:43:45 -04:00
Daniel Moran cc6accf106
chore: delete old-style DBRP mapping (#22339) 2021-08-30 18:27:11 -04:00
Daniel Moran 22c6b78c62
build: fix generation when vendoring dependencies (#22089) 2021-08-06 15:06:36 -04:00
William Baker 8e80798c33
fix: field metaqueries take fast path if predicate is only on `_measurement` (#21962)
* fix: field metaqueries take fast path if predicate is only on `_measurement`

* chore: update CHANGELOG

* test: add test for fields with measurement predicate
2021-07-29 13:57:06 -06:00
Daniel Moran 9d81f4a096
fix: prevent silently dropped writes with overlapping shards (#21946)
Co-authored-by: Geoffrey Wossum <gwossum@influxdata.com>
2021-07-27 12:15:08 -04:00
Daniel Moran 858be90139
fix: regenerate protos using gogo 1.3.2 (#21880)
* Regenerate protos using gogo 1.3.2
* Add protos to generate, add checkgenerate to CI
* Address proto warning
* Add generator tooling to Makefile
* Delete recursive Makefiles, simplify generation run by goreleaser
* Use env bash for fetch-ui-assets
* Add static-data to clean target
2021-07-20 15:11:46 -04:00
Daniel Moran d747e7ec4e
feat: add config parameters to toggle WAL concurrency and timeouts (#21621)
* feat: add context parameter to Take() method on fixed limiter
* refactor: plumb context through to uses of Take()
* test: update tests to pass context as needed
* feat: add config toggles for setting WAL write concurrency & timeout
2021-06-09 11:03:53 -04:00
Yun Zhao c22fe2f0d7
fix(coordinator): avoid unnecessary copy of FieldDimensions result data (#21390) 2021-06-04 13:09:42 -04:00
William Baker b0ea3b6675
feat: backup and restore sqlite database (#21584)
* feat: new metadata backup endpoint

* feat: added restore/sql API endpoint

* fix: content-type is multipart/mixed, part names are kv and sql

* fix: changed multipart manifest to buckets and made it .json

* feat: added lock for backing up sqlite and bolt dbs

* fix: use read lock instead of write lock on kv during backup

* fix: use filepath.Join for temp dirs
2021-06-02 19:07:53 -04:00
Daniel Moran 00420fb54c
fix(influxql): make meta queries respect query timeout (#21545)
Co-authored-by: davidby-influx <dbyrne@influxdata.com>
2021-05-24 21:10:53 -04:00
Sam Arnold d89b9cbaca
refactor: last cursor optimization function name (#21353)
Closes: #21330
2021-05-13 14:44:00 -04:00
Faith Chikwekwe 7bde3413b3
feat(query): enable min/max pushdown (#20994)
* feat(query): enable min/max pushdown

* fix(query): fix the group last pushdown to use descending cursors

* test(storage): add read group test with no agg

Co-authored-by: Jonathan A. Sternberg <jonathan@influxdata.com>
2021-04-20 12:56:43 -07:00
Sean Brickley f862f8cb1f
fix(storage): Detect need for descending cursor in WindowAggregate (#21140)
* fix(storage): Detect need for descending cursor in WindowAggregate

* chore: Format + comments

* chore: PR cosmetic feedback (#21141)

Co-authored-by: Phil Bracikowski <pbracikowski+git@influxdata.com>

* chore: rename testcase and fix comments

Co-authored-by: Phil Bracikowski <pbracikowski+git@influxdata.com>
2021-04-05 18:33:07 -04:00
Daniel Moran f6b073348b
build: delete vendored copy of gogo protos (#21098) 2021-03-30 14:26:05 -04:00
Daniel Moran 00afd95cb7
refactor: automated move of errors and id from root to kit (#21101)
Co-authored-by: Sam Arnold <sarnold@influxdata.com>
2021-03-30 14:10:02 -04:00
Daniel Moran b1283bc998
build: fix 'go generate' and regenerate the universe (#21089) 2021-03-30 10:32:14 -04:00
Daniel Moran ba4099ddb9
feat(cmd/influx): allow setting shard-group durations for buckets via API and CLI (#20911)
Co-authored-by: Alexander Savinykh <asavinykh@nbrx.de>
2021-03-11 14:51:03 -05:00
Daniel Moran 5fc9240dc5
refactor: delete unused v1 monitoring service (#20818) 2021-03-02 12:39:38 -05:00
Daniel Moran 727a7b58c1
test: replace influxlogger with zaptest logger (#20589) 2021-02-11 10:12:39 -05:00
Daniel Moran b5bbc503a0
fix(onboarding): don't ignore failures to set initial password (#20317) 2020-12-16 06:43:43 -08:00
Daniel Moran 5b29ef7060
fix(v1): optimize shard lookup in groups containing only one shard (#20201)
Co-authored-by: Yun Zhao <zhaoyun2316@gmail.com>
2020-11-30 15:16:03 -05:00
Stuart Carnie 3ac530edd2 fix: Track seen databases in map and skip duplicates 2020-11-17 09:17:28 +11:00
Daniel Moran 15b9531273
fix: correct various typos (#19987)
Co-authored-by: kumakichi <xyesan@gmail.com>
2020-11-11 13:54:21 -05:00
Ben Johnson 419b0cf76b feat: Implement full restore 2020-11-05 10:05:01 -07:00
Ben Johnson 23679c2375 feat: Implement backup/restore CLI subcommands. 2020-11-05 10:05:01 -07:00
Christopher M. Wolff 14443a5c39
fix: return empty iterator instead of null in tagValues (#19894)
Fixes influxdata/flux#3300.
2020-11-04 08:09:27 -08:00
Yiqun (Ethan) Zhang 17fcd2dce2
feat: make sure the query plan nodes have unique ids (#19879) 2020-11-02 11:15:39 -06:00
Stuart Carnie 13b3bb81cd fix: PR feedback to move defer logic to its own function for clarity 2020-10-30 11:56:09 -07:00