Commit Graph

344 Commits (1.11)

Author SHA1 Message Date
Jamie Strandboge bf8374320d
chore: use github.com/golang-jwt/jwt/v4 and update golang.org/x/net to v0.38.0 (1.11) (#26312)
* chore: update to supported github.com/golang-jwt/jwt/v4
* chore(dep): update golang.org/x/net to v0.38.0

(cherry-picked from 58475a1)
2025-04-22 14:45:04 -05:00
davidby-influx 3570db0b75
fix: strip double quotes from measurement names in v2/delete (#25407) (#25477)
When searching for a measurement in the v2/delete API,
remove any quotes put on the measurement name (e.g. to
alllow special characters) before string comparisons.

closes https://github.com/influxdata/influxdb/issues/25406

(cherry picked from commit e3e9232de5)

closes https://github.com/influxdata/influxdb/issues/25476
2024-10-31 08:57:46 -07:00
davidby-influx 734d25f3fd
fix: do not escape CSV output (#24311) (#24313)
CSV output is incorrectly escaped.
Add a boolean flag to tag output
functions to prevent this.

closes https://github.com/influxdata/influxdb/issues/24309

(cherry picked from commit 2dc3dcb3d1)

closes https://github.com/influxdata/influxdb/issues/24310
2023-06-29 13:35:38 -07:00
davidby-influx e05cc9667e
fix: /api/v2/delete - user nil sources field, do not set measurement db or rp (#24223)
* fix: do not set measurement db or rp in V2 delete (#24141)

Setting the DB or RP in measurements extracted from
the predicate and used as sources causes Enterprise
/api/v2/delete to fail. The DB must explicitly be passed
to Store.Delete without being set in the measurement
struct passed as a source.

closes https://github.com/influxdata/influxdb/issues/24139

(cherry picked from commit c31e99574f)

closes https://github.com/influxdata/influxdb/issues/24140

* fix: use nil Sources field in /api/v2/delete (#24145)

An empty slice in Sources causes an empty
WHERE clause to be generated by
Statement.String(). Use a nil in the field
to omit the WHERE clause completely

closes https://github.com/influxdata/influxdb/issues/24144

(cherry picked from commit f365bb7e3a)

closes https://github.com/influxdata/influxdb/issues/24146
2023-05-03 09:37:36 -07:00
Geoffrey Wossum ac350884d7
feat: include `crypto` diagnostics in `/debug/vars` output (#23948)
* feat: include `crypto` diagnostics in `/debug/vars` output

Pulls `crypto` diagnostics and includes them in `/debug/vars` output.
If no `crypto` diagnostics are available, then OSS crypto information will
be shown instead.

closes: #23947
2022-12-06 12:02:30 -06:00
Brandon Pfeifer e484c4d871
chore: upgrade Go to v1.19.3 (1.x) (#23941)
* chore: upgrade Go to 1.19.3

This re-runs ./generate.sh and ./checkfmt.sh to format and update
source code (this is primarily responsible for the huge diff.)

* fix: update tests to reflect sorting algorithm change
2022-11-28 12:15:47 -05:00
Brandon Pfeifer e68b64c57b
chore: make "flux-testing" public (#23935) 2022-11-21 14:10:10 -05:00
Brandon Pfeifer 5976e41d54
feat: upgrade flux to v0.188.0 (#23911)
* feat: upgrade flux to 0.171.0

Tests failing, safety commit

First step in https://github.com/influxdata/influxdb/issues/23815

* fix: remove "org" parameter" from writeOptSource

I attempted to implement the "orgOpt" argument in a similar fashion
to f6669f7512. However, it looks like Flux doesn't accept "org" as
a parameter to "load". It responds with:

Error calling function \"load\" @113:16-113:30: error calling function \"to\" @6:19-6:47: unused arguments [org]

This brings us from 194 passing to 570 passing.

* fix: temporarily disable broken flux tests

These tests expect rows to be stored in a certain order. However,
nothing is specifying the sort order. This has been fixed in a
later update to flux: (see 3d6f47ded).

Temporarily disable these tests until we include a fixed
version of the flux tests.

* chore: add tests from a492993012

This fixes "test-flux.sh" so it runs tests within the "flux/"
directory. This uncovered some other issues with the tests
located within "flux/". These also needed to be updated
to match the newer flux API.

* feat: upgrade flux to 0.172.0

This includes changes made in "cbbf4b27da". Since "test.go" in 2.x
diverged from 1.x, some modifications were required to make this
compatible.

* feat: upgrade flux to 0.173.0

* feat: upgrade flux to v0.174.0

* fix: Update the condition when reseting cursor (#23522)

Filters that contain `or` may change between cursor resets so we must remember to update the condition in the read cursor.

```flux
|> filter(fn: (r) => ((r["_field"] == "field1" and r["_value"]==true) or (r["_field"] == "field2" and r["_value"] == false)))
```

Closes https://github.com/influxdata/flux/issues/4804

* feat: upgrade flux to 0.174.1

* feat: upgrade flux to 0.175.0

* chore: remove end-to-end tests

These were removed in a492993 for 2.x. These tests prevent "go test ./..."
from completing. As stated in the original commit, these tests should now be
handled by the "fluxtest" harness.

* feat: upgrade flux to 0.176.0

Some tests needed to be disabled within the flux harness. This is a
result of enabling "Optimize Aggregate Window" in flux@05a1065f.
These tests are not present in 2.x. Therefore, I am unsure if
the breakage is resolved in a later commit.

* feat: upgrade flux to 0.177.0

* feat: upgrade flux to 0.178.0

* feat: upgrade flux to v0.179.0

This removes all invocations of "flux.RegisterOpSpec". According
to flux@e39096d5, "flux.RegisterOpSpec" does nothing in the
current version of flux and was removed.

* chore: update fluxtest skip list (#23633)

* chore: manually backport 785a465e9a

This removes the reference to "flux.Spec".

* build(flux): update flux to v0.181.0 (#23682)

* build(flux): update flux to v0.184.2

* chore: skip more Flux acceptance tests

There are issues for each skip detailed in test-flux.sh.

* feat: upgrade flux to v0.185.0

This adds "FluxTesting" to the "HTTPD" configuration. This option is
hidden and disabled by default. When "FluxTesting" is set, it
enables the default testing flags for "Flux".

These flags allow the "vectorized float tests" and tests requiring
the "removeRedundantSortNodes" and "labelPolymorphism" flag
enabled to work. These changes are based off of d8553c002e.

flux@3d6f47ded is included within this version of Flux. Therefore
we can now include the "group_*" tests.

* feat: upgrade flux to 0.186.0

* feat: upgrade flux to 0.187.0

* feat: upgrade flux to 0.188.0

* fix: re-run ./generate.sh with updated protoc

* fix: restrict cores to match CircleCI documentation

Co-authored-by: davidby-influx <dbyrne@influxdata.com>
Co-authored-by: Markus Westerlind <marwes91@gmail.com>
Co-authored-by: Sean Brickley <sean@wabr.io>
Co-authored-by: Jonathan A. Sternberg <jonathan@influxdata.com>
Co-authored-by: Christopher M. Wolff <chris.wolff@influxdata.com>
2022-11-15 15:20:27 -05:00
davidby-influx be9a3d4a07
fix: :db/:rp -> :dbrp, fixed create bucket, update bucket (#23898) (#23900)
Fix URL processing for V2 buckets API, add error checking.

(cherry picked from commit 636fbb3f79)

Co-authored-by: Vlasta Hajek <vlastimil.hajek@bonitoo.io>
2022-11-11 14:19:05 -08:00
davidby-influx b0e44a4659
fix: listBuckets properly returns wrapped response (#23879) (#23884)
closes https://github.com/influxdata/influxdb/issues/23861

(cherry picked from commit e6e0fd2d9c)

closes https://github.com/influxdata/influxdb/issues/23883

Co-authored-by: Vlasta Hajek <vlastimil.hajek@bonitoo.io>
2022-11-09 09:57:46 -08:00
davidby-influx d862a82515
feat: add version number to debug/vars (#23795)
closes https://github.com/influxdata/influxdb/issues/23793
2022-10-13 11:49:30 -07:00
davidby-influx d01ea61465
feat: stub out bucket members, owners, and labels (#23294)
For the V2 buckets API, stub out the unsupportable endpoints.
2022-04-25 11:42:01 -07:00
davidby-influx ce40acb298
feat: partial /v2/api/buckets support: create, delete, list, retrieve, update (#23284)
Partially implements the /v2/api/buckets
POST for create a bucket
DELETE for deleting a bucket
GET for listing buckets
GET for retrieving one bucket
PATCH for modifying a bucket

See here for API details:
https://docs.influxdata.com/influxdb/cloud/api/#tag/Buckets
2022-04-21 16:36:19 -07:00
Yun Zhao 997698f891
fix(httpd): abort processing write request when encountering a precision error (#21746) 2022-04-14 13:56:24 -05:00
Dane Strandboge 0574163566
build: upgrade to go1.18 (#23250) 2022-03-31 16:17:57 -05:00
davidby-influx 2fe52d8edc
fix: implement v2/api/delete (#23247)
Support to V2 delete api

Does not support deletion by field, only by tag value, time, and measurement.
2022-03-31 13:40:14 -07:00
Dane Strandboge 449319a964
chore: upgrade `influx_tools` to new protobuf library (#22717) 2021-10-21 14:00:53 -05:00
Dane Strandboge 8b38d0e2bf
build: upgrade protobuf library (#22606) 2021-10-15 11:42:47 -05:00
Sam Arnold 38de69cc1c
fix: flux error properly read by cloud (#22348) 2021-08-31 17:43:12 -04:00
Jamie Strandboge dec56f9c31
chore: use community maintained golang-jwt (#21934)
The dgrijalva/jwt-go project is no longer maintained[1] and they have
transferred ownership to golang-jwt/jwt[2][3][4]. We should move to the
supported golang-jwt/jwt.

The following was performed:

1. update services/httpd/handler*.go to import golang-jwt/jwt
2. revert testcase string comparison changes from 225bcecd (back to v3)
2. go mod edit -require github.com/golang-jwt/jwt@v3.2.1+incompatible
3. go mod edit -droprequire github.com/dgrijalva/jwt-go
4. go mod tidy  # see note
5. go clean ./... && go build ./...
6. go test ./...

Note: 'go mod tidy' had unrelated changes (perhaps it wasn't run in
recent commits) so I removed the unrelated delta to keep this PR focused
on the dgrijalva/jwt-go to golang-jwt/jwt changes.

References:
[1] dgrijalva/jwt-go#462
[2] dgrijalva/jwt-go#463
[3] https://github.com/dgrijalva/jwt-go/blob/master/README.md
[4] https://github.com/golang-jwt/jwt
[5] https://github.com/influxdata/influxdb/issues/21926
2021-07-23 11:41:14 -05:00
Sam Arnold c6fbc2e66f
fix: FGA enablement (#21516)
* fix: FGA enablement

Closes #21514

* fix: FGA enablement

Closes #21515
2021-05-19 17:03:21 -04:00
Sam Arnold fed8ca1ace
fix: redundant registration for prometheus collector metrics (#21156) 2021-04-07 12:38:00 -04:00
davidby-influx 70755bf42c
feat: Log query text for POST requests (#20993)
The HTTP handler logs URLs, but not body values for POST requests.
This means that queries sent by GET are logged, because the query
is in the URL, but queries sent by POST have no query text in the
log.  This feature prints all the key-value pairs in the post body,
which includes the query text, except passwords, which are redacted. 

Closes https://github.com/influxdata/influxdb/issues/20653
2021-03-22 10:31:56 -07:00
Sam Arnold b7e7de24d6
refactor: separate coarse and fine permission interfaces (#20996) 2021-03-22 09:52:33 -04:00
Sam Arnold dd6abe6f19
fix(flux): add durations to Flux logging (#19697) (#20940)
(cherry picked from commit 25fb1077e2)

Co-authored-by: Christopher M. Wolff <chris.wolff@influxdata.com>
2021-03-15 12:33:57 -04:00
Sam Arnold bf7dddaec5
fix: Add back flux CLI (#20889)
Closes:
https://github.com/influxdata/influxdb/issues/18947
https://github.com/influxdata/influxdb/issues/20852

Similarly to 2.x, the flux CLI now only supports the -execute argument
or commands on stdin. Examples:

```
➜ echo 'import "influxdata/influxdb/v1" v1.databases()' | ./influx -type flux
Result: _result
Table: keys: [organizationID]
 organizationID:string     databaseName:string  retentionPolicy:string         retentionPeriod:int  default:bool         bucketId:string
----------------------  ----------------------  ----------------------  --------------------------  ------------  ----------------------
                                     _internal                 monitor             604800000000000          true
                                      telegraf                 autogen                           0          true
                                            db                 autogen                           0          true

➜ ./influx -type flux -execute 'import "influxdata/influxdb/v1" v1.databases()'
Result: _result
Table: keys: [organizationID]
 organizationID:string     databaseName:string  retentionPolicy:string         retentionPeriod:int  default:bool         bucketId:string
----------------------  ----------------------  ----------------------  --------------------------  ------------  ----------------------
                                     _internal                 monitor             604800000000000          true
                                      telegraf                 autogen                           0          true
                                            db                 autogen                           0          true

➜ ./influx -type flux -execute 'import "influxdata/influxdb/v1"'
Error (500): error in query specification while starting program: this Flux script returns no streaming data. Consider adding a "yield" or invoking streaming functions directly, without performing an assignment
```

Note that tty input is explicitly forbidden, unlike 2.x:

```
➜ ./influx -type flux
Connected to http://localhost:8086 version unknown
InfluxDB shell version: unknown
Interactive flux is not supported. Provide your flux script via stdin or the -execute argument.
```
2021-03-09 09:56:52 -05:00
Sam Arnold d6f7716924
fix: infinite recursion bug (#20862)
* Revert "fix(error): unsupported value: +Inf" error not handled gracefully (#20250)"

This reverts commit 6ac0bb3fe3.

* fix: No infinite recursion on write error

If there is some error writing to the response writer, we
would previous have infinite recursion.

Re-closes https://github.com/influxdata/influxdb/issues/20249
2021-03-04 16:31:06 -05:00
Sam Arnold 7210fa6355
chore: Upgrade 1.x to the latest flux (#20767)
* chore: Update flux to 0.67

* chore: Builds against 0.68 flux

* chore: Builds against 0.80.0

* chore: Builds against 0.90.0

* chore: Everything builds on latest flux

* chore: goimports fixed

* chore: fix tests locally

* chore: fix CI dockerfiles

* chore: clean up some unused code

* chore: remove flux repl and Spec in flux query json

* chore: port flux end to end tests from 2.x

* chore: fix up goimports

* chore: remove 32 bit build support
2021-03-03 16:24:49 -05:00
davidby-influx a8b2129df5 chore: move context.Context to first argument in methods per convention 2021-02-23 12:52:44 -08:00
Sam Arnold 117341fb0f fix: Move value metric down to tsdb store
Previously we tracked values on the http ingress, but the tsdb store is the correct
place to track total values written for the instance.
2021-02-02 10:58:47 -05:00
Sam Arnold 6795ec6c01 refactor: do not use context value anti-pattern
Extending the context instead of fixing the API breaks type safety.
For tracking the number of points / values written, it is much clearer
to pass an explicit tracker.
2021-02-01 14:34:11 -05:00
davidby-influx 6ac0bb3fe3
fix(error): unsupported value: +Inf" error not handled gracefully (#20250)
JSON marshalling errors should be returned properly formatted in JSON
like other errors. This fix formats marshalling errors the same way
influxdb formats other query errors.

Fixes https://github.com/influxdata/influxdb/issues/20249
2020-12-07 13:03:55 -08:00
Ayan George e75e83314b
fix: Reuse http server (#20191)
Once applied, this patch will use the same net/http.Server value to
handle all http requests.

This simplifies cleanly shutting down the server.
2020-11-29 21:03:19 -05:00
Ayan George 8d90d953d7
fix: Properly shutdown http server on Close() (#20171) 2020-11-25 12:46:55 -05:00
Ayan George 225bcecd73
fix: Upgrade version of jwt-go package to v4.0.0 (#19893)
* fix: Upgrade version of jwt-go package to v4.0.0

This commit updates the dependencies for influxdb to require v4.0.0-preview1 of
the jwt-go package.  This required updating the go.mod and go.sum files as well
as any source file that directly imported that package.

Prior to this commit, the TestHandler_Query_Auth() tests would fail as it
checked for specific error strigns returned by the jwt-go package.

Version 4.0.0-preview1 of the package changed the verbiage of those errors a
bit.  This patch updates the test to detect the new error string.
2020-11-05 10:55:24 -05:00
Ayan George b1def70670
feat: generate modern profiles (#19655)
* feat: generate modern profiles

Prior to this commit, influxd was writing legacy profiling data which
often (always?) required an accompanying executable to use.

This commit instructs influxd to write profiles in the new format which
can be examined without a binary.

While we're at it, this commit also adds the allocs and threadcreate
profiles.

Finally, this patch also changes the format of the downloaded tar in the
following ways:

* The profiles are added to the profile/ directory -- so instead of
  extracting the profiles into your current directory, they're placed in
  a "profiles" directory.

* This commit adds the .pb.gz extension to each of the files since
  they're gzipped protobuf files and not .txt.
2020-10-21 09:26:15 -04:00
Pavel Zavora e8f7b78d68 fix(CORS): allow PATCH 2020-09-24 11:55:22 +02:00
Ayan George 6ce0e11738
feat: Collect values written stats (#19187)
* feat(engine/tsm1): Add WritePointsWithContext()

Add WritePontsWithContext() and make WritePoints() a thin wrapper for
it.

The purpose is to add statistics context values that we'll use to
propagate the number of fields and points written to calls up the call
chain.

* feat(tsdb): Add WriteToShardWithContext()

When applied, this patch adds WriteToShardWithContext() and wraps it
with WriteToShard() to preserve the API.

The the purpose of this addition is to propagate a context.Context value
to Shard.WritePointsWithContext().

* feat(tsdb/shard): Add WritePointsWithContext()

The purpose of adding WritePointsWithContext() is to propage context
values down to engine code and propage statistics via the context.Value
up to callers.

This patch also adds values written statistics to the shard.

* feat(http): Gather values written stats

WritePointsWithContext() was added to propagate context values down to
the engine and communicate stats to the caller.

* feat(http): Gather values written stats

WritePointsWithContext() was added to propagate context values down to
the engine and communicate stats to the caller.

* refactor: Change MetricKey to ContextKey

This patch gives the type we're useing for context keys a better name.
2020-08-12 11:26:12 -04:00
David Norton 619f0ab78e
Merge pull request #18667 from influxdata/new-http-headers
feat(service/httpd): Add user configurable HTTP headers
2020-07-08 13:59:59 -04:00
Tristan Su 6910c53440
feat(prometheus): update prometheus remote protocol (#17814)
Fetched up-to-date protocol from prometheus project
2020-07-08 07:12:52 -07:00
Ayan George 04536858d7
Merge branch 'master-1.x' into new-http-headers 2020-06-30 11:03:04 -04:00
Ayan George dde8231d5c feat(http): Allow user supplied HTTP headers
This patch adds the [http.headers] subsection to the configuration file
that allows users to supply headers that will be returned in all HTTP
responses.

Applying this patch will:

* Add code to implement new configuration items.
* Add test to ensure configuration is properly parsed.
* Add test to ensure http response headers are set
* Update sample configuration file
2020-06-30 10:59:25 -04:00
Pavel Závora 2ff73114a8
Merge branch 'master-1.x' into 18391/cors_v1 2020-06-26 06:29:30 +02:00
Tristan Su 57ea78e984
fix(httpd): add option to authenticate prometheus remote read (#18429) 2020-06-23 15:03:19 -07:00
Pavel Zavora 0c6940d0a3 fix(handler): allow CORS in v2 compatibility endpoints 2020-06-09 09:09:30 +02:00
Pavel Zavora 0e458e7175 fix(handler): add User-Agent to allowed CORS headers 2020-06-09 07:09:27 +02:00
Jakub Bednář 61606289db chore(handler): add 2.0 compatible health endpoint from v1.8 (#17252) 2020-06-09 06:53:32 +02:00
Ayan George b7d3d21313
refactor(http): replace string(w.Body.Bytes()) with w.Body.String() (#18143)
Instead of type converting the Body.Bytes() to a string, we can simply
call Body.String().

While we're at it, this patch also uses a simple string comparison
instead of cmp.Equal() for two strings.

According to the docs, they're equivalent.
2020-05-18 17:31:15 -04:00
Ayan George 3b9be0145c
fix: address static check warning s1039 (#18135)
This commit quiets staticcheck's warnings about "unnecessary use of
fmt.Sprintf" and "unnecessary use of fmt.Sprint".

Prior to this commit we were wrapping simple constant strings without
any formatting verbs with fmt.Sprintf().
2020-05-18 13:55:05 -04:00
Stuart Carnie 8843a01ebf
fix(httpd): Fixes key collisions when serializing /debug/vars 2020-05-07 11:49:59 -07:00