Commit Graph

14781 Commits (e68b64c57bb0acfe34722b979e628a771cce7aba)

Author SHA1 Message Date
Brandon Pfeifer e68b64c57b
chore: make "flux-testing" public (#23935) 2022-11-21 14:10:10 -05:00
davidby-influx fd7e4aa0f7
chore: fix trace message text (#23917) 2022-11-16 08:40:10 -05:00
Brandon Pfeifer 5976e41d54
feat: upgrade flux to v0.188.0 (#23911)
* feat: upgrade flux to 0.171.0

Tests failing, safety commit

First step in https://github.com/influxdata/influxdb/issues/23815

* fix: remove "org" parameter" from writeOptSource

I attempted to implement the "orgOpt" argument in a similar fashion
to f6669f7512. However, it looks like Flux doesn't accept "org" as
a parameter to "load". It responds with:

Error calling function \"load\" @113:16-113:30: error calling function \"to\" @6:19-6:47: unused arguments [org]

This brings us from 194 passing to 570 passing.

* fix: temporarily disable broken flux tests

These tests expect rows to be stored in a certain order. However,
nothing is specifying the sort order. This has been fixed in a
later update to flux: (see 3d6f47ded).

Temporarily disable these tests until we include a fixed
version of the flux tests.

* chore: add tests from a492993012

This fixes "test-flux.sh" so it runs tests within the "flux/"
directory. This uncovered some other issues with the tests
located within "flux/". These also needed to be updated
to match the newer flux API.

* feat: upgrade flux to 0.172.0

This includes changes made in "cbbf4b27da". Since "test.go" in 2.x
diverged from 1.x, some modifications were required to make this
compatible.

* feat: upgrade flux to 0.173.0

* feat: upgrade flux to v0.174.0

* fix: Update the condition when reseting cursor (#23522)

Filters that contain `or` may change between cursor resets so we must remember to update the condition in the read cursor.

```flux
|> filter(fn: (r) => ((r["_field"] == "field1" and r["_value"]==true) or (r["_field"] == "field2" and r["_value"] == false)))
```

Closes https://github.com/influxdata/flux/issues/4804

* feat: upgrade flux to 0.174.1

* feat: upgrade flux to 0.175.0

* chore: remove end-to-end tests

These were removed in a492993 for 2.x. These tests prevent "go test ./..."
from completing. As stated in the original commit, these tests should now be
handled by the "fluxtest" harness.

* feat: upgrade flux to 0.176.0

Some tests needed to be disabled within the flux harness. This is a
result of enabling "Optimize Aggregate Window" in flux@05a1065f.
These tests are not present in 2.x. Therefore, I am unsure if
the breakage is resolved in a later commit.

* feat: upgrade flux to 0.177.0

* feat: upgrade flux to 0.178.0

* feat: upgrade flux to v0.179.0

This removes all invocations of "flux.RegisterOpSpec". According
to flux@e39096d5, "flux.RegisterOpSpec" does nothing in the
current version of flux and was removed.

* chore: update fluxtest skip list (#23633)

* chore: manually backport 785a465e9a

This removes the reference to "flux.Spec".

* build(flux): update flux to v0.181.0 (#23682)

* build(flux): update flux to v0.184.2

* chore: skip more Flux acceptance tests

There are issues for each skip detailed in test-flux.sh.

* feat: upgrade flux to v0.185.0

This adds "FluxTesting" to the "HTTPD" configuration. This option is
hidden and disabled by default. When "FluxTesting" is set, it
enables the default testing flags for "Flux".

These flags allow the "vectorized float tests" and tests requiring
the "removeRedundantSortNodes" and "labelPolymorphism" flag
enabled to work. These changes are based off of d8553c002e.

flux@3d6f47ded is included within this version of Flux. Therefore
we can now include the "group_*" tests.

* feat: upgrade flux to 0.186.0

* feat: upgrade flux to 0.187.0

* feat: upgrade flux to 0.188.0

* fix: re-run ./generate.sh with updated protoc

* fix: restrict cores to match CircleCI documentation

Co-authored-by: davidby-influx <dbyrne@influxdata.com>
Co-authored-by: Markus Westerlind <marwes91@gmail.com>
Co-authored-by: Sean Brickley <sean@wabr.io>
Co-authored-by: Jonathan A. Sternberg <jonathan@influxdata.com>
Co-authored-by: Christopher M. Wolff <chris.wolff@influxdata.com>
2022-11-15 15:20:27 -05:00
davidby-influx be9a3d4a07
fix: :db/:rp -> :dbrp, fixed create bucket, update bucket (#23898) (#23900)
Fix URL processing for V2 buckets API, add error checking.

(cherry picked from commit 636fbb3f79)

Co-authored-by: Vlasta Hajek <vlastimil.hajek@bonitoo.io>
2022-11-11 14:19:05 -08:00
Sam Arnold 9e9f1be574
fix: remove dead iterator (#23888) 2022-11-09 16:24:01 -05:00
davidby-influx a2fca83af0
fix: don't write skipped shard messages to the line protocol output destination (#23727) (#23885)
This switches so that the message

    skipped missing file: /path/to/tsm.tsm

is written to stdErr instead of stdout (or the output file if `-out` has been provided)

(cherry picked from commit a9bf1d54c1)

closes https://github.com/influxdata/influxdb/issues/23866

Co-authored-by: Ben Tasker <88340935+btasker@users.noreply.github.com>
2022-11-09 09:58:45 -08:00
davidby-influx b0e44a4659
fix: listBuckets properly returns wrapped response (#23879) (#23884)
closes https://github.com/influxdata/influxdb/issues/23861

(cherry picked from commit e6e0fd2d9c)

closes https://github.com/influxdata/influxdb/issues/23883

Co-authored-by: Vlasta Hajek <vlastimil.hajek@bonitoo.io>
2022-11-09 09:57:46 -08:00
davidby-influx cc26b7653c
fix: remove breaking argument validation for _fieldKeys iterator (#23875)
New argument validation code for _fieldKeys system iterator 
broke Enterprise tests because it is misused all over the 
place. Back out the safety check.
2022-11-09 09:04:44 -08:00
davidby-influx f5da0f50f4
fix: Optimize SHOW FIELD KEY CARDINALITY (#23871)
Use the _fieldKeys system iterator

closes https://github.com/influxdata/influxdb/issues/23840
2022-11-08 08:32:10 -08:00
Sam Arnold 1d92b7a8fc
fix: response writer simplification (#23846)
Also a performance improvement for when we don't need to write
series keys that have no data.
2022-11-04 15:24:11 -04:00
davidby-influx bfccddc7b6
fix: log errors in continuous query statistics storage (#23822) 2022-10-19 08:33:44 -07:00
Jamie Strandboge d31e74c07c
chore: upgrade to Go 1.18.7 (#23821) 2022-10-18 14:30:47 -05:00
Brandon Pfeifer 1253ff8fea
chore: update to use scheduled pipeline (1.x) (#23808)
* chore: update to use scheduled pipeline

* chore: add documentation to scheduled pipelines
2022-10-18 12:21:09 -04:00
davidby-influx d862a82515
feat: add version number to debug/vars (#23795)
closes https://github.com/influxdata/influxdb/issues/23793
2022-10-13 11:49:30 -07:00
davidby-influx bc8d9ea9f3
fix: add tests for file rename across volumes (#23787)
Also move shared code from file_unix.go
2022-10-13 09:26:59 -07:00
davidby-influx 0913276ff0
fix: use copy when a rename spans volumes (#23785)
When a file rename fails with EXDEV
(cross device or volume error), copy the
file and delete the original instead

Differs from master branch by overwriting
existing files instead of erring.

closes https://github.com/influxdata/influxdb/issues/22997
2022-10-12 09:50:27 -07:00
Brandon Pfeifer 03ad3443ab
chore: upgrade to Go 1.18.6 (#23745) 2022-09-26 10:10:27 -04:00
davidby-influx b17f27a5d9
fix: incorrect error message concatenation (#23729) 2022-09-15 09:26:51 -07:00
davidby-influx 80c10c8c04
feat: optimize saving changes to fields.idx (#23701)
Instead of writing out the complete fields.idx
file when it changes, write out incremental
changes that will be applied to the file on
close and startup.

closes https://github.com/influxdata/influxdb/issues/23653
2022-09-14 13:14:09 -07:00
Brandon Pfeifer df6a5e68e0
chore: add protoc-gen script to releng (#23694) 2022-08-31 16:42:29 -04:00
Brandon Pfeifer a52893479e
feat: enable static-pie builds (1.x) (#23642)
* feat: enable static pie builds

* fix: update cross-builder to fix 'rdynamic'
2022-08-19 12:34:59 -04:00
Brandon Pfeifer 787a168f32
chore: use 1.x signing key (#23611) 2022-08-10 13:10:52 -04:00
davidby-influx 84c4f676b0
feat: add type conflict checker to influx_inspect (#23616)
adds two commands "check-schema" and
"merge-schema" to influx_inspect.
These test for field type conflicts
in all fields.idx beneath a directory
and merges the derived schemas if
"check-schema" has been run multiple
times on different directories
2022-08-10 09:36:58 -07:00
Brandon Pfeifer 1ed48ae699
feat: upgrade musl to 1.2.3 (#23603) 2022-08-02 12:32:03 -04:00
davidby-influx eb3cc88772
fix: generalize test for Windows (#23580)
Also eliminate race condition in tests

(cherry picked from commit 7e37a7ad16)
2022-07-21 13:28:10 -07:00
davidby-influx a8732dcf52
fix: restore in-memory Manifest on write error (#23552)
Do not update the `FileSet` or `activeLogFile` field in the in-memory
Partition structure if the Manifest file is not correctly saved to
the disk.

closes https://github.com/influxdata/influxdb/issues/23553
2022-07-20 12:59:15 -07:00
Brandon Pfeifer 771e3647ee
build: upgrade to Go 1.18.4 (#23568) 2022-07-20 12:18:36 -04:00
davidby-influx 5c22d6c729
fix: add reporttsi to the help text (#23566)
reporttsi was not listed as a command in the influx_inspect help text.
2022-07-19 13:24:22 -07:00
davidby-influx 25cea95beb
fix: add paths to tsi log and index file errors (#23557)
Add paths to various TSI errors on opening and unmarshaling files
to help poinpoint the corrupt files.

Closes https://github.com/influxdata/influxdb/issues/23556
2022-07-19 09:02:20 -07:00
Brandon Pfeifer b4908528a8
chore: Publish Packages and CHANGELOG.md to S3 (1.x) (#23549)
* chore: split workflow into "release" and "snapshot"

* feat: sign release packages

* feat: publish changelog to S3

* feat: publish packages to S3
2022-07-14 15:17:26 -04:00
davidby-influx 061cf55f2a
fix: create TSI MANIFEST files atomically (#23539)
When a MANIFEST file is created in TSI, it
should be written to a temp file, then
atomically renamed, to avoid overwriting
the existing file only to fail on the
later write.

closes https://github.com/influxdata/influxdb/issues/23536
2022-07-13 10:11:49 -07:00
davidby-influx a2dd708a26
fix: improve error messages opening index partitions (#23532)
Where possible, add the file path path to any errors
on opening, reading, (un)marshaling, or validating
the various files comprising a partition

closes https://github.com/influxdata/influxdb/issues/23506
2022-07-12 14:22:36 -07:00
Brandon Pfeifer a33493531e
feat: use new changelogger (#23511)
This also regenerates CHANGELOG_frozen.md (from version 1.8.0 - 1.9.7)
so that it is compatible with the new changelogger.
2022-07-01 12:24:54 -04:00
davidby-influx 521adbdcb4
chore: update flux to 0.170.1 (#23489)
Last flux version before test framework change.
2022-06-23 12:11:20 -07:00
Geoffrey Wossum dd356e0bcc
fix: eliminate race condition on Monitor.globalTags (#23467)
fixes #23466
2022-06-16 18:04:59 -05:00
davidby-influx a428043f84
fix: lost TSI reference / close TagValueSeriesIDIterator in error case (#23461) (#23462)
(cherry picked from commit 8bd4fc502d)

closes https://github.com/influxdata/influxdb/issues/23460

Co-authored-by: Dane Strandboge <dstrandboge@influxdata.com>
2022-06-16 11:54:04 -07:00
davidby-influx 54ac7e54ed
fix: remember shards that fail Open(), avoid repeated attempts (#23437)
If a shard cannot be opened, store its ID and last error.
Prevent future attempts to open during this invocation of
influxDB. This information is not persisted.

closes https://github.com/influxdata/influxdb/issues/23428
closes https://github.com/influxdata/influxdb/issues/23426
2022-06-13 10:32:47 -07:00
davidby-influx d3db48e93d
fix: fully clean up partially opened TSI (#23430)
When one partition in a TSI fails to open, all previously opened
partitions should be cleaned up, and remaining partitions 
should not be opened

closes https://github.com/influxdata/influxdb/issues/23427
2022-06-10 11:31:29 -07:00
Sam Arnold febf7a927c
feat: log the log level regardless of log level (#23425) 2022-06-10 08:38:55 -04:00
Dane Strandboge c44ec5974d
build: upgrade to Go 1.18.3 (#23408) 2022-06-07 13:09:04 -05:00
davidby-influx ec412f793b
fix: do not rename files on mmap failure (#23396)
If NewTSMReader() fails because mmap fails, do not
rename the file, because the error is probably
caused by vm.max_map_count being too low

closes https://github.com/influxdata/influxdb/issues/23172
2022-06-07 08:37:00 -07:00
davidby-influx 0ae0bd6e2e
fix: replace unprintable and invalid characters in errors (#23387)
Replace unprintable and invalid characters with '?'
in logged errors.  Truncate consecutive runs of them to
only 3 repeats of '?'

closes https://github.com/influxdata/influxdb/issues/23386
2022-06-01 13:45:24 -07:00
Jamie Strandboge df65c01ba2
chore: update yaml.v3 to 3.0.1 (#23392)
$ go mod edit -require gopkg.in/yaml.v3@v3.0.1
$ go mod tidy
$ go test -v ./...
2022-06-01 12:33:46 -05:00
davidby-influx ef90bc830f
feat: estimate Cloud2 cardinality on 1.X databases (#23351)
feat: estimate Cloud2 cardinality on 1.X databases

To ease migrations to Cloud 2 installations from
1.X databases, estimate Cloud 2 cardinality for
a data node (or OSS system).

closes https://github.com/influxdata/influxdb/issues/23356
2022-05-26 11:27:17 -07:00
Brandon Pfeifer 522c32754c
build: update changelogger to ignore case when parsing verbs (#23368) 2022-05-24 16:07:57 -04:00
Brandon Pfeifer 3074e62784
chore: upgrade flux to v1.167.0 for 1.x (#23349)
* chore: backport 8334dd0a23 to 1.x

* chore: upgrade flux to v0.167.0

* chore: update flux to latest version (#23249)

* chore: update flux to latest version

* fix: backport "convert allocator to interface"

* fix: construct `span` during dependency injection
2022-05-17 15:59:24 -04:00
Andrew Lee 7c31f92067
fix: remove data directory appending for influx_inspect verify (#23336)
influx_inspect verify -dir will no longer append the "/data" path to the dir.  Files are checked recursively, so this will still include files in the "/data" path as well as other subdirectories.
 
closes https://github.com/influxdata/influxdb/issues/22572
2022-05-13 18:58:44 -06:00
davidby-influx 1c89102276
feat: log slow queries even without query logging (#23320)
Log long-running queries if "log-queries-after" > 0,
even if general query logging is not enabled.

closes https://github.com/influxdata/influxdb/issues/23147
2022-05-11 13:12:09 -07:00
J. Emrys Landivar a894717c2c
fix(flux): inject sane defaults dependency for flux (#23309) (#23313)
This is to prevent an error and also to remove the size limit for
queries.

Also to prevent the following error:
```
/ # cat broke.flux 
import "types"

host="http://127.0.0.1:8086"
token="myuser:mypass"

from(bucket: "vehicle_communication/30days", host: host, token: token)
    |> range(start: -1h)
    |> filter(fn: (r) => types.isType(v: r["_value"], type: "int"))
    |> aggregateWindow(every: 1m, fn: mean)



/ # cat broke.flux | /influx -username influx_support -type flux -password <pass>
{"error":"failed to initialize execute state: Provider.ReaderFor called on an error dependency"}
```
2022-04-28 15:09:03 -05:00
Brandon Pfeifer 8f78c4bb02
chore: backport semantic PR and commit message checks (#23306)
* chore: use common semantic PR and commit message checks (#23300)

After this is merged, the old semantic checker can be disabled, and its config file removed from the repository.

* chore: remove previous semantic pull request config (#23305)

* chore: remove previous semantic pull request config

* chore: update pull request template, conventional commit version

Co-authored-by: Jacob Marble <jacobmarble@influxdata.com>
2022-04-27 11:09:06 -04:00