Commit Graph

14918 Commits (db/6263/compaction-debug-logging)

Author SHA1 Message Date
davidby-influx d862a82515
feat: add version number to debug/vars (#23795)
closes https://github.com/influxdata/influxdb/issues/23793
2022-10-13 11:49:30 -07:00
davidby-influx bc8d9ea9f3
fix: add tests for file rename across volumes (#23787)
Also move shared code from file_unix.go
2022-10-13 09:26:59 -07:00
davidby-influx 0913276ff0
fix: use copy when a rename spans volumes (#23785)
When a file rename fails with EXDEV
(cross device or volume error), copy the
file and delete the original instead

Differs from master branch by overwriting
existing files instead of erring.

closes https://github.com/influxdata/influxdb/issues/22997
2022-10-12 09:50:27 -07:00
Brandon Pfeifer 03ad3443ab
chore: upgrade to Go 1.18.6 (#23745) 2022-09-26 10:10:27 -04:00
davidby-influx b17f27a5d9
fix: incorrect error message concatenation (#23729) 2022-09-15 09:26:51 -07:00
davidby-influx 80c10c8c04
feat: optimize saving changes to fields.idx (#23701)
Instead of writing out the complete fields.idx
file when it changes, write out incremental
changes that will be applied to the file on
close and startup.

closes https://github.com/influxdata/influxdb/issues/23653
2022-09-14 13:14:09 -07:00
Brandon Pfeifer df6a5e68e0
chore: add protoc-gen script to releng (#23694) 2022-08-31 16:42:29 -04:00
Brandon Pfeifer a52893479e
feat: enable static-pie builds (1.x) (#23642)
* feat: enable static pie builds

* fix: update cross-builder to fix 'rdynamic'
2022-08-19 12:34:59 -04:00
Brandon Pfeifer 787a168f32
chore: use 1.x signing key (#23611) 2022-08-10 13:10:52 -04:00
davidby-influx 84c4f676b0
feat: add type conflict checker to influx_inspect (#23616)
adds two commands "check-schema" and
"merge-schema" to influx_inspect.
These test for field type conflicts
in all fields.idx beneath a directory
and merges the derived schemas if
"check-schema" has been run multiple
times on different directories
2022-08-10 09:36:58 -07:00
Brandon Pfeifer 1ed48ae699
feat: upgrade musl to 1.2.3 (#23603) 2022-08-02 12:32:03 -04:00
davidby-influx eb3cc88772
fix: generalize test for Windows (#23580)
Also eliminate race condition in tests

(cherry picked from commit 7e37a7ad16)
2022-07-21 13:28:10 -07:00
davidby-influx a8732dcf52
fix: restore in-memory Manifest on write error (#23552)
Do not update the `FileSet` or `activeLogFile` field in the in-memory
Partition structure if the Manifest file is not correctly saved to
the disk.

closes https://github.com/influxdata/influxdb/issues/23553
2022-07-20 12:59:15 -07:00
Brandon Pfeifer 771e3647ee
build: upgrade to Go 1.18.4 (#23568) 2022-07-20 12:18:36 -04:00
davidby-influx 5c22d6c729
fix: add reporttsi to the help text (#23566)
reporttsi was not listed as a command in the influx_inspect help text.
2022-07-19 13:24:22 -07:00
davidby-influx 25cea95beb
fix: add paths to tsi log and index file errors (#23557)
Add paths to various TSI errors on opening and unmarshaling files
to help poinpoint the corrupt files.

Closes https://github.com/influxdata/influxdb/issues/23556
2022-07-19 09:02:20 -07:00
Brandon Pfeifer b4908528a8
chore: Publish Packages and CHANGELOG.md to S3 (1.x) (#23549)
* chore: split workflow into "release" and "snapshot"

* feat: sign release packages

* feat: publish changelog to S3

* feat: publish packages to S3
2022-07-14 15:17:26 -04:00
davidby-influx 061cf55f2a
fix: create TSI MANIFEST files atomically (#23539)
When a MANIFEST file is created in TSI, it
should be written to a temp file, then
atomically renamed, to avoid overwriting
the existing file only to fail on the
later write.

closes https://github.com/influxdata/influxdb/issues/23536
2022-07-13 10:11:49 -07:00
davidby-influx a2dd708a26
fix: improve error messages opening index partitions (#23532)
Where possible, add the file path path to any errors
on opening, reading, (un)marshaling, or validating
the various files comprising a partition

closes https://github.com/influxdata/influxdb/issues/23506
2022-07-12 14:22:36 -07:00
Brandon Pfeifer a33493531e
feat: use new changelogger (#23511)
This also regenerates CHANGELOG_frozen.md (from version 1.8.0 - 1.9.7)
so that it is compatible with the new changelogger.
2022-07-01 12:24:54 -04:00
davidby-influx 521adbdcb4
chore: update flux to 0.170.1 (#23489)
Last flux version before test framework change.
2022-06-23 12:11:20 -07:00
Geoffrey Wossum dd356e0bcc
fix: eliminate race condition on Monitor.globalTags (#23467)
fixes #23466
2022-06-16 18:04:59 -05:00
davidby-influx a428043f84
fix: lost TSI reference / close TagValueSeriesIDIterator in error case (#23461) (#23462)
(cherry picked from commit 8bd4fc502d)

closes https://github.com/influxdata/influxdb/issues/23460

Co-authored-by: Dane Strandboge <dstrandboge@influxdata.com>
2022-06-16 11:54:04 -07:00
davidby-influx 54ac7e54ed
fix: remember shards that fail Open(), avoid repeated attempts (#23437)
If a shard cannot be opened, store its ID and last error.
Prevent future attempts to open during this invocation of
influxDB. This information is not persisted.

closes https://github.com/influxdata/influxdb/issues/23428
closes https://github.com/influxdata/influxdb/issues/23426
2022-06-13 10:32:47 -07:00
davidby-influx d3db48e93d
fix: fully clean up partially opened TSI (#23430)
When one partition in a TSI fails to open, all previously opened
partitions should be cleaned up, and remaining partitions 
should not be opened

closes https://github.com/influxdata/influxdb/issues/23427
2022-06-10 11:31:29 -07:00
Sam Arnold febf7a927c
feat: log the log level regardless of log level (#23425) 2022-06-10 08:38:55 -04:00
Dane Strandboge c44ec5974d
build: upgrade to Go 1.18.3 (#23408) 2022-06-07 13:09:04 -05:00
davidby-influx ec412f793b
fix: do not rename files on mmap failure (#23396)
If NewTSMReader() fails because mmap fails, do not
rename the file, because the error is probably
caused by vm.max_map_count being too low

closes https://github.com/influxdata/influxdb/issues/23172
2022-06-07 08:37:00 -07:00
davidby-influx 0ae0bd6e2e
fix: replace unprintable and invalid characters in errors (#23387)
Replace unprintable and invalid characters with '?'
in logged errors.  Truncate consecutive runs of them to
only 3 repeats of '?'

closes https://github.com/influxdata/influxdb/issues/23386
2022-06-01 13:45:24 -07:00
Jamie Strandboge df65c01ba2
chore: update yaml.v3 to 3.0.1 (#23392)
$ go mod edit -require gopkg.in/yaml.v3@v3.0.1
$ go mod tidy
$ go test -v ./...
2022-06-01 12:33:46 -05:00
davidby-influx ef90bc830f
feat: estimate Cloud2 cardinality on 1.X databases (#23351)
feat: estimate Cloud2 cardinality on 1.X databases

To ease migrations to Cloud 2 installations from
1.X databases, estimate Cloud 2 cardinality for
a data node (or OSS system).

closes https://github.com/influxdata/influxdb/issues/23356
2022-05-26 11:27:17 -07:00
Brandon Pfeifer 522c32754c
build: update changelogger to ignore case when parsing verbs (#23368) 2022-05-24 16:07:57 -04:00
Brandon Pfeifer 3074e62784
chore: upgrade flux to v1.167.0 for 1.x (#23349)
* chore: backport 8334dd0a23 to 1.x

* chore: upgrade flux to v0.167.0

* chore: update flux to latest version (#23249)

* chore: update flux to latest version

* fix: backport "convert allocator to interface"

* fix: construct `span` during dependency injection
2022-05-17 15:59:24 -04:00
Andrew Lee 7c31f92067
fix: remove data directory appending for influx_inspect verify (#23336)
influx_inspect verify -dir will no longer append the "/data" path to the dir.  Files are checked recursively, so this will still include files in the "/data" path as well as other subdirectories.
 
closes https://github.com/influxdata/influxdb/issues/22572
2022-05-13 18:58:44 -06:00
davidby-influx 1c89102276
feat: log slow queries even without query logging (#23320)
Log long-running queries if "log-queries-after" > 0,
even if general query logging is not enabled.

closes https://github.com/influxdata/influxdb/issues/23147
2022-05-11 13:12:09 -07:00
J. Emrys Landivar a894717c2c
fix(flux): inject sane defaults dependency for flux (#23309) (#23313)
This is to prevent an error and also to remove the size limit for
queries.

Also to prevent the following error:
```
/ # cat broke.flux 
import "types"

host="http://127.0.0.1:8086"
token="myuser:mypass"

from(bucket: "vehicle_communication/30days", host: host, token: token)
    |> range(start: -1h)
    |> filter(fn: (r) => types.isType(v: r["_value"], type: "int"))
    |> aggregateWindow(every: 1m, fn: mean)



/ # cat broke.flux | /influx -username influx_support -type flux -password <pass>
{"error":"failed to initialize execute state: Provider.ReaderFor called on an error dependency"}
```
2022-04-28 15:09:03 -05:00
Brandon Pfeifer 8f78c4bb02
chore: backport semantic PR and commit message checks (#23306)
* chore: use common semantic PR and commit message checks (#23300)

After this is merged, the old semantic checker can be disabled, and its config file removed from the repository.

* chore: remove previous semantic pull request config (#23305)

* chore: remove previous semantic pull request config

* chore: update pull request template, conventional commit version

Co-authored-by: Jacob Marble <jacobmarble@influxdata.com>
2022-04-27 11:09:06 -04:00
davidby-influx d01ea61465
feat: stub out bucket members, owners, and labels (#23294)
For the V2 buckets API, stub out the unsupportable endpoints.
2022-04-25 11:42:01 -07:00
davidby-influx ce40acb298
feat: partial /v2/api/buckets support: create, delete, list, retrieve, update (#23284)
Partially implements the /v2/api/buckets
POST for create a bucket
DELETE for deleting a bucket
GET for listing buckets
GET for retrieving one bucket
PATCH for modifying a bucket

See here for API details:
https://docs.influxdata.com/influxdb/cloud/api/#tag/Buckets
2022-04-21 16:36:19 -07:00
Geoffrey Wossum 160cf678d5
fix: MeasurementsCardinality should not be less than 0 (#23286)
Clamp the value of Store.MeasurementsCardinality so that it can not be less
than 0. This primarily shows up as a negative `numMeasurements` value in
/debug/vars under some circumstances.

refs #23285
2022-04-21 13:32:12 -05:00
Yun Zhao 997698f891
fix(httpd): abort processing write request when encountering a precision error (#21746) 2022-04-14 13:56:24 -05:00
Nicolas Hennion 717d93ff96
chore: fix typo in config.sample.toml (#21125) 2022-04-14 13:26:21 -05:00
Shiwen Cheng 476111c40f
fix(influx_inspect): multiple retention policies bug in influx_inspect export (#23197) 2022-04-14 12:59:06 -05:00
Dane Strandboge 753f503424
build: upgrade to latest crossbuilder (#23273)
- Upgrades Go 1.18 -> 1.18.1
- Upgrades Rust 1.53 -> 1.58.1
- Fixes an issue with OSXCross and Darwin builds. This results in the new minimum OSX version being `MacOSX10.14`/`darwin18`
2022-04-13 14:48:07 -05:00
Dane Strandboge 1d268cf9c3
build: bump xbuilder for parity with enterprise (#23254) 2022-04-04 16:04:10 -05:00
Dane Strandboge 0574163566
build: upgrade to go1.18 (#23250) 2022-03-31 16:17:57 -05:00
davidby-influx 2fe52d8edc
fix: implement v2/api/delete (#23247)
Support to V2 delete api

Does not support deletion by field, only by tag value, time, and measurement.
2022-03-31 13:40:14 -07:00
davidby-influx c3a958cb5f
fix: return underlying error creating a subscription (#23217)
When creating a subscription, return the wrapped error
on failure

closes https://github.com/influxdata/influxdb/issues/23216
2022-03-22 13:04:08 -07:00
Sam Arnold ba1c573115
chore: upgrade flux to v0.159.0 (#23192)
* chore: upgrade flux to v0.159.0

* chore: skip flux database connection tests
2022-03-15 10:49:43 -04:00
davidby-influx 7d182158f4
fix: add database to MaxSeriesPerDatabase error message (#23113)
To simplify debugging, print the database name when the
max-series-per-database limit is exceeded in InMem indices.

closes https://github.com/influxdata/influxdb/issues/23112
2022-02-08 11:52:14 -08:00