Commit Graph

14901 Commits (8c908c547b9812e31ab8cf3fa5923ef8ede27631)

Author SHA1 Message Date
davidby-influx 061cf55f2a
fix: create TSI MANIFEST files atomically (#23539)
When a MANIFEST file is created in TSI, it
should be written to a temp file, then
atomically renamed, to avoid overwriting
the existing file only to fail on the
later write.

closes https://github.com/influxdata/influxdb/issues/23536
2022-07-13 10:11:49 -07:00
davidby-influx a2dd708a26
fix: improve error messages opening index partitions (#23532)
Where possible, add the file path path to any errors
on opening, reading, (un)marshaling, or validating
the various files comprising a partition

closes https://github.com/influxdata/influxdb/issues/23506
2022-07-12 14:22:36 -07:00
Brandon Pfeifer a33493531e
feat: use new changelogger (#23511)
This also regenerates CHANGELOG_frozen.md (from version 1.8.0 - 1.9.7)
so that it is compatible with the new changelogger.
2022-07-01 12:24:54 -04:00
davidby-influx 521adbdcb4
chore: update flux to 0.170.1 (#23489)
Last flux version before test framework change.
2022-06-23 12:11:20 -07:00
Geoffrey Wossum dd356e0bcc
fix: eliminate race condition on Monitor.globalTags (#23467)
fixes #23466
2022-06-16 18:04:59 -05:00
davidby-influx a428043f84
fix: lost TSI reference / close TagValueSeriesIDIterator in error case (#23461) (#23462)
(cherry picked from commit 8bd4fc502d)

closes https://github.com/influxdata/influxdb/issues/23460

Co-authored-by: Dane Strandboge <dstrandboge@influxdata.com>
2022-06-16 11:54:04 -07:00
davidby-influx 54ac7e54ed
fix: remember shards that fail Open(), avoid repeated attempts (#23437)
If a shard cannot be opened, store its ID and last error.
Prevent future attempts to open during this invocation of
influxDB. This information is not persisted.

closes https://github.com/influxdata/influxdb/issues/23428
closes https://github.com/influxdata/influxdb/issues/23426
2022-06-13 10:32:47 -07:00
davidby-influx d3db48e93d
fix: fully clean up partially opened TSI (#23430)
When one partition in a TSI fails to open, all previously opened
partitions should be cleaned up, and remaining partitions 
should not be opened

closes https://github.com/influxdata/influxdb/issues/23427
2022-06-10 11:31:29 -07:00
Sam Arnold febf7a927c
feat: log the log level regardless of log level (#23425) 2022-06-10 08:38:55 -04:00
Dane Strandboge c44ec5974d
build: upgrade to Go 1.18.3 (#23408) 2022-06-07 13:09:04 -05:00
davidby-influx ec412f793b
fix: do not rename files on mmap failure (#23396)
If NewTSMReader() fails because mmap fails, do not
rename the file, because the error is probably
caused by vm.max_map_count being too low

closes https://github.com/influxdata/influxdb/issues/23172
2022-06-07 08:37:00 -07:00
davidby-influx 0ae0bd6e2e
fix: replace unprintable and invalid characters in errors (#23387)
Replace unprintable and invalid characters with '?'
in logged errors.  Truncate consecutive runs of them to
only 3 repeats of '?'

closes https://github.com/influxdata/influxdb/issues/23386
2022-06-01 13:45:24 -07:00
Jamie Strandboge df65c01ba2
chore: update yaml.v3 to 3.0.1 (#23392)
$ go mod edit -require gopkg.in/yaml.v3@v3.0.1
$ go mod tidy
$ go test -v ./...
2022-06-01 12:33:46 -05:00
davidby-influx ef90bc830f
feat: estimate Cloud2 cardinality on 1.X databases (#23351)
feat: estimate Cloud2 cardinality on 1.X databases

To ease migrations to Cloud 2 installations from
1.X databases, estimate Cloud 2 cardinality for
a data node (or OSS system).

closes https://github.com/influxdata/influxdb/issues/23356
2022-05-26 11:27:17 -07:00
Brandon Pfeifer 522c32754c
build: update changelogger to ignore case when parsing verbs (#23368) 2022-05-24 16:07:57 -04:00
Brandon Pfeifer 3074e62784
chore: upgrade flux to v1.167.0 for 1.x (#23349)
* chore: backport 8334dd0a23 to 1.x

* chore: upgrade flux to v0.167.0

* chore: update flux to latest version (#23249)

* chore: update flux to latest version

* fix: backport "convert allocator to interface"

* fix: construct `span` during dependency injection
2022-05-17 15:59:24 -04:00
Andrew Lee 7c31f92067
fix: remove data directory appending for influx_inspect verify (#23336)
influx_inspect verify -dir will no longer append the "/data" path to the dir.  Files are checked recursively, so this will still include files in the "/data" path as well as other subdirectories.
 
closes https://github.com/influxdata/influxdb/issues/22572
2022-05-13 18:58:44 -06:00
davidby-influx 1c89102276
feat: log slow queries even without query logging (#23320)
Log long-running queries if "log-queries-after" > 0,
even if general query logging is not enabled.

closes https://github.com/influxdata/influxdb/issues/23147
2022-05-11 13:12:09 -07:00
J. Emrys Landivar a894717c2c
fix(flux): inject sane defaults dependency for flux (#23309) (#23313)
This is to prevent an error and also to remove the size limit for
queries.

Also to prevent the following error:
```
/ # cat broke.flux 
import "types"

host="http://127.0.0.1:8086"
token="myuser:mypass"

from(bucket: "vehicle_communication/30days", host: host, token: token)
    |> range(start: -1h)
    |> filter(fn: (r) => types.isType(v: r["_value"], type: "int"))
    |> aggregateWindow(every: 1m, fn: mean)



/ # cat broke.flux | /influx -username influx_support -type flux -password <pass>
{"error":"failed to initialize execute state: Provider.ReaderFor called on an error dependency"}
```
2022-04-28 15:09:03 -05:00
Brandon Pfeifer 8f78c4bb02
chore: backport semantic PR and commit message checks (#23306)
* chore: use common semantic PR and commit message checks (#23300)

After this is merged, the old semantic checker can be disabled, and its config file removed from the repository.

* chore: remove previous semantic pull request config (#23305)

* chore: remove previous semantic pull request config

* chore: update pull request template, conventional commit version

Co-authored-by: Jacob Marble <jacobmarble@influxdata.com>
2022-04-27 11:09:06 -04:00
davidby-influx d01ea61465
feat: stub out bucket members, owners, and labels (#23294)
For the V2 buckets API, stub out the unsupportable endpoints.
2022-04-25 11:42:01 -07:00
davidby-influx ce40acb298
feat: partial /v2/api/buckets support: create, delete, list, retrieve, update (#23284)
Partially implements the /v2/api/buckets
POST for create a bucket
DELETE for deleting a bucket
GET for listing buckets
GET for retrieving one bucket
PATCH for modifying a bucket

See here for API details:
https://docs.influxdata.com/influxdb/cloud/api/#tag/Buckets
2022-04-21 16:36:19 -07:00
Geoffrey Wossum 160cf678d5
fix: MeasurementsCardinality should not be less than 0 (#23286)
Clamp the value of Store.MeasurementsCardinality so that it can not be less
than 0. This primarily shows up as a negative `numMeasurements` value in
/debug/vars under some circumstances.

refs #23285
2022-04-21 13:32:12 -05:00
Yun Zhao 997698f891
fix(httpd): abort processing write request when encountering a precision error (#21746) 2022-04-14 13:56:24 -05:00
Nicolas Hennion 717d93ff96
chore: fix typo in config.sample.toml (#21125) 2022-04-14 13:26:21 -05:00
Shiwen Cheng 476111c40f
fix(influx_inspect): multiple retention policies bug in influx_inspect export (#23197) 2022-04-14 12:59:06 -05:00
Dane Strandboge 753f503424
build: upgrade to latest crossbuilder (#23273)
- Upgrades Go 1.18 -> 1.18.1
- Upgrades Rust 1.53 -> 1.58.1
- Fixes an issue with OSXCross and Darwin builds. This results in the new minimum OSX version being `MacOSX10.14`/`darwin18`
2022-04-13 14:48:07 -05:00
Dane Strandboge 1d268cf9c3
build: bump xbuilder for parity with enterprise (#23254) 2022-04-04 16:04:10 -05:00
Dane Strandboge 0574163566
build: upgrade to go1.18 (#23250) 2022-03-31 16:17:57 -05:00
davidby-influx 2fe52d8edc
fix: implement v2/api/delete (#23247)
Support to V2 delete api

Does not support deletion by field, only by tag value, time, and measurement.
2022-03-31 13:40:14 -07:00
davidby-influx c3a958cb5f
fix: return underlying error creating a subscription (#23217)
When creating a subscription, return the wrapped error
on failure

closes https://github.com/influxdata/influxdb/issues/23216
2022-03-22 13:04:08 -07:00
Sam Arnold ba1c573115
chore: upgrade flux to v0.159.0 (#23192)
* chore: upgrade flux to v0.159.0

* chore: skip flux database connection tests
2022-03-15 10:49:43 -04:00
davidby-influx 7d182158f4
fix: add database to MaxSeriesPerDatabase error message (#23113)
To simplify debugging, print the database name when the
max-series-per-database limit is exceeded in InMem indices.

closes https://github.com/influxdata/influxdb/issues/23112
2022-02-08 11:52:14 -08:00
Brandon Pfeifer e9f44a401a
build: allow circleci to build 1.x packages (#23104)
* build: enable version 1.x builds in circleci (#23087)

* wip: oss build first try

* build: remove unused release-engineering files

* build: update alpine and fpm versions

* build: remove static package builds

* fix: allow for ubuntu and centos package builds

* fix(build): supply INFLUXDB_VERSION to go and package buildsystems

rebaseme

* build: test debian and rpm packages

Co-authored-by: Sam Arnold <sarnold@influxdata.com>

* fix: resolve quoting issues in go build

* fix: rework version detection for circleci builds

Co-authored-by: Sam Arnold <sarnold@influxdata.com>
2022-02-07 12:15:05 -05:00
davidby-influx f27df39c03
fix: add additional testing for MaxSeriesPerDatabase (#23094)
Added test to ensure new code path taken for inmem index
2022-02-02 13:16:09 -08:00
davidby-influx 0c3dca883e
fix: correctly handle MaxSeriesPerDatabaseExceeded (#23091)
Check for the correctly returned PartialWriteError
in (*shard).validateSeriesAndFields, allow partial
writes.

closes https://github.com/influxdata/influxdb/issues/23090
2022-02-01 19:08:51 -08:00
Geoffrey Wossum 1ab50d7557
fix: `influxd-ctl backup` will create a working backup when only `-shard` given (#22998)
`influxd-ctl backup` will now create a working backup when only the `-shard`
option is given. Previously this would create a backup that could not be
restored.

fixes #16751
2022-01-28 14:59:20 -06:00
davidby-influx eb3bc7069f
feat: configurable DELETE concurrency (#23055)
Currently, deletion of series or measurements are
serialized. This new feature will add
max-concurrent-deletes to the [data] section of the
 configuration file. Legal values are any positive
 number, defaulting to 1, the current behavior.

 closes https://github.com/influxdata/influxdb/issues/23054
2022-01-13 11:04:57 -08:00
Sam Arnold d2da5c5088
build: use latest build container (#23057) 2022-01-13 10:05:53 -05:00
Sam Arnold 3ce6cb45aa
fix: allow limiting shards during flux iterator reads (#23052) 2022-01-12 19:29:32 -05:00
J. Emrys Landivar 004438e91f
build: bigger stack for musl builds (#23043) 2022-01-07 12:33:12 -06:00
Sam Arnold e8c9f54532
test: increase timeout in continuous query test (#22955)
Closes: #22954
2021-12-01 13:26:39 -08:00
Dane Strandboge b2984faef7
chore: upgrade Flux, restore `rules_test.go` (#22931)
Upgrade Flux to v0.140.0
2021-11-25 08:13:46 -05:00
Sam Arnold 956915296a
fix: use defer for unlock (#22929) 2021-11-24 14:43:46 -05:00
davidby-influx 08a4a710eb
chore: upgrade Go to 1.17 (#22921)
* chore: upgrade Go to 1.17

* fix: update circleci container to go1.17

* fix: correct unsafe conversion of []byte to string

Co-authored-by: Sam Arnold <sarnold@influxdata.com>
2021-11-24 12:47:26 -05:00
lifeibo 5be1c044c3
fix(tsi): sync index file before close (#21932) 2021-11-24 08:36:03 -05:00
Sam Arnold d4fd1cb4ca
chore: update changelog container (#22834) 2021-11-10 11:12:58 -05:00
davidby-influx 3f7192ab3c
fix: log at Debug level in influx_tools compact-shard (#22848)
Closes https://github.com/influxdata/influxdb/issues/22846
2021-11-08 16:50:48 -08:00
Sam Arnold 1b03f7ec35
chore: don't load approximate cache before saving new version (#22827) 2021-11-05 11:51:12 -04:00
Geoffrey Wossum 91609fdd3f
fix(restore): fix race condition which causes restore command to fail (#22796)
* fix(restore): fix race condition which causes restore command to fail

Fixes a race condition in the restore code path that causes shard data restores
to fail. When the bug occurs, `Error while freeing cold shard resources`
appears in the log files.

fixes issue #15323
2021-11-03 14:21:33 -05:00