Commit Graph

14730 Commits (1.9)

Author SHA1 Message Date
davidby-influx 7f8441e996
feat: optionally dump queries to log on SIGTERM (#22638) (#22683)
Dump all active queries to the log when a SIGTERM
is received and the termination-query-log flag is
true in the coordinator section of the config. The
default is false.

(cherry picked from commit fd1d51690d)

closes https://github.com/influxdata/influxdb/issues/22637
2021-10-15 12:20:03 -07:00
davidby-influx 875c538bcd
feat: add thread-safe access to CountingWriter byte total (#22620) (#22682)
Use atomic operations to update and report CountingWriter.Total through a new method.

closes https://github.com/influxdata/influxdb/issues/22618

(cherry picked from commit 022b6e866b)

closes https://github.com/influxdata/influxdb/issues/22619
2021-10-15 12:17:45 -07:00
davidby-influx 993bdb244b
fix(restore): enforce the -db parameter when -newdb used (#22542) (#22677)
closes https://github.com/influxdata/influxdb/issues/15901

(cherry picked from commit 1dde65bb75)

closes https://github.com/influxdata/influxdb/issues/22543
2021-10-14 20:01:16 -07:00
davidby-influx 159b10543c
fix: for Windows, copy snapshot files being backed up (#22551) (#22676)
On Windows, make copies of files for snapshots, because
Go does not support the FILE_SHARE_DELETE flag which
allows files (and links) to be deleted while open. This
causes temporary directories to be left behind after
backups.

closes https://github.com/influxdata/influxdb/issues/16289

(cherry picked from commit 3702fe8e76)

Closes https://github.com/influxdata/influxdb/issues/22556
2021-10-14 16:54:34 -07:00
davidby-influx 9d3508c54d
fix: For Windows, close temp file before removing (#22492) (#22675)
closes https://github.com/influxdata/influxdb/issues/21470

(cherry picked from commit d2199efcd7)

Closes https://github.com/influxdata/influxdb/issues/22493
2021-10-14 16:52:35 -07:00
Sam Arnold a083fa90d0
test: fix DiskSizeBytes flakiness (#22641) (#22672)
(cherry picked from commit 59fe8e515e)
2021-10-14 15:25:28 -04:00
Sam Arnold b5825b3aeb
fix: better error for no data from snapshots (#22452) (#22666)
Closes: #16739
(cherry picked from commit 84b785b78c)
2021-10-14 13:51:32 -04:00
Sam Arnold be940bcfc5
fix: create shards without overlaps (#22601) (#22665)
(cherry picked from commit 3c811bcd81)
2021-10-14 10:52:57 -04:00
Dane Strandboge 92a524ef9e
chore: increase timer to 5 seconds (#22663) 2021-10-13 14:04:00 -05:00
Cody Shepherd 10c242c19b
chore: Revert "fix(tsdb): sync series segment to disk after writing (#22565)" (#22605)
This reverts commit 6692a14cf4.
2021-09-30 14:27:53 -07:00
Cody Shepherd 6493ce1619
fix: update flux dependency to v0.131.0 (#22603) 2021-09-30 13:51:31 -07:00
Dane Strandboge 6692a14cf4
fix(tsdb): sync series segment to disk after writing (#22565) 2021-09-23 14:10:04 -05:00
Cody Shepherd 995e5301d8
build: automate changelog (#22533) 2021-09-21 13:38:07 -07:00
Cody Shepherd daa4bc4672
chore: update changelog for 1.9.4rc2 (#22468) 2021-09-13 14:18:18 -07:00
Cody Shepherd 3037d9f44b
fix: influxdb packages should depend on curl bc of use in systemd script (#22250) 2021-09-07 08:22:01 -07:00
davidby-influx 25e21c90f2
fix: discard excessive errors (#22379) (#22389)
The tsmBatchKeyIterator discards excessive errors to avoid
out-of-memory crashes when compacting very corrupt files.
Any error beyond DefaultMaxSavedErrors (100) will be
discarded instead of appended to the error slice.

closes https://github.com/influxdata/influxdb/issues/22328

(cherry picked from commit e53f75e06d)

Closes https://github.com/influxdata/influxdb/issues/22380
2021-09-03 12:30:34 -07:00
Cody Shepherd d9a144b796
build: upgrade flux to 0.127.3 (#22375) (#22376)
Co-authored-by: Sam Arnold <sarnold@influxdata.com>
2021-09-02 11:56:50 -07:00
Sam Arnold 190728180a
fix: flux error properly read by cloud (#22348) (#22359)
(cherry picked from commit 38de69cc1c)
2021-09-01 13:03:28 -04:00
davidby-influx b519b630aa
fix: correct error return shadowing (#22354) 2021-08-31 11:47:31 -07:00
Sam Arnold 65aba7b65b
fix: TSI logfile race (#22338) (#22341)
modTime should be protected by the read lock.

Fixes #22337

(cherry picked from commit 1755b8f6d2)
2021-08-30 17:59:02 -04:00
Sam Arnold ff2c99a938
build: update circle jobs to use cross build container (#22286) (#22332)
* build: update circle jobs to use cross build container

* build: fixes for review comments

(cherry picked from commit 27e5f9709e)
2021-08-30 15:37:16 -04:00
Sam Arnold d565159edf
test: add extra logging when disk size test fails (#22103) (#22296)
(cherry picked from commit 3ae389b359)
2021-08-24 16:32:05 -04:00
davidby-influx d6aa0e69de
fix: return correct count of ErrNotExecuted (#22273) (#22294)
executeQuery() iterates over statements until each is
processed or if an error is encountered that causes
the loop to exit pre-maturely. It should return
ErrNotExecuted for each remaining statement in the
query

closes https://github.com/influxdata/influxdb/issues/19136

(cherry-picked from commit 0090c5b111)

closes https://github.com/influxdata/influxdb/issues/22274
2021-08-24 12:09:28 -07:00
davidby-influx 509d34e784
fix: require database authorization to see continuous queries (#22283) (#22289)
SHOW CONTINUOUS QUERIES now requires the same permissions
as SHOW DATABASES in order to see the continuous queries
in a database.

closes https://github.com/influxdata/influxdb/issues/10292

(cherry picked from commit 71b3d06494)

closes https://github.com/influxdata/influxdb/issues/22284
2021-08-24 11:15:20 -07:00
davidby-influx 0ff2196808
fix: avoid compaction queue stats flutter. (#22195) (#22214)
When the compaction planner runs, if it cannot acquire
a lock on the files it plans to compact, it returns a
nil list of compaction groups. This, in turn, sets the
engine statistics for compactions queues to zero,
which is incorrect. Instead, use the length of pending
files which would have been returned.

closes https://github.com/influxdata/influxdb/issues/22138

(cherry picked from commit 7d3efe1e9e)

closes https://github.com/influxdata/influxdb/issues/22139
2021-08-16 10:56:00 -07:00
Sam Arnold 9d6944553b
test: expose tcpaddr for enterprise tests (#22172) (#22189)
* docs: update comment for series updates

* fix: expose TCP address for Enterprise test harness

* refactor: remove dead RemoteServer code

(cherry picked from commit fd81373937)
2021-08-12 15:34:24 -04:00
Cody Shepherd 372fbef605
fix: systemd service -- handle https, 40x, and block indefinitely (#22091)
* fix: handle https in systemd wrapper, and prevent it from looping forever

* fix: fixes to systemd startup script

* accept 40x responses as evidence that API is up and running
* block indefinitely while waiting for API

* chore: update changelog
2021-08-10 14:36:04 -07:00
davidby-influx 3630bfee38
fix: ensure log formatting (JSON) is respected (#22106) (#22107)
Replace usage of the Go log package with the zap
logger and respect the formatting specified in
the configuration file.

closes https://github.com/influxdata/influxdb/issues/22104

(cherry picked from commit 0315d2ede1)

closes https://github.com/influxdata/influxdb/issues/22105
2021-08-09 09:30:48 -07:00
Cody Shepherd 1a321b3a08
chore: update flux dependency (#22102)
* chore: update flux dependency

* chore: update changelog
2021-08-09 07:56:51 -07:00
Sam Arnold 5fcbf37018
fix: hard limit on field size while parsing line protocol (#21843) (#22095)
Per https://docs.influxdata.com/enterprise_influxdb/v1.9/write_protocols/line_protocol_reference/
we only support 64KB, but 1MB is a more realistic practical limit. Before this commit there was
no enforcement of field value size.

Closes #22094

(cherry picked from commit 6d22e69ef1)
2021-08-06 16:03:06 -04:00
Sam Arnold 906631b0dd
chore: update CHANGELOG.md (#22074)
Closes #22042
2021-08-06 14:35:04 -04:00
Sam Arnold fcdc980711
fix: old tsl files should be compacted without new writes (#22006) (#22067)
Closes #22030

* fix: old tsl files should be compacted wihout new writes

* chore: update changelog.md

(cherry picked from commit e62efaf751)
2021-08-06 09:24:18 -04:00
Sam Arnold 9892f01c3d
feat: update flux components to use flux/array instead of arrow/array (#21979) (#22043)
Closes #22031

This updates the flux integration to use the `flux/array` package rather
than directly using the `arrow/array` package.

Flux recently switched to wrapping the array types from arrow and
creating its own array package to be used for table columns instead of
directly referencing the arrow package. This allows us to keep a
consistent interface, but potentially change internal implementations
without changing downstream consumers. Most recently, the
`*array.String` type has some of its own optimizations for certain array
patterns.

This change updates the flux integration to use the new API.

(cherry picked from commit 8fae484a66)

Co-authored-by: Jonathan A. Sternberg <jonathan@influxdata.com>
2021-08-05 09:36:09 -04:00
Sam Arnold 1585d0faa1
fix: tsi index should compact old or too-large log files (#21943) (#22044)
Closes #22033

* fix: tsi index should compact old log files that are too large

* chore: run automated formatter

* chore: update changelog

* fix: review comments

(cherry picked from commit b64c2c3dcf)
2021-08-05 09:35:53 -04:00
davidby-influx 05744ea48b
fix: copy names from mmapped memory before closing iterator (#22040) (#22058)
This fix ensures that memory-mapped files are not released
before pointers into them are copied into heap memory.
MeasurementNamesByExpr() and MeasurementNamesByPredicate() can
cause panics by copying memory from mmapped files that have been
released. The functions they call use iterators to files which
are closed (releasing the mmapped files) before the memory is
safely copied to the heap.

closes https://github.com/influxdata/influxdb/issues/22000

(cherry picked from commit a989f8f8b6)

Closes https://github.com/influxdata/influxdb/issues/22001
2021-08-04 14:27:10 -07:00
Sam Arnold 24d5c24fff
fix: error instead of panic when enterprise tries to restore with OSS (#22019) (#22048)
Closes #22029

(cherry picked from commit d0fe2ba2ff)
2021-08-04 17:08:20 -04:00
Sam Arnold 7bb93f14be
test: fix order of index teardown (#22038) (#22056)
Closes #22055

(cherry picked from commit 444c22b67d)
2021-08-04 17:06:34 -04:00
Sam Arnold 6fc7a8e4a4
fix(tsi): clean up FileSet fields (#22047)
Closes #22036

(cherry picked from commit 108e2600b3)

Co-authored-by: Tristan Su <sooqing@gmail.com>
2021-08-04 11:40:27 -04:00
Sam Arnold 71a1e5ab60
feat: add subscription buffer size usage metric (#22049)
Closes #22027

(cherry picked from commit 733a6b4245)
2021-08-04 11:35:46 -04:00
Sam Arnold 9c42ea8d11
fix: export example and fix adjacent shards (#22046)
Closes #22034

* fix: export example and fix adjacent shards

* fix: Cody's review

(cherry picked from commit 22809a33aa)
2021-08-04 11:31:58 -04:00
Sam Arnold 86166692fa
test: ensure compile takes time (#22045)
Closes #22032

(cherry picked from commit eef37b96b9)
2021-08-04 11:29:42 -04:00
Sam Arnold 08dd23f986
fix: restore portable backup error (#21993)
Originally committed by rickif in https://github.com/influxdata/influxdb/pull/20115/files

This is manual copy of the changes to master-1.x

(cherry picked from commit 5bbb61b815)
2021-08-04 11:23:27 -04:00
davidby-influx 34c1b5c607
feat: SHOW TAG VALUES should produce results from one specific RP (#22011)
* feat: SHOW TAG VALUES should produce results from one specific RP (#21983)

Ensure that the Sources field of the ShowTagValuesStatement is
filled in. Then use the sources to limit the retention policies,
and thus the shards from which tag values are collected.

This fix only works on TSI databases; INMEM shards share
indices, so restricting shard indices used does not restrict the
tag values returned.

This will not permit multiple retention policies to be specified in
a query; either all RPs or one are permitted.

Closes https://github.com/influxdata/influxdb/issues/21981

(cherry picked from commit 519e23b86a)

* chore: update CHANGELOG.md
2021-08-03 12:38:56 -04:00
Cody Shepherd bf616e9309
fix: systemd-start script should be executable by group and others (#21988)
* fix: systemd-start script should be executable by group and others

* chore: update changelog
2021-08-03 08:21:54 -07:00
Cody Shepherd c522965331
chore: pin rust version (#21995) (#22005) 2021-08-02 15:06:49 -07:00
Daniel Moran 2ffd71a0fb
fix: prevent silently dropped writes with overlapping shards (#21952)
Co-authored-by: Geoffrey Wossum <gwossum@influxdata.com>
2021-07-27 20:37:45 -04:00
Jamie Strandboge 9cf7b8853d
chore: use community maintained golang-jwt (#21935)
The dgrijalva/jwt-go project is no longer maintained[1] and they have
transferred ownership to golang-jwt/jwt[2][3][4]. We should move to the
supported golang-jwt/jwt.

The following was performed:

1. update services/httpd/handler*.go to import golang-jwt/jwt
2. revert testcase string comparison changes from 225bcecd (back to v3)
2. go mod edit -require github.com/golang-jwt/jwt@v3.2.1+incompatible
3. go mod edit -droprequire github.com/dgrijalva/jwt-go
4. go mod tidy  # see note
5. go clean ./... && go build ./...
6. go test ./...

Note: 'go mod tidy' had unrelated changes (perhaps it wasn't run in
recent commits) so I removed the unrelated delta to keep this PR focused
on the dgrijalva/jwt-go to golang-jwt/jwt changes.

References:
[1] dgrijalva/jwt-go#462
[2] dgrijalva/jwt-go#463
[3] https://github.com/dgrijalva/jwt-go/blob/master/README.md
[4] https://github.com/golang-jwt/jwt
[5] https://github.com/influxdata/influxdb/issues/21927
2021-07-23 11:41:26 -05:00
davidby-influx 4e2fa689e3
fix: don't access a field in a nil struct (#21693) (#21921)
(cherry picked from commit aca69e530f)

Closes https://github.com/influxdata/influxdb/issues/21698
2021-07-22 14:59:33 -07:00
Cody Shepherd 3cbab8e7f1
fix: systemd unit should block on startup until http endpoint is ready (#21897)
* fix: systemd unit should block on startup until http endpoint is ready

* chore: update changelog

* chore: adjust packaging procedure to enable correct systemd behavior (#21905)
2021-07-22 11:29:36 -07:00
Sam Arnold ad65da2d76
chore: update protobuf library versions and remove influx_tsm (#21882) (#21890)
* chore: update protobufs

* fix: run codegen during build

* fix: fully remove influx_tsm
2021-07-20 13:05:29 -04:00