Commit Graph

2284 Commits (1.10)

Author SHA1 Message Date
Cody Shepherd 380dade9b3
build: automate changelog (#22522) 2021-09-21 08:30:50 -07:00
Cody Shepherd 24f9f05264
fix: influxdb packages should depend on curl (#22229)
* fix: influxdb packages should depend on curl bc of use in systemd script

* chore: update changelog

Co-authored-by: Daniel Moran <danxmoran@gmail.com>
2021-09-07 08:03:26 -07:00
Sam Arnold 38de69cc1c
fix: flux error properly read by cloud (#22348) 2021-08-31 17:43:12 -04:00
Sam Arnold 1755b8f6d2
fix: TSI logfile race (#22338)
modTime should be protected by the read lock.

Fixes #22337
2021-08-30 17:43:37 -04:00
davidby-influx 0090c5b111
fix: return correct count of ErrNotExecuted (#22273)
executeQuery() iterates over statements until each is
processed or if an error is encountered that causes
the loop to exit pre-maturely. It should return
ErrNotExecuted for each remaining statement in the
query

closes https://github.com/influxdata/influxdb/issues/19136
2021-08-24 11:27:10 -07:00
davidby-influx 71b3d06494
fix: require database authorization to see continuous queries (#22283)
SHOW CONTINUOUS QUERIES now requires the same permissions
as SHOW DATABASES in order to see the continuous queries
in a database.

closes https://github.com/influxdata/influxdb/issues/10292
2021-08-24 08:56:56 -07:00
davidby-influx 7d3efe1e9e
fix: avoid compaction queue stats flutter. (#22195)
When the compaction planner runs, if it cannot acquire
a lock on the files it plans to compact, it returns a
nil list of compaction groups. This, in turn, sets the
engine statistics for compactions queues to zero,
which is incorrect. Instead, use the length of pending
files which would have been returned.

closes https://github.com/influxdata/influxdb/issues/22138
2021-08-16 09:21:07 -07:00
Cody Shepherd f71ab14888
fix: systemd service -- handle https, 40x, and block indefinitely (#22090)
* fix: systemd service --handle 40x and block indefinitely

* chore: update changelog
2021-08-10 14:35:12 -07:00
Cody Shepherd 37faac9755
chore: update flux dependency (#22097)
* chore: update flux dependency

* chore: update changelog
2021-08-09 09:32:19 -07:00
davidby-influx 0315d2ede1
fix: ensure log formatting (JSON) is respected (#22106)
Replace usage of the Go log package with the zap
logger and respect the formatting specified in
the configuration file.

closes https://github.com/influxdata/influxdb/issues/22104
2021-08-06 16:32:30 -07:00
davidby-influx a989f8f8b6
fix: copy names from mmapped memory before closing iterator (#22040)
This fix ensures that memory-mapped files are not released
before pointers into them are copied into heap memory.
MeasurementNamesByExpr() and MeasurementNamesByPredicate() can
cause panics by copying memory from mmapped files that have been
released. The functions they call use iterators to files which
are closed (releasing the mmapped files) before the memory is
safely copied to the heap.

closes https://github.com/influxdata/influxdb/issues/22000
2021-08-04 13:16:00 -07:00
Sam Arnold d0fe2ba2ff
fix: error instead of panic when enterprise tries to restore with OSS (#22019) 2021-08-03 15:56:19 -04:00
Sam Arnold 733a6b4245
feat: add subscription buffer size usage metric (#22020) 2021-08-03 14:32:41 -04:00
Cody Shepherd 8f8e9209c6
fix: systemd-start script should be executable by group and others (#21992)
* fix: systemd unit should block on startup until http endpoint is ready

* fix: systemd-start script should be executable by group and others

* chore: update changelog
2021-08-03 08:22:11 -07:00
davidby-influx 519e23b86a
feat: SHOW TAG VALUES should produce results from one specific RP (#21983)
Ensure that the Sources field of the ShowTagValuesStatement is
filled in. Then use the sources to limit the retention policies,
and thus the shards from which tag values are collected.

This fix only works on TSI databases; INMEM shards share
indices, so restricting shard indices used does not restrict the
tag values returned.

This will not permit multiple retention policies to be specified in 
a query; either all RPs or one are permitted.

Closes https://github.com/influxdata/influxdb/issues/21981
2021-08-02 11:11:39 -07:00
Sam Arnold e62efaf751
fix: old tsl files should be compacted without new writes (#22006)
* fix: old tsl files should be compacted wihout new writes

* chore: update changelog.md
2021-08-02 13:36:23 -04:00
Sam Arnold 5bbb61b815
fix: restore portable backup error (#21978)
Originally committed by rickif in https://github.com/influxdata/influxdb/pull/20115/files

This is manual copy of the changes to master-1.x
2021-07-30 13:06:54 -04:00
Daniel Moran 62484387cf
fix: prevent silently dropped writes with overlapping shards (#21947)
Co-authored-by: Geoffrey Wossum <gwossum@influxdata.com>
2021-07-27 12:29:01 -04:00
Sam Arnold b64c2c3dcf
fix: tsi index should compact old or too-large log files (#21943)
* fix: tsi index should compact old log files that are too large

* chore: run automated formatter

* chore: update changelog

* fix: review comments
2021-07-26 17:40:15 -04:00
Jamie Strandboge dec56f9c31
chore: use community maintained golang-jwt (#21934)
The dgrijalva/jwt-go project is no longer maintained[1] and they have
transferred ownership to golang-jwt/jwt[2][3][4]. We should move to the
supported golang-jwt/jwt.

The following was performed:

1. update services/httpd/handler*.go to import golang-jwt/jwt
2. revert testcase string comparison changes from 225bcecd (back to v3)
2. go mod edit -require github.com/golang-jwt/jwt@v3.2.1+incompatible
3. go mod edit -droprequire github.com/dgrijalva/jwt-go
4. go mod tidy  # see note
5. go clean ./... && go build ./...
6. go test ./...

Note: 'go mod tidy' had unrelated changes (perhaps it wasn't run in
recent commits) so I removed the unrelated delta to keep this PR focused
on the dgrijalva/jwt-go to golang-jwt/jwt changes.

References:
[1] dgrijalva/jwt-go#462
[2] dgrijalva/jwt-go#463
[3] https://github.com/dgrijalva/jwt-go/blob/master/README.md
[4] https://github.com/golang-jwt/jwt
[5] https://github.com/influxdata/influxdb/issues/21926
2021-07-23 11:41:14 -05:00
Cody Shepherd ca97bb2b28
chore: update master-1.x changelog to reflect v1.8.7 release (#21907) 2021-07-21 12:25:24 -07:00
Sam Arnold 22809a33aa
fix: export example and fix adjacent shards (#21863)
* fix: export example and fix adjacent shards

* fix: Cody's review
2021-07-21 09:39:31 -04:00
Cody Shepherd d0772d7d67
fix: systemd unit should block on startup until http endpoint is ready (#21895)
* fix: systemd unit should block on startup until http endpoint is ready

* chore: update changelog
2021-07-20 11:08:03 -07:00
Sam Arnold 23c3d35aab
chore: update protobuf library versions and remove influx_tsm (#21882)
* chore: update protobufs

* fix: run codegen during build

* fix: fully remove influx_tsm
2021-07-20 09:42:52 -04:00
Sam Arnold 6d22e69ef1
fix: hard limit on field size while parsing line protocol (#21843)
Per https://docs.influxdata.com/enterprise_influxdb/v1.9/write_protocols/line_protocol_reference/
we only support 64KB, but 1MB is a more realistic practical limit. Before this commit there was
no enforcement of field value size.

Closes #21841
2021-07-14 17:11:09 -04:00
Sam Arnold 8ea5900ad6
chore: update flux to v0.120.1 (#21820) 2021-07-09 11:35:58 -04:00
Sam Arnold ac96fd45fb
feat: un-revert add total-buffer-bytes config parameter to subscriptions (#21804)
This reverts commit 4d82ed2c07.
2021-07-08 15:00:40 -04:00
Sam Arnold 4d82ed2c07
revert: revert add total-buffer-bytes config parameter to subscriptions (#21803)
Reverts PR #21752

This reverts commit 85abb3af36.
2021-07-07 15:37:37 -04:00
Sam Arnold 85abb3af36
feat: add total-buffer-bytes config parameter to subscriptions (#21752)
* chore: harmonize OSS and Enterprise subscriber initialization

* refactor: chanwriter close blocks until chanwriter is flushed

* refactor: remove redundant state

* test: add test for blocked subscriber during subscription update

* fix: blocked subscriber does not fail subscription update

* fix: only iterate matching subscriptions

* fix: use serialized points in subscriber queues

* feat: add total-buffer-bytes config parameter to subscriptions

* fix: put subscription serialization on the write path

* fix: subscription service only needs regular mutex, not RWMutex

* fix: review comments

* chore: update changelog
2021-07-06 15:56:01 -04:00
Sam Arnold 8fa4d82ce7
fix: show shards gives empty expiry time for inf duration shards (#21795) 2021-07-06 15:53:07 -04:00
Sam Arnold 98361e2073
fix: error instead of panic for statement rewrite failure (#21792) 2021-07-06 11:05:21 -04:00
Cody Shepherd 639527cd57
fix: convert arm arch names for rpms during builds via docker (#21777)
* fix: convert arm arch names for rpms during builds via docker

* chore: update changelog
2021-07-02 08:23:05 -07:00
Cody Shepherd e7edf9fc5f
fix: rename arm rpms with yum-compatible names (#21750) 2021-06-28 07:45:58 -07:00
Sam Arnold 4268cf3432
chore: update changelog to sync with 1.9 branch (#21738) 2021-06-23 18:34:15 -04:00
davidby-influx 73bdb2860e
chore: add logging to compaction (#21707)
Compaction logging will generate intermediate information on 
volume of data written and output files created, as well as 
improve some of the anti-entropy messages related to compaction.

This will also apply to `influx_tools compact`

Closes https://github.com/influxdata/influxdb/issues/21704
2021-06-16 15:28:44 -07:00
davidby-influx acc4105b8c
fix: do not panic on cleaning up failed iterators (#21666)
We have seen occasional panics in Iterators.Close()
when cleaning up after failed iterator creation.
This commit checks for nil on any iterator to be
closed, and now returns any errors generated by
that Close().

Closes https://github.com/influxdata/influxdb/issues/19579
Closes https://github.com/influxdata/influxdb/issues/19476
2021-06-11 11:28:59 -07:00
davidby-influx bce6553459
fix: Do not close connection twice in DigestWithOptions (#21659)
tsm1.DigestWithOptions closes its network connection
twice. This may cause broken pipe errors on concurrent
invocations of the same procedure, by closing a reused
i/o descriptor. This fix also captures errors from TSM
file closures, which were previously ignored.

Closes https://github.com/influxdata/influxdb/issues/21656
2021-06-10 12:41:42 -07:00
Sam Arnold 894f54e6ac
fix: group by returns multiple results per group in some circumstances (#21631)
* fix: Revert performance improvement for sorted merge iterator

This reverts commit af8e66cd25.

* test: add end to end regression test for broken group-by

* chore: update changelog
2021-06-08 10:41:58 -04:00
davidby-influx f64be286be
fix: avoid rewriting fields.idx unnecessarily (#21592)
Under heavy write load creating new fields and measurements
the rewrite of the fields.idx file is a bottleneck. This
enhancement combines multiple writes into a single one and
shares any error return value with all of the combined
invocations. MeasurementFieldSet and the new 
MeasurementFieldSetWriter must both now be explicitly
closed.

Closes #21577
2021-06-04 09:21:33 -07:00
Phillip Steinbachs 87fbb3fec9
chore: update changelog for 1.8.6 (#21532)
* chore: update changelog for 1.8.6

* chore: update changelog for 1.8.6
2021-05-21 18:07:31 -07:00
Sam Arnold c6fbc2e66f
fix: FGA enablement (#21516)
* fix: FGA enablement

Closes #21514

* fix: FGA enablement

Closes #21515
2021-05-19 17:03:21 -04:00
davidby-influx c8da9bafbf
chore(ae): add more logging (#21381) (#21452)
tsdb.Engine.IsIdle and tsdb.Engine.Digest now return a reason string for why the engine & shard are not idle.
Callers can then use this string for logging, if desired. The returned reason does not allocate memory, so the
caller may want to add the shard ID and path for more information in the log. This is intended to be used in
calls from the anti-entropy service in Enterprise.

(cherry picked from commit bf45841359)

fixes https://github.com/influxdata/influxdb/issues/21448
2021-05-11 09:46:45 -07:00
Sam Arnold 1f201472d9
fix: disable MergeFiltersRule until it is more stable (#21358)
Add a test to make sure this stays fixed.
2021-05-03 15:33:48 -04:00
Sam Arnold 8edf7a4e2f
fix(storage): cursor requests are [start, stop] instead of [start, stop) (#21347)
* fix: backport tsdb fix for window pushdowns

From https://github.com/influxdata/influxdb/pull/19855

* fix(storage): cursor requests are [start, stop] instead of [start, stop)

The cursors were previously [start, stop) to be consistent with how flux
requests data, but the underlying storage file store was [start, stop]
because that's how influxql read data. This reverts back the cursor
behavior so that it is now [start, stop] everywhere and the conversion
from [start, stop) to [start, stop] is performed when doing the cursor
request to get the next cursor.

cherry-pick from #21318

Co-authored-by: Sam Arnold <sarnold@influxdata.com>
(cherry picked from commit 7766672797)

* chore: fix formatting

Co-authored-by: Jonathan A. Sternberg <jonathan@influxdata.com>
2021-04-30 15:26:31 -04:00
Sam Arnold 4cf19f8254
fix: summation should be in native type for mean,count iterator (#21334) 2021-04-29 18:10:14 -04:00
Sam Arnold 32aa970eba
feat: mean,count aggregation for WindowAggregate pushdown in enterprise (#21291)
We support only one aggregate list [mean,count]. All other aggregates
still must be single-element lists.
2021-04-29 14:30:13 -04:00
davidby-influx 5d04f0ae20
chore: update CHANGELOG.md (#21311) 2021-04-27 16:47:10 -07:00
davidby-influx f646cbc540
fix(models): grow tag index buffer if needed (#20137) (#21303) (#21306)
* fix(models): grow tag index buffer if needed (#20137)

(cherry picked from commit cae14176aa)

Co-authored-by: Tristan Su <foobar@users.noreply.github.com>

fixes #21303
2021-04-27 10:44:51 -07:00
Sam Arnold e97c5d9785
feat: flux upgrade to v0.112.1 (#21226) 2021-04-15 13:41:51 -04:00
Sam Arnold 461464f180
fix: flux regression in buckets query (#21183) 2021-04-12 08:30:36 -04:00