tsdb.Engine.IsIdle and tsdb.Engine.Digest now return a reason string for why the engine & shard are not idle.
Callers can then use this string for logging, if desired. The returned reason does not allocate memory, so the
caller may want to add the shard ID and path for more information in the log. This is intended to be used in
calls from the anti-entropy service in Enterprise.
The anti-entropy service will loop trying to copy an empty shard to a
data node missing that shard. This fix is one of two changes that
correctly create an empty shard on a new node. This fix will set the
LastModified date of an empty shard directory to the modification time
of that directory, instead of to the Unix epoch.
Fixes: https://github.com/influxdata/influxdb/issues/21273
(cherry picked from commit 7f300dc248)
The HTTP handler logs URLs, but not body values for POST requests.
This means that queries sent by GET are logged, because the query
is in the URL, but queries sent by POST have no query text in the
log. This feature prints all the key-value pairs in the post body,
which includes the query text, except passwords, which are redacted.
Closes https://github.com/influxdata/influxdb/issues/20653
(cherry picked from commit 70755bf42c)
Closes https://github.com/influxdata/influxdb/issues/21000
* fix: infinite recursion bug (#20862)
* Revert "fix(error): unsupported value: +Inf" error not handled gracefully (#20250)"
This reverts commit 6ac0bb3fe3.
* fix: No infinite recursion on write error
If there is some error writing to the response writer, we
would previous have infinite recursion.
Re-closes https://github.com/influxdata/influxdb/issues/20249
(cherry picked from commit d6f7716924)
* chore: update CHANGELOG.md
Co-authored-by: Sam Arnold <sarnold@influxdata.com>
fields.idx frequent writes cause lock contention and fields.idx is recreated
when a field or measurement is added in a WritePointsWithContext()
This eliminates locking during the actual file rewrite, and limits it to
the times when the MeasurementFieldSet is actually being read or written
in memory and when the new file is being renamed.
Test verification of correct behavior by checking the fields.idx
file matches the in-memory copy after heavily parallel measurement addition.
Fixes https://github.com/influxdata/influxdb/issues/20500
(cherry picked from commit fe3af66c54)
Meta queries (SHOW TAG VALUES, SHOW TAG KEYS, SHOW SERIES CARDINALITY, etc.) do not respect
the QueryTimeout config parameter. Meta queries should check the query context when possible
to allow cancellation and timeout. This will not be as frequent as regular queries, which
use iterators, because meta queries return data in batches.
Add a context.Context to
(*Store).MeasurementNames()
(*Store).MeasurementsCardinality()
(*Store).SeriesCardinality()
(*Store).TagValues()
(*Store).TagKeys()
(*Store).SeriesSketches()
(*Store).MeasurementsSketches()
which is tested for timeout or cancellation
to allow limitation of time spent in meta queries
https://github.com/influxdata/influxdb/issues/20736
(cherry picked from commit 092c7a9976)
* chore: move context.Context to first argument in methods per convention
(cherry picked from commit a8b2129df5)
* fix(error): SELECT INTO doesn't return error with unsupported value (#20429)
When a SELECT INTO query generates an illegal value that cannot be inserted,
like +/- Inf, it should return an error, rather than failing silently.
This adds a boolean parameter to the [data] section of influxdb.conf:
* strict-error-handling
When false, the default, the old behavior is preserved. When true,
unsupported values will return an error from SELECT INTO queries
Fixes https://github.com/influxdata/influxdb/issues/20426
(cherry picked from commit 9e33be2619)
Fixes https://github.com/influxdata/influxdb/issues/20427
* fix: cp.Mux.Serve() closes all net.Listener instances silently on error. (#20278)
A customer has seen a rash of "connection refused" errors to the meta node.
This fix ensures that when net.Listener instances are closed because of an
error in Accept(), influxdb logs the error which caused the closures, as well
as any errors in closing the Listeners.
Fixes https://github.com/influxdata/influxdb/issues/20256
(cherry picked from commit 5b98166b05)+
Customer reported that a GROUP BY query with an offset that caused an interval
to cross a daylight savings change inserted an extra output row off by one hour.
This fix ensured that the start time for the interval of a GROUP BY operator is
correctly set before calculating the time zone offset for that date and time.
Add TestGroupByIterator_DST() in query/iterator_test.go
for regression testing of this bug.
Fixes https://github.com/influxdata/influxdb/issues/20238
(cherry picked from commit df39b1e71c)
Fixes https://github.com/influxdata/influxdb/issues/20252
* fix(error): unsupported value: +Inf" error not handled gracefully
JSON marshalling errors should be returned properly formatted in JSON
like other errors. This fix formats marshalling errors the same way
influxdb formats other query errors.
Fixes https://github.com/influxdata/influxdb/issues/20249
(cherry picked from commit 2407077576)
* chore(changelog): update CHANGELOG.md for PR #20276
When an InfluxDB database is very busy writing new points the backup
the process can fail because it can not write a new snapshot.
The error is: `operation timed out with error: create snapshot: snapshot in progress`.
This happens because InfluxDB takes almost "continuously" a snapshot
from the cache caused by the high number of points ingested.
This PR skips snapshots if the `snapshotter` does not come available
after three attempts when a backup is requested.
The backup won't contain the data in the cache or WAL.
Signed-off-by: Gianluca Arbezzano <gianarb92@gmail.com>
davidby-influx
Cody Shepherd
Sam Arnold
Daniel Moran
Hercules Mango Churchill
Gershon Shif
Yun Zhao
davidby-influx
Christopher M. Wolff
David Norton
Gunnar
timhallinflux
Gunnar Aasen
Ben Johnson
Stuart Carnie
Gianluca Arbezzano
Sean Brickley