A write lock was being taken to read the memory size to determine if writes
should be paused. What happens is that writers get blocked indefintely when
trying to acquire a write lock which makes writes pause (or stop) for long periods
of time.
The log was deferring the release of the read lock on the WAL. This had
the affect that a read-lock was held until after the partition finished writing
(which maintains it's own locks). The read lock is only needed around the call
to pointsToPartions so it can get a consistent copy of the points to write. After
that calls returns, a lock is not needed so free it immediatedly.
addToCache is called in a goroutine and can panic if the server is closed while opening. If
part of the open func errors, it returns an error and immediately calls close. close sets
p.cache to nil which causes the goroutine trying to initialized the cache to panic as well. The
goroutine should run under a write lock to avoid this race/panic.
If LoadMetadataIndex() tries to log an error, it causes a panic because the
logger is not set until Open() is called, which is after LoadMetaDataIndex() returns.
Instead, just set the logger up when the WAL is created.
From
516f0d1c90
"Note that we use a channel send here and not a close.
The race detector doesn't know that we're waiting for a timeout
and thinks that the waitgroup inside httptest.Server is added to
concurrently with us closing it. If we timed out immediately, we could close
the testserver before we entered the handler. We're not timing out immediately
and there's no way we would be done before we entered the handler, but the
race detector doesn't know this, so synchronize explicitly.
Writes could timeout and when adding new measurement names to the
index if the sort took a long time. The names slice was never
actually used (except a test) so keeping it in index wastes memory
and sort it wastes CPU and increases lock contention. The sorting
was happening while the shard held a write-lock.
Fixes#3869
If we've seeked a cursor, then we can be sure that there will be no
data between it and the point that was seeked to. Take advantage of
this fact to only seek when it would yield us a value that would be
different from the last.
In addition, only init the pointsheap when doing a raw query. For
aggregate queries, it is reinitialized on every time bucket, so no
need to seek through all the cursors
For a synthetic database where there was only entries for a tiny
slice of time, it cut queries from 112 seconds to 30 seconds doing
`select mean(value) from cpu where time > now - 2h group by time(1h)`
Unless a valid shell for the user running influxdb, the command su fails with
the error:
This account is currently not available.
The user shouldn't have a valid shell although the package creates influxdb
user with /bin/bash
Philip O'Toole
Nick Robinson
Philip O'Toole
Jason Wilder
Daniel Morsing
dgnorton
aviau
Swann Croiset
Sean Beckett
Paul Dix