This commit adds initial empty sketches back to the tsi1 index, as well
as ensuring that ephemeral sketches in the index `LogFile` are updated
accordingly.
The commit also adds a test that verifies that the merged sketches at
the store level produce the correct results under writes, deletions and
re-opening of the store.
This commit does not provide working sketches for post-compaction on the
tsi1 index.
Because of a race between the index and series file lookups, empty
keys can be returned for series which are tombstoned after the
series ids are obtained but before the caller looks up the key.
The default of 4096 results in writes to the WAL still requiring muliple
IOs. We had previously bumped this to 1M, but that was too high when
there are many shards. Increasing to around 16k reduces the IOs to
one or two for the workloads tested. We may want to make this
configurable in the future.
The large number of partitions cause big HeapInUse swings at higher
cardinality which can lead to OOMs. Reducing this to 16 lowers
write throughput to some extent at lower cardinalities, keeps memory
more stable over the long run.
If all the series in a measurement were tombstone, MeasurementHasSeries
would return true because the ok var was re-used from a prior check
earlier in the func. This caused it to be true all the time unless
the measurment was actually tombstoned.
The Store.Delete series held an RLock while deleting from each shard.
While deleting, the Engine uses shardSet to see if a series is fully
deleted. The shardSet.ForEach also takes and RLock. If a Lock is
requested between these two calls, a deadlock occurs.
To fix, we don't need to hold an RLock for the duration of the delete
in the store as each Shard handles concurrency itself and we have a
snapshot of the shards we need to access.
Under concurrent writes and deletes of the same series, a nil panic
could occur in bytes.Compare. Instead of setting the seriesKeys to
nil, set them to an 0 length slice which prevents the panic.
This separates out the dropping of a measurement from the series
to avoid frequent checks to see if a measurement still has series.
The series are dropped individually and we keep track of which
measurements are involved and then delete each measurment afterwards.
If the fields.idx was corrupted in someway, it would cause the shard
to fail to load. Deleting the file will allow it to be rebuilt.
This change handles this automatically so it's rebuilt if necessary
without user intervention.
Stuart Carnie
Edd Robinson
Ben Johnson
Stuart Carnie
Jason Wilder
Joe LeGasse
Jason Wilder