Writes could timeout and when adding new measurement names to the
index if the sort took a long time. The names slice was never
actually used (except a test) so keeping it in index wastes memory
and sort it wastes CPU and increases lock contention. The sorting
was happening while the shard held a write-lock.
Fixes#3869
The interaction of continuous query service, the meta-store loading
and initializing raft state, and syncing node info could cause a
deadlock in some instances. There was an extra read-lock taken by isLeader()
when it already had a read-lock. Removing this extra lock fixes the startup
deadlock.
Fixes#3607
This commit changes the default block size from 64KB to 4KB for
bz1. This was lowered because small blocks were being uncompressed,
merged, recompressed, and inserted for a large portion of updates.
This became slower and slower over time until it reached the 64KB
threshold. We moved to the 4KB threshold in order to lower the
impact of this recompression.
If 2 or more points during this map-and-reduce share the same timestamp,
the tie is broken by looking at the value. This ensures that these
functions operate in a deterministic manner.
This solution due to @jwilder
Since we already got a pointsHeapItem, let's just reuse it instead
of allocating a new one. This cuts allocated memory of a 1 million
points aggregate query from 4881.97MB to 4139.86MB
Some queries may be subject to race conditions. This variable allows the
test to request that a query is executed multiple times, increasing the
chance that any race will be exposed.
The buffer allocation in bz1 was unused and I'm fairly certain that it
was harmful to performance if used. For queries that run through a bz1
block, needing to hold on to a 64kb block is expensive. Better to churn
on the allocator and have the blocks be released when they are unused
than to have 64kb hanging around for each series regardless of size.
Thanks to @jwilder for brainstorming this issue with me.
By using preallocated buffers for marshaling WAL entries, we can
reduce the amount of memory we allocate.
On a run of `influx_stress -series 10000 -points 1000` this cuts
total allocations from 18684.15MB to 15200.73MB
Jason Wilder
dgnorton
Philip O'Toole
Philip O'Toole
Sean Beckett
Paul Dix
Ben Johnson
Daniel Morsing
Cory LaNou
KoeSystems