currently we use /etc/lsb-release to detect debian,
but this file is only present on ubuntu.
/etc/debian_version is installed by default by debian and ubuntu
This commit enforces a limit on `RawMapper` so that it will not
produce more values than are specified by the LIMIT clause.
Previously the mapper would read up to the chunk size and the
values would be limited afterward.
We were closing the cursor when we read the last block which caused
the internal state to be cleared. In a group by query, we seeked multiple
times so depending on the group by interval and how the data was laid out
in the blocks, we woudl close the cursor and the last block would get skipped.
Fixes#5193
Add --pkgarch option to build.py to specify the packaging architecture which can be different to GOARCH.
Example: build for debian on raspberry pi. GOARCH will be arm but the packaging architecture on debian will be armhf (arm hard float). The --pkgarch option is passed to fpm to specify the required architecture which is reflected in the package manifest and also in the result filename.
This may be causing slow restart times for systems with many large TSM files.
What I believe is happening at startup in these cases is that multiple goroutines
are started to load each TSM file concurrently. The kernel appears to serialize
mmap calls from the same process so all of the goroutines end up getting blocked
on the actual mmap system call. MAP_POPULATE instruct the kernel to pre-fault the
page table for the files and triggers read-ahead of the pages. For larger, 2GB files,
this makes the mmap call more expensive and slower. When there are many of these files
and calls it is possible to fill all available memory with pagecache. In this case,
the OS will end up pre-faulting pages from one file and have to remove pages that it just
loaded from another files causing slowness. MAP_POPULATE may also be cause much more data
to be pre-faulted than necessary. To load a file, we just need to scan the index at the end
of the file. MAP_POPULATE is likely causing the whole file to be loaded when it won't actually
be accessed for a while (or at all).
Might fix issue #5311.
Some data shapes would cause files to grow larger than the max size more
quickly which resulted in them getting skipped by the full compaction planner
at times. Some datasets that could make this happen are very large keys or
very large numbers of keys (10M). When this happened, multiple max sized
files would accumulate but the blocks would not be full. When the shard went
cold for writes, these files would get recompacted down to the optimal size, but
a lot of space would be wasted in the mean time.
- Re-enabling support for ARM builds in `build.py`.
- Improved `build.py`'s iteration and package version.
- Fixed minor bug with logrotate file path in `build.py`.
- Improved installation pre and post install/uninstall scripts by adding distribution-specific logic to account for quirks between the different package managers.
- Added post-install as post-trans script for RPM builds due to the order of RPM upgrade commands (issue where init scripts were removed when upgrading).
- Added explicit backup to pre-install script to ensure legacy configuration survive an upgrade.
David Norton
Jason Wilder
Jason Wilder
joelegasse
Jonathan A. Sternberg
Edd Robinson
Ross McDonald
Jonathan A. Sternberg
Ross McDonald
Alexandre Derumier
Ben Johnson
Marcus Geiger
Philip O'Toole
Philip O'Toole
Paulo Pires
Mark Rushakoff