This commit adds `mincore.Limiter` which throttles page faults caused
by mmap() data. It works by periodically calling `mincore()` to determine
which pages are not resident in memory and using `rate.Limiter` to
throttle accessing using a token bucket algorithm.
* chore: remove tsi1 testdata and add go generate file to download
* chore: fix testdata url and rename gen file
* fix: add testdata generate command to Makefile
* chore: add testdata dir to gitignore
* refactor(tsdb): improve error message when missing testdata
* refactor(tsdb): tagged testdata and avoid stacktrace when missing
Checking a channel too regularly could cause
context switching to other goroutines. In tight loops,
it is prudent to check, but to do so less frequently so
as to avoid thrashing.
* feat(tsdb): SHOW TAG KEYS (no time) query using only TSI data.
* fix(tsdb): Allow for earlier return when scanning during show tag keys.
* fix(tsdb): Speed things up by using the key merger to reduce allocs.
* chore(tsm1): Fix golint.
* fix(tsdb): Remove sorting, because these keys should already be sorted.
* fix(tsdb): Remove dead code to placate the linter.
* fix(tsm1): delimit tsmKeyPrefix with appended comma
Fixes#7589.
Append a comma to the TSM key prefix when matching a full measurement name to avoid erroneously matching other measurement names that include the prefix in their own name. For example, this prevents matching a measurement "cpu1" when targeting "cpu" by updating the prefix to "cpu,". This relies on the fact that tag key-value pairs are separated by commas.
* fix(tsm1): regression tests for tsmKeyPrefix comma delimiting
* fix(storage): Push-down a predicate to match tags for SHOW MEASUREMENTS calls.
* chore: Address feedback.
* fix(tsm1): Split behavior based on existence of predicate for show measurements.
* fix(tsm1): Allow parenthesis expression on the LHS of a predicate.
* fix(tsm1): Create a separate tag predicate verifier that rejects negative comparisons.
* fix(tsm1): Additional test cases for show measurements with predicate.
This commit adds ref counting for files that we pull tag keys from.
Previously, files were only ref counted during the time we extracted
tag keys but this commit adds additional ref counting for the life of
the `Engine.tagKeysNoPredicate()` function.
This commit
* adds new request and response data types for schema gRPC calls
* adds fmt.Stringer implementation to cursors.FieldType
* adds APIs to sort a slice of MeasurementField values,
* upgrades the gogo protobuf package to v1.3.1, which
includes improvements to serialization.
This commit adds a new API to `Cache` to address data races
with the `TagKeys` and `TagValues` APIs.
`Cache` and `entry` provide `AppendTimestamps`, which
appends the current timestamps to the provided slice
to reduce allocations. As noted in the documentation,
it is the responsibility of the caller to sort and deduplicate
the values, if required.
The `cursors.TimestampArray` type was extended to permit
use of the `sort.Sort` API.
This commit introduces a new API for finding the maximum
timestamp of a series when iterating over the keys in a
set of TSM files.
This API will be used to determine the field type of a single
field key by selecting the series with the maximum timestamp.
It has also refactored the common functionality for iterating
TSM keys into `timeRangeBlockReader`, which is shared
between `TimeRangeIterator` and `TimeRangeMaxTimeIterator`.
These APIs require a measurement, permitting an additional optimization
to reduce the search space against the TSM index. Specifically, the
search key prefix is extended from `org+bucket` to
`org+bucket,\x00=<measurement>`
* MeasurementNames
* MeasurementTagKeys
* MeasurementTagValues
* Adds an api to the models package for efficiently parsing the
measurement tag (\x00) from a normalized series key
The root cause is that the Unsigned data type has no representation
in the valueType function in the cache and falls back to the default
case of 0.
0 is also a sentinel value in the entry#add function that will
result in skipping the value type check.
It therefore is possible that unsigned values followed by some other
data type is stored in the cache.
It is suspected that the write may be rejected before reaching the
cache, and therefore may not occur in practice. Specifically, the
series file stores the data types on a per-series basis and would
reject the write.
This commit turns the value types into explicit constants and
ensures all existing block types are represented. In addition,
it adds a mapping function to convert these to a known Block type,
which will be used by the `MeasurementFields` schema request to
determine the type of a series in the cache.
* refactor(storage): move type ByTagKey to the only package that uses it
* refactor(tsdb): use types in tsdb/cursors
* refactor(tsdb): remove unused type SeriesIDElems
* refactor(tsdb): inline only use of tsdb.ReadAllSeriesIDIterator
* refactor(tsdb): move series file to its own package
* refactor(storage): remove platform->influxdb aliases
Ben Johnson
Gavin Cabbage
ricky
Phil Bracikowski
Stuart Carnie
Brett Buddin
jlapacik
Jonathan A. Sternberg
Patrik Helia
Edd Robinson
Jacob Marble