This is a backport of #14262 to the 1.x storage engine. The 1.x storage
engine is now the primary engine for open source so when we switched we
regressed to the old behavior.
This also fixes `go generate` for the tsm1 package by running `tmpl`
with `go run` instead of assuming the correct one is installed in the
path.
This fixes multi measurement queries that go through the storage service
to correctly pick up all series that apply with the filter. Previously,
negative queries such as `!=`, `!~`, and predicates attempting to match
empty tags did not work correctly with the storage service when multiple
measurements or `OR` conditions were included.
This was because these predicates would be categorized as "multiple
measurements" and then it would attempt to use the field keys iterator
to find the fields for each measurement. The meta queries for these did
not correctly account for negative equality operators or empty tags when
finding appropriate measurements and those could not be changed because
it would cause a breaking change to influxql too.
This modifies the storage service to use new methods that correctly
account for the above situations rather than the field keys iterator.
Some queries that appeared to be single measurement queries also get
considered as multiple measurement queries. Any query with an `OR`
condition will be considered a multiple measurement query.
This bug did not apply to single measurement queries where one
measurement was selected and all of the logical operators were `AND`
values. This is because it used a different code path that correctly
handled these situations.
* refactor: Replace ctx.Done() with ctx.Err()
Prior to this commit we checked for context cancellation with a select
block and context.Context.Done() without multiplexing over any other
channel like:
select {
case <-ctx.Done():
// handle cancellation
default:
// fallthrough
}
This commit replaces those type of blocks with a simple check of
ctx.Err(). This has the following benefits:
* Calling ctx.Err() is much faster than entering a select block.
* ctx.Done() allocates a channel when called for the first time.
* Testing the result of ctx.Err() is a reliable way of determininging if
a context.Context value has been canceled.
* fix: Fix data race in execDeleteTagValueEntry()
This commit adds `mincore.Limiter` which throttles page faults caused
by mmap() data. It works by periodically calling `mincore()` to determine
which pages are not resident in memory and using `rate.Limiter` to
throttle accessing using a token bucket algorithm.
* chore: remove tsi1 testdata and add go generate file to download
* chore: fix testdata url and rename gen file
* fix: add testdata generate command to Makefile
* chore: add testdata dir to gitignore
* refactor(tsdb): improve error message when missing testdata
* refactor(tsdb): tagged testdata and avoid stacktrace when missing
Checking a channel too regularly could cause
context switching to other goroutines. In tight loops,
it is prudent to check, but to do so less frequently so
as to avoid thrashing.
* feat(tsdb): SHOW TAG KEYS (no time) query using only TSI data.
* fix(tsdb): Allow for earlier return when scanning during show tag keys.
* fix(tsdb): Speed things up by using the key merger to reduce allocs.
* chore(tsm1): Fix golint.
* fix(tsdb): Remove sorting, because these keys should already be sorted.
* fix(tsdb): Remove dead code to placate the linter.
* fix(tsm1): delimit tsmKeyPrefix with appended comma
Fixes#7589.
Append a comma to the TSM key prefix when matching a full measurement name to avoid erroneously matching other measurement names that include the prefix in their own name. For example, this prevents matching a measurement "cpu1" when targeting "cpu" by updating the prefix to "cpu,". This relies on the fact that tag key-value pairs are separated by commas.
* fix(tsm1): regression tests for tsmKeyPrefix comma delimiting
* fix(storage): Push-down a predicate to match tags for SHOW MEASUREMENTS calls.
* chore: Address feedback.
* fix(tsm1): Split behavior based on existence of predicate for show measurements.
* fix(tsm1): Allow parenthesis expression on the LHS of a predicate.
* fix(tsm1): Create a separate tag predicate verifier that rejects negative comparisons.
* fix(tsm1): Additional test cases for show measurements with predicate.
This commit adds ref counting for files that we pull tag keys from.
Previously, files were only ref counted during the time we extracted
tag keys but this commit adds additional ref counting for the life of
the `Engine.tagKeysNoPredicate()` function.
This commit
* adds new request and response data types for schema gRPC calls
* adds fmt.Stringer implementation to cursors.FieldType
* adds APIs to sort a slice of MeasurementField values,
* upgrades the gogo protobuf package to v1.3.1, which
includes improvements to serialization.
Ben Johnson
Daniel Moran
jl
Mark Rushakoff
sans
Roger Peppe
Jonathan A. Sternberg
Stuart Carnie
Ayan George
Brett Buddin
Edd Robinson
Gavin Cabbage
ricky
Phil Bracikowski
jlapacik
Patrik Helia