This commit
* adds new request and response data types for schema gRPC calls
* adds fmt.Stringer implementation to cursors.FieldType
* adds APIs to sort a slice of MeasurementField values,
* upgrades the gogo protobuf package to v1.3.1, which
includes improvements to serialization.
Filter cursors buffer points in between calls to Next() if the number
of read points exceeds 1000. Previously, this buffer was being cleared
out before being iterated over which caused queries to return a resultset
which had a number of rows divisable by 1000.
This change moves the clearing of the buffer until after the points have
been read. This change affects any queries which read more than 1000 points
from a single series & have a filter that can be successfully applied to at
least one of those points.
These APIs require a measurement, permitting an additional optimization
to reduce the search space against the TSM index. Specifically, the
search key prefix is extended from `org+bucket` to
`org+bucket,\x00=<measurement>`
* MeasurementNames
* MeasurementTagKeys
* MeasurementTagValues
* Adds an api to the models package for efficiently parsing the
measurement tag (\x00) from a normalized series key
* refactor(storage): move type ByTagKey to the only package that uses it
* refactor(tsdb): use types in tsdb/cursors
* refactor(tsdb): remove unused type SeriesIDElems
* refactor(tsdb): inline only use of tsdb.ReadAllSeriesIDIterator
* refactor(tsdb): move series file to its own package
* refactor(storage): remove platform->influxdb aliases
* refactor(storage): add readSource field accessors
* refactor(storage): remove unused limitSeriesCursor
* refactor(storage): export IndexSeriesCursor
This allows IDPE to use the same implementation, rather than duplicate
code. Also copied unit tests from IDPE.
* chore: go fmt
* refactor(storage): move unused code to repo that needs it
Turns out that a bunch of code is only needed in IDPE. This change
removes that code, and another PR adds it to IDPE.
* refactor(storage): export KeyMerger
* refactor(storage): export NilSortHi and NilSortLo
* refactor(storage): move StringIterator & friends to IDPE
* refactor(storage): unexport a few test helper funcs
* fix(storage): simplify storage/seriesCursor
storage/seriesCursor releases series file and TSI references sooner.
Remove unhelpful request object, inherited from 1.x
* chore(storage): replace SeriesCursor interface with sole implementation
* feat(backup): `influx backup` creates data backup
* feat(backup): initial restore work
* feat(restore): initial restore impl
Adds a restore tool which does offline restore of data and metadata.
* fix(restore): pr cleanup
* fix(restore): fix data dir creation
* fix(restore): pr cleanup
* chore: amend CHANGELOG
* fix: restore to empty dir fails differently
* feat(backup): backup and restore credentials
Saves the credentials file to backups and restores it from backups.
Additionally adds some logging for errors when fetching backup files.
* fix(restore): add missed commit
* fix(restore): pr cleanup
* fix(restore): fix default credentials restore path
* fix(backup): actually copy the credentials file for the backup
* fix: dirs get 0777, files get 0666
* fix: small review feedback
Co-authored-by: tmgordeeva <tanya@influxdata.com>
This adds an lru cache for the columns that are produced as tags. When
producing the columns that are part of the group key, it will generate
the column and then keep it in an lru cache to reuse for future tables.
The start and stop column are effectively cached for every table because
they are special and will be the same for all of the tables.
For the tags, it retains the most recently used since they may be used
by a future table. That way most of the columns will get shared with
each other.
When the size differs, a slice is used so the underlying data is still
shared, but the size is different.
This removes the duplicate filter that is used by the reader. The
storage engine shouldn't be sending us duplicate tables anyway and this
code hurts performance in high cardinality queries because of the memory
it uses to keep track of all of the keys that have been seen.
By default this feature is disabled; the full compaction behaviour does
not change. When this feature is enabled compactions can be limited
across multiple storage engines running in multiple processes.
The mechanism by which this happens is not part of the abstraction added
here.
Stuart Carnie
kun
jlapacik
Faith Chikwekwe
Ben Johnson
Jonathan A. Sternberg
Yiqun (Ethan) Zhang
Jacob Marble
Sebastian Borza
Edd Robinson
Johnny Steenbergen
David McKay
Greg
Chris Goller
Brandon Farmer
Kelvin Wang
elbehery