influxdb/tsdb
Sam Arnold 8edf7a4e2f
fix(storage): cursor requests are [start, stop] instead of [start, stop) (#21347)
* fix: backport tsdb fix for window pushdowns

From https://github.com/influxdata/influxdb/pull/19855

* fix(storage): cursor requests are [start, stop] instead of [start, stop)

The cursors were previously [start, stop) to be consistent with how flux
requests data, but the underlying storage file store was [start, stop]
because that's how influxql read data. This reverts back the cursor
behavior so that it is now [start, stop] everywhere and the conversion
from [start, stop) to [start, stop] is performed when doing the cursor
request to get the next cursor.

cherry-pick from #21318

Co-authored-by: Sam Arnold <sarnold@influxdata.com>
(cherry picked from commit 7766672797)

* chore: fix formatting

Co-authored-by: Jonathan A. Sternberg <jonathan@influxdata.com>
2021-04-30 15:26:31 -04:00
..
cursors fix(storage): cursor requests are [start, stop] instead of [start, stop) (#21347) 2021-04-30 15:26:31 -04:00
engine fix(storage): cursor requests are [start, stop] instead of [start, stop) (#21347) 2021-04-30 15:26:31 -04:00
index refactor: separate coarse and fine permission interfaces (#20996) 2021-03-22 09:52:33 -04:00
internal chore: run goimports -w ./ 2021-01-29 11:40:02 -05:00
testdata Reduce allocations in TSI TagSets implementation 2018-09-18 15:58:38 -07:00
README.md Merge pull request #5678 from jonseymour/typo 2016-02-25 09:33:41 -07:00
batcher.go add in some optimization 2017-10-02 12:02:38 -06:00
batcher_test.go rename influxdb/influxdb to influxdata/influxdb 2016-02-10 10:26:18 -07:00
config.go feat: measurement metrics by login (#20687) 2021-02-04 11:52:53 -05:00
config_test.go Allow TSI bitset cache size to be configured 2019-01-24 17:41:45 +00:00
cursor.go Upgrade flux to the latest version and remove the platform dependency 2019-04-04 10:55:09 -05:00
engine.go feat: series creation ingress metrics (#20700) 2021-02-05 14:52:43 -04:00
epoch_tracker.go tsdb: conflict based concurrency resolution 2018-11-21 19:19:53 -07:00
epoch_tracker_test.go tsdb: conflict based concurrency resolution 2018-11-21 19:19:53 -07:00
field_validator.go remove bool return param from dataTypeFromModelsFieldType 2018-04-25 09:48:24 -06:00
guard.go tsdb: conflict based concurrency resolution 2018-11-21 19:19:53 -07:00
guard_test.go tsdb: conflict based concurrency resolution 2018-11-21 19:19:53 -07:00
index.go refactor: separate coarse and fine permission interfaces (#20996) 2021-03-22 09:52:33 -04:00
index_test.go fix(services/storage): multi measurement queries return all applicable series (#19592) (#20934) 2021-03-12 16:34:14 -05:00
ingress_metrics.go feat: series creation ingress metrics (#20700) 2021-02-05 14:52:43 -04:00
ingress_metrics_test.go feat: series creation ingress metrics (#20700) 2021-02-05 14:52:43 -04:00
meta.go buildtsi: Do not escape measurement names 2018-05-30 15:20:56 -07:00
meta_test.go create iterator benchmark 2018-07-02 16:47:44 +01:00
series_cursor.go Prevent a panic from occuring when CreateSeriesCursor fails 2018-08-07 22:55:02 -05:00
series_file.go feat: series creation ingress metrics (#20700) 2021-02-05 14:52:43 -04:00
series_file_test.go feat: series creation ingress metrics (#20700) 2021-02-05 14:52:43 -04:00
series_index.go Fix series file tombstoning. 2018-10-05 08:23:25 -06:00
series_index_test.go Segment series file 2017-12-29 11:57:45 -07:00
series_partition.go feat: series creation ingress metrics (#20700) 2021-02-05 14:52:43 -04:00
series_segment.go feat(storage): Offline series file compaction 2020-02-03 13:57:31 -07:00
series_segment_test.go Improve series segment recovery. 2018-06-26 10:56:44 -06:00
series_set.go Remove copy-on-write when caching bitmaps 2019-01-25 18:02:48 +00:00
series_set_test.go chore: Quiet static analysis tools (#19509) 2020-09-05 12:43:29 -04:00
shard.go fix(services/storage): multi measurement queries return all applicable series (#19592) (#20934) 2021-03-12 16:34:14 -05:00
shard_internal_test.go feat: Ingress metrics by measurement 2021-02-02 15:58:28 -05:00
shard_test.go feat: Ingress metrics by measurement 2021-02-02 15:58:28 -05:00
store.go refactor: separate coarse and fine permission interfaces (#20996) 2021-03-22 09:52:33 -04:00
store_test.go feat: Make meta queries respect QueryTimeout values 2021-02-23 12:52:44 -08:00

README.md

Line Protocol

The line protocol is a text based format for writing points to InfluxDB. Each line defines a single point. Multiple lines must be separated by the newline character \n. The format of the line consists of three parts:

[key] [fields] [timestamp]

Each section is separated by spaces. The minimum required point consists of a measurement name and at least one field. Points without a specified timestamp will be written using the server's local timestamp. Timestamps are assumed to be in nanoseconds unless a precision value is passed in the query string.

Key

The key is the measurement name and any optional tags separated by commas. Measurement names, tag keys, and tag values must escape any spaces or commas using a backslash (\). For example: \ and \,. All tag values are stored as strings and should not be surrounded in quotes.

Tags should be sorted by key before being sent for best performance. The sort should match that from the Go bytes.Compare function (http://golang.org/pkg/bytes/#Compare).

Examples

# measurement only
cpu

# measurement and tags
cpu,host=serverA,region=us-west

# measurement with commas
cpu\,01,host=serverA,region=us-west

# tag value with spaces
cpu,host=server\ A,region=us\ west

Fields

Fields are key-value metrics associated with the measurement. Every line must have at least one field. Multiple fields must be separated with commas and not spaces.

Field keys are always strings and follow the same syntactical rules as described above for tag keys and values. Field values can be one of four types. The first value written for a given field on a given measurement defines the type of that field for all series under that measurement.

  • integer - Numeric values that do not include a decimal and are followed by a trailing i when inserted (e.g. 1i, 345i, 2015i, -10i). Note that all values must have a trailing i. If they do not they will be written as floats.
  • float - Numeric values that are not followed by a trailing i. (e.g. 1, 1.0, -3.14, 6.0+e5, 10).
  • boolean - A value indicating true or false. Valid boolean strings are (t, T, true, TRUE, f, F, false, and FALSE).
  • string - A text value. All string values must be surrounded in double-quotes ". If the string contains a double-quote or backslashes, it must be escaped with a backslash, e.g. \", \\.
# integer value
cpu value=1i

cpu value=1.1i # will result in a parse error

# float value
cpu_load value=1

cpu_load value=1.0

cpu_load value=1.2

# boolean value
error fatal=true

# string value
event msg="logged out"

# multiple values
cpu load=10,alert=true,reason="value above maximum threshold"

Timestamp

The timestamp section is optional but should be specified if possible. The value is an integer representing nanoseconds since the epoch. If the timestamp is not provided the point will inherit the server's local timestamp.

Some write APIs allow passing a lower precision. If the API supports a lower precision, the timestamp may also be an integer epoch in microseconds, milliseconds, seconds, minutes or hours.

Full Example

A full example is shown below.

cpu,host=server01,region=uswest value=1 1434055562000000000
cpu,host=server02,region=uswest value=3 1434055562000010000

In this example the first line shows a measurement of "cpu", there are two tags "host" and "region, the value is 1.0, and the timestamp is 1434055562000000000. Following this is a second line, also a point in the measurement "cpu" but belonging to a different "host".

cpu,host=server\ 01,region=uswest value=1,msg="all systems nominal"
cpu,host=server\ 01,region=us\,west value_int=1i

In these examples, the "host" is set to server 01. The field value associated with field key msg is double-quoted, as it is a string. The second example shows a region of us,west with the comma properly escaped. In the first example value is written as a floating point number. In the second, value_int is an integer.

Distributed Queries