Commit Graph

49 Commits (c889d0f22bf3d3c4a207d4800bc9f8fb1ccf6a08)

Author SHA1 Message Date
Sam Arnold a74e05177c
feat: disk size metrics per shard (#22912) 2021-11-22 16:53:55 -05:00
Sam Arnold dece95d1dd
feat: tsm compaction metrics via prometheus (#22904)
* feat: tsm compaction metrics via prometheus

* chore: fix formatting

* chore: make activeCompactions a pointer
2021-11-19 14:51:22 -05:00
Stuart Carnie d165d76fc3
feat: influx CLI Packager support for explicit schema buckets (#21465)
This commit adds the minimal requirements to process specs containing
explicit schema buckets, so they may be exported and applied from
Cloud 2.
2021-05-31 10:01:29 +10:00
Daniel Moran b1283bc998
build: fix 'go generate' and regenerate the universe (#21089) 2021-03-30 10:32:14 -04:00
Tristan Su cae14176aa
fix(models): grow tag index buffer if needed (#20137) 2020-11-23 14:06:12 -05:00
Cuong Manh Le 51ff6f7b5c
fix: misuse of reflect.SliceHeader (#19875)
Currently, unsafeBytesToString function violates the 6th rule of unsafe
pointer usage. That is, reflect.SliceHeader/String header should never
be used as plain struct. This misuse can make to silent memory
corruption, which can be difficult to track down when problem occurred.

Instead, use the more (right) idiom to convert slice of byte to string
without heap allocation.

goos: linux
goarch: amd64
cpu: Intel(R) Core(TM) i7-8665U CPU @ 1.90GHz
BenchmarkInvalid-8   	1000000000	         0.497 ns/op	       0 B/op	       0 allocs/op
BenchmarkValid-8     	1000000000	         0.239 ns/op	       0 B/op	       0 allocs/op
PASS
ok  	command-line-arguments	0.815s
2020-11-09 09:37:20 -05:00
Mark Rushakoff 0e0a784e29
fix(models): correctly parse float as 64-bits (#19870)
Go tip now enforces that strconv.ParseFloat receives a bitSize parameter
of 32 or 64. Previously, any value that was not 32, was treated as 64.

The previous value of 10 was likely a copy-paste error from integer
parsing.
2020-10-30 11:27:37 -07:00
Stuart Carnie f38f19787f
fix: Ensure precision is passed to points parsing
In addition, adds additional precision parameters for v1 /write
compatibility.

Should address e2e bucket tests failing
2020-08-31 17:39:52 -07:00
Stuart Carnie 89532f0c29
refactor: move v2/v1/models to v2/models 2020-08-03 09:20:51 -07:00
Edd Robinson 2b175291be
refactor: WIP removing tsbd 2020-08-03 09:18:34 -07:00
Ben Johnson 9784ebb781 feat(storage): Add option to disable WritePoints() validation.
This commit adds the `WithWritePointsValidationEnabled()` option
to disable validation in `Engine.WritePoints()` as the same
validation is performed earlier in the call stack by cloud.
2020-06-23 09:07:49 -06:00
Ben Johnson 914255d374 fix(models): Optimize allocations in VerifyPoint() 2020-04-10 13:39:30 -06:00
Jonathan A. Sternberg 0ae8bebd75
refactor: rewrite imports to include the /v2 suffix for version 2 2020-04-03 12:39:20 -05:00
Stuart Carnie 069820ba4b
fix(models): Added error return value; use iota; fix spelling 2020-04-02 08:34:22 -07:00
Stuart Carnie a9a9535aea
feat: Amend tests to use real org+bucket prefixes, including escaped 2020-04-02 08:33:58 -07:00
Stuart Carnie d424d7d1f5
feat(tsdb): Add new measurement based schema APIs
These APIs require a measurement, permitting an additional optimization
to reduce the search space against the TSM index. Specifically, the
search key prefix is extended from `org+bucket` to
`org+bucket,\x00=<measurement>`

* MeasurementNames
* MeasurementTagKeys
* MeasurementTagValues
* Adds an api to the models package for efficiently parsing the
  measurement tag (\x00) from a normalized series key
2020-04-02 08:33:58 -07:00
Kelvin Wang 60a3922446 fix(models): fix the line breaker 2020-03-19 18:32:29 -04:00
Chris Goller 6af35aec42
feat: render models.Points to string (#17266)
Signed-off-by: Chris Goller <goller@gmail.com>
2020-03-13 14:11:10 -05:00
Michael Angerman 4c1e682117
fix(models): Remove tsdb package dependency from tests (#16918) 2020-02-25 14:06:47 -06:00
George 281c08ec50
fix(models): filter out reserved tag keys in points parser (#16412)
* fix(models): filter out reserved tag keys in points parser

* chore(models): update changelog to reflect reserved tag key change
2020-01-24 17:09:16 +00:00
Stuart Carnie 195a554353
fix(models): Fix bug that did not return ErrLimitMaxBytesExceeded
Adds additional allocation check when constructing errors, to ensure
these do not exceed total allocations.
2020-01-15 17:50:37 -07:00
Stuart Carnie 76c4f90f0b
feat(models): Add limits options to line-protocol parser
The behavior of the line-protocol parser remains unchanged. This PR
moves the functions to a struct in order to simplify the tracking of
limits whilst parsing a request.

Limits may be set using the ParsePointsWithOptions API and the
associated WithParser functional options, all of which are documented.

It should be noted that the behavior of ParsePointsWithPrecision remains
unchanged.

ParsePointsWithPrecisionV1 was no longer used and removed to simplify
code.
2020-01-10 14:42:02 -07:00
Ayan George d2bf6b02f8 refactor: permanently enable uint64 support
* Remove all instances of EnableUintSupport()

* Remove all instances of enableUint64Support

Closes #15711
2019-12-09 14:45:34 -05:00
Edd Robinson 270f917fcf fix: enable uint64 support 2019-11-01 10:55:54 +00:00
Edd Robinson afcb440d02
Merge pull request #14182 from zhulongcheng/unsigned-type-name
fix(storage): add string representation of the Unsigned FieldType
2019-09-23 14:04:30 +01:00
Stuart Carnie e0542fa89b
chore(models): Updates based on PR feedback 2019-09-11 17:04:04 -07:00
Stuart Carnie 4da079410f
feat(models): New APIs to create Tags from a list of key, value pairs
* New API to convert `models.Tags` to a slice of key, value pairs

```
BenchmarkNewTagsKeyValues/sorted/no_dupes/preallocate-8         	20000000	        63.0 ns/op	       0 B/op	       0 allocs/op
BenchmarkNewTagsKeyValues/sorted/no_dupes/allocate-8            	10000000	       124 ns/op	     144 B/op	       1 allocs/op
BenchmarkNewTagsKeyValues/sorted/dupes-8                        	10000000	       181 ns/op	     240 B/op	       1 allocs/op
BenchmarkNewTagsKeyValues/unsorted/no_dupes-8                   	10000000	       204 ns/op	     176 B/op	       2 allocs/op
BenchmarkNewTagsKeyValues/unsorted/dupes-8                      	 5000000	       308 ns/op	     272 B/op	       2 allocs/op
```
2019-09-11 16:37:57 -07:00
zhulongcheng 199f2001a6 fix(storage): add string representation of the Unsigned FieldType 2019-06-25 00:35:19 +08:00
Christopher Wolff 90a5d88fc5 fix(query): skip failing end to end tests 2019-05-14 12:52:37 -07:00
Stuart Carnie 87bfa06a1e
feat(storage): Add ParseTagsWithTags API
This API is a benefit when the caller is only interested in parsing
tags and reusing an existing Tags buffer. It will be used by the
storage schema APIs.
2019-04-24 11:14:22 -07:00
Stuart Carnie 35e137e1f6
feat(storage): TagKeysSet is a set of tag keys derived from models.Tags
This type will be used by the storage TagKeys API to quickly determine
the distinct set of tag keys over a set of series keys.

It is also intended to replace the implementation of keyMerger in
the reads package in a separate PR.
2019-04-24 11:14:21 -07:00
Ben Johnson 272f340c30
Merge point parse & explode. 2019-04-24 10:12:15 -06:00
Todd Persen cd64ec8718 Fix typos in miscellaneous packages 2019-04-17 13:30:22 -07:00
Edd Robinson 582ed6834c ddress PR feedback 2019-03-07 09:56:07 +00:00
Edd Robinson 9949fef2a5 Only validate tag pairs are valid unicode
We will want to validate that all tag key/value data is valid unicode.
This commit changes the validation helper to only validate provided
tags, since measurements are currently very likely to contain invalid
utf-8 characters.

There are two exceptions to the tag validation: the validation of the
special tag keys for measurements and field keys.
2019-03-07 09:56:07 +00:00
Edd Robinson f029f1645d Change location and value for internal tag keys 2019-03-07 09:56:07 +00:00
Ben Johnson 12d35f1a50
Revert "Merge point parse & explode."
This reverts commit 1004abc3e1.
2019-03-02 06:23:04 -07:00
Ben Johnson 1004abc3e1
Merge point parse & explode. 2019-03-01 15:55:37 -07:00
Jeff Wendling aa12144fc7 storage: replay the WAL through the whole engine 2019-02-04 10:32:52 -07:00
Mark Rushakoff d73d73c0d4 chore: rename imports from platform to influxdb
I did this with a dumb editor macro, so some comments changed too.

Also rename root package from platform to influxdb.

In interest of minimizing risk, anyone importing the root package has
now aliased it to "platform" so that no changes beyond imports were
necessary in those files.

Lastly, replace the old platform module to local path /dev/null so that
nobody can accidentally reintroduce a platform dependency while
migrating platform code to influxdb.
2019-01-09 20:51:47 -08:00
Chris Goller 65bf4bb383 feat(models): only use common SI time units for timestamp precision
Closes #1959
2018-12-20 22:39:24 -06:00
Edd Robinson 186e0392ed Address PR feedback 2018-11-30 10:54:24 +00:00
Edd Robinson 9403c1ec8e Ensure error strings not capitalised ST1005 2018-11-30 10:54:24 +00:00
Mark Rushakoff ed4bba83f5 test(models): don't reassign package variable
Prior to this change, `go test -count=2 ./models` would fail like:

--- FAIL: TestMarshal (0.00s)
    points_test.go:37: got:  ,A\ FOO=bar,APPLE=orange,host=serverA,region=uswest
    points_test.go:38: exp:  ,apple=orange,foo=bar,host=serverA,region=uswest
    points_test.go:39: invalid match

because a later test reassigned the package-level variable that
TestMarshal depends on. Don't reassign that variable anymore.

Also cleaned up some whitespace, and moved an init function to the top
of the file for visibility.
2018-11-12 08:24:14 -08:00
Mark Rushakoff 39b9c26db4 chore: add String method to FieldType
Type conflicts should be rare, but when they do happen, printing out the
string name should save developers a couple minutes of digging compared
to looking up which numeric value means which type.
2018-10-26 08:24:18 -07:00
Chris Goller d2ecf8aad6 http(write): support timestamp precision 2018-10-25 21:23:50 -05:00
Jacob Marble 85c6cb5d23 Export models.UnescapeMeasurement 2018-10-23 11:09:47 -07:00
Edd Robinson fb0db04bc1 Initial import pkg package 2018-10-01 12:03:20 +01:00
Edd Robinson 04818c7859 Initial import of models package 2018-10-01 12:03:19 +01:00