Commit Graph

13736 Commits (48797873ee24fc1dcb5a63f23474d3210f6d68c5)

Author SHA1 Message Date
Edd Robinson 652bb3d3c3 Keep __name__ tag in results 2018-06-13 11:12:56 +01:00
Edd Robinson 62dd27b5e8 Remove unused code 2018-06-13 00:25:59 +01:00
Edd Robinson 1eca0093de Remove __name__ tag once parsed for measurement 2018-06-13 00:20:00 +01:00
Edd Robinson 8a78e64868 Make zero results work properly 2018-06-12 23:49:04 +01:00
Edd Robinson 28b6df7afb Ensure remote read can handle no data in time 2018-06-12 23:10:18 +01:00
Edd Robinson 522e509709 Add further tests 2018-06-12 15:54:18 +01:00
Edd Robinson b314c316a9 Add regex support to storage service 2018-06-12 15:54:18 +01:00
Edd Robinson 524f400836 Make testing of handler easier 2018-06-12 15:54:18 +01:00
Edd Robinson 806464d9e7 Add storage package mocks 2018-06-12 15:54:18 +01:00
Edd Robinson 40fa4eddc0 Fix overflow 2018-06-12 15:54:18 +01:00
Paul Dix 4f7b93342c Update Prometheus read/write to use new storage query layer.
* Update Prometheus remote write to use metric name as measurement name and value as the field name.
* Update Prometheus remote read to use the storage.Read method to bypass the InfluxQL query engine.
2018-06-12 15:54:18 +01:00
Peng Liu 403559154f Add dep for pgzip 2018-06-12 00:15:26 -07:00
Ben Johnson 3eb5349c70
gofmt 2018-06-06 14:50:43 -06:00
Ben Johnson 28c7164e3e
Merge pull request #9937 from influxdata/bj-export-shard-engine
Export Shard.Engine()
2018-06-05 12:34:49 -06:00
Ben Johnson d54a580bba
Merge pull request #9835 from huhue/master
Performance optimization for testcode
2018-06-04 15:06:25 -06:00
Ben Johnson cfaaf39d8b
Export Shard.Engine() 2018-06-04 13:25:03 -06:00
Jeff Wendling 5ec7b901bb
Merge pull request #9847 from influxdata/jmw-docker-on-windows
fix(tsdb): attempt to work on docker on windows
2018-06-01 15:16:51 -06:00
Jeff Wendling e6aec771b0 fix(tsdb): attempt to work on docker on windows
multiple users have attempted to run influxdb in a docker container
with a windows host and a volume mounted from windows. that causes
problems because it apparently uses samba/cifs which does not
support fsync on directories. this patchset will, if it receives an EINVAL
on directory fsync, as is what appears to happen on samba/cifs, then it
will ignore it. this should help.

fixes #9833.
fixes #9630.
2018-06-01 14:57:18 -06:00
Stuart Carnie e4b3204328 fix(services/storage): Don't serialize empty group frames 2018-06-01 13:28:34 -07:00
Jeff Wendling 84e6a25b62
Merge pull request #9933 from influxdata/jgm-fix-tests
tests: Fix OpenLimiter is nil
2018-06-01 12:38:03 -06:00
Jacob Marble 9c4073ce75 tests: Fix OpenLimiter is nil 2018-06-01 11:20:44 -07:00
Hercules Mango Churchill 1a16a4ec61 Update changelog 2018-06-01 15:32:21 +00:00
Jacob Marble 82551a70e7
Merge pull request #9921 from influxdata/jgm-escape
buildtsi: Do not escape measurement names
2018-06-01 08:32:01 -07:00
Stuart Carnie 4b7a4868a9 fix(services/storage): Add nil checks to prevent panics
Fixes #9925
2018-05-31 14:22:19 -07:00
Hercules Mango Churchill 3ccc076b0f Update changelog 2018-05-31 20:49:41 +00:00
Jacob Marble 426a9fe765
Merge pull request #9928 from influxdata/jgm-limiter
TSM: Enforce OpenLimiter must not be nil
2018-05-31 13:49:24 -07:00
Jacob Marble 9a7b652a1c TSM: OpenLimiter must not be nil 2018-05-31 13:43:16 -07:00
Jacob Marble 44c5da060b buildtsi: Do not escape measurement names
When `influx_inspect buildtsi` is used to create a new `tsi1` index, spaces in measurement names are escaped, so measurement "a b" is changed to "a\ b".

This change modifies `models.ParseKeyBytes()` and `models.ParseName()` to unescape measurement names. `models.ParseKeyBytes()` returns unescaped tag keys, so this seems like the natural place to unescape measurement names.

Also followed `scanMeasurement()` to see what other code could be problematic, and this should be everything (the result of one other use of `scanMeasurement()` is later escaped).

Removed `tsdb.MeasurementFromSeriesKey()`. These methods are exported, so checked for side effects in other InfluxData repositories.
2018-05-30 15:20:56 -07:00
Hercules Mango Churchill 8efed00569 Update changelog 2018-05-30 19:44:28 +00:00
Ben Johnson cec2a2d988
Merge pull request #9918 from influxdata/bj-tsm-open-limiter
TSM1 Open Limiter
2018-05-30 13:13:14 -06:00
Stuart Carnie d42abde836 refactor(services/storage): Enhanced group support for Read RPC 2018-05-30 11:32:20 -07:00
Stuart Carnie 0253f6fe05 fix(tsdb/engine/tsm1): Fix panic when closing cursor multiple times
Fixes #9909
2018-05-29 09:59:52 -07:00
Jacob Marble bb313765e4 tsdb/tsm1: Clean up TSM filename format/parse 2018-05-29 09:57:48 -07:00
Ben Johnson d3e3b05a49
Add tsm1 open limiter
This commit restricts the number of TSM1 files that can be opened
concurrently across the entire `tsdb.Store`. There is currently
a limit for the number of shards that can be opened concurrently,
however, this limit does not help when the number of CPU cores
is higher than the number of shards. Because TSM1 files have a 2GB
limit and there is no limit on the number of files per shard,
extremely large shards (1TB+) can load 1,000s of files simultaneously.
2018-05-29 10:21:53 -06:00
Jonathan A. Sternberg 4e1228fce3
Merge pull request #9855 from influxdata/js-megacheck-importer
Remove an unused variable caught by megacheck in the importer
2018-05-29 08:44:06 -05:00
Jonathan A. Sternberg efb811d156
Merge pull request #9907 from influxdata/js-fix-influx-inspect-manpage
Fix the influx_inspect man page
2018-05-25 11:56:38 -05:00
Jonathan A. Sternberg 6c418546bc Fix the influx_inspect man page 2018-05-25 09:36:20 -05:00
Jeff Wendling 2b3cd8406f
Merge pull request #9892 from influxdata/jmw-tombstone-notifications
tsdb: observe tombstone files as well
2018-05-24 09:56:09 -06:00
Ben Johnson 67030d1dae
Merge pull request #9898 from influxdata/pprof
Add optional pprof http endpoint immediately on startup.
2018-05-24 09:08:38 -06:00
Ben Johnson 8b44e3142c
Add optional pprof http endpoint immediately on startup.
This commit adds `debug-pprof-enabled` which will start the default
`net/http/pprof` endpoint and bind against `localhost:6060`. This
will help to debug startup performance issues.
2018-05-23 14:32:15 -06:00
Stuart Carnie e3d7095d14 fix(tsm1): Avoid searching index if key outside index bounds
This improvement avoids performing a binary search on the index by
first checking the key against the lower and upper bounds. Particularly
useful for multiple, fully-compacted TSM files.
2018-05-23 13:29:49 -07:00
Jeff Wendling ce565965a4 tsdb: avoid nil checks on the observer
this avoids nil panics in the case that someone eventually forgets.
2018-05-23 13:15:41 -06:00
Jeff Wendling 8ad515b387 tsdb: remove the shard id again
callers can always ensure that the observer set on the engine options
is appropriate for that shard id. this simplifies the api and reduces
the chance of bugs due to mixing up shard ids.
2018-05-23 13:04:54 -06:00
Stuart Carnie e209a0a1f2 Restore "Performance optimization suggestions"
CLA confirmed
PR: https://github.com/influxdata/influxdb/pull/9836

This reverts commit 7215bad
2018-05-23 08:54:20 -07:00
Jeff Wendling 15ae0bd98d tsdb: observe tombstone files as well 2018-05-22 22:07:16 -06:00
Jeff Wendling e62b1a02fb
Merge pull request #9879 from influxdata/jmw-add-shard-number-to-observer
tsdb: add shard number to the observer
2018-05-21 20:00:47 -06:00
Stuart Carnie 7215badfcd Revert "Performance optimization suggestions"
This reverts commit f82d53f75d.
2018-05-21 14:10:03 -07:00
chenjian.cj f82d53f75d Performance optimization suggestions 2018-05-21 13:30:32 -07:00
Ben Johnson ff070313ed
Merge pull request #9877 from influxdata/bj-http-throttle
Add http write throttling.
2018-05-21 13:22:19 -06:00
Ben Johnson 8a74c6759f
Add http write throttling.
This commit adds throttling to the HTTP write endpoints based on
queue depth and, optionally, timeout. Two queues exist: `enqueued`
and `current`. The `current` queue is the number of concurrent
requests that can be processed. The `enqueued` queue limits the
maximum number of requests that can be waiting to be processed.

If the timeout is exceeded or the `enqueued` queue is full then
a `"503 Service unavailable"` code is returned and the error is
logged.

By default these options are turned off.
2018-05-21 13:08:24 -06:00