Commit Graph

988 Commits (8d679cf0c36edf5aaede7a0ea068cf9d0c9be409)

Author SHA1 Message Date
Edd Robinson 8d679cf0c3
Merge pull request #10265 from influxdata/er-go1.11
Update to Go 1.11
2018-09-21 12:04:57 -07:00
Stuart Carnie 62b9791da8 feat(flux): Add initial support for executing Flux queries 2018-09-17 22:38:27 -07:00
Edd Robinson 18cb95f858 Update Go to 1.11 2018-09-06 16:40:20 +01:00
Stuart Carnie badc1d8ed3 fix(storage): Fix potential panic if nil is returned 2018-08-24 15:05:48 -07:00
Stuart Carnie 59bd822af9 refactor(storage): Export subset of APIs to facilitate reuse
* the protocol service definition, ReadRequest and ReadResponse is
  reused across projects, rather than requiring redefinition.

* the ReadRequest protocol buffer definition removes the concept of a
  database and retention policy, replacing it with a field named
  ReadSource of type google.protobuf.Any. OSS requests will use the
  ReadSource message structure defined in local to this package, which
  defines fields to represent a Database and RetentionPolicy. Other
  implementations can provide their own data structure allowing the
  remainder of the ReadRequest to be reused.

* The RPC service and Store are expected to be redefined to handle their
  specific requirements for resolving a ReadSource

* ResultSet and GroupResultSet are interfaces representing non-grouping
  and grouping read behavior respectively. Calling NewResultSet or
  NewGroupResultSet will construct instances of these types

* The ResponseWriter type is exported to deal with serialization of
  the ResultSet and GroupResultSet types
2018-08-24 11:56:04 -07:00
Stuart Carnie 2f4fcd8255 chore: Remove BatchCursor references 2018-08-24 11:56:04 -07:00
Edd Robinson ca1b780906
Merge pull request #10213 from influxdata/er-test-go1.11
Test InfluxDB on Go 1.11rc1
2018-08-21 18:19:25 +01:00
Edd Robinson 663566e3e0 Ensure go fmt passes on 1.10/11 2018-08-21 17:39:42 +01:00
Tanya Gordeeva 9d711d9417 httpd/meta: use open auth when unrestricted
This changes adds additional auth shortcutting, primarily for enterprise
usecases with simple FGA setups. OSS users won't see any changes.
2018-08-20 12:55:40 -07:00
Edd Robinson 9eece563b1 Simplify loops 2018-08-05 15:16:33 +01:00
Jeff Wendling 767b991089 implement using tls configuration 2018-08-02 14:29:07 -06:00
Stuart Carnie 468497c11f fix(storage): Check for Array cursor types 2018-07-18 12:43:53 -07:00
Edd Robinson f03101e660 PR feedback 2018-07-18 12:29:03 +01:00
Edd Robinson afa581ea08 Testing 2018-07-18 12:21:54 +01:00
Edd Robinson 80dc07cbcb Efficient means of getting fields for measurement
If it's known that the read request only needs to use a single
measurement, then we can avoid the need to get field keys via the query
engine.

However, that means that a new method of getting the field keys for a
measurement would be needed. This commit exposes a method to efficiently
get field key names for a measurement across multiple shards.

name
2018-07-18 12:21:54 +01:00
Edd Robinson 9c5c1c7001 Optimisation for expressions with single measument 2018-07-18 12:21:54 +01:00
Stuart Carnie d977c0ac24 fix(tsdb): Fix existing Prometheus tests based on batch cursors 2018-07-16 08:55:37 -07:00
Stuart Carnie 497fc42779 pr(tsdb): Feedback items from megacheck
* batch cursors and cursorIterator will be removed in a follow up
  PR using Arrow array data structures
2018-07-16 08:55:37 -07:00
Stuart Carnie 910d0fe5e6 feat(tsm1): ArrayCursor interfaces and implementations
Array cursors are enabled for storage RPC calls

tsm1:

* Implemented cursors that utilize Array decoders

storage:

* Abstractions to easily switch to Array cursors
2018-07-16 08:55:37 -07:00
Gunnar e9f0dc48ca
Merge pull request #10051 from influxdata/ga-config
Update example config with UDP precision option
2018-07-10 08:16:28 -07:00
Gunnar Aasen 8870512e6b Update example config with UDP precision option
Also add a test for precision in the UDP configuration.
2018-07-09 10:07:05 -07:00
Jeff Wendling 07e5465cb8 httpd: fix flaky test in timeout handler
there were two problems with this code:

1. the send on pending did not imply that the handler was running
2. there was a race starting the handler with timing out

1 is fixed by sending to a begin channel inside the handler. it is
then guaranteed that the timeout handler code has been entered.

2 is fixed by attempting to acquire the semaphore channel once before
checking the timeout channel. in this way, if there is capacity, which
in this test there is known to be, it is guaranteed to be taken. if
we check with the timer at the same time and the timer has already
fired, there is a pseudorandom chance the timer will be taken even
if there is capacity.
2018-07-05 12:09:09 -06:00
Jonathan A. Sternberg 88b81941ac Modify the storage service to expose a grpc interface instead of yarpc 2018-06-28 14:03:09 -05:00
Edd Robinson 8fd00853e7 Ensure read service regexes get optimised 2018-06-22 19:32:17 +01:00
Jonathan A. Sternberg 87d2469877
Merge pull request #9964 from influxdata/js-enable-storage-service
Enable the storage service by default
2018-06-13 16:28:27 -05:00
Edd Robinson 3cb9e13d58 Address PR feedback 2018-06-13 17:41:50 +01:00
Jonathan A. Sternberg 17ca220f33 Enable the storage service by default 2018-06-13 10:56:50 -05:00
Edd Robinson 8a78e64868 Make zero results work properly 2018-06-12 23:49:04 +01:00
Edd Robinson 28b6df7afb Ensure remote read can handle no data in time 2018-06-12 23:10:18 +01:00
Edd Robinson 522e509709 Add further tests 2018-06-12 15:54:18 +01:00
Edd Robinson 524f400836 Make testing of handler easier 2018-06-12 15:54:18 +01:00
Edd Robinson 806464d9e7 Add storage package mocks 2018-06-12 15:54:18 +01:00
Edd Robinson 40fa4eddc0 Fix overflow 2018-06-12 15:54:18 +01:00
Paul Dix 4f7b93342c Update Prometheus read/write to use new storage query layer.
* Update Prometheus remote write to use metric name as measurement name and value as the field name.
* Update Prometheus remote read to use the storage.Read method to bypass the InfluxQL query engine.
2018-06-12 15:54:18 +01:00
Jeff Wendling 5ec7b901bb
Merge pull request #9847 from influxdata/jmw-docker-on-windows
fix(tsdb): attempt to work on docker on windows
2018-06-01 15:16:51 -06:00
Jeff Wendling e6aec771b0 fix(tsdb): attempt to work on docker on windows
multiple users have attempted to run influxdb in a docker container
with a windows host and a volume mounted from windows. that causes
problems because it apparently uses samba/cifs which does not
support fsync on directories. this patchset will, if it receives an EINVAL
on directory fsync, as is what appears to happen on samba/cifs, then it
will ignore it. this should help.

fixes #9833.
fixes #9630.
2018-06-01 14:57:18 -06:00
Stuart Carnie e4b3204328 fix(services/storage): Don't serialize empty group frames 2018-06-01 13:28:34 -07:00
Stuart Carnie 4b7a4868a9 fix(services/storage): Add nil checks to prevent panics
Fixes #9925
2018-05-31 14:22:19 -07:00
Stuart Carnie d42abde836 refactor(services/storage): Enhanced group support for Read RPC 2018-05-30 11:32:20 -07:00
Ben Johnson 8b44e3142c
Add optional pprof http endpoint immediately on startup.
This commit adds `debug-pprof-enabled` which will start the default
`net/http/pprof` endpoint and bind against `localhost:6060`. This
will help to debug startup performance issues.
2018-05-23 14:32:15 -06:00
Ben Johnson 8a74c6759f
Add http write throttling.
This commit adds throttling to the HTTP write endpoints based on
queue depth and, optionally, timeout. Two queues exist: `enqueued`
and `current`. The `current` queue is the number of concurrent
requests that can be processed. The `enqueued` queue limits the
maximum number of requests that can be waiting to be processed.

If the timeout is exceeded or the `enqueued` queue is full then
a `"503 Service unavailable"` code is returned and the error is
logged.

By default these options are turned off.
2018-05-21 13:08:24 -06:00
Jeff Wendling e89438f7c2 fix flaky subscriber tests
Fixes #9554.
2018-04-24 12:34:45 -06:00
Jonathan A. Sternberg a7e1da5f86 Add suppress-write-log option to disable the write log when the log is enabled 2018-04-23 12:45:48 -05:00
Stuart Carnie a8692a9e24 services/meta: improve readability of Contains function, add unit tests 2018-04-19 18:05:55 -07:00
Jacob Marble 232be14aef respect rp parameter in /query 2018-04-19 08:31:43 -07:00
Jacob Marble 321ae4ff04
update CircleCI config to 2.0 syntax (#9711)
* enable flaky test, see if CircleCI fails

* Use CircleCI 2.0 with docker layer caching

* update CONTRIBUTING
2018-04-16 12:00:44 -07:00
Jonathan A. Sternberg 8334693b47 Properly track the response bytes written for queries in all format types
The number of bytes written for CSV responses and probably MessagePack
responses was incorrect.
2018-04-09 12:42:58 -05:00
Jonathan A. Sternberg 243ed2ea5e Avoid a panic when using show diagnostics with text/csv
If the columns change between series, it will now act as if it was a new
statement id and reprint the headers. This only happens with show
diagnostics at the moment and we shouldn't add this functionality
anywhere else anyway.
2018-04-09 09:09:42 -05:00
Jonathan A. Sternberg 1b738d3991 Allow customizing the unix socket group and permissions created by the server 2018-04-05 14:40:12 -05:00
Ben Johnson 1fe9abd66f
Delete deleted shards in retention service. 2018-03-28 10:44:14 -06:00