Commit Graph

39 Commits (497fc42779af7312f8b22a2102e69c13d3bfe721)

Author SHA1 Message Date
Stuart Carnie 497fc42779 pr(tsdb): Feedback items from megacheck
* batch cursors and cursorIterator will be removed in a follow up
  PR using Arrow array data structures
2018-07-16 08:55:37 -07:00
Stuart Carnie 910d0fe5e6 feat(tsm1): ArrayCursor interfaces and implementations
Array cursors are enabled for storage RPC calls

tsm1:

* Implemented cursors that utilize Array decoders

storage:

* Abstractions to easily switch to Array cursors
2018-07-16 08:55:37 -07:00
Jonathan A. Sternberg 88b81941ac Modify the storage service to expose a grpc interface instead of yarpc 2018-06-28 14:03:09 -05:00
Edd Robinson 8fd00853e7 Ensure read service regexes get optimised 2018-06-22 19:32:17 +01:00
Jonathan A. Sternberg 87d2469877
Merge pull request #9964 from influxdata/js-enable-storage-service
Enable the storage service by default
2018-06-13 16:28:27 -05:00
Jonathan A. Sternberg 17ca220f33 Enable the storage service by default 2018-06-13 10:56:50 -05:00
Edd Robinson 8a78e64868 Make zero results work properly 2018-06-12 23:49:04 +01:00
Edd Robinson 28b6df7afb Ensure remote read can handle no data in time 2018-06-12 23:10:18 +01:00
Edd Robinson 522e509709 Add further tests 2018-06-12 15:54:18 +01:00
Edd Robinson 806464d9e7 Add storage package mocks 2018-06-12 15:54:18 +01:00
Stuart Carnie e4b3204328 fix(services/storage): Don't serialize empty group frames 2018-06-01 13:28:34 -07:00
Stuart Carnie 4b7a4868a9 fix(services/storage): Add nil checks to prevent panics
Fixes #9925
2018-05-31 14:22:19 -07:00
Stuart Carnie d42abde836 refactor(services/storage): Enhanced group support for Read RPC 2018-05-30 11:32:20 -07:00
Stuart Carnie ef2ba80ce2 don't overwrite `_measurement` for multi-tenant reads 2018-03-23 13:59:42 -07:00
Stuart Carnie 813cb1a2f6 to var () or not to var (), that is the question 2018-03-23 12:26:55 -07:00
Stuart Carnie ee3e2ad67f rename Tenant -> OrgID 2018-03-23 12:26:55 -07:00
Stuart Carnie 2cc1f5137e support for tenant+bucket
NOTE: to match storage service, values for database and rp are
hard-coded to `db` and `rp` respectively
2018-03-23 12:26:55 -07:00
Stuart Carnie aa61359cc7 Storage RPC API improvements. See PR for details
* reduce # allocations (115M -> 22M)
* reduce size allocations (53GB -> 1.3GB)
* reduce RPC query time (45s -> 12.9s)
2018-03-21 13:46:09 -07:00
Stuart Carnie 0a5a07dc3a series keys are produced in ascending order 2018-02-22 13:08:36 -07:00
Stuart Carnie 41dc96ca91 restore `MetaClient`, which is needed by store
* Switch from an anonymous type to avoid false positives with
  `megacheck`
2018-02-08 12:13:13 -07:00
Edd Robinson 6a66b5faf0 Cleanup services package 2018-01-21 10:52:37 -08:00
Stuart Carnie 5dfe3b2645 inmem startup improvments
* only call ParseTags when necessary
* remove dependency on inmem.Series in tsdb test package
* Measurement and Series are no longer exported. Their use is restricted
  to the inmem package
* improve Measurement and Series types by exporting immutable
  fields and removing unnecessary APIs and locks

Reduced startup time from 28s to 17s. Overall improvement including
#9162 reduces startup from 46s to 17s for 1MM series across 14 shards.
2017-12-29 07:58:52 -07:00
Stuart Carnie 7cdfd95966 initial opentrace implementation for ifql interface
NOTE: does not include a default tracer until configuration across
projects is standardized
2017-11-22 14:42:26 -07:00
Stuart Carnie 89877d7764 ifql: writer tracks estimated size (bytes) to limit memory between Send 2017-11-20 11:33:37 -07:00
Edd Robinson c098081c7d Don't initialise a new Authorizer each query 2017-11-17 11:06:43 +00:00
Stuart Carnie 2e04e871c9 fix descending queries
* did not handle cached values correctly
* sort shards by time in either ascending or descending
  order depending on the RPC request ordering to ensure they
  are traversed in the correct order.
2017-11-13 17:14:36 -08:00
Jonathan A. Sternberg 0b7c56bcd8 Update the zap logger dependency
The previous sha was taken from a revision on a devel branch that I
thought would continue staying in the tree after it was merged. That
revision was rebased away and the API was changed for the logger.

This updates the usage of the logger and adds a simple package for
constructing the base logger.

The 1.0 version of zap changed the format of the default console logger
so this change moves over to this new logger instead of attempting to
retain backwards compatibility with the old format.
2017-11-10 16:27:16 -06:00
Stuart Carnie 57677be010 don't panic here; nil cursor is handled for now. fixes #9090 2017-11-10 11:21:31 -07:00
Edd Robinson 126db1b5f9
Merge pull request #9068 from influxdata/er-show-query-perf
Add time support to some meta queries
2017-11-07 15:48:58 +00:00
Stuart Carnie 6ee0c6ee0c check and flush frames while streaming points for a series
TODO(sgc): implement `writer` type that handles all the details
of writing frames to the RPC stream. Additional responsibilities
of writer include

* point frame recycling to reduce memory pressure
* skip empty point frames
* skip series frames with no points
2017-11-06 13:00:57 -07:00
Edd Robinson 98d584b63f Use index for SHOW X meta queries
When a meta query does not include a time component then it can be
answered exclusively by the index. This should result in a much faster
query execution that if the TSM engine was engaged.

This commit rewrites the following queries such that they make use
of the index where no time component is present:

  - SHOW MEASUREMENTS
  - SHOW SERIES
  - SHOW TAG KEYS
  - SHOW FIELD KEYS
2017-11-06 19:15:00 +00:00
Stuart Carnie cf2227def1 add expected data type to series frame 2017-11-06 11:12:27 -07:00
Stuart Carnie 728f5cc6ac strip series frame if no points returned 2017-11-03 17:04:33 -07:00
Stuart Carnie 10a0bb8f73 don't send empty response 2017-11-02 16:27:05 -07:00
Stuart Carnie f3d45ba301 influxdata/influxdb/influxql -> influxdata/influxql 2017-10-30 14:40:26 -07:00
Stuart Carnie d99cabb5d2 handle nil *indexSeriesCursor 2017-10-26 13:32:05 -07:00
Stuart Carnie ab17e15caf check nil iterator; check nil cursor when no data 2017-10-26 12:54:59 -07:00
Stuart Carnie dc04eaa8f3 Amendments based on feedback
* Fprint* functions
* No nakedness
* clarify panic messages
* spacing between case statements
* remove break in favor of return
* remove goto in favor of for { continue }
2017-10-25 13:38:07 -07:00
Stuart Carnie 415ed14c53 storage service
* storage service is disabled by default
* default port 8082
* RPC interface generated using yarpc via service.proto
2017-10-25 13:38:07 -07:00