Commit Graph

947 Commits (35a64dee993ea1855dcb5bfe0d43548b7a74172b)

Author SHA1 Message Date
Jeff Wendling e89438f7c2 fix flaky subscriber tests
Fixes #9554.
2018-04-24 12:34:45 -06:00
Jonathan A. Sternberg a7e1da5f86 Add suppress-write-log option to disable the write log when the log is enabled 2018-04-23 12:45:48 -05:00
Stuart Carnie a8692a9e24 services/meta: improve readability of Contains function, add unit tests 2018-04-19 18:05:55 -07:00
Jacob Marble 232be14aef respect rp parameter in /query 2018-04-19 08:31:43 -07:00
Jacob Marble 321ae4ff04
update CircleCI config to 2.0 syntax (#9711)
* enable flaky test, see if CircleCI fails

* Use CircleCI 2.0 with docker layer caching

* update CONTRIBUTING
2018-04-16 12:00:44 -07:00
Jonathan A. Sternberg 8334693b47 Properly track the response bytes written for queries in all format types
The number of bytes written for CSV responses and probably MessagePack
responses was incorrect.
2018-04-09 12:42:58 -05:00
Jonathan A. Sternberg 243ed2ea5e Avoid a panic when using show diagnostics with text/csv
If the columns change between series, it will now act as if it was a new
statement id and reprint the headers. This only happens with show
diagnostics at the moment and we shouldn't add this functionality
anywhere else anyway.
2018-04-09 09:09:42 -05:00
Jonathan A. Sternberg 1b738d3991 Allow customizing the unix socket group and permissions created by the server 2018-04-05 14:40:12 -05:00
Ben Johnson 1fe9abd66f
Delete deleted shards in retention service. 2018-03-28 10:44:14 -06:00
Stuart Carnie ef2ba80ce2 don't overwrite `_measurement` for multi-tenant reads 2018-03-23 13:59:42 -07:00
Stuart Carnie 813cb1a2f6 to var () or not to var (), that is the question 2018-03-23 12:26:55 -07:00
Stuart Carnie ee3e2ad67f rename Tenant -> OrgID 2018-03-23 12:26:55 -07:00
Stuart Carnie 2cc1f5137e support for tenant+bucket
NOTE: to match storage service, values for database and rp are
hard-coded to `db` and `rp` respectively
2018-03-23 12:26:55 -07:00
Stuart Carnie aa61359cc7 Storage RPC API improvements. See PR for details
* reduce # allocations (115M -> 22M)
* reduce size allocations (53GB -> 1.3GB)
* reduce RPC query time (45s -> 12.9s)
2018-03-21 13:46:09 -07:00
Edd Robinson bdd61298ea Skip flaky test 2018-03-12 16:18:35 +00:00
Jonathan A. Sternberg 733d842812 Turn the ExecutionContext into a context.Context
Along with modifying ExecutionContext to be a context and have the
TaskManager return the context itself, this also creates a Monitor
interface and exposes the Monitor through the Context. This way, we can
access the monitor from within the query.Select method and keep all of
the limits inside of the query package instead of leaking them into the
statement executor.

An eventual goal is to remove the InterruptCh from the IteratorOptions
and use the Context instead, but for now, we'll just assign the done
channel from the Context to the IteratorOptions so at least they refer
to the same channel.
2018-03-08 14:03:20 -06:00
Jonathan A. Sternberg de4390ae83 Rename some of the structs and interfaces in the query package
Remove the `Query` prefix from some structs and interfaces. They were
there so when the query engine was in the same package as influxql,
these would be differentiated. Now that the package name is query, the
extra prefix seems redundant.
2018-03-02 09:44:12 -06:00
Stuart Carnie a74d296200 use underscore vs period, fix doc comment, add database name to CQ 2018-02-26 10:08:43 -07:00
Stuart Carnie 0a5a07dc3a series keys are produced in ascending order 2018-02-22 13:08:36 -07:00
Stuart Carnie d135aecf02 Generate trace logs for a number of significant influx operations
* tsdb Store.Open traces all events related to opening files
    * op.name : tsdb.open
* retention policy shard deletions
    * op.name : retention.delete_check
* all TSM compaction strategies
    * op.name : tsm1.compact_group
* series file compactions
    * op.name : series_partition.compaction
* continuous query execution (if logging enabled)
    * op.name : continuous_querier.execute
* TSI log file compaction
    * op_name: index.tsi.compact_log_file
* TSI level compaction
    * op.name: index.tsi.compact_to_level
2018-02-21 15:08:49 -07:00
Jonathan A. Sternberg d38413a849
Merge pull request #9454 from influxdata/js-structured-logging
Update logging calls to take advantage of structured logging
2018-02-21 09:14:40 -06:00
Jonathan A. Sternberg 2bbd96768d Update logging calls to take advantage of structured logging
Includes a style guide that details the basics of how to log.
2018-02-20 10:04:19 -06:00
Stuart Carnie 584e7ac09a Added option to write HTTP request logs to separate file. 2018-02-14 23:11:01 -07:00
Mark Rushakoff f7fc6a6501 Remove IsAdmin() method from meta.User interface
The method was only called in tests.
2018-02-13 16:59:22 -08:00
Stuart Carnie 8f978068f9
Merge pull request #9415 from influxdata/sgc-storage
restore `MetaClient`, which is needed by store
2018-02-09 07:51:36 -07:00
Andrew Hare d21ebfe531 Do not report an error when dropping a CQ on a non-existent DB/RP
This makes the behvior similar to other places in the DB where we
don't return an error when we try to drop an object from a non-
existent database.
2018-02-08 13:28:47 -06:00
Stuart Carnie 41dc96ca91 restore `MetaClient`, which is needed by store
* Switch from an anonymous type to avoid false positives with
  `megacheck`
2018-02-08 12:13:13 -07:00
Edd Robinson f19588360e
Merge pull request #9349 from influxdata/er-the-purge
Cleanup of codebase using static analysis tools
2018-01-25 17:11:53 -08:00
Patrick Hemmer 2dc2c53093 fix nil err panic in msgpack httpd WriteResponse 2018-01-23 19:54:00 -05:00
Edd Robinson 6a66b5faf0 Cleanup services package 2018-01-21 10:52:37 -08:00
Adam 938db68198
Update restore functionality to run in online mode, consume Enterprise backup files. (#9207)
* Live Restore + Enterprise data format compatability

* Extended ImportData to import all DB's if no db name given

* Added a new enterprise data test, and backup command now prints the backup file paths at conclusion

* Added whole-system backup test

* Update to use protobuf in all enterprise data cases

* Update to test to do cross-testing with enterprise version

* incremental enterprise backup format support
2018-01-10 13:59:18 -05:00
Edd Robinson 1f3352efbd
Merge pull request #9153 from influxdata/er-prom-parsing
Fix Prometheus regex parsing
2018-01-02 18:39:46 +00:00
Stuart Carnie 5dfe3b2645 inmem startup improvments
* only call ParseTags when necessary
* remove dependency on inmem.Series in tsdb test package
* Measurement and Series are no longer exported. Their use is restricted
  to the inmem package
* improve Measurement and Series types by exporting immutable
  fields and removing unnecessary APIs and locks

Reduced startup time from 28s to 17s. Overall improvement including
#9162 reduces startup from 46s to 17s for 1MM series across 14 shards.
2017-12-29 07:58:52 -07:00
Ben Johnson d8b1d208c0
rebase 2017-12-20 15:13:34 -07:00
Edd Robinson c476a0b4a1 Merge branch 'master' into er-tsi-index-part 2017-12-15 18:31:24 +00:00
Jonathan A. Sternberg 5fcf57a764 Remove extraneous newlines from the log
The newlines were accidentally kept when changing the logger. They are
not necessary and mess up the log output.
2017-12-14 16:41:42 -06:00
Stuart Carnie 0d29dc1121 add Prometheus metrics HTTP endpoint 2017-12-11 08:51:40 -07:00
Edd Robinson 7d13bf3262 merge master 2017-12-08 17:21:58 +00:00
Edd Robinson f6835632e7 Merge master into branch 2017-12-08 17:11:07 +00:00
Adam a0b2195d6b
Pulled in backup-relevant code for review (#9193)
for issue #8879
2017-12-07 11:35:20 -05:00
Jonathan A. Sternberg 95e1e3b332
Merge pull request #8015 from influxdata/js-code-coverage
Expand code coverage for undercovered packages
2017-11-29 19:30:47 -06:00
Andrew Hare d7e328050c
Merge branch 'master' into ah-truncate-shards 2017-11-28 17:25:31 -07:00
Jonathan A. Sternberg b775ad3d5d Expand unit test code coverage in services that were undercovered
This expands code coverage for the following packages:
* monitor (3.5% -> 86.9%)
* services/precreator (31.6% -> 83.8%)
* services/retention (83.0% -> 84.9%)
* services/snapshotter (0.0% -> 82.1%)
* tcp (48.7% -> 60.0%)
2017-11-28 15:44:35 -06:00
Edd Robinson 8e3d29ec7a Fixes #9134.
This converts Prometheus' regex syntax for a condition value into InfluxDB's.
2017-11-27 11:19:01 +00:00
Stuart Carnie 7cdfd95966 initial opentrace implementation for ifql interface
NOTE: does not include a default tracer until configuration across
projects is standardized
2017-11-22 14:42:26 -07:00
Ben Johnson fc966a1b67
Add series file backup/restore. 2017-11-22 08:55:54 -07:00
Stuart Carnie 89877d7764 ifql: writer tracks estimated size (bytes) to limit memory between Send 2017-11-20 11:33:37 -07:00
Edd Robinson c098081c7d Don't initialise a new Authorizer each query 2017-11-17 11:06:43 +00:00
Jonathan A. Sternberg 97ab61addb
Merge pull request #9092 from influxdata/jenkinsfile
Initial jenkinsfile
2017-11-14 11:12:32 -06:00
Stuart Carnie 2e04e871c9 fix descending queries
* did not handle cached values correctly
* sort shards by time in either ascending or descending
  order depending on the RPC request ordering to ensure they
  are traversed in the correct order.
2017-11-13 17:14:36 -08:00