Commit Graph

13761 Commits (ba38447e60d0da47f2e37b531e9b17a2289ab4ba)

Author SHA1 Message Date
Stuart Carnie 9e79b5cbaf influx-tools/format: implementation of various series and value writers
A format.Writer is an abstraction for reading data from
storage.ResultSet and writing to various formats. Those included are

* binary: efficient binary format using protocol buffers. This is the
  expected format for the import tooling. The data is written in the
  desired shard group shape so that it can be read and streams to
  TSM files without further transformation.

* line: line protocol use for exporting field type conflicts or as an
  alternative, lossless export format

* text: two debugging modes for outputting series or series and values
  in a more efficient format that line protocol.

* discard: reads and discards the source data. This can be useful for
  benchmarking and profiling the read and decode performance.
2018-04-19 18:05:55 -07:00
Stuart Carnie 338d6b463d influx-tools/server: abstractions to simplify usage in Enterprise
This package is the only component that must be implemented to
use the influx-tools in the Enterprise edition.
2018-04-19 18:05:55 -07:00
Stuart Carnie a8f3c8ed5b import-tools: minimal storage API to read shard data efficiently
* this implementation produces a cursor per shard to permit the export
  tool to distinguish field type conflicts
2018-04-19 18:05:55 -07:00
Stuart Carnie a8692a9e24 services/meta: improve readability of Contains function, add unit tests 2018-04-19 18:05:55 -07:00
Stuart Carnie e7389b18c0 tsdb: add additional engine options
* filters allow specific combinations of database, retention policy and
  shard groups to be opened. This was added to reduce the start-up time
  of the export tool and limit the memory usage.
2018-04-19 18:05:55 -07:00
Ben Johnson 8c9d179a94
Add 'influx_inspect deletetsm' 2018-04-19 10:36:08 -06:00
Jacob Marble 232be14aef respect rp parameter in /query 2018-04-19 08:31:43 -07:00
Hercules Mango Churchill 12b1ecc40e Update changelog 2018-04-17 19:49:05 +00:00
Jonathan A. Sternberg 672a32a5ef
Merge pull request #9620 from Tomcat-Engineering/more-math-functions
Add more math functions to influxql
2018-04-17 13:07:12 -05:00
Tom Young 42581c7432 Add new math functions:
- abs
- asin, acos, atan, atan2
- exp, ln, log, log2, log10
- pow, sqrt
2018-04-17 12:56:36 -05:00
Jonathan A. Sternberg 7463195035
Merge pull request #9710 from jangaraj/master
Update client README - close client resources
2018-04-16 16:56:18 -05:00
Jacob Marble 321ae4ff04
update CircleCI config to 2.0 syntax (#9711)
* enable flaky test, see if CircleCI fails

* Use CircleCI 2.0 with docker layer caching

* update CONTRIBUTING
2018-04-16 12:00:44 -07:00
Jan Garaj fd3156e4c1 Update client README - close client resources 2018-04-16 18:45:58 +01:00
Stuart Carnie 7ebfc9c544 add default to avoid blocking 2018-04-12 15:42:33 -07:00
Adam a00e4dc70b
Changed file download retry rate to have exponential wait times (#9706)
* Changed file download retry rate to have exponential wait times

* Fixes from PR review
2018-04-12 15:21:02 -04:00
Jacob Marble 10a7ffb647
Check for errors from binary.Uvarint when reading TSI logs (#9705)
* Check for errors from binary.Uvarint when reading TSI logs

* also check len(parsed) == len(input)

* wrap binary.Uvarint

* make uvarint() more generally useful/used
2018-04-12 09:59:56 -07:00
Jonathan A. Sternberg a70cd46102
Merge pull request #9696 from influxdata/js-9690-properly-track-response-bytes-written
Properly track the response bytes written for queries in all format types
2018-04-10 13:00:47 -05:00
Edd Robinson 5fd75fbbbc
Merge pull request #9699 from influxdata/jw-delete-predicate
Return time range from delete predicate func
2018-04-10 17:06:39 +01:00
Jonathan A. Sternberg 1f9227e20c Allow math functions to be used in the condition 2018-04-10 10:55:34 -05:00
Jason Wilder 97ecf62ffb Return time range from delete predicate func
This moves the time range to delete to be returned by the predicate
func in DeleteSeriesRangeWithPredicate.  It allows for a single delete
to delete different ranges of times per series instead of a single
range of time for all series.
2018-04-09 20:01:33 -06:00
Jonathan A. Sternberg 8334693b47 Properly track the response bytes written for queries in all format types
The number of bytes written for CSV responses and probably MessagePack
responses was incorrect.
2018-04-09 12:42:58 -05:00
Jonathan A. Sternberg bf0eb140ec
Merge pull request #9686 from influxdata/js-tsm1-aggregate-benchmarks
Adding additional aggregate benchmarks for tsm1
2018-04-09 11:08:57 -05:00
Jonathan A. Sternberg 117aac4b9e Adding additional aggregate benchmarks for tsm1
This will help us address performance problems in the underlying tsm1
implementations of the aggregate iterators.
2018-04-09 10:37:33 -05:00
Jonathan A. Sternberg 529c028603
Merge pull request #9682 from influxdata/js-9504-csv-show-diagnostics-panic
Avoid a panic when using show diagnostics with text/csv
2018-04-09 10:16:16 -05:00
Jonathan A. Sternberg 243ed2ea5e Avoid a panic when using show diagnostics with text/csv
If the columns change between series, it will now act as if it was a new
statement id and reprint the headers. This only happens with show
diagnostics at the moment and we shouldn't add this functionality
anywhere else anyway.
2018-04-09 09:09:42 -05:00
Edd Robinson 2e46cd525f
Merge pull request #9687 from influxdata/jw-tombstone-panic
Fix panic in readTombstoneV4
2018-04-06 14:30:39 +01:00
Jason Wilder 8cc2e68d3b Fix panic in readTombstoneV4
The length check was backwards so if a series key was longer than
4096 bytes, it would cause a slice out of bounds panic.
2018-04-05 22:15:54 -07:00
Adam 72bceca888
Fix stream package to allow for renaming the file before writing it to the stream (#9684)
* Fix stream package to allow for renaming the file before writing it to the stream

* updated test to make sure that the final tsm file has more than one block
2018-04-05 16:24:29 -04:00
Hercules Mango Churchill 4337f5a547 Update changelog 2018-04-05 20:03:07 +00:00
Jonathan A. Sternberg b24afed528
Merge pull request #9399 from influxdata/js-9266-unix-socket-permissions
Allow customizing the unix socket group and permissions created by the server
2018-04-05 15:02:49 -05:00
Jonathan A. Sternberg 1b738d3991 Allow customizing the unix socket group and permissions created by the server 2018-04-05 14:40:12 -05:00
Hercules Mango Churchill 38d2d29f79 Update changelog 2018-04-04 21:16:18 +00:00
Jonathan A. Sternberg 733f2862f5
Merge pull request #9632 from Ahmah2009/master
Implement floor, ceil, and round functions
2018-04-04 16:12:25 -05:00
ahmah2009 7968e21881 Implement floor, ceil, and round functions 2018-04-04 23:53:55 +03:00
Ben Johnson ef8f6b71b6
Merge pull request #9679 from influxdata/bj-buildtsi-require-root
Check for root user when running buildtsi.
2018-04-04 12:11:00 -06:00
Ben Johnson fd491b28a6
Check for root user when running buildtsi.
This commit also adds the `max-log-file-size` flag.
2018-04-04 09:58:02 -06:00
Jonathan A. Sternberg 729c8ad135
Merge pull request #9673 from influxdata/js-9667-explain-analyze-output
Update explain analyze to output data related to the iterator scanners
2018-04-02 18:05:23 -05:00
Jonathan A. Sternberg 8aeb0fa0c6 Update explain analyze to output data related to the iterator scanners 2018-04-02 14:49:22 -05:00
Ben Johnson 891736d22b
Merge pull request #9670 from influxdata/bj-adjustable-tsi-compaction-threshold
Add adjustable TSI log file size.
2018-04-02 13:34:14 -06:00
Ben Johnson 92d38414f2
Add adjustable TSI log file size.
This commit adds the `max-index-log-file-size` configuration flag so
that users can restrict the maximum size of log files before compaction.
The default limit was also lowered from `5MB` to `1MB`. The original
size was set before we partitioned the index so the change reflects this.
2018-04-02 11:47:59 -06:00
Jonathan A. Sternberg dc71a8d82b
Merge pull request #9666 from influxdata/js-simplify-call-valuer
Update the interface for the simplified call valuer
2018-04-02 11:47:41 -05:00
Hercules Mango Churchill 7576bed1ec Update changelog 2018-03-31 16:16:51 +00:00
Jonathan A. Sternberg a383dc982c
Merge pull request #9665 from influxdata/js-9624-subquery-cast
Enable casting values from a subquery
2018-03-31 11:16:34 -05:00
Jonathan A. Sternberg f7bfae4044 Update the interface for the simplified call valuer 2018-03-31 00:21:36 -05:00
Jonathan A. Sternberg 0f304690c5 Enable casting values from a subquery
This also fixes the cursor system to abandon iterators that will not
produce meaningful results since the variables are all unknown types.

This creates a weird behavior that existed in previous releases and we
are keeping here for backwards compatibility. If a subquery referenced a
field that didn't exist in the subquery, it will return nothing. But, if
there are two subqueries and one of them has the field exist and the
other doesn't, the second will return all null values.
2018-03-30 16:58:37 -05:00
Mark Rushakoff 02fd678483
Merge pull request #9663 from influxdata/mr-walread-error
Log error encountered when reading WAL files
2018-03-30 11:22:56 -07:00
Mark Rushakoff b3c2d9290f Log error encountered when reading WAL files
Inspired by #9657.
2018-03-30 09:40:58 -07:00
Jonathan A. Sternberg 5e08187cb6
Merge pull request #9658 from ch33hau/fix-contribution-doc
Fix curl commands in contributing doc
2018-03-29 22:10:24 -05:00
Hercules Mango Churchill 8682fa9c3e Update changelog 2018-03-29 21:17:54 +00:00
Ben Johnson 556a4f77ef
Merge pull request #9661 from influxdata/bj-buildtsi-key
Fix buildtsi partition key.
2018-03-29 15:15:47 -06:00