Commit Graph

1563 Commits (623bb71b0121c8ec65535190ee65966c1cc6248e)

Author SHA1 Message Date
Gustav Westling 69c5354d98
Use length instead of removing it 2016-12-30 12:23:40 +01:00
Gustav Westling 56d98325da
Removed ineffective assignments, and added checks for errors that previsouly was not checked 2016-12-29 20:26:15 +01:00
Cory LaNou bc5736f59d Merge pull request #7672 from influxdata/cjl-7563-rp-duration-inf
Enforce minimum shard duration when creating retention policies
2016-12-20 12:16:07 -06:00
Cory LaNou 880c7cdcc8 Merge branch 'master' into cjl-3188-cli-rp-context 2016-12-20 09:55:40 -06:00
Cory LaNou 0cbdea531a add the ability for retention policy context in cli with use command 2016-12-20 09:15:38 -06:00
Cory LaNou fbc9e3cfcc add clear command to cli 2016-12-20 09:14:20 -06:00
Cory LaNou 572da8985c enforce minimum shard duration when creating retention policies 2016-12-20 09:11:43 -06:00
Mark Rushakoff 295a29d4ea Fix quoting on exported string fields
The previous implementation was wrong and double-escaped quotes and
backslashes.
2016-12-18 00:37:48 -08:00
Mark Rushakoff ff78c84b0f Improve export performance
Benchmark improvements with this change:

benchmark                                   old ns/op     new ns/op     delta
BenchmarkExportTSMFloats_100s_250vps-4      23206480      10279106      -55.71%
BenchmarkExportTSMInts_100s_250vps-4        17995000      5762310       -67.98%
BenchmarkExportTSMBools_100s_250vps-4       17067605      4235467       -75.18%
BenchmarkExportTSMStrings_100s_250vps-4     54846997      34682568      -36.76%
BenchmarkExportWALFloats_100s_250vps-4      23459937      10436297      -55.51%
BenchmarkExportWALInts_100s_250vps-4        18747150      6236062       -66.74%
BenchmarkExportWALBools_100s_250vps-4       17988273      4814358       -73.24%
BenchmarkExportWALStrings_100s_250vps-4     59700802      35815739      -40.01%

benchmark                                   old allocs     new allocs     delta
BenchmarkExportTSMFloats_100s_250vps-4      201442         51738          -74.32%
BenchmarkExportTSMInts_100s_250vps-4        201442         51728          -74.32%
BenchmarkExportTSMBools_100s_250vps-4       201441         51638          -74.37%
BenchmarkExportTSMStrings_100s_250vps-4     404092         201584         -50.11%
BenchmarkExportWALFloats_100s_250vps-4      250322         75627          -69.79%
BenchmarkExportWALInts_100s_250vps-4        250323         75617          -69.79%
BenchmarkExportWALBools_100s_250vps-4       250321         75527          -69.83%
BenchmarkExportWALStrings_100s_250vps-4     452868         225291         -50.25%

benchmark                                   old bytes     new bytes     delta
BenchmarkExportTSMFloats_100s_250vps-4      5170539       2351789       -54.52%
BenchmarkExportTSMInts_100s_250vps-4        5143189       2331276       -54.67%
BenchmarkExportTSMBools_100s_250vps-4       3724951       2143780       -42.45%
BenchmarkExportTSMStrings_100s_250vps-4     17131400      10796281      -36.98%
BenchmarkExportWALFloats_100s_250vps-4      4487868       1468109       -67.29%
BenchmarkExportWALInts_100s_250vps-4        4458395       1452359       -67.42%
BenchmarkExportWALBools_100s_250vps-4       2838719       1258755       -55.66%
BenchmarkExportWALStrings_100s_250vps-4     16787201      10010700      -40.37%

Also, after improving those benchmarks, I did a time-filtered export on
a 450MB TSM file to a 21GB plain text output, with and without the
bufio.BufferedWriter.

Without buffering, it took about 263s, and with buffering, it took about
60s, for a delta of about -77%.
2016-12-17 20:15:39 -08:00
Mark Rushakoff da45aab52c Clean up export code, add tests and benchmarks
The export code was moved around a bit, particularly to ease testing
export of a single TSM or WAL file. The functionality should not have
changed.
2016-12-17 18:17:18 -08:00
Jonathan A. Sternberg ec57108520 Use proper uber-go/zap import path
It looks like the real import path to the project is go.uber.org/zap
instead of github.com/uber-go/zap since the example in the project
references that path.
2016-12-15 08:54:14 -06:00
Jonathan A. Sternberg 74d474594f Remove the override of GOMAXPROCS
This was needed when we were on go 1.4 but hasn't been needed since go
1.5. It was kept because we weren't sure if we were going to have to
rollback to an older version of Go at that time and we kept it so we
wouldn't forget to readd it.

Now that we are on go 1.7 with go 1.4 deprecated, there is no going back
so we might as well remove this so people can set GOMAXPROCS to a custom
value using environment variables.
2016-12-14 12:00:40 -06:00
Jonathan A. Sternberg 21502a39e8 Switch logging to use structured logging everywhere
The logging library has been switched to use uber-go/zap. While the
logging has been changed to use structured logging, this commit does not
change any of the logging statements to take advantage of the new
structured log or new log levels. Those changes will come in future
commits.
2016-12-14 10:45:15 -06:00
Cory LaNou 6f027ac60e Merge pull request #7702 from influxdata/cjl-6527-cli-precision-case-sensitive
fix precision being case sensitive in cli
2016-12-07 12:15:34 -06:00
Cory LaNou dddc3c6690
fix precision being case sensitive in cli 2016-12-07 12:04:19 -06:00
Cory LaNou 4442615f15 use spaces instead of tabs for CLI column output format 2016-12-07 11:21:04 -06:00
Cory LaNou a90a1f4b7f
cli was caching db/rp for insert into statements 2016-12-07 09:22:48 -06:00
Edd Robinson a98ad483ee Refactor how CLI manages configuration options 2016-12-02 18:26:47 +00:00
Edd Robinson 840fb6af38 Ensure unsafessl option passed to importer 2016-12-02 18:26:47 +00:00
Jonathan A. Sternberg bffc759cf9 Return the time from a percentile call on an integer
`percentile()` is supposed to be a selector and return the time of the
point, but that only got changed when the input was a float. Updating
the integer processor to also return the time of the point rather than
the beginning of the interval.
2016-12-01 12:34:48 -06:00
Jonathan A. Sternberg 3dc5165b3c Merge pull request #7505 from oiooj/pr-influx-unixsocket
support unix socket connection for influx CLI
2016-11-30 09:02:32 -06:00
oiooj 8ddb11cee2 support unix socket connect for influx CLI 2016-11-30 22:37:56 +08:00
Allen Petersen 31129ab0e9 Use slash separator for filenames in tar archives
NO-OP on platforms with unix path separator.
On Windows paths get converted to slashes before adding to archive and back to backslashes during restore.
2016-11-29 09:44:08 -08:00
Jonathan A. Sternberg b4db76cee2 Introduce syntax for marking a partial response with chunking
The `partial` tag has been added to the JSON response of a series and
the result so that a client knows when more of the series or result will
be sent in a future JSON chunk.

This helps interactive clients who don't want to wait for all of the
data to know if it is done processing the current series or the current
result. Previously, the client had to guess if the next chunk would
refer to the same result or a new result and it had to match the name
and tags of the two series to know if they were the same series. Now,
the client just needs to check the `partial` field included with the
response to know if it should expect more.

Fixed `max-row-limit` so it counts rows instead of results and it
truncates the response when the `max-row-limit` is reached.
2016-11-22 11:16:22 -06:00
Jonathan A. Sternberg e885fe5117 Expand string and boolean fields when using a wildcard with sample() 2016-11-15 15:56:47 -06:00
Marc 7d277d0def Add support for collectd security policy
There are 2 new keys in the configuration file.
- security-level: "none", "sign", or "encrypt".
- auth-file: The location of the user/password file.

Please see the collectd network doc for more details.
2016-11-08 08:42:09 +08:00
Mark Rushakoff d52eb01c17 Merge pull request #7492 from influxdata/mr-influx_inspect-help-verify
Mention verify subcommand in influx_inspect help
2016-11-07 13:23:39 -08:00
Jason Wilder 3c0aaae8ff Merge pull request #7556 from influxdata/rk-readme-fix
Update example in influx_inspect README.md
2016-11-07 14:16:43 -07:00
Jonathan A. Sternberg be49f3ee2c Merge pull request #7586 from influxdata/js-7575-fix-execute-flag-with-no-tty
Fix the `-execute` and `-import` when there is no TTY
2016-11-07 15:07:21 -06:00
Jason Wilder b6863b04d1 Disable/deprecate admin interface 2016-11-07 12:04:51 -07:00
Jonathan A. Sternberg 8a57f27eb9 Fix the `-execute` and `-import` when there is no TTY 2016-11-04 16:35:38 -05:00
Cory LaNou 6e290040bb
remove SetDefaultRetentionPolicy method from meta.Client 2016-11-03 09:39:41 -05:00
Cory LaNou cd272ce6c3 fix retention policy creation inconsistencies 2016-11-03 09:09:43 -05:00
Jonathan A. Sternberg 1b2fa645ee Fix incorrect grouping when multiple aggregates are used with sparse data
When a query would use a grouping with two different aggregates, it was
possible for one of the aggregates to return a value from a different
series key than the second aggregate. When these series keys didn't
match, the returned grouping would be screwed up because it sorted by
time before checking for name and tags.

This did not happen when the aggregates returned values for the same
series keys because then the iterators were aligned with each other.
2016-11-02 13:35:22 -05:00
Regan Kuchan 3ebaba4714 Update example in influx_inspect README.md 2016-11-01 10:16:23 -07:00
Jason Wilder 0b6f5441b9 Add config option to messages when limits exceeded
When a limit is exceeded, we return errors and sometimes log (if appropriate)
that a limit was exceeded.  The messages don't always provide an indication
as to where or how they are configured.

Instead, return the config option (easily searchable for) as well as the limit
currently set and the value that exceeded it when possible.
2016-10-28 14:54:45 -06:00
Jonathan A. Sternberg 3e29d3d9ca Truncate the version string when linking to the documentation
The admin console would dynamically discover the version from the
InfluxDB server, but for patch releases, it included the patch in the
link to the documentation and that wasn't a valid link.

Truncate the version so the documentation url is correct since we only
do documentation for `major.minor`.
2016-10-27 16:40:08 -05:00
joelegasse 05c252696e Merge pull request #7499 from influxdata/jl-export-doc
Update `influx_inspect export` doc
2016-10-26 14:17:27 -04:00
Jonathan A. Sternberg f1fbb55909 Change default time boundaries for raw queries
Changes the default time boundaries for raw queries so raw queries will
range until the end of time. Aggregate queries continue to have their
default end time be `now()`.
2016-10-25 15:08:51 -05:00
Cory LaNou cc8e34886c escape fields when exporting tsm/wal files 2016-10-25 13:36:45 -05:00
Joe LeGasse ad62d04ce3 influx_inspect: update flag descriptions 2016-10-25 11:05:06 -04:00
Edd Robinson b12b0d12fb Add regex benchmarks and fix existing approach 2016-10-25 11:10:03 +01:00
Jason Wilder 686d1a7ba4 Remove unused config options 2016-10-24 15:32:38 -06:00
Mark Rushakoff ab7bfe2473 Mention verify subcommand in influx_inspect help 2016-10-24 08:46:48 -07:00
Edd Robinson 0ae775f230 Prevent influx from panicking on connect 2016-10-24 16:09:06 +01:00
Joe LeGasse aa9f832166 influx_inspect: make 'go vet' happy 2016-10-21 10:52:13 -04:00
Joe LeGasse 5be37c014d influx_inspect: Update usage doc for 'export' 2016-10-21 10:35:54 -04:00
Cory LaNou 5b72b874d8
remove ProcessContinousQueries from httpd endpoint 2016-10-20 11:22:36 -05:00
Jason Wilder 6fd74a6a2a Merge pull request #7423 from mnuessler/usage-help
Fix help command usage string
2016-10-19 16:22:21 -06:00
Matthias Nüßler d2b987eac8 Fix help command usage string
The information in the usage string for the `help` command about
how to get more information about a specific command was incorrect:

"influxd help [command]" does not work, but "influxd [command] --help"
does.
2016-10-19 22:30:36 +02:00