Commit Graph

12534 Commits (9fecaa0eb07549391f9fa2ba7fe8a308912eb89e)

Author SHA1 Message Date
Nathaniel Cook 9fecaa0eb0 chore: Update for Flux renaming 2018-11-05 19:30:22 -07:00
Mark Rushakoff 188db01b19 fix(task): consume table iterators when executing query
Leaving tables unconsumed seems to cause a variety of problems,
including deadlocks and queries that don't actually execute.
2018-11-05 14:10:37 -08:00
Christopher M. Wolff e95fb9cc9a
Fix error where _start or _stop where redundantly added to group key (#1308) 2018-11-05 14:06:08 -08:00
Stuart Carnie a1df0f2f17
Merge pull request #1285 from influxdata/sgc-deps
Update flux dependency with distinct push down support
2018-11-05 07:36:45 -07:00
Stuart Carnie 04733339cc
chor(deps): Update flux dependency with distinct push down support 2018-11-03 13:50:11 -07:00
Jeff Wendling 6549f24b16
Merge pull request #1286 from influxdata/jmw-backport-tsm-index-fix
Fix TSM index maxtime modification
2018-11-02 18:49:53 -06:00
Edd Robinson 1857bf1084 Fix TSM index maxtime modification 2018-11-02 18:39:30 -06:00
Mark Rushakoff d8dedc5644 ci: use -p=8 in GOFLAGS to reduce linker OOMs
The environment Go runs in thinks there are 32 CPUs, but according to
https://circleci.com/docs/2.0/configuration-reference/#resource_class
that's a known issue, and only 2 CPUs are actually available.

We're still overallocating with -p=8, so if we continue to get OOMs we
may need to reduce that further.
2018-11-02 17:21:33 -07:00
Michael Desa cff0634001
Merge pull request #1195 from influxdata/feat/bolt-cli
Add local flag to influx command
2018-11-02 14:49:28 -04:00
Michael Desa 459f2cbfbb feat(cmd/influx): add local flag for operating directly with boltdb
feat(internal/fs): move influxDir function to internal package in base of repo

feat(cmd/influx): add local flag that operates on local file system

feat(cmd/influx): add local support for org command

feat(cmd/influx): add local support for user command

eat(cmd/influx): add local support for bucket command

feat(cmd/influx): add error for local with query/task/setup cmds

fix(cmd/influx): unshadow error in delete bucket command

fix(cmd/influx): update copy for local flag commands
2018-11-02 14:34:48 -04:00
Michael Desa c70c884fb9
Merge pull request #1258 from influxdata/feat/operation-log
WIP: add operation log for resources
2018-11-02 14:34:12 -04:00
Chris Goller 9d507a220a
Merge pull request #1280 from influxdata/feature/query-plan
refactor(http): preallocate query plans
2018-11-02 13:25:42 -05:00
Michael Desa 7227bf8fa2 feat(platform): add operation log for resources
feat(platform): add dashboard operation log interface

feat(bolt): add dashboard operation log to bolt client

feat(platform): add links to dashboard load response

fix(http): update links in json response of dashboards

feat(bolt): add operation logs for each resource

feat(http): add routes for operation logs

feat(cmd/influxd): set operation logs on http api backend
2018-11-02 14:21:14 -04:00
Adam 612005cad1
add output to to function (#1282)
* added (unused) stats struct, and piped written data to output table

* updated to tests since to returns data now

* changes based on review
2018-11-02 13:40:25 -04:00
kelwang 3fed14dabc
Merge pull request #943 from influxdata/feat/inmem_onboarding_basic_auth
add inmem service for basic auth, http onboarding testing
2018-11-02 13:22:00 -04:00
Chris Goller 84af710e42 refactor(http): preallocate query plans 2018-11-02 12:03:45 -05:00
Kelvin Wang 63da5d1e9f (feat/testing) add onboarding and basic auth service 2018-11-02 12:20:45 -04:00
kelwang 3dec916178
Merge pull request #1277 from influxdata/fix/400_for_unknown_error_code
fix the error code for 400
2018-11-02 12:16:17 -04:00
Kelvin Wang cb4e095463 fix the error code for 400 2018-11-02 11:28:39 -04:00
Chris Goller 3e54ef9f53
Merge pull request #1275 from influxdata/feature/query-plan
feat(http): add plan endpoint to analyze queries
2018-11-02 09:25:51 -05:00
Chris Goller 0d7b77d3dd feat(http): add query plan validation 2018-11-02 09:11:57 -05:00
kelwang 8f52f9e1dc
Merge pull request #1273 from influxdata/fix/http_error
(fix/http): make error compatible with new and old lib
2018-11-02 10:03:21 -04:00
Chris Goller 7ca5187dc6 feat(http): add plan endpoint to analyze queries 2018-11-01 21:00:37 -05:00
kelwang d022a26250
Merge pull request #1252 from influxdata/link_reset_user_password
fix user response
2018-11-01 21:53:15 -04:00
Kelvin Wang 5378fe877a (fix/http): make error compatible with new and old lib 2018-11-01 20:28:40 -04:00
Kelvin Wang 5b7b123d1e fix user response 2018-11-01 20:12:04 -04:00
Jorge Landivar 34ab06634f
Merge pull request #1266 from influxdata/opentracing
open tracing context handling for tasks and for HTTP endpoints for tasks and query
2018-11-01 16:33:19 -05:00
docmerlin (j. Emrys Landivar) bcbd7b5c24 context handling for tasks and for HTTP endpoints for tasks and query 2018-11-01 16:29:52 -05:00
Stuart Carnie 56510ba0dc
Merge pull request #1262 from influxdata/sgc-fix-race
Fix race condition as `Close` was called by multiple goroutines
2018-11-01 14:22:32 -07:00
Nathaniel Cook 95b2bfb15b Merged pull request #1269 from influxdata/nc-flux-0.7.0-rc1
chore: Update to Flux 0.7.0-rc1
2018-11-01 14:57:32 -06:00
Nathaniel Cook 89e6c6adbd chore: Update to Flux 0.7.0-rc1 2018-11-01 14:32:56 -06:00
Mark Rushakoff 1d42012c8d
Merge pull request #1267 from influxdata/chore/megacheck
Re-enable megacheck
2018-11-01 13:29:04 -07:00
Mark Rushakoff 68c0ad29e2 ci: add megacheck to tools.go 2018-11-01 13:25:22 -07:00
Mark Rushakoff b37279c50e ci: remove 2/3 disabled exit-non-zero flags for megacheck
The one remaining failing static check is in a pigeon-generated promql
file. It looks like megacheck is supposed to ignore generated files by
default, so it's unclear at this time if this is a bug in megacheck, or
if fixing pigeon is the only path forward.
2018-11-01 12:54:46 -07:00
Mark Rushakoff ff2998dcf8 chore(chronograf): fix megacheck errors 2018-11-01 12:54:46 -07:00
Mark Rushakoff dae3dc8a7e chore(bolt,http,mock,query,testing): fix megacheck errors 2018-11-01 12:54:46 -07:00
Mark Rushakoff 4c129e0044 chore(context,gather,inmem,kit): fix megacheck errors 2018-11-01 12:54:46 -07:00
Mark Rushakoff 89813b8aa4 chore(cmd): fix megacheck errors 2018-11-01 12:54:46 -07:00
Mark Rushakoff 985c260af7 chore(storage,tsdb): fix megacheck errors 2018-11-01 12:54:46 -07:00
Mark Rushakoff fe84ff7357 chore(task): fix megacheck errors 2018-11-01 12:46:33 -07:00
Mark Rushakoff 9acc251ab0 ci: re-enable megacheck 2018-11-01 12:46:33 -07:00
Edd Robinson f998c3369d
Merge pull request #1104 from influxdata/er-batch-decoders
Batch oriented encoders
2018-11-01 19:20:12 +00:00
Stuart Carnie a0300064df feat(tsm1): Improve performance of Gorilla float block decoding
```
name                        old time/op   new time/op    delta
FloatArrayDecodeAll/1-8      45.9ns ± 1%    13.8ns ± 1%   -70.00%  (p=0.000 n=9+9)
FloatArrayDecodeAll/55-8      686ns ± 0%     232ns ± 1%   -66.10%  (p=0.000 n=9+8)
FloatArrayDecodeAll/550-8    5.78µs ± 0%    2.22µs ± 1%   -61.61%  (p=0.000 n=9+9)
FloatArrayDecodeAll/1000-8   10.2µs ± 2%     4.0µs ± 5%   -60.47%  (p=0.000 n=10+10)

name                        old speed     new speed      delta
FloatArrayDecodeAll/1-8     414MB/s ± 1%  1383MB/s ± 1%  +233.76%  (p=0.000 n=9+9)
FloatArrayDecodeAll/55-8    144MB/s ± 0%   424MB/s ± 1%  +194.19%  (p=0.000 n=9+9)
FloatArrayDecodeAll/550-8   133MB/s ± 0%   346MB/s ± 1%  +160.09%  (p=0.000 n=9+10)
FloatArrayDecodeAll/1000-8  135MB/s ± 2%   340MB/s ± 5%  +153.03%  (p=0.000 n=10+10)
```
2018-11-01 18:59:20 +00:00
Edd Robinson 353df7edca Fix imports 2018-11-01 18:59:20 +00:00
Edd Robinson e282d012c8 Address PR feedback 2018-11-01 18:59:20 +00:00
Stuart Carnie c21336af0a fix(encoding): Improve array string encoding perf a little more
Encode the compressed data at the start internal buffer. This ensures
the returned slice maintains the entire capacity and is available for
subsequent use.

When we pool / reuse string buffers, this will help considerably.

Improvements over previous commit:

```
name                        old time/op    new time/op    delta
EncodeStrings/10/batch-8       542ns ± 1%     355ns ± 2%   -34.53%  (p=0.008 n=5+5)
EncodeStrings/100/batch-8     5.29µs ± 1%    3.58µs ± 2%   -32.20%  (p=0.008 n=5+5)
EncodeStrings/1000/batch-8    48.6µs ± 0%    36.2µs ± 2%   -25.40%  (p=0.008 n=5+5)

name                        old alloc/op   new alloc/op   delta
EncodeStrings/10/batch-8        704B ± 0%        0B       -100.00%  (p=0.008 n=5+5)
EncodeStrings/100/batch-8     9.47kB ± 0%    0.00kB       -100.00%  (p=0.008 n=5+5)
EncodeStrings/1000/batch-8    90.1kB ± 0%     0.0kB       -100.00%  (p=0.008 n=5+5)

name                        old allocs/op  new allocs/op  delta
EncodeStrings/10/batch-8        0.00           0.00           ~     (all equal)
EncodeStrings/100/batch-8       1.00 ± 0%      0.00       -100.00%  (p=0.008 n=5+5)
EncodeStrings/1000/batch-8      1.00 ± 0%      0.00       -100.00%  (p=0.008 n=5+5)
```
2018-11-01 18:59:20 +00:00
Stuart Carnie 296d39059a fix(encoding): Improve simple8b another 6%; fix inconsequential bug
simple8b encodes deltas[1:], thus deltas[0] >= simple8b.MaxValue is
invalid.

Also changed loop calculating deltas, RLE and max to be similar to
batch timestamp, for greater consistency.

Improvements over previous commit:

```
name                             old time/op    new time/op    delta
name                             old time/op    new time/op    delta
EncodeIntegers/1000_seq/batch-8    1.50µs ± 1%    1.48µs ± 1%  -1.40%  (p=0.008 n=5+5)
EncodeIntegers/1000_ran/batch-8    6.10µs ± 0%    5.69µs ± 2%  -6.58%  (p=0.008 n=5+5)
EncodeIntegers/1000_dup/batch-8    1.50µs ± 1%    1.49µs ± 0%  -1.21%  (p=0.008 n=5+5)
```

Improvements overall:

```
name                             old time/op    new time/op    delta
EncodeIntegers/1000_seq/batch-8    2.04µs ± 0%    1.48µs ± 1%  -27.25%  (p=0.008 n=5+5)
EncodeIntegers/1000_ran/batch-8    8.80µs ± 2%    5.69µs ± 2%  -35.29%  (p=0.008 n=5+5)
EncodeIntegers/1000_dup/batch-8    2.03µs ± 1%    1.49µs ± 0%  -26.93%  (p=0.008 n=5+5)
```
2018-11-01 18:59:20 +00:00
Stuart Carnie 9fa01f7115 feat(encoding): Improve timestamp encoding
Timestamp improvements prior to any improvements to simple8b

```
name                               old time/op    new time/op    delta
name                               old time/op    new time/op    delta
EncodeTimestamps/1000_seq/batch-8    2.64µs ± 1%    1.36µs ± 1%  -48.25%  (p=0.008 n=5+5)
EncodeTimestamps/1000_ran/batch-8    64.0µs ± 1%    32.2µs ± 1%  -49.64%  (p=0.008 n=5+5)
EncodeTimestamps/1000_dup/batch-8    9.32µs ± 0%    1.30µs ± 1%  -86.06%  (p=0.008 n=5+5)
```
2018-11-01 18:59:20 +00:00
Stuart Carnie a339f8f620 feat(encoding): Improve integer and simple8b encoding performance
simple8b EncodeAll improvements should

```
name                     old time/op  new time/op  delta
EncodeAll/1_bit-8        28.5µs ± 1%  28.6µs ± 1%     ~     (p=0.133 n=9+10)
EncodeAll/2_bits-8       28.9µs ± 2%  28.7µs ± 0%     ~     (p=0.068 n=10+8)
EncodeAll/3_bits-8       29.3µs ± 1%  28.8µs ± 0%   -1.70%  (p=0.000 n=10+10)
EncodeAll/4_bits-8       29.6µs ± 1%  29.1µs ± 1%   -1.85%  (p=0.000 n=10+10)
EncodeAll/5_bits-8       30.6µs ± 1%  29.8µs ± 2%   -2.70%  (p=0.000 n=10+10)
EncodeAll/6_bits-8       31.3µs ± 1%  30.0µs ± 1%   -4.08%  (p=0.000 n=9+9)
EncodeAll/7_bits-8       32.6µs ± 1%  30.8µs ± 0%   -5.49%  (p=0.000 n=9+9)
EncodeAll/8_bits-8       33.6µs ± 2%  31.0µs ± 1%   -7.77%  (p=0.000 n=10+9)
EncodeAll/10_bits-8      34.9µs ± 0%  31.9µs ± 2%   -8.55%  (p=0.000 n=9+10)
EncodeAll/12_bits-8      36.8µs ± 1%  32.6µs ± 1%  -11.35%  (p=0.000 n=9+10)
EncodeAll/15_bits-8      39.8µs ± 1%  34.1µs ± 2%  -14.40%  (p=0.000 n=10+10)
EncodeAll/20_bits-8      45.2µs ± 3%  36.2µs ± 1%  -19.97%  (p=0.000 n=10+9)
EncodeAll/30_bits-8      55.0µs ± 0%  40.9µs ± 1%  -25.62%  (p=0.000 n=9+9)
EncodeAll/60_bits-8      86.2µs ± 1%  55.2µs ± 1%  -35.92%  (p=0.000 n=10+10)
EncodeAll/combination-8   582µs ± 2%   502µs ± 1%  -13.80%  (p=0.000 n=9+9)
```

EncodeIntegers:

```
name                             old time/op    new time/op    delta
EncodeIntegers/1000_seq/batch-8    2.04µs ± 0%    1.50µs ± 1%  -26.22%  (p=0.008 n=5+5)
EncodeIntegers/1000_ran/batch-8    8.80µs ± 2%    6.10µs ± 0%  -30.73%  (p=0.008 n=5+5)
EncodeIntegers/1000_dup/batch-8    2.03µs ± 1%    1.50µs ± 1%  -26.04%  (p=0.008 n=5+5)
```

EncodeTimestamps (ran is improved due to simple8b improvements)

```
name                               old time/op    new time/op    delta
EncodeTimestamps/1000_seq/batch-8    2.64µs ± 1%    2.65µs ± 2%     ~     (p=0.310 n=5+5)
EncodeTimestamps/1000_ran/batch-8    64.0µs ± 1%    33.8µs ± 1%  -47.23%  (p=0.008 n=5+5)
EncodeTimestamps/1000_dup/batch-8    9.32µs ± 0%    9.28µs ± 1%     ~     (p=0.087 n=5+5)
```
2018-11-01 18:59:20 +00:00
Stuart Carnie dd83bd230f chore(simple8b): Add more unit tests and benchmarks 2018-11-01 18:59:20 +00:00