Commit Graph

11891 Commits (eeaad877bce4cc810d2feb76fa8b310f868e94ca)

Author SHA1 Message Date
Jason Wilder 517df0f939 Make integration tests able to run against remote server
URL=http://localhost:8086 go test -parallel 1 ./cmd/influxd/run

will run the tests over HTTP against localhost:8086.  They currently
need to be run serially since they all write to the same DB.
2017-02-15 10:07:40 -07:00
Ben Johnson 7e6f33ef48
TSI1 Index file compaction. 2017-02-15 08:44:22 -07:00
Ben Johnson 71d13fed6d
Immediate, parallel tsl compactions. 2017-02-15 08:44:21 -07:00
Ben Johnson 673143a0ad
Remove .tsl file. 2017-02-15 08:44:01 -07:00
Jonathan A. Sternberg 0e0057d1da Merge pull request #8002 from influxdata/js-8001-map-types-with-empty-measurement
Map types correctly when using a regex and one of the measurements is empty
2017-02-13 18:24:30 -06:00
Jonathan A. Sternberg 71f62d33e6 Map types correctly when using a regex and one of the measurements is empty 2017-02-13 18:14:29 -06:00
Mark Rushakoff c762ab49ee Merge pull request #7974 from influxdata/mr-4785-show-databases
Allow non-admin users to execute SHOW DATABASES
2017-02-13 15:04:00 -08:00
Jason Wilder f45a58937c Merge pull request #7998 from influxdata/jw-merge-12
Merge 1.2.1-rc3 to master
2017-02-13 14:24:17 -07:00
Jonathan A. Sternberg 9636960ce6 Merge pull request #7990 from influxdata/js-modulo-operator-fix
Fixed String() output for MOD operator and added MOD op precedence
2017-02-13 14:46:12 -06:00
Jason Wilder c3de210ded Merge branch '1.2' into jw-merge-12 2017-02-13 11:45:27 -07:00
Edd Robinson 51afdd22ab Merge pull request #7997 from influxdata/er-logfile-series
Store series in sorted order in LogFile
2017-02-13 18:17:02 +00:00
Edd Robinson 7749bc1c67 Store sorted series in logFile 2017-02-13 17:03:42 +00:00
Mark Rushakoff 53699aa24f Allow non-admin users to execute SHOW DATABASES
This commit introduces a new interface type, influxql.Authorizer, that
is passed as part of a statement's execution context and determines
whether the context is permitted to access a given database. In the
future, the Authorizer interface may be expanded to other resources
besides databases. In this commit, the Authorizer interface is
specifically used to determine which databases are returned when
executing SHOW DATABASES.

When HTTP authentication is enabled, the existing meta.UserInfo struct
implements Authorizer, meaning admin users can SHOW every database, and
non-admin users can SHOW only databases for which they have read and/or
write permission.

When HTTP authentication is disabled, all databases are visible through
SHOW DATABASES.

This addresses a long-standing issue where Chronograf or Grafana would
be unable to list databases if the logged-in user did not have admin
privileges.

Fixes #4785.
2017-02-13 08:59:16 -08:00
Jonathan A. Sternberg 19f331a450 Merge pull request #7989 from jgeiger/master
Add chunked processing back into v2 client
2017-02-13 10:54:08 -06:00
jgeiger 43117a94d6 Add chunked processing back into v2 client
- Moving the to v2 client removed this functionality. This copies
  code back into the client. The tests were also added back into
  the test suite.
2017-02-13 09:21:13 -07:00
Jonathan A. Sternberg 55e64e1edd Fixed String() output for MOD operator and added MOD op precedence 2017-02-10 16:48:05 -06:00
Jason Wilder 8d0f2c3ca9 Merge pull request #7983 from influxdata/jw-parallel-buffers
Increase parallel iterator buffers to improve group by query speed
2017-02-10 11:22:59 -07:00
Jason Wilder 5e42ac411a Increase buffer to improve group by query speed 2017-02-10 11:07:49 -07:00
Jonathan A. Sternberg 5395bd3bcf Merge pull request #7765 from influxdata/js-7553-modulo-operator
Add modulo operator to the query language
2017-02-10 11:46:24 -06:00
Jason Wilder 4b6289ce58 Merge pull request #7942 from influxdata/jw-cache-partitions
Reduce write timeouts
2017-02-10 10:07:08 -07:00
Jonathan A. Sternberg a0d8c1ca9f Add modulo operator to the query language 2017-02-10 10:16:37 -06:00
Edd Robinson c94858e53b Merge pull request #7982 from influxdata/er-meta-load
Don't load meta data when using TSI
2017-02-10 15:15:54 +00:00
Jason Wilder c88b1c3a96 Merge pull request #7981 from influxdata/jw-simple8b-update
Use simple8b.CountBytes to avoid allocations
2017-02-09 11:05:08 -07:00
Edd Robinson 38eb6d5994 Don't load meta data for tsi 2017-02-09 18:04:23 +00:00
Edd Robinson a6a2f9d5f0 Don't load meta data for tsi 2017-02-09 17:59:14 +00:00
Jason Wilder 2f74e3f3d5 Use simple8b.CountBytes to avoid allocations 2017-02-09 10:47:03 -07:00
Edd Robinson 1aebb6d58e Merge pull request #7976 from influxdata/er-tsi
Reduce allocations
2017-02-09 16:42:22 +00:00
Jason Wilder 0d9fd8a37b Merge pull request #7948 from CAFxX/gzip_encoder_pool
[influxd] Use sync.Pool to reuse gzip.Writers across requests
2017-02-09 09:04:24 -07:00
Edd Robinson 2d59fb788c Modify arguments to reduce allocations 2017-02-09 15:01:32 +00:00
Edd Robinson c2bbc18e4b Reduce repeated string allocations 2017-02-09 11:23:23 +00:00
Carlo Alberto Ferraris 005e480b55 [influxd] add utility functions to make it harder to misuse gzipWriterPool 2017-02-09 09:31:11 +09:00
Jonathan A. Sternberg afbfa0086f Merge pull request #7966 from influxdata/js-7898-prevent-panic-when-aggregate-returns-no-aux-fields
Prevent a panic when aggregates are used in an inner query with a raw query
2017-02-08 12:49:14 -06:00
Edd Robinson 21e821e5f9 Reduce memory footprint by pre-allocating 2017-02-08 18:49:03 +00:00
Edd Robinson 4fbba8234e Add Size to models.Tags 2017-02-08 18:44:48 +00:00
Jonathan A. Sternberg 2ad1668c2a Prevent a panic when aggregates are used in an inner query with a raw query
The following types of queries will panic:

    SELECT mean, host FROM (SELECT mean(value) FROM cpu GROUP BY host)
    SELECT top(sum, host, 3) FROM (SELECT sum(value) FROM cpu GROUP BY host)

These queries _should_ work, but due to a current limitation with
aggregate functions, the aggregate functions won't return any auxiliary
fields. So even if a tag is not an auxiliary field, it is treated that
way by the query engine and this query will fail.

Fixing this properly will take a longer period of time. This fix just
prevents the panic from killing the server while we fix this for real.
2017-02-08 11:44:56 -06:00
Edd Robinson a9d58cc879 Add log file benchmarks 2017-02-08 16:50:46 +00:00
Ben Johnson d6fedcacd5
Refactor tsi.LogFile compaction. 2017-02-08 09:00:08 -07:00
Jason Wilder 8c25f0104e Merge pull request #7958 from influxdata/jw-merge-12
Merge 1.2.1-rc2 to master
2017-02-07 13:29:00 -07:00
Jason Wilder 1bc0f68490 Merge branch '1.2' into jw-merge-12 2017-02-07 12:48:36 -07:00
Ben Johnson c4e9430de3
Recover after partial tsi log file write. 2017-02-07 10:02:31 -07:00
Jonathan A. Sternberg b3930f2a3f Merge pull request #7907 from influxdata/js-7905-order-by-desc-fix
Fix ORDER BY time DESC with ordering series keys
2017-02-06 15:57:35 -06:00
Jonathan A. Sternberg e1fa48d0dd Fix ORDER BY time DESC with ordering series keys
The order of series keys is in ascending alphabetical order, not
descending alphabetical order, when it is ordered by descending time.
This fixes the ordering so points are returned in descending order. The
emitter also had the conditions for choosing which iterator to use in
the wrong direction (which only affects aggregates with `FILL(none)`).
2017-02-06 15:49:12 -06:00
Jonathan A. Sternberg 11a49db662 Merge pull request #7954 from influxdata/js-7885-fix-limit-and-offset-in-subqueries
Fix LIMIT and OFFSET when they are used in a subquery
2017-02-06 15:46:49 -06:00
Carlo Alberto Ferraris a6a7782e04 [influxd] Use a sync.Pool to reuse gzip.Writer across requests
This brings alloc_space down from ~20200M to ~10700M in a run of
go test ./cmd/influxd/run -bench=Server -memprofile=mem.out -run='^$'
2017-02-07 05:23:58 +09:00
Jonathan A. Sternberg 95831b3307 Fix LIMIT and OFFSET when they are used in a subquery
This fixes LIMIT and OFFSET when they are used in a subquery where the
grouping of the inner query is different than the grouping of the outer
query. When organizing tag sets, the grouping of the outer query is
used so the final result is in the correct order. But, unfortunately,
the optimization incorrectly limited the number of points based on the
grouping in the outer query rather than the grouping in the inner query.

The ideal solution would be to use the outer grouping to further
organize it by the grouping for the inner subquery, but that's more
difficult to do at the moment. As an easier fix, the query engine now
limits the output of each series. This may result in these types of
queries being slower in some situations like this one:

    SELECT mean(value) FROM (SELECT value FROM cpu GROUP BY host LIMIT 1)

This will be slower in a situation where the `cpu` measurement has a
high cardinality and many different tags.

This also fixes `last()` and `first()` when they are used in a subquery
because those functions use `LIMIT 1` as an internal optimization.
2017-02-06 14:04:34 -06:00
Jonathan A. Sternberg 4d97dd495d Merge pull request #7952 from influxdata/js-7946-subquery-authorization
Fix authentication when subqueries are present
2017-02-06 14:03:50 -06:00
Jason Wilder 93a9d01643 Increase default waiting WAL writes 2017-02-06 11:48:51 -07:00
Jason Wilder 38a649fc40 Batch multiple WAL fsyncs
Every write to the WAL current runs and fsync before returning.  When
there are lot of concurrent writes, this can cause the WAL to bottleneck
write throughput since fsyncs are very expensive.

This changes the writeToLog to fsync on an interval to allow multiple fsyncs
calls to be batched up into one.  The writeToLog behavior is the same in that
it won't return until an fsync has been performed.
2017-02-06 11:48:45 -07:00
Ben Johnson d91e6eabac
Add max-values-per-tag to inmem index. 2017-02-06 11:14:13 -07:00
Jonathan A. Sternberg caaad60dcf Fix authentication when subqueries are present
The code that checked if a query was authorized did not account for
sources that were subqueries. Now, the check for the required privileges
will descend into the subquery and add the subqueries required
privileges to the list of required privileges for the entire query.
2017-02-06 09:43:14 -06:00