Commit Graph

13795 Commits (adbfee28b321990d024fe1242b957aee3346666d)

Author SHA1 Message Date
Jonathan A. Sternberg d4db76508f Add some unit tests to subqueries
This is not complete, but it is a starting point for more thorough tests
of subqueries.

This also reorders the use of `cmp.Diff` so the `want` is first and
`got` is second. This way, the `want` shows up as a minus sign in the
diff rather than, confusingly, as a plus sign.
2018-03-27 14:56:27 -05:00
Jonathan A. Sternberg 4044d41e10
Merge pull request #9637 from influxdata/js-use-null-for-float-nan
Use a null placeholder for NaN results
2018-03-27 12:23:49 -05:00
Hercules Mango Churchill fd3c9c7de2 Update changelog 2018-03-27 16:19:59 +00:00
Jonathan A. Sternberg 5d9f6519ad
Merge pull request #9633 from influxdata/js-9142-group-by-offset-now
Fix regression to allow now() to be used as the group by offset again
2018-03-27 11:10:27 -05:00
Adam ca9f2510b8
check for failure case where backup dir has no manifest files (#9634) 2018-03-27 12:05:29 -04:00
Jonathan A. Sternberg 41bc1ab241 Use a null placeholder for NaN results
This ensures that NaN gets serialized as a null value and that it does
not get replaced with the fill value.
2018-03-27 08:44:44 -05:00
Jonathan A. Sternberg 92dd6ea978 Fix regression to allow now() to be used as the group by offset again 2018-03-26 10:52:32 -05:00
Stuart Carnie ef2ba80ce2 don't overwrite `_measurement` for multi-tenant reads 2018-03-23 13:59:42 -07:00
Stuart Carnie 813cb1a2f6 to var () or not to var (), that is the question 2018-03-23 12:26:55 -07:00
Stuart Carnie ee3e2ad67f rename Tenant -> OrgID 2018-03-23 12:26:55 -07:00
Stuart Carnie f69681c7c2 store command: add support for multi-tenant queries; fix validation 2018-03-23 12:26:55 -07:00
Stuart Carnie 2cc1f5137e support for tenant+bucket
NOTE: to match storage service, values for database and rp are
hard-coded to `db` and `rp` respectively
2018-03-23 12:26:55 -07:00
Jason Wilder e7e0cf0322
Merge pull request #9621 from influxdata/jw-9583
Fix flaky test TestServer_ConcurrentPointsWriter_Subscriber
2018-03-22 17:08:30 -06:00
Jason Wilder 8730be1e9f Fix flaky test TestServer_ConcurrentPointsWriter_Subscriber
The LocalServer could be closed before the goroutine writing points
exited causing nil pointer panic.

Fixes #9583
2018-03-22 14:33:55 -06:00
Jason Wilder 477de23e35
Merge pull request #9609 from influxdata/jw-compaction-filter
Add capability change compaction planner
2018-03-22 07:30:52 -06:00
Jason Wilder 0eb6564e79 Add extension point to swap out the compaction planner 2018-03-21 15:51:00 -06:00
Stuart Carnie aa61359cc7 Storage RPC API improvements. See PR for details
* reduce # allocations (115M -> 22M)
* reduce size allocations (53GB -> 1.3GB)
* reduce RPC query time (45s -> 12.9s)
2018-03-21 13:46:09 -07:00
Ben Johnson da8669f3e2
Remove error for series file when no shards exist 2018-03-21 14:41:11 -06:00
Hercules Mango Churchill 50ea3f8506 Update changelog 2018-03-21 20:36:45 +00:00
Ben Johnson 57af20ceb8
Merge pull request #9612 from influxdata/bj-revert-madvise-willneed
Revert "Use MADV_WILLNEED when loading TSM files"
2018-03-21 14:36:25 -06:00
Ben Johnson 2a8ca9a10e
Revert "Use MADV_WILLNEED when loading TSM files"
This reverts commit ee270e1dd2.
2018-03-21 13:26:45 -06:00
Hercules Mango Churchill 307aac873a Update changelog 2018-03-21 00:02:49 +00:00
Jonathan A. Sternberg ea1e9955e5
Merge pull request #9499 from influxdata/js-9424-implement-trigonometry-functions
Implement basic trigonometry functions
2018-03-20 19:02:32 -05:00
Jonathan A. Sternberg 6e627cfdbf Implement basic trigonometry functions
This adds support for math functions into the query language. Math
functions are special because they are transformations and do not access
the filesystem in the same way aggregate functions do. A transformation
takes one point and always outputs one point making it more similar to
binary expressions so these math functions follow the same rules as
binary expressions.

This also supports using math literals (so you can do `sin(1)`) and the
math functions can be used anywhere such as in a field or an expression.
Both of the following are supported:

    SELECT sin(value) FROM cpu
    SELECT value FROM cpu WHERE sin(value) > 0.5

Arguments are in radians. Degrees is not supported.
2018-03-20 14:13:52 -05:00
Jonathan A. Sternberg 49ed5f3f5b
Merge pull request #9563 from influxdata/js-9511-math-eval-iterators
Refactor the math engine to compile the query and use eval
2018-03-20 14:13:07 -05:00
Jonathan A. Sternberg f8d60a881d Refactor the math engine to compile the query and use eval
This change makes it so that we simplify the math engine so it doesn't
use a complicated set of nested iterators. That way, we have to change
math in one fewer place.

It also greatly simplifies the query engine as now we can create the
necessary iterators, join them by time, name, and tags, and then use the
cursor interface to read them and use eval to compute the result. It
makes it so the auxiliary iterators and all of their complexity can be
removed.

This also makes use of the new eval functionality that was recently
added to the influxql package.

No math functions have been added, but the scaffolding has been included
so things like trigonometry functions are just a single commit away.

This also introduces a small breaking change. Because of the call
optimization, it is now possible to use the same selector multiple times
as a selector. So if you do this:

    SELECT max(value) * 2, max(value) / 2 FROM cpu

This will now return the timestamp of the max value rather than zero
since this query is considered to have only a single selector rather
than multiple separate selectors. If any aspect of the selector is
different, such as different selector functions or different arguments,
it will consider the selectors to be aggregates like the old behavior.
2018-03-19 15:01:15 -05:00
Mark Rushakoff c720b3b40c
Merge pull request #9591 from influxdata/mr-toml-embedded-ignored
Handle embedded and ignored fields in toml env override
2018-03-19 12:01:47 -07:00
Jonathan A. Sternberg f701adce27
Merge pull request #9600 from influxdata/js-dep-update
Updated all dependencies
2018-03-19 13:40:01 -05:00
Jonathan A. Sternberg 035c537486 Updated all dependencies
Locked the prometheus client to a specific revision because we use a
revision more recent than the latest released version.
2018-03-19 13:08:58 -05:00
Jonathan A. Sternberg d89d5823bd
Merge pull request #9595 from influxdata/js-dep-update
Update dependencies and remove unnecessary constraints from Gopkg.toml
2018-03-19 12:10:40 -05:00
Jonathan A. Sternberg 196eea8cb5 Update dependencies and remove unnecessary constraints from Gopkg.toml
This also adds a small script to write out the dependencies to gdm.
These can be written with:

    $ ./write-gdm-deps.sh > Godeps

This is a temporary script during the migration.

The dependency overrides have also been removed in favor of them being
locked by Gopkg.lock.
2018-03-19 11:12:49 -05:00
Mark Rushakoff 4dc92875ef Handle embedded and ignored fields in toml env override
One of the closed source copies of applyEnvOverrides needed this
behavior.
2018-03-16 08:38:56 -07:00
Mark Rushakoff a65e51e295
Merge pull request #9588 from influxdata/mr-math-bits
Use math/bits exclusively instead of go-bits
2018-03-15 14:29:55 -07:00
Hercules Mango Churchill aeb4060cf7 Update changelog 2018-03-15 20:36:04 +00:00
Ben Johnson 5bd29a03de
Merge pull request #9586 from influxdata/bj-interrupt-show-series
Allow SHOW SERIES kill
2018-03-15 14:35:46 -06:00
Mark Rushakoff 426a9a0b8b Use math/bits exclusively instead of go-bits
We won't be rolling back to pre-Go1.9, so prefer the standard library
over a dependency that provides backwards compatibility.
2018-03-15 12:03:24 -07:00
Hercules Mango Churchill 7b1c0e75d3 Update changelog 2018-03-15 18:00:18 +00:00
Edd Robinson 8ab8b5a428
Merge pull request #9580 from influxdata/er-wal-race
Fix data race in WAL
2018-03-15 17:59:55 +00:00
Ben Johnson f6fdba2590
Allow SHOW SERIES kill. 2018-03-15 11:22:34 -06:00
Stuart Carnie 6cf6ae7af4 Use combined IndexSet when executing meta queries
* removed unused fieldset field
2018-03-15 09:59:11 -07:00
Edd Robinson 0fc7643d59 Fix data race in WAL
This commit fixes a data race in the WAL, which can occur when writes
and deletes are being executed concurrently. The WAL uses a buffer pool
of `[]byte` when reading the WAL. WAL entries are unmarshaled into these
buffers and passed along to the relevant methods handling the different
types of entry (write, delete etc).

In the case of deletes, the keys that need to be deleted were being
stored for later processing, however these keys were part of the backing
array of initial buffer from the pool. As such, those keys could be
written to at a future time when handling other parts of the WAL.
2018-03-15 12:51:30 +00:00
Mark Rushakoff 8116cded0c
Merge pull request #9376 from influxdata/mr-toml-env
Move ApplyEnvOverrides to toml package
2018-03-14 21:19:45 -07:00
Mark Rushakoff fbddcf7cad Move ApplyEnvOverrides to toml package
This code has been duplicated to other projects and its implementations
have grown out of sync. Now the code can live as a package-level
function rather than a method coupled with particular structs.
2018-03-14 15:23:46 -07:00
Jonathan A. Sternberg c9e7c5ad2b
Merge pull request #9568 from influxdata/js-update-influxql
Update influxql to include the function type evaluators in the query package
2018-03-14 16:03:28 -05:00
Jonathan A. Sternberg c8b0c6e166 Update influxql to include the function type evaluators in the query package 2018-03-14 15:42:28 -05:00
Hercules Mango Churchill 9213546bb2 Update changelog 2018-03-14 17:12:00 +00:00
Edd Robinson 3e7c80dd66
Merge pull request #9573 from influxdata/er-tag-parsing
Ensure correct number of tags parsed when commas used
2018-03-14 17:06:48 +00:00
Adam b213b31cd4
Bugfix for shard collisions resulting from attempted _internal import (#9569) 2018-03-14 11:51:57 -04:00
Jonathan A. Sternberg 6820dd8d13
Merge pull request #9406 from influxdata/js-9404-use-dep-for-dependencies
Use dep for dependency management instead of gdm
2018-03-14 09:34:48 -05:00
Edd Robinson 6b350edf59 Ensure correct number of tags parsed
This commit fixes a parsing bug that was causing extra tags to be
generated when the incoming point contained escaped commas.

As an optimisation, the slice of tags associated with a point was
being pre-allocated using the number of commas in the series key as a hint to
the appropriate size. The hinting did not consider literal comma values in
the key though, and so it was possible for extra (empty) tag key and
value pairs to be part of the tags structure associated with a parsed
point.
2018-03-14 13:28:55 +00:00