Commit Graph

1452 Commits (afb56c33c182f78c6b3555f105249fc8bfdae3a9)

Author SHA1 Message Date
Jonathan A. Sternberg a05e2b164e Support booleans for min() and max()
Fixes #6494.
2016-04-29 14:56:22 -04:00
Jonathan A. Sternberg 043bd6031e Return the time with a selector when there is no group by interval
The time of the point will be returned with a selector when there is no
group by interval and when there is only one selector. Any other
conditions will return the start time of the interval.

Fixes #5890.
2016-04-29 11:27:26 -04:00
Jonathan A. Sternberg 6f61c0ea4a Add POST /query endpoint and warning messages for using GET with write operations
In order to follow REST a bit more carefully, all write operations
should go through a POST in the future. We still allow read operations
through either GET or POST (similar to the Graphite /render endpoint),
but write operations will trigger a returned warning as part of the JSON
response and will eventually return an error.

Also updates the Golang client libraries to always use POST instead of
GET.

Fixes #6290.
2016-04-29 09:00:23 -04:00
David Norton 4df6483db9 update CHANGELOG.md 2016-04-28 23:00:31 -04:00
lvheyang 751a81f414 fix show statement's rewriting bug 2016-04-29 08:23:50 +08:00
Jason Wilder 2bd5880d7a Remove series from index when shard is closed
When a shard is closed and removed due to retention policy enforcement,
the series contained in the shard would still exists in the index causing
a memory leak.  Restarting the server would cause them not to be loaded.

Fixes #6457
2016-04-28 12:34:46 -06:00
Todd Persen 337d49bea3 Merge pull request #5720 from Vidhuran/bug4987-query-permalinks
added a button to generate URL for queries created in admin UI.
2016-04-28 10:34:06 -07:00
Jonathan A. Sternberg 2ccbe26447 Fix the CLI not to enter an infinite loop when the liner has an error
This also removes the dependency on `os/user` and uses the `HOME`
environment variable which is more common on Linux and Mac OS X for
customizing the history file location.

Removing this import also lets the `influx` binary be cross-compiled as
`os/user` relies on cgo.
2016-04-28 08:35:54 -04:00
Ben Johnson f7af787aef
add DELETE query support
This commit adds query language support for deleting series with a
`DELETE` query.
2016-04-27 15:16:23 -06:00
Jason Wilder e7891cbe4a Update changelog 2016-04-27 13:09:53 -06:00
Edd Robinson 289fe5bfcb Don't catch SIGQUIT; safer close of Quit channel 2016-04-27 14:38:48 +01:00
Vidhuran Harichandra Babu fedb7bfdbf added a button in admin ui to genearate URL for the query 2016-04-27 11:30:47 +03:00
Ben Johnson 291dd08cd4
remove SHOW SERVERS & DROP SERVER
This commit removes support for `SHOW SERVERS` and `DROP SERVER`
from the `influxql` package. It also removes extraneous cluster
testing code from `cmd/influxd/run`.

Fixes #6465
2016-04-26 10:28:42 -06:00
Pierre Fersing 9f9a3fc98e Fix cluster/pool release of connection 2016-04-25 20:06:51 +02:00
Ben Johnson f608e4cdb0 Merge pull request #6462 from oldmantaiter/measurement-fields-deadlock
Add safer locking to CreateFieldIfNotExists
2016-04-25 11:07:33 -06:00
Tait Clarridge df0e16a92f Add safer unlock to CreateFieldIfNotExists
A deadlock can occur if the field was created while we were waiting for the lock.
2016-04-25 12:44:58 -04:00
Kostas Botsas c39ff515ca Merge pull request #6450 from influxdata/changelog-update-0122
Changelog update for 0.12.2
2016-04-25 19:18:36 +03:00
Kostas Botsas f8885a49ee removed duplicates from v0.13
removed duplicate GH PR's from v0.13 which were merged in v0.12
2016-04-25 15:48:05 +03:00
Philip O Toole ed7229fa2d Make it clear when CLI version is unknown
This mimics the behavior of the database binary.
2016-04-22 21:35:42 -07:00
Jonathan A. Sternberg 9b6d5dfd78 Allow derivative() function to be used with ORDER BY desc
The derivative function had an arbitrary limitation that would cause it
to set the value to zero if the previous value was after the next value.
This caused all `ORDER BY desc` queries with `derivative()` to always
return zero values.

Fixes #4675.
2016-04-22 12:20:32 -04:00
Jonathan A. Sternberg e28d16cfcf Log slow queries if they pass a configurable threshold
Fixes #6429.
2016-04-22 12:04:15 -04:00
Jonathan A. Sternberg b6beec4f57 Merge pull request #6411 from influxdata/js-3558-allow-math-in-where-clause
Pass binary expressions to the underlying query
2016-04-22 12:03:27 -04:00
Jonathan A. Sternberg d55fb1b16f Merge pull request #6421 from influxdata/js-3883-sanitize-query
Improve query sanitization to prevent a password leak in the logs
2016-04-22 11:40:17 -04:00
Jonathan A. Sternberg d26e4e3650 Pass binary expressions to the underlying query
Binary math inside of a where condition was previously disallowed. Now,
these types of queries are just passed verbatim down to the underlying
query engine which can handle it.

We may want to revisit this when it comes to tags at some point as it
prevents the more efficient filtering of tags that a simple expression
allows, but it allows a query like this to be done:

    SELECT * FROM cpu WHERE value + 2 < 5

So while it can be better, this is a good initial implementation to
provide this functionality. There are very rare situations where a tag
may be used appropriately in one of these circumstances.

Fixes #3558.
2016-04-22 11:30:36 -04:00
Jonathan A. Sternberg 5070f69bb2 Merge pull request #6444 from influxdata/js-config-from-env-var
Allow setting the config path through an environment variable
2016-04-22 11:28:57 -04:00
Jonathan A. Sternberg 62c66b788c Improve query sanitization to prevent a password leak in the logs
Sanitizing is now done through pattern matching rather than parsing the
query and replacing the password in the query. This prevents
accidentally redacting the wrong part of a query and revealing what the
password is through association.

Fixes #3883.
2016-04-22 11:27:09 -04:00
Jonathan A. Sternberg 9c09023508 Allow setting the config path through an environment variable
The config path previously could only be specified through the command
line options. This made it very difficult to set a default config path
without using any option.

Now the environment variable can be set so the default configuration
path is set to a specific place, but can be overwritten using the
command line option.

The primary purpose of this is so the Docker container can have a
default configuration file, but not have to parse the command line
options to figure out if a different configuration file has been
specified while still allowing the user to only type `influxd` and have
the program start correctly.

This might also help #6392 as it would allow a default configuration
location to be included with the package by setting an environment
variable.

A default search path is also provided now with checking the following
paths for a config file when none is specified:

* `~/.influxdb/influxdb.conf`
* `/etc/influxdb/influxdb.conf`

The config command has also been modified to read this config file
before outputting a sample config.
2016-04-22 09:33:53 -04:00
Kostas Botsas d56ec7ef8a Changelog for 0.12.2
Updated Changelog with details for 0.12.2
2016-04-22 14:21:00 +03:00
Jason Wilder caa9592163 Fix setting uint config options via env vars 2016-04-20 14:38:08 -06:00
Jason Wilder bfa225f149 Merge pull request #6430 from influxdata/jw-cache-load-size
Disable cache max memory size when reloading the cache
2016-04-20 14:35:23 -06:00
Stephen Gutekanst 9dc09c5257 Make logging output location more programmatically configurable (#6213)
This has various benefits:

- Users embedding InfluxDB within other Go programs can specify a different logger / prefix easily.
- More consistent with code used elsewhere in InfluxDB (e.g. services, other `run.Server.*` fields, etc).
- This is also more efficient, because it means `executeQuery` no longer allocates a single `*log.Logger` each time it is called.
2016-04-20 21:07:08 +01:00
Jason Wilder f679787080 Disable cache max memory size when reloading the cache
The cache max memory size is an approximate size and can prevent a
shard from loading at startup.  This change disable the max size
at startup to prevent this problem and sets the limt back after
reloading.

Fixes #6109
2016-04-20 10:41:30 -06:00
thbourlove 0e3964d8c2 fix panic in transform iterator on division
if left point is nil in division expr, the influxd server would panic. Now check left point before use it.
2016-04-21 00:07:20 +08:00
Seif Lotfy c6e3c87e00 Add Block checksum validation and "influx_inspect verify" tool
Fixes #5502
2016-04-19 22:33:03 +02:00
Nathaniel Cook 7def8bc0c9 Merge pull request #6423 from influxdata/nc-elapsed
Add elapsed function
2016-04-19 13:43:30 -06:00
Cameron Sparr 21db76b337 Prevent InfluxDB client from leaking TCP connections
If users properly call client.Close(), then this will make sure that
established tcp connections dont continually grow when creating new http
clients.

This fixes the case where users are creating new http clients on top of
existing _valid_ connections.

This was encountered in Telegraf when we were recreating our http
clients after getting write failures that were unrelated to the actual
connection being severed (such as typos in the retention policy, see
https://github.com/influxdata/telegraf/issues/1058)
2016-04-19 13:18:33 -06:00
Nathaniel Cook 465f5a375f add elapsed function 2016-04-19 12:54:54 -06:00
Jonathan A. Sternberg 132b9e5496 Merge pull request #6414 from influxdata/js-prevent-query-pipelining
Send the "Connection: close" header for queries
2016-04-19 14:54:07 -04:00
Jonathan A. Sternberg 09c46a451a Sort the series keys inside of a tag set so the output is deterministic
The series keys within a tag set were previously not sorted which would
cause the output to be non-deterministic. This sorts the output series
by their keys so it has a consistent output especially when using
limits.

Fixes #3166.
2016-04-18 17:45:31 -04:00
Jonathan A. Sternberg 8bdd44107f Send the "Connection: close" header for queries
CloseNotify() is not compatible with pipelining. Adding this header
prevents the client from attempting to pipeline requests.
2016-04-18 15:38:50 -04:00
Aaron Knister 77db7ad618 Prevent goroutine leak in http service for persistent query connections 2016-04-18 15:11:04 -04:00
Jonathan A. Sternberg 34544d2c4b Fix CREATE RETENTION POLICY parsing so it doesn't consume tokens it shouldn't
The optional sections of the command consumed the semicolon token and
didn't put it back for the outer loop. The code shouldn't explicitly
check for a semicolon or EOF anyway, so these checks were removed and
the token gets unscanned if it doesn't match the optional token that the
parser is looking for.

Fixes #6398.
2016-04-17 02:41:17 -04:00
Jonathan A. Sternberg 86046bb2d0 Implement derivatives across intervals for aggregate queries
For aggregate queries, derivatives will now alter the start time to one
interval behind and will use that interval to find the derivative of the
first point instead of giving no value for that interval. Null values
will still be discarded so if the interval before the one you are
querying is null, then it will be discarded like if it were in the
middle of the query. You can use `fill(0)` to fill in these values.

This does not apply to raw queries yet.

Also modified the derivative and difference aggregates to use the stream
iterator instead of the reduce slice iterator for space efficiency.

Fixes #3247. Contributes to #5943.
2016-04-15 18:16:08 -04:00
Jonathan A. Sternberg f4ce20c92a Merge pull request #6394 from influxdata/js-integer-literal-duration-math
Allow time math with integer timestamps
2016-04-15 18:15:43 -04:00
Jonathan A. Sternberg 0144903686 Merge pull request #6272 from influxdata/js-3369-query-time-range-overflow
Detect when a time literal will overflow or underflow the query engine
2016-04-15 18:15:13 -04:00
Jonathan A. Sternberg a85188bd61 Allow time math with integer timestamps 2016-04-15 17:36:37 -04:00
Jonathan A. Sternberg 425e315f45 Detect when a time literal will overflow or underflow the query engine
Fixes #3369.
2016-04-15 17:35:54 -04:00
Jonathan A. Sternberg 6dbfe81e6b Update the changelog in master from v0.12.1
Removed bugfixes from 0.13.0 that were included in 0.12.1.
2016-04-15 17:34:56 -04:00
Jonathan A. Sternberg 1160b52a3a Merge pull request #6383 from influxdata/js-query-executor-panic-recovery
Recover from a panic during query execution
2016-04-15 17:30:56 -04:00
gunnaraasen d983a8c0d3 Support env overrides for all inputs of the same type 2016-04-15 09:39:22 -07:00
Jonathan A. Sternberg 207a4b9d07 Recover from a panic during query execution 2016-04-15 11:13:54 -04:00
Jonathan A. Sternberg 9c2e430a72 Removing no longer used code in the query engine 2016-04-15 10:22:34 -04:00
Jonathan A. Sternberg 42b68d915f Return a deprecated message when IF NOT EXISTS is used
The deprecated message is now attached to a new attribute returned with
the results. This message can then be read by clients to warn a user
about upcoming changes to the query engine.

The `influx` client has already been modified to read this message and
print it out for every format except CSV.

The first warning message is a deprecated message about removing `IF NOT
EXISTS` from `CREATE DATABASE`.

The message will also be printed to the server log.

Fixes #5707.
2016-04-15 09:17:59 -04:00
Seif Lotfy 64a4808dcb
Merge branch 'fix6294' of https://github.com/seiflotfy/influxdb into seiflotfy-fix6294
Conflicts:
	CHANGELOG.md
2016-04-14 14:08:38 -06:00
Jonathan A. Sternberg 03bbff1a65 Merge pull request #6380 from influxdata/js-6379-validate-percentile-args
Added checking for the first argument in percentile()
2016-04-14 15:57:07 -04:00
Seif Lotfy 7a2dcd3131 Check FieldCodec f before calling FieldCoded.Fields()
If FieldCodec f == nil don't call FieldCoded.Fields().

Fixes #6294
2016-04-14 20:30:30 +02:00
Jonathan A. Sternberg 1b6ffa1d2d Merge pull request #6363 from influxdata/js-6292-percentile-as-selector
Allow percentile to be used as a selector
2016-04-14 12:28:16 -04:00
Jonathan A. Sternberg d36698ebd0 Added checking for the first argument in percentile()
The `percentile()` call previously did not validate that the first
argument was a variable reference and that would let an invalid query
slip by that would panic the query engine.

Added checking for this case and also included test cases for the other
calls that require a variable reference as the first argument.

Fixes #6379.
2016-04-14 12:12:31 -04:00
Jonathan A. Sternberg 9d01f3aa56 Allow the implicit time field to be renamed
Fixes #6296.
2016-04-14 11:04:06 -04:00
Chris Ramón 93aa287d0a updates CHANGELOG 2016-04-13 13:10:40 -05:00
Jonathan A. Sternberg 66a599825b Allow percentile to be used as a selector
Fixes #6292.
2016-04-13 13:29:14 -04:00
Todd Persen 00ed9ebaf3 Remove TSDB listener accept message 2016-04-12 16:09:31 -07:00
Gunnar cfa50bbd3d Merge pull request #6228 from influxdata/ga-multiple-inputs
Support multiple listeners per input
2016-04-12 15:31:38 -07:00
Edd Robinson 592d668e1b Ensure Client is safe for concurrent use
Fixes #6287.
2016-04-12 14:23:32 +01:00
Pierre Fersing d4668af4d7 Updated changelog 2016-04-12 09:39:27 +02:00
Valeriy Simonov 1e6b76e0a6 Changelog update 2016-04-12 09:11:55 +05:00
Ben Johnson 525e22c92b
tsm1 query engine alloc reduction
This commit makes a number of performance improvements to
reduce allocations during query execution. Several objects
and buffers are now reused across the components to avoid
allocations.

Previously a simple `count(value)` query across 1M points
would require 26,000+ allocations. After the changes in
this commit that number has been reduced to 88.
2016-04-11 14:50:59 -06:00
Jonathan A. Sternberg 5bdd61bde7 Support empty tags for all WHERE equality operations
A missing tag on a point was sometimes treated as `""` and sometimes
treated as a separate `null` entity. This change modifies the equality
operations to always treat a missing tag as an empty string.

Empty tags are *not* indexed and do not have the same performance as a
tag that exists.

Fixes #3773.
2016-04-11 12:01:35 -04:00
Jonathan A. Sternberg ca534bf09f Fix the subset method on tags
If a point had no tags at all and was asked for the subset of tags with
at least one key, it would return a new set of tags that was completely
empty. In contrast, if the point had any tags at all, it would return a
set of tags with the tag value being an empty string. This lead to
a point with no tags being treated differently than a point with at
least one tag.

Fixing this so the tag value will always be an empty string for
consistency. A missing tag should always be empty.
2016-04-11 12:00:55 -04:00
gunnaraasen 915fc10591 Update CHANGELOG.md 2016-04-11 08:48:27 -07:00
Cameron Sparr 23e2c40923 Reduce UDP service per-packet allocation size
This will reduce memory pressure and number of GC cycles, my results
sending 100,000 UDP points were:

- udp-payload-size=0: 242 GC cycles
- udp-payload-size=1500: 142 GC cycles
- udp-payload-size=0 (with change): 114 GC cycles
- udp-payload-size=1500 (with change): 112 GC cycles
2016-04-08 13:29:47 -06:00
Jonathan A. Sternberg fa5a38dcd4 Fixing aggregate queries with no GROUP BY to include the end time
Queries with a time constraint but no group by would not include the
final point from the underlying iterator.

Fixes #6229.
2016-04-07 14:11:28 -04:00
Jonathan A. Sternberg e2eb93f6d4 Move changelog lines erroneously added to 0.12.0 to 0.13.0 2016-04-07 13:47:20 -04:00
Cory LaNou 7eaf7f8855 CreateShardGroup was incrementing meta data index even when it was idempotent 2016-04-07 11:39:45 -05:00
Jonathan A. Sternberg 61aa2980ce Merge pull request #6255 from influxdata/js-6248-strings-in-select-panic
Throw an error when an invalid expression is used with aux iterators
2016-04-07 12:32:52 -04:00
Jonathan A. Sternberg d176c8babb Throw an error when an invalid expression is used with aux iterators
The following query was fixed previously:

    SELECT 'value' FROM cpu

This ended up hitting the `buildExprIterator()` code path and was
handled properly. But this query:

    SELECT 'value', value FROM cpu

This took a different code path that would trigger a panic because it
triggered a panic instead of an error condition. This code path has now
been modified to trigger an error instead of a panic.

Fixes #6248.
2016-04-07 11:01:45 -04:00
Edd Robinson 09e08e8f55 Merge pull request #6249 from mvadu/Add_AppVeyor_ci_windows
Add AppVeyor ci config to enable continuous integration tests on Windows
2016-04-07 15:52:43 +01:00
mvadu b670ca1936 Add AppVeyor ci config to enable ci tests on Windows
Influxdb uses gdm for the dependency management. For gdm to work on Windows, it needs go 1.6. Influx still uses go 1.4.3. So go 1.6 will be sed for the pre-build stages, and go 1.4.3 to run the tests.
This arrangements is working in AppVeyor now.
2016-04-07 19:47:01 +05:30
Jonathan A. Sternberg ccac9858f1 Update changelog for issue #6206 2016-04-06 11:53:38 -04:00
mvadu 94524a7699 Fix failure on Windows
Windows does not allow a file to be renamed while its in use. f.Sync does not close the handle. It needs to be explicitly closed before renaming.
2016-04-06 07:32:37 +05:30
Paul Dix 092cf4ee4a Update CHANGELOG.md with release notes for 0.12.0 2016-04-04 17:54:19 -04:00
Todd Persen ffa0ece3ac Update CHANGELOG.md 2016-04-04 11:12:35 -07:00
Edd Robinson 712cc3f290 ShardGroup Duration is checked when recreating rp
Fixes #6178.
2016-04-01 11:59:41 +01:00
Cory LaNou 7bda8102ac update changelog 2016-03-31 17:15:58 -05:00
Jonathan A. Sternberg 8752d1b1e3 Support chunked queries in the Go InfluxDB client
Modify the CLI to always use chunked queries.
2016-03-31 15:30:43 -04:00
Jonathan A. Sternberg c193bde61c Throw an error when time is compared to an invalid literal
A bigger refactor of these functions is needed to support #3290, but
this will work for the more common case that someone uses double quotes
instead of single quotes when surrounding a time literal.

Fixes #3932.
2016-03-31 11:29:20 -06:00
Edd Robinson f68637205c Check ShardGroup Duration when recreating database
Fixes #6153
2016-03-31 18:11:50 +01:00
Edd Robinson 105bcd3bb3 Merge pull request #6174 from influxdata/er-fix-shard-duration
Let SHARD DURATION be specified in isolation
2016-03-31 17:54:42 +01:00
Jason Wilder 328b39ae9b Add v0.11.1 changelog entries
[ci skip]
2016-03-31 10:50:32 -06:00
Edd Robinson 9cd0bc65f5 Let SHARD DURATION be specified in isolation
Fixed #6152.
2016-03-31 17:42:50 +01:00
Jason Wilder 34f1f4a1fb Update changelog 2016-03-31 10:19:46 -06:00
Ross McDonald 630996e2a1 Re-add RPM upgrade logic for 0.9 branch. 2016-03-31 11:07:02 -05:00
Edd Robinson 63b89b9215 Merge pull request #6151 from influxdata/er-iterator-panic
Ensure safe access to Shard engine
2016-03-31 16:36:50 +01:00
Jonathan A. Sternberg d8807ffaff Merge pull request #6167 from influxdata/js-6078-max-group-by-buckets
Limit bucket count in selection
2016-03-31 11:16:24 -04:00
Edd Robinson 8e2d1e48c7 Check if engine closed. Fixes #6140 2016-03-31 15:59:04 +01:00
Jason Wilder 3f34ae69c1 Update changelog 2016-03-31 00:06:12 -06:00
Jonathan A. Sternberg ad7480e64b Limit bucket count in selection
Fixes #6078.
2016-03-30 22:57:09 -04:00
Jonathan A. Sternberg eb467d8d7f Merge pull request #6126 from influxdata/js-6115-chunked-query-support-mid-series
Support chunking in the middle of a series in the emitter
2016-03-30 20:59:24 -04:00
Jonathan A. Sternberg 178a6e2f0a Merge pull request #6113 from influxdata/js-6112-simple-moving-average
Implement simple moving average
2016-03-30 20:57:55 -04:00
joelegasse de94acc79d Merge pull request #6163 from influxdata/jl-content-type
Don't use Request.FormValue() in HTTP handler
2016-03-30 17:34:59 -04:00