Commit Graph

27 Commits (0cb74eedbf730bd0a7c921d115ffb121d0a65ff3)

Author SHA1 Message Date
Ben Johnson 409b0165f5
shared in-memory index 2017-01-05 10:09:57 -07:00
Jonathan A. Sternberg ec57108520 Use proper uber-go/zap import path
It looks like the real import path to the project is go.uber.org/zap
instead of github.com/uber-go/zap since the example in the project
references that path.
2016-12-15 08:54:14 -06:00
Jonathan A. Sternberg 21502a39e8 Switch logging to use structured logging everywhere
The logging library has been switched to use uber-go/zap. While the
logging has been changed to use structured logging, this commit does not
change any of the logging statements to take advantage of the new
structured log or new log levels. Those changes will come in future
commits.
2016-12-14 10:45:15 -06:00
Jonathan A. Sternberg 64c2d704da Avoid deadlock when max-row-limit is hit
When the `max-row-limit` was hit, the goroutine reading from the results
channel would stop reading from the channel, but it didn't signal to the
sender that it was no longer reading from the results. This caused the
sender to continue trying to send results even though nobody would ever
read it and this created a deadlock.

Include an `AbortCh` on the `ExecutionContext` that will signal when
results are no longer desired so the sender can abort instead of
deadlocking.
2016-11-08 13:12:28 -06:00
Jason Wilder 0b6f5441b9 Add config option to messages when limits exceeded
When a limit is exceeded, we return errors and sometimes log (if appropriate)
that a limit was exceeded.  The messages don't always provide an indication
as to where or how they are configured.

Instead, return the config option (easily searchable for) as well as the limit
currently set and the value that exceeded it when possible.
2016-10-28 14:54:45 -06:00
Jonathan A. Sternberg 3afdf3cd94 Merge tag 'v1.0.1' 2016-09-27 17:53:33 -05:00
Jonathan A. Sternberg dbc4a9150f Prevent manual use of system queries
Manual use of system queries could result in a user using the query
incorrect. Rather than check to make sure the query was used correctly,
we're just going to prevent users from using those sources so they can't
use them incorrectly.
2016-09-23 10:00:18 -05:00
Edd Robinson 90ff713f21 Fix base64 encoding issue in stats
Fixes #7177.
2016-08-22 15:21:31 +01:00
Ben Johnson 8aa224b22d
reduce memory allocations in index
This commit changes the index to point to index data in the shards
instead of keeping it in-memory on the heap.
2016-08-16 14:09:00 -06:00
Jonathan A. Sternberg 32d10de94f Check in between query statements to see if the query was interrupted
This allows a long series of uninterruptible statements to still be
interrupted for a long running query that might do something like create
or drop many databases.
2016-08-10 15:36:02 -05:00
Jonathan A. Sternberg 3959656968 Add additional statistics to query executor
The query executor would only store the number of active queries and the
query duration so it was impossible to determine how many queries were
actually executed during that timeframe because quick queries would be
gone before the call to gather statistics was made.

This adds two new statistics so track when queries start and when
queries finish and doesn't decrement the counter so the number of
executed queries can be obtained using `derivative()` and
`difference()`.
2016-08-10 11:35:06 -05:00
Jonathan A. Sternberg 7b8bcfb173 Add option to suppress logging query statements in the query executor 2016-07-26 13:53:45 -05:00
Jonathan A. Sternberg 837a9804cf Refactoring the monitor service to avoid expvar
Truncate the time interval output of the monitor service to be on even
time intervals rather than on every minute based on the start time. This
normalizes the output from the monitor service.
2016-07-07 11:13:58 -05:00
Ben Johnson 48f1a6d858 Merge pull request #6820 from benbjohnson/http-query-node-id
Add NodeID to execution options
2016-06-10 13:10:15 -06:00
Jonathan A. Sternberg 9db82e6bf0 Switch ExecutionContext to be passed by value 2016-06-10 12:31:51 -05:00
Jonathan A. Sternberg 55973d2815 Separate the task manager from the query executor
The task manager now acts as its own statement executor so that a custom
statement executor can perform custom actions for KillQueryStatement and
ShowQueriesStatement.
2016-06-10 12:30:51 -05:00
Ben Johnson 7d4bea7153
add node id to execution options
This commit changes the `ExecutionOptions` and `SelectOptions` to
allow a `NodeID` for specifying an exact node to query against.
2016-06-10 09:20:44 -06:00
Jonathan A. Sternberg 71c8e9e567 Refactor ExecuteQuery to take options as a struct
This allows us to add additional options to ExecuteQuery without
creating parameter bloat.

Removing the unused Series structs. Their necessity was removed by a
previous commit, but the structs were not removed yet.

Add another type of interrupt iterator that monitors the interrupt
channel and calls `Close()` on the iterator when the interrupt happens.
It will primarily be used for asynchronously closing the ReaderIterator,
but it will only close the read side of the connection properly. More
work needs to be done to allow closing the write side efficiently.
2016-06-01 12:30:52 -05:00
Edd Robinson 67e793e512 Add more context to recovered panics 2016-05-23 12:24:05 +01:00
Jonathan A. Sternberg c01cda58e8 Print the query executor's stack trace on a panic to the log
Fixes #6644.
2016-05-17 09:23:55 -04:00
Jonathan A. Sternberg 6f61c0ea4a Add POST /query endpoint and warning messages for using GET with write operations
In order to follow REST a bit more carefully, all write operations
should go through a POST in the future. We still allow read operations
through either GET or POST (similar to the Graphite /render endpoint),
but write operations will trigger a returned warning as part of the JSON
response and will eventually return an error.

Also updates the Golang client libraries to always use POST instead of
GET.

Fixes #6290.
2016-04-29 09:00:23 -04:00
Jonathan A. Sternberg e28d16cfcf Log slow queries if they pass a configurable threshold
Fixes #6429.
2016-04-22 12:04:15 -04:00
Stephen Gutekanst 9dc09c5257 Make logging output location more programmatically configurable (#6213)
This has various benefits:

- Users embedding InfluxDB within other Go programs can specify a different logger / prefix easily.
- More consistent with code used elsewhere in InfluxDB (e.g. services, other `run.Server.*` fields, etc).
- This is also more efficient, because it means `executeQuery` no longer allocates a single `*log.Logger` each time it is called.
2016-04-20 21:07:08 +01:00
Jonathan A. Sternberg 207a4b9d07 Recover from a panic during query execution 2016-04-15 11:13:54 -04:00
Jonathan A. Sternberg 42b68d915f Return a deprecated message when IF NOT EXISTS is used
The deprecated message is now attached to a new attribute returned with
the results. This message can then be read by clients to warn a user
about upcoming changes to the query engine.

The `influx` client has already been modified to read this message and
print it out for every format except CSV.

The first warning message is a deprecated message about removing `IF NOT
EXISTS` from `CREATE DATABASE`.

The message will also be printed to the server log.

Fixes #5707.
2016-04-15 09:17:59 -04:00
Jonathan A. Sternberg 37b63cedec Cleanup QueryExecutor and split statement execution code
The QueryExecutor had a lot of dead code made obsolete by the query
engine refactor that has now been removed. The TSDBStore interface has
also been cleaned up so we can have multiple implementations of this
(such as a local and remote version).

A StatementExecutor interface has been created for adding custom
functionality to the QueryExecutor that may not be available in the open
source version. The QueryExecutor delegate all statement execution to
the StatementExecutor and the QueryExecutor will only keep track of
housekeeping. Implementing additional queries is as simple as wrapping
the cluster.StatementExecutor struct or replacing it with something
completely different.

The PointsWriter in the QueryExecutor has been changed to a simple
interface that implements the one method needed by the query executor.
This is to allow different PointsWriter implementations to be used by
the QueryExecutor. It has also been moved into the StatementExecutor
instead.

The TSDBStore interface has now been modified to contain the code for
creating an IteratorCreator. This is so the underlying TSDBStore can
implement different ways of accessing the underlying shards rather than
always having to access each shard individually (such as batch
requests).

Remove the show servers handling. This isn't a valid command in the open
source version of InfluxDB anymore.

The QueryManager interface is now built into QueryExecutor and is no
longer necessary. The StatementExecutor and QueryExecutor split allows
task management to much more easily be built into QueryExecutor rather
than as a separate struct.
2016-04-04 13:27:17 -04:00
Ben Johnson e3b4b71c13 refactor query executor
This commit moves the `QueryExecutor` to the `cluster` package
and provides an interface to it inside the `influxql` package.
2016-02-17 15:13:56 -07:00