Commit Graph

720 Commits (3ecc913c887f49c86579cedc24e113d4534fe13d)

Author SHA1 Message Date
Edd Robinson 90ff713f21 Fix base64 encoding issue in stats
Fixes #7177.
2016-08-22 15:21:31 +01:00
Jonathan A. Sternberg 9b28f7704f Merge pull request #7171 from influxdata/js-1.0-merge-response-formatter-bugfix
Merge branch '1.0'
2016-08-17 19:13:09 -05:00
Jonathan A. Sternberg 5e468fac0b Merge branch '1.0' into master 2016-08-17 13:27:02 -05:00
Jonathan A. Sternberg d2746ee8f2 The number of bytes recorded when using chunking was off by one
Previously, we implicitly added a newline and had to add one to the
number of bytes transmitted because we added that byte. That was removed
at some point and the metric was not updated to record the correct
value.
2016-08-17 11:54:19 -05:00
Jonathan A. Sternberg 71b54768d0 Remove redundant code the from response formatter and fix CloseNotify
The query killing functionality depends on the ResponseWriter exposing a
CloseNotify method. Since we wrap the http.ResponseWriter, the new
struct does not have that method and the HTTP handler would skip past
calling that method.

Instead of duplicating `Flush()` and `CloseNotify()` for every response
formatter, we will unify all of that under a single struct and create
formatters instead.

Also, fixes a bug where the header information from a query would not be
returned until some other data was returned with it because of
buffering and another bug in the gzipResponseWriter that wouldn't flush
the actual underlying ResponseWriter.
2016-08-16 15:54:03 -05:00
Ben Johnson 8aa224b22d
reduce memory allocations in index
This commit changes the index to point to index data in the shards
instead of keeping it in-memory on the heap.
2016-08-16 14:09:00 -06:00
Edd Robinson cebeda817c Update jwt-go to v3 2016-08-12 17:35:57 +01:00
Jonathan A. Sternberg 87f7c66b8a Merge pull request #7119 from influxdata/js-create-database-use-defaults
Use defaults from `meta` package for `CREATE DATABASE`
2016-08-11 10:34:22 -05:00
Jonathan A. Sternberg f58a50c231 Allow queries to be uploaded as a file rather than forcing a text parameter
The query can be uploaded from a file using `multipart/form-data` and
setting the file name to `q`. An example of using curl to execute an
async query would be:

    curl -F "q=@database.iql" -F "async=true" http://localhost:8086/query
2016-08-10 15:34:04 -05:00
Jonathan A. Sternberg 760767b033 Support running a command in async mode
It will return a 204 No Content as long as the query is accepted
(immediate errors will be returned, but not individual errors with
specific queries). The only way to kill the query is by using the task
manager.
2016-08-10 15:34:04 -05:00
Jonathan A. Sternberg a4e49963f5 Implement text/csv content encoding for the response writer
CSV doesn't offer a way to separate different sheets from each other and
it doesn't really have a standard format. We separate sheets with a
newline so they can be imported into something like Excel or LibreOffice
more easily.

The number of columns for each sheet is inferred from the first returned
row in each statement since they should all be the same.
2016-08-10 15:15:33 -05:00
kun 6945655be2 add support for unix socket binding service 2016-08-11 02:20:54 +08:00
Jonathan A. Sternberg 530b00bd76 Use defaults from `meta` package for `CREATE DATABASE`
Instead of having the parser set the defaults, the command will set the
defaults so that the constants for that are actually used. This way we
can also identify which things the user provided and which ones we are
filling with default values.

This allows the meta client to be able to make smarter decisions when
determining if the user requested a conflict or if the requested
capabilities match with what is currently available. If you just say
`CREATE DATABASE WITH NAME myrp`, the user doesn't really care what the
duration of the retention policy is and just wants to use the default.
Now, we can use that information to determine if an existing retention
policy would conflict with what the user requested rather than returning
an error if a default value ever gets changed since the meta client
command can communicate intent more easily.
2016-08-09 12:00:06 -05:00
Jonathan A. Sternberg b98763a3d8 Merge pull request #7118 from influxdata/js-go-generate
go generate on every package to ensure they are generated with the correct dependency
2016-08-08 09:02:32 -05:00
Jonathan A. Sternberg 4cdfc3280d Move the CQ interval by the group by offset
This will make the period selected by the CQ system work correctly for a
query with an offset.
2016-08-05 14:39:52 -05:00
Jonathan A. Sternberg 0879d43b74 Do not run continuous queries that have no time span 2016-08-05 14:39:52 -05:00
Jonathan A. Sternberg ed2f81357f go generate on every package to ensure they are generated with the correct dependency 2016-08-05 14:35:07 -05:00
Ben Johnson 6c7d56d4bc
limit shard concurrency
This commit limits queries to only process one shard at a time.
However, within a shard, multiple series can still be processed in
parallel. Shard iterators are lazily instantiated during query
execution to limit the amount of memory a given query uses.
2016-08-05 09:45:57 -06:00
Jonathan A. Sternberg 73bbbf24b1 Use the Accept header for determining the Content-Type instead of Content-Type
The `Content-Type` header is meant to say what the content type of the
request is supposed to be and `Accept` is used to ask for a specific
content type. I messed this up and used `Content-Type` instead of
`Accept`. This works with `GET` requests accidentally, but `POST`
requests stop working.
2016-07-29 14:29:39 -05:00
Jonathan A. Sternberg d85072ada9 Setup an interface for writing http responses based on the Content-Type header
Only `application/json` is supported right now, but this opens up the
easier possibility of additional content types to be returned from the
server.
2016-07-29 13:11:51 -05:00
Edd Robinson 9a2efafa13 Ensure CQ id names can't clash 2016-07-28 16:26:05 +01:00
Cameron Sparr d688676266 Update UDP buffer setting doc to 25MB 2016-07-28 12:02:56 +01:00
Cameron Sparr 6afdcf9572 Fix incrementing pointsRx twice 2016-07-28 11:06:50 +01:00
David Norton e3328117a9 fix #7081: hardcode auto gen RP name 2016-07-27 16:49:46 -04:00
David Norton b1079c9080 revert previous DefaultRetentionPolicyName change 2016-07-27 11:38:42 -04:00
David Norton 8fbfdc5616 fix #7068: use DefaultRetentionPolicyName from cfg
Use DefaultRetentionPolicyName from the config instead of passing
through meta data.
2016-07-26 18:53:55 -04:00
Jonathan A. Sternberg 4bdf57534b Log the CQ execution time when continuous query logging is enabled 2016-07-21 12:16:53 -05:00
Jonathan A. Sternberg f6ff8a572d Run continuous query for multiple buckets rather than one per bucket 2016-07-14 11:23:05 -05:00
Jonathan A. Sternberg 83caeec3a4 Support loading a folder for collectd typesdb files 2016-07-07 20:57:57 -05:00
Jonathan A. Sternberg 7a3bd19926 Properly use the 401 and 403 HTTP status codes
According to the HTTP standard, a lack of authentication credentials or
incorrect authentication credentials should send back a 401
(Unauthorized) with a `WWW-Authenticate` header with a challenge that
can be used to authenticate. This is because a 401 status should be sent
when an authentication attempt can be retried by the browser.

The 403 (Forbidden) status code should be sent when authentication
succeeded, but the user does not have the necessary authorization.
Previously, the server would always send a 401 status code.
2016-07-07 20:30:04 -05:00
Jonathan A. Sternberg 837a9804cf Refactoring the monitor service to avoid expvar
Truncate the time interval output of the monitor service to be on even
time intervals rather than on every minute based on the start time. This
normalizes the output from the monitor service.
2016-07-07 11:13:58 -05:00
Edd Robinson 2c85d498ef Remove unused errors 2016-07-06 15:04:39 +01:00
Mikael Taina b9d3ae085c Don't force root path on getClientVersion
Similar to #1339 .

Fix getClientVersion() on setups where the admin interface resides on some
other path than root path due to some reverse proxy setup.
2016-07-05 22:47:48 +03:00
Cole Mackenzie 6d926722b5 added favicon.ico and regenerated statik.go 2016-07-01 08:53:56 +00:00
Jonathan A. Sternberg b8e52ce39a Merge pull request #6889 from influxdata/js-update-config-options
Update help and remove unused config options from the configuration file
2016-06-23 11:49:10 -05:00
kun 8aa2fe8ede fix ClusterID generation bug 2016-06-23 14:52:58 +08:00
Jonathan A. Sternberg 22173acb70 Update help and remove unused config options from the configuration file
Normalize the output for the various help options so they all follow the
same format and display all relevant options.

Removing some of the unused config options from the configuration file
and updating the help documentation. Removing some remaining references
to clustering within the open source version.
2016-06-21 14:06:05 -05:00
Jonathan A. Sternberg 497db2a6d3 Removing dead code from every package except influxql
The tsdb package had a substantial amount of dead code related to the
old query engine still in there. It is no longer used, so it was removed
since it was left unmaintained. There is likely still more code that is
the same, but wasn't found as part of this code cleanup.

influxql has dead code show up because of the code generation so it is
not included in this pruning.
2016-06-20 22:41:07 -05:00
Edd Robinson 9603bad14c Merge pull request #6868 from smunier01/continuous-queries-admin-interface
Add continuous queries templates in admin interface
2016-06-20 11:35:47 +01:00
Edd Robinson 3d8cde2346 Merge pull request #6774 from minkikim89/patch-1
Fixed admin page reloaded when navbar-brand clicked
2016-06-20 11:34:08 +01:00
simon f007fccb4a Add continuous queries templates in admin interface 2016-06-18 16:33:41 +02:00
Jonathan A. Sternberg 8e1b036b0a Modify the max nanosecond time to be one nanosecond less
The highest time represented by a nanosecond needs to be used for an
exclusive range, so the maximum time needs to be one less than the
possible maximum number of nanoseconds representable by an int64 so that
we don't lose a point at that one time.

Previously worked in the open source version because the timestamp used
for finding a shard would be truncated by the retention policy so the
lookup time didn't run into this edge case because it didn't rest on the
truncation boundary. Since that point didn't really belong in that shard
group and was placed there by mistake, it's best to fix this bug since
the timestamp used to create the shard group should be capable of
retrieving it.
2016-06-16 12:15:41 -05:00
Jack Zampolin a96f96c024 Merge pull request #6834 from influxdata/jz-graphite
Add `port` to `[graphite]` log output
2016-06-14 11:35:19 -07:00
Cameron Sparr b9a3f2e1e5 Make httpd logger closer to Common Log Format
changes the httpd log lines from this:

    [httpd] 2016/06/08 14:06:39 ::1 - - [08/Jun/2016:14:06:39 +0100] POST /write?consistency=any&db=telegraf&precision=s&rp= HTTP/1.1 204 0 - InfluxDBClient d6aa01fc-2d79-11e6-8024-000000000000 2.751391ms

to this:

    [httpd] ::1 - - [08/Jun/2016:14:06:39 +0100] "POST /write?consistency=any&db=telegraf&precision=s&rp= HTTP/1.1" 204 0 "-" "InfluxDBClient" d6aa01fc-2d79-11e6-8024-000000000000 2751

So it changes a few things:

1. Remove the logger timestamp at the beginning which isn't very relevant anyways
2. adds quotes around "METHOD URI PROTOCOL", because this is part of the
common log format.
3. adds quotes around "AGENT" and "REFERRER" because this is part of the
"combined" log format.
4. Puts the response time in integer microseconds, because this is
consistent with apache's %D config mod option.

Compared with CLF, our logs now look like this:

    [httpd] %{COMMON_LOG_FORMAT} "<agent>" "<referrer>" <request_uuid> <response_time_µs>

For reference, see:
https://en.wikipedia.org/wiki/Common_Log_Format
http://httpd.apache.org/docs/current/mod/mod_log_config.html
2016-06-14 14:48:24 +01:00
Jack Zampolin f1db2f2ca5 Add port to all graphite log output to help with debugging multiple endpoints 2016-06-13 11:59:56 -07:00
Jason Wilder fd262697b9 Initialize subscriber channel on open
If the subscriber service was closed and re-open, it would panic
with panic: send on closed channel
2016-06-10 18:00:44 -06:00
Ben Johnson 48f1a6d858 Merge pull request #6820 from benbjohnson/http-query-node-id
Add NodeID to execution options
2016-06-10 13:10:15 -06:00
Jonathan A. Sternberg 9db82e6bf0 Switch ExecutionContext to be passed by value 2016-06-10 12:31:51 -05:00
Jonathan A. Sternberg 55973d2815 Separate the task manager from the query executor
The task manager now acts as its own statement executor so that a custom
statement executor can perform custom actions for KillQueryStatement and
ShowQueriesStatement.
2016-06-10 12:30:51 -05:00
Ben Johnson 7d4bea7153
add node id to execution options
This commit changes the `ExecutionOptions` and `SelectOptions` to
allow a `NodeID` for specifying an exact node to query against.
2016-06-10 09:20:44 -06:00