Commit Graph

9482 Commits (2ab79e75eb7d3449eebe67d234f60208e69da8f4)

Author SHA1 Message Date
Mark Rushakoff 2ab79e75eb Merge pull request #5775 from jonseymour/jss-5499-extend-tsm-cache-stats
tsm: cache: during writes, update the memSize statistic outside the lock
2016-02-21 14:36:56 -08:00
Jon Seymour 510ee2c790 tsm: cache: during writes, update the memSize statistic outside the lock
Since we are not locking but relying on atomic arithmetic,
use Add rather than Set. Will also result in slightly less garbage
being created.

Signed-off-by: Jon Seymour <jon@wildducktheories.com>
2016-02-22 08:26:35 +11:00
Mark Rushakoff feceb4dae1 Merge pull request #5769 from jonseymour/jss-5499-extend-tsm-cache-stats
tsm: cache: ensure all statistics are initialised on cache creation.
2016-02-21 07:25:49 -08:00
Jon Seymour 9c6efe99f1 tsm: cache: ensure all statistics are initialised on cache creation.
The intent of this change is to ensure that all statistic fields of the
resulting tsm1_cache measurement are initialized on initialization of
the cache. That way, any consumer of those measurements doesn't
have to deal with the null case.

Signed-off-by: Jon Seymour <jon@wildducktheories.com>
2016-02-21 15:33:50 +11:00
Mark Rushakoff 04645188fa Merge pull request #5762 from jonseymour/jss-5499-extend-tsm-cache-stats
tsm: cache: add cache throughput related statistics.
2016-02-20 10:15:17 -08:00
Jon Seymour 6697c721fb tsm: cache: add cache throughput related statistics.
Complementing and extending the changes in #5758.

Add 2 level statistics:

  * snapshotCount
  * cacheAgeMs

Add 2 counter statistics

  * cachedBytes
  * WALCompactionTimeMs

snapshotCount can be used to measure transient write errors that are causing snapshots to accumulate

cacheAgeMs can be used to guage the level of write activity into the cache

The differences between cachedBytes stats sampled at different times can be used to calculate cache throughput rates

The ratio (cachedBytes-diskBytes)/WALCompactionTimeMs can be used calculate WAL compaction throughput.

The ratio of difference between first and last WAL compaction time over the interval
length is an estimate of percentage of cache throughput consumed.

Signed-off-by: Jon Seymour <jon@wildducktheories.com>
2016-02-20 22:18:57 +11:00
Mark Rushakoff 602043e11b Add disk stats for FileStore 2016-02-19 16:37:34 -08:00
Mark Rushakoff d99c09cedd Add stats for current and old WAL segment sizes 2016-02-19 16:37:34 -08:00
Mark Rushakoff e76967efb6 Add stats to tsm1.Cache 2016-02-19 16:37:34 -08:00
Gunnar c1de263858 Merge pull request #5710 from influxdata/ga-admin
Move admin UI assets to the admin service directory
2016-02-19 15:36:23 -08:00
Jonathan A. Sternberg 328f505c93 Merge pull request #5745 from influxdata/js-5740-mixed-float-integer-math
Update binary expressions to handle mixed math between integers and floats
2016-02-19 16:05:16 -05:00
Jonathan A. Sternberg f84556fe22 Update binary expressions to handle mixed math between integers and floats
This also changes it so all NumericLiterals are treated as a float
rather than sometimes being cast to an integer.

Fixes #5740.
2016-02-19 14:32:22 -05:00
Jonathan A. Sternberg 8fc6a0f648 Merge pull request #5732 from influxdata/js-5730-binary-expr-column-name
Assign a name to columns with binary expressions in them
2016-02-19 14:30:43 -05:00
Edd Robinson 2da8321128 Merge pull request #5691 from influxdata/er-retention-policies
Clean up shard data when dropping retention policies
2016-02-19 11:32:54 +00:00
Edd Robinson 99a7341701 Wire up DROP retention policy to TSDB store.
Fixes #5653 and #5394.

Previously dropping retention policies did not propogate to local TSDB
shards. Instead, the retention policiess would just be removed from the
Meta Store.

This PR adds ensures that data associated with retention policies is
removed, when the retention policy is dropped.

Also, it cleans up a couple of other methods in `tsdb`, including the
requirement to provide (redundant) shardIDs when deleting databases.
2016-02-19 11:15:00 +00:00
Jason Wilder 645eb44ea9 Merge pull request #5735 from influxdata/kostasb-screen-influx_tsm
screen suggestion for influx_tsm
2016-02-18 16:08:40 -07:00
Jonathan A. Sternberg e58f66845c Merge pull request #5658 from influxdata/js-mean-iterator
Optimize the mean() call by moving the calculation into the shard iterator
2016-02-18 17:27:46 -05:00
Mark Rushakoff b72c684afb Merge pull request #5681 from influxdata/mr-expand-stats
Add more stats for httpd, query executor
2016-02-18 14:11:09 -08:00
Jason Wilder 26163cb03b Merge pull request #5706 from influxdata/jw-cluster
Cluster Setup
2016-02-18 15:08:15 -07:00
Mark Rushakoff c2d2e23832 Update changelog 2016-02-18 13:55:46 -08:00
Jason Wilder 04ba79407a Add join config option back 2016-02-18 14:45:35 -07:00
Jason Wilder 886a62008e Fix dropping nodes
Dropping a meta node that had already been removed from the config
would fail because the raft.RemovePeers call would return an error
that the address was unknown.  This change skips calling RemovePeer
if it doesn't exist.

Dropping a non-existing ID would hang for 10 seconds becuase the
meta.Client retryUntilExec didn't differentiate before command errors
and redirect errors.  In this case, the command would return an error
but we'd try 10 more times and ultimately give up and return the error.
We now return immediately if the command returned and error because
retrying it will not succeed.

Finally, the join loop had no delay and would immediately try to join
the other nodes hundreds of times a second.  We now pause a second if we've
tried every node at least once.
2016-02-18 14:45:34 -07:00
Jason Wilder dedd6f2e9e Fix hostname/bind addresses
This fixes several issues related to the bind address and hostname:
* Allows bind addresses where a hostname or IP is not specified to
work correct and bind to all interfaces by default.
* Fixes the top-level "hostname" config option to allow overridding
all bind address hostnames.  This allows a node to advertise a different
hostname than what is defined in the bind address setting.
* Adds the -hostname command-line option back to allow specifing
both -join and -hostname as command-line flags.
* Enforces a configuration precedence and overriding ability defined
as config file is overridden by env vars which are overriden by command-line
flags.

Fixes #5670 #5671
2016-02-18 14:45:34 -07:00
Jason Wilder a90114aa4d Use same node ID for meta and data nodes
Fixes #5669
2016-02-18 14:45:34 -07:00
Ben Johnson ae7b027742 Merge pull request #5366 from gabelev/lint_influxql
Refactoring influxql package to enable golint
2016-02-18 14:24:37 -07:00
Jason Wilder 04b3914b33 Merge pull request #5743 from influxdata/mr-meta-httpd-headers
Update HTTP headers in meta service's httpd
2016-02-18 13:59:46 -07:00
joelegasse b709f3279b Merge pull request #5542 from influxdata/jl-binary-cleanup
Remove custom binary-conversion functions
2016-02-18 15:57:25 -05:00
Gabriel Levine a9fcc3229a Enable golint for influxql package. 2016-02-18 15:52:37 -05:00
Mark Rushakoff c295db0a49 Fix Content-Type headers in meta httpd 2016-02-18 12:34:29 -08:00
Mark Rushakoff 57dd54250f Correctly set X-Influxdb-Version in meta httpd 2016-02-18 12:34:03 -08:00
Jason Wilder 84eac81ba1 Add 0.10.1 release nodes
[ci skip]
2016-02-18 12:42:49 -07:00
Jason Wilder d8af75f7f3 Merge pull request #5741 from influxdata/mr-cleanup-hh-diagnostics
Prefer bool over string yes/no for HH diagnostics
2016-02-18 12:15:10 -07:00
Mark Rushakoff 8e2d157cef Prefer bool over string yes/no for HH diagnostics 2016-02-18 11:02:14 -08:00
Joe LeGasse dc8ed7953d Remove custom binary-conversion functions
Also cleaned up some excess allocations, and other cruft from the code
2016-02-18 13:56:35 -05:00
Jonathan A. Sternberg 983f810539 Assign a name to columns with binary expressions in them
The name of the column will be every measurement located inside of the
math expression in the order they are encountered in within the
expression.

Also handle `*influxql.ParenExpr` in the function
`(*influxql.Field).Name()`

Fixes #5730.
2016-02-18 10:11:28 -05:00
joelegasse 7479584afd Merge pull request #5725 from influxdata/jl-influx-tsm
influx_tsm: close bolt databases properly
2016-02-18 10:01:00 -05:00
Kostas Botsas b7544c6e7e screen suggestion for influx_tsm
Added a suggestion to run influx_tsm under a screen session to avoid interruptions due to console disconnects.
2016-02-18 11:41:30 +02:00
Jason Wilder 0cb47cc790 Merge pull request #5562 from chrusty/conjoined-field-names-for-graphite-templates
Adding conjoined-field-names for graphite templates
2016-02-17 18:05:19 -07:00
Mark Rushakoff 691a2a97f0 Track queries active, query duration stats 2016-02-17 16:11:30 -08:00
bastard a5cbc43afd Merge remote-tracking branch 'upstream/master' into conjoined-field-names-for-graphite-templates 2016-02-18 00:10:11 +00:00
Jason Wilder b4490788a0 Merge pull request #5512 from influxdata/ga-remove-json-write
Remove JSON write path
2016-02-17 16:52:22 -07:00
Mark Rushakoff 759538da2a Add httpd stats for durations, num active requests 2016-02-17 15:38:55 -08:00
Ben Johnson eb221a5adb Merge pull request #5663 from benbjohnson/query-executor
Refactor QueryExecutor (WIP)
2016-02-17 16:30:20 -07:00
Jason Wilder 7e64e7720e Merge pull request #5608 from mvadu/mvadu-patch-ErrDialTimeout
Windows: fix test failure - TestShardWriter_Write_ErrDialTimeout
2016-02-17 15:51:45 -07:00
Ben Johnson e3b4b71c13 refactor query executor
This commit moves the `QueryExecutor` to the `cluster` package
and provides an interface to it inside the `influxql` package.
2016-02-17 15:13:56 -07:00
Ben Johnson 979bd771f4 Merge pull request #5726 from benbjohnson/nan
Remove NaN from query engine
2016-02-17 15:12:29 -07:00
Ben Johnson f7e04abef7 remove NaN from query engine
This commit removes `math.NaN` returns from float iterators.
2016-02-17 14:11:31 -07:00
Joe LeGasse 58033b1efa influx_tsm: close bolt databases properly
Fixes #5724
2016-02-17 14:20:28 -05:00
Edd Robinson f315444f86 Merge pull request #5255 from titilambert/github4752
Workaround for #4752
2016-02-17 11:00:58 +00:00
gunnaraasen 426bc0aa3e Move admin UI assets to the admin service directory 2016-02-16 17:12:39 -08:00