Commit Graph

10950 Commits (fcd425c8c62ccce9e549be7705d5a29c668e10f2)

Author SHA1 Message Date
Jonathan A. Sternberg a101f52e34 Merge pull request #7067 from influxdata/js-add-quiet-execution-option
Add option to suppress logging query statements in the query executor
2016-07-26 15:38:05 -05:00
Jason Wilder 8e1f238861 Merge pull request #7055 from influxdata/jw-compact-full
Prevent full compactions from conflicting with other compactions
2016-07-26 13:46:33 -06:00
Jason Wilder 7c3d1aac68 Simplify purger.add logic 2016-07-26 13:02:08 -06:00
Jason Wilder cab84ae279 Prevent concurrent compactions from stepping on each other
Normally, compactions do not conflict on the files they are compacting.
If the full cold threshold is set very low, it can cause conflicts where
two compactions compact the same files.  The full compaction was the
only place this could happen as it's planning is greedy.

To make this safer for concurrent execution, the compaction tracks which
files are current being compacted and prevents any new compactions from
starting if the file set overlaps.

Fixes #6595
2016-07-26 12:58:25 -06:00
Jason Wilder ded6e40d47 Remove lastPlanCheck var
This causes full compactions to not run if the server is running, but
after a restart they do run.
2016-07-26 12:58:25 -06:00
Jason Wilder 2f78c4ec83 Fix race when creating temp file
Using os.O_EXCL is safer than checking and then creating the file.
2016-07-26 12:58:25 -06:00
Jonathan A. Sternberg 7b8bcfb173 Add option to suppress logging query statements in the query executor 2016-07-26 13:53:45 -05:00
Jason Wilder 115fe89d65 Merge pull request #7047 from influxdata/jw-tsminspect
Add report command to influx_inspect
2016-07-26 11:34:12 -06:00
Jason Wilder 6bee354530 Add Load Time column 2016-07-26 10:23:45 -06:00
Jason Wilder 78cfe441aa Skip TSM files that fail to load
This can be used a quick way to determine if any TSM files are
corrupted.
2016-07-26 10:23:44 -06:00
Jason Wilder 5bdc01a3b2 Replace info w/ report command
info was not safe to run and not really useful.
2016-07-26 10:23:44 -06:00
Jason Wilder bfe1710d72 Merge pull request #7063 from influxdata/jw-heap-template
Add heap profile to issue template
2016-07-26 10:21:13 -06:00
Jason Wilder fadc227c93 Flush tab writer after each row
tab writer was flushed after all the rows were processed which consumed
a lot of memory and was very slow.
2016-07-26 10:17:58 -06:00
Jason Wilder 4543724eb5 Add heap profile to issue template 2016-07-26 09:54:46 -06:00
Cory LaNou 7ec80e05ce update influx inspect readme with DDL information 2016-07-25 14:09:53 -05:00
Jason Wilder ec34549ef7 Merge pull request #7056 from influxdata/jw-template
Add additional debug commands
2016-07-25 12:08:58 -06:00
Cory LaNou 2c1a65f47f Merge pull request #7050 from influxdata/cjl-dep-updates
update go dependencies
2016-07-25 08:52:24 -05:00
Jason Wilder b20bbf5064 Add additional debug commands 2016-07-24 22:33:59 -06:00
Cory LaNou 71085e9a39 update changelog 2016-07-22 14:33:20 -05:00
Cory LaNou 063675b928 updates to make snappy compression tests work again 2016-07-22 14:33:20 -05:00
Cory LaNou 9ef95ccb5a update go dependencies 2016-07-22 14:33:20 -05:00
Jonathan A. Sternberg 680de727b9 Merge pull request #7020 from influxdata/js-7011-create-man-pages
Create man pages for commands
2016-07-22 10:59:42 -05:00
Jonathan A. Sternberg e50850da5b Update pull request template to reference man pages and update changelog 2016-07-22 10:34:40 -05:00
Jonathan A. Sternberg 94f6c19eab Install man pages inside of the packages 2016-07-22 10:34:40 -05:00
Jonathan A. Sternberg e091b6d51a Initial work creating man pages 2016-07-22 10:34:40 -05:00
Cory LaNou 3c24504370 Merge pull request #7046 from influxdata/cjl-tsm-export
Finish tsm file exporter
2016-07-21 18:02:07 -05:00
Cory LaNou 968d322d6d finish tsm file exporter 2016-07-21 17:20:51 -05:00
Jason Wilder da10220016 Merge pull request #7038 from influxdata/jw-compact-query
Fix queries against compacted files
2016-07-21 12:26:06 -06:00
Jason Wilder fb5a143b08 Fix typos 2016-07-21 12:13:04 -06:00
Jason Wilder 13147efb24 Close underlying cursors when closing iterators
If a query is interrupted via kill query, the tsm files managed
by the file store purger would never get removeed because
KeyCursor.Close was never called.

KeyCursor.Close should always be called now.
2016-07-21 12:13:04 -06:00
Jason Wilder 822f409b31 Allow queries to complete before closing TSM files
If a query was running against a file being compacted, we close the file
and the query would end wherever it had read up to.  This could result
in queries that randomly lost data, but running them again showed the
full results.

We now use a reference counting approach and move the in-use files out
of the way in the filestore and allow the queries to complete against
the old tsm files.  The new files are installed and new queries will
use them.

Fixes #5501
2016-07-21 12:13:04 -06:00
Jonathan A. Sternberg 92825b418f Merge pull request #6920 from influxdata/js-6909-log-cq-execution-time
Log the CQ execution time when continuous query logging is enabled
2016-07-21 12:56:03 -05:00
Jonathan A. Sternberg 4bdf57534b Log the CQ execution time when continuous query logging is enabled 2016-07-21 12:16:53 -05:00
Cory LaNou e3ecbf0d19 Merge pull request #7043 from influxdata/cjl-walkshards-remove-limiter
remove limiter from walkShards
2016-07-21 11:31:14 -05:00
Cory LaNou fd86670518 remove limiter from walkShards 2016-07-21 11:23:31 -05:00
Edd Robinson 785e50b546 Merge pull request #6980 from influxdata/er-tsm-log
Add trace logging options to tsm engine
2016-07-21 12:55:20 +01:00
Edd Robinson ff9d38540f Update release notes and sample config 2016-07-21 11:50:21 +01:00
Edd Robinson f37e726869 Add trace logging statements to tsdb 2016-07-21 11:14:29 +01:00
Edd Robinson 44231abcbd Add trace logger controlled via DataLoggingEnabled 2016-07-21 11:14:29 +01:00
Edd Robinson 217bd4de84 Disable trace logging by default 2016-07-21 11:14:29 +01:00
Edd Robinson 83cc580ff8 Tidy up logging 2016-07-21 11:14:29 +01:00
Jason Wilder 6b71474434 Merge pull request #6995 from influxdata/mr-fuzz-crash-repros
Check slice bounds in tsm1 to avoid panics
2016-07-20 16:38:30 -06:00
Mark Rushakoff 518bd3b565 Micro-optimize BooleanDecoder for 20% speedup
benchmark                          old ns/op     new ns/op     delta
BenchmarkBooleanDecoder_2048-4     9954          7846          -21.18%

benchmark                          old allocs     new allocs     delta
BenchmarkBooleanDecoder_2048-4     0              0              +0.00%

benchmark                          old bytes     new bytes     delta
BenchmarkBooleanDecoder_2048-4     0             0             +0.00%
2016-07-20 08:43:05 -07:00
Mark Rushakoff 523aea715a Protect against bounds errors in FloatDecoder 2016-07-19 15:59:27 -07:00
Mark Rushakoff e483689563 Protect against bounds errors in BooleanDecoder 2016-07-19 15:59:27 -07:00
Mark Rushakoff 35e3adc890 Protect against bounds errors in IntegerDecoder 2016-07-19 15:43:27 -07:00
Mark Rushakoff 42b35ca068 Protect against bounds errors in TimeDecoder 2016-07-19 15:43:27 -07:00
Mark Rushakoff be589a6760 Protect against bounds errors in StringDecoder 2016-07-19 15:43:27 -07:00
Mark Rushakoff 5b549ffdfe Handle bounds errors in UnpackBlock 2016-07-19 15:43:27 -07:00
Mark Rushakoff 39f12e376c Defend against some boundary errors in TSM reading 2016-07-19 15:43:27 -07:00