602a2e80ce
Ensure aux and cond cursors are closed when iterator is closed
2016-07-28 20:25:37 -06:00
0b60862248
Close drained iterators
...
Aux and condition iterators where not closed which could
cause TSM files to leak if they were queried against while
a compaction was running.
2016-07-28 20:25:37 -06:00
5764a730d5
Prevent tombstoning series keys more than once
...
If there were multiple TSM files and a delete/drop was run,
we would write the delete series to the tombstone file N
times for each file. This occurred because FileStore.WalkKeys walks
every key in every TSM file which can return duplicate keys.
This issue caused TSM files to be much larger than they should be
and also cause large memory usage during the delete.
2016-07-28 20:25:36 -06:00
ef8ecf0e90
Apply reload tombstones in batches
...
This keeps some memory bounds when reloading a TSM files tombstones
so that the heap does not grow exceedintly fast and stay there
after the deletes are applied.
2016-07-28 20:25:36 -06:00
4436e65fb9
Apply deletes to TSM files concurrently
2016-07-28 20:25:36 -06:00
a8c69e222a
Use scanner for reading v1 tombstones
...
Use a bufio.Scanner to read v1 tombstones instead of reading in
the whole file and parsing it from memory.
2016-07-28 20:25:36 -06:00
7b8959f6f2
Apply tombstones iteratively at startup
...
Tombstone were read fully into memory at startup which could consume
a lot of RAM and OOM the process if there were a lot of deleted
series and many TSM files.
This now walks the tombstone file and iteratively applies the tombstone
which uses significantly less RAM. This may be slightly slower in the
generate cause, but should scale better.
2016-07-28 20:25:36 -06:00
644693ad4f
Merge pull request #7086 from influxdata/jw-6990
...
Fix panic with parsing empty key
2016-07-28 19:29:32 -06:00
d432aaa84d
Fix panic with parsing empty key
...
Fixes #6990
2016-07-28 18:38:17 -06:00
ae4a1a4d00
Merge pull request #7049 from influxdata/cjl-influx-inspect-doc-update
...
Update influx inspect readme with DDL information
2016-07-28 18:19:12 -05:00
94e07158d5
Merge pull request #7009 from influxdata/js-5750-wildcard-support-for-aggregates
...
Support wildcards in aggregate functions
2016-07-28 18:13:59 -05:00
86364a85b4
Merge pull request #7090 from influxdata/js-show-measurements-and-tag-values-refactor
...
Switch SHOW MEASUREMENTS and SHOW TAG VALUES to directly access the tsdb.Store
2016-07-28 18:04:13 -05:00
c657f48b06
Merge pull request #7092 from influxdata/js-update-dependency-license
...
Update LICENSE_OF_DEPENDENCIES.md with github.com/retailnext/hllpp
2016-07-28 18:04:03 -05:00
23ef9484a4
Support wildcards in aggregate functions
2016-07-28 17:56:32 -05:00
86bd97f3b9
Switch SHOW MEASUREMENTS and SHOW TAG VALUES to directly access the tsdb.Store
...
The `SHOW MEASUREMENTS` and `SHOW TAG VALUES` cannot go through the
query engine to get the speed they need. They also only need access to
the database index and do not need access to specific shards. This
removes the query rewriting that was done to turn these two queries into
a select statement and reimplements them inside of the coordinator as an
interface on the TSDBStore.
2016-07-28 17:38:11 -05:00
839a567604
Update LICENSE_OF_DEPENDENCIES.md with github.com/retailnext/hllpp
...
Also reorganizing Godeps so it lists the dependencies in alphabetical
order.
2016-07-28 17:22:54 -05:00
a1c7d9c8f2
Merge pull request #7080 from influxdata/er-cq-delimiter
...
Ensure CQ ID names can't clash
2016-07-28 22:00:34 +01:00
9a2efafa13
Ensure CQ id names can't clash
2016-07-28 16:26:05 +01:00
d688676266
Update UDP buffer setting doc to 25MB
2016-07-28 12:02:56 +01:00
6afdcf9572
Fix incrementing pointsRx twice
2016-07-28 11:06:50 +01:00
d6a726b5e8
Merge pull request #7085 from influxdata/mr-fix-series-keys-length
...
Fix length of (*tsdb.DatabaseIndex).SeriesKeys()
2016-07-27 16:25:40 -07:00
f34a7430e3
Fix length of (*DatabaseIndex).SeriesKeys()
...
Previously, it would return as many empty strings in the first half of
the slice as valid values at the end of the slice.
2016-07-27 16:07:39 -07:00
b2725ca9d4
Merge pull request #7082 from influxdata/dgn-hardcode-rp-name
...
fix #7081 : hardcode auto gen RP name
2016-07-27 17:25:34 -04:00
e3328117a9
fix #7081 : hardcode auto gen RP name
2016-07-27 16:49:46 -04:00
19ff8a1b67
Merge pull request #7033 from benbjohnson/create-iterator-check
...
Check for nil iterator creation
2016-07-27 16:15:48 -04:00
5df6f75545
check for nil iterator creation
...
This commit checks if an iterator is `nil` before adding to an
iterator list during creation.
2016-07-27 13:54:56 -06:00
1579dd1e75
Merge pull request #7078 from influxdata/dgn-revert-change
...
revert previous DefaultRetentionPolicyName change
2016-07-27 11:50:03 -04:00
b1079c9080
revert previous DefaultRetentionPolicyName change
2016-07-27 11:38:42 -04:00
429c4c5c4a
Merge pull request #7069 from influxdata/dgn-fix-create-database
...
fix #7068 : use DefaultRetentionPolicyName from cfg
2016-07-26 19:56:15 -04:00
8fbfdc5616
fix #7068 : use DefaultRetentionPolicyName from cfg
...
Use DefaultRetentionPolicyName from the config instead of passing
through meta data.
2016-07-26 18:53:55 -04:00
a101f52e34
Merge pull request #7067 from influxdata/js-add-quiet-execution-option
...
Add option to suppress logging query statements in the query executor
2016-07-26 15:38:05 -05:00
8e1f238861
Merge pull request #7055 from influxdata/jw-compact-full
...
Prevent full compactions from conflicting with other compactions
2016-07-26 13:46:33 -06:00
7c3d1aac68
Simplify purger.add logic
2016-07-26 13:02:08 -06:00
cab84ae279
Prevent concurrent compactions from stepping on each other
...
Normally, compactions do not conflict on the files they are compacting.
If the full cold threshold is set very low, it can cause conflicts where
two compactions compact the same files. The full compaction was the
only place this could happen as it's planning is greedy.
To make this safer for concurrent execution, the compaction tracks which
files are current being compacted and prevents any new compactions from
starting if the file set overlaps.
Fixes #6595
2016-07-26 12:58:25 -06:00
ded6e40d47
Remove lastPlanCheck var
...
This causes full compactions to not run if the server is running, but
after a restart they do run.
2016-07-26 12:58:25 -06:00
2f78c4ec83
Fix race when creating temp file
...
Using os.O_EXCL is safer than checking and then creating the file.
2016-07-26 12:58:25 -06:00
7b8bcfb173
Add option to suppress logging query statements in the query executor
2016-07-26 13:53:45 -05:00
115fe89d65
Merge pull request #7047 from influxdata/jw-tsminspect
...
Add report command to influx_inspect
2016-07-26 11:34:12 -06:00
6bee354530
Add Load Time column
2016-07-26 10:23:45 -06:00
78cfe441aa
Skip TSM files that fail to load
...
This can be used a quick way to determine if any TSM files are
corrupted.
2016-07-26 10:23:44 -06:00
5bdc01a3b2
Replace info w/ report command
...
info was not safe to run and not really useful.
2016-07-26 10:23:44 -06:00
bfe1710d72
Merge pull request #7063 from influxdata/jw-heap-template
...
Add heap profile to issue template
2016-07-26 10:21:13 -06:00
fadc227c93
Flush tab writer after each row
...
tab writer was flushed after all the rows were processed which consumed
a lot of memory and was very slow.
2016-07-26 10:17:58 -06:00
4543724eb5
Add heap profile to issue template
2016-07-26 09:54:46 -06:00
7ec80e05ce
update influx inspect readme with DDL information
2016-07-25 14:09:53 -05:00
ec34549ef7
Merge pull request #7056 from influxdata/jw-template
...
Add additional debug commands
2016-07-25 12:08:58 -06:00
2c1a65f47f
Merge pull request #7050 from influxdata/cjl-dep-updates
...
update go dependencies
2016-07-25 08:52:24 -05:00
b20bbf5064
Add additional debug commands
2016-07-24 22:33:59 -06:00
71085e9a39
update changelog
2016-07-22 14:33:20 -05:00
063675b928
updates to make snappy compression tests work again
2016-07-22 14:33:20 -05:00
Jason Wilder
Jason Wilder
Cory LaNou
Jonathan A. Sternberg
Edd Robinson
Cameron Sparr
Mark Rushakoff
Mark Rushakoff
David Norton
Ben Johnson