Commit Graph

10950 Commits (fcd425c8c62ccce9e549be7705d5a29c668e10f2)

Author SHA1 Message Date
Jason Wilder e8e6bc44a7 Remove defers in TSM reader read path 2016-08-02 16:39:45 -06:00
Jonathan A. Sternberg c65693a34b Merge pull request #7032 from influxdata/js-fix-influx-stress-concurrent-write-panic
Copy tags in influx_stress to avoid a concurrent write panic on a map
2016-08-01 13:39:59 -05:00
Jonathan A. Sternberg ccd1bd91f4 Copy tags in influx_stress to avoid a concurrent write panic on a map
Removing lock from the same object as that object never does anything
except for having attributes read.
2016-08-01 13:18:21 -05:00
Jonathan A. Sternberg 155ef811ad Merge pull request #6547 from influxdata/js-6543-parse-fill-bad-query
Fix parseFill to check for fill ident before attempting to parse an expression
2016-08-01 11:52:01 -05:00
Jonathan A. Sternberg 2c739c0532 Fix parseFill to check for fill ident before attempting to parse an expression
The previous parseFill would try to parse an expression and only unscan
one token when it failed. This caused it to not put back the correct
number of tokens with some expression.

Now it has been modified to check for the fill ident ahead of time and
then use ParseExpr() to parse the call. If the expression fails to parse
into a call, it will send an error instead of trying to continue with an
invalid parser state.

Fixes #6543.
2016-08-01 11:38:44 -05:00
David Norton 0c4559722c feat #6679: add series limit config setting 2016-08-01 08:28:46 -04:00
Jonathan A. Sternberg cac6765364 Merge pull request #7098 from influxdata/js-use-proper-header-for-content-type
Use the Accept header for determining the Content-Type instead of Content-Type
2016-07-29 15:00:13 -05:00
Jonathan A. Sternberg 73bbbf24b1 Use the Accept header for determining the Content-Type instead of Content-Type
The `Content-Type` header is meant to say what the content type of the
request is supposed to be and `Accept` is used to ask for a specific
content type. I messed this up and used `Content-Type` instead of
`Accept`. This works with `GET` requests accidentally, but `POST`
requests stop working.
2016-07-29 14:29:39 -05:00
Jonathan A. Sternberg 4a89ab4482 Merge pull request #6815 from influxdata/js-content-type-response-writer
Setup an interface for writing http responses based on the Content-Type header
2016-07-29 13:28:37 -05:00
Jonathan A. Sternberg d85072ada9 Setup an interface for writing http responses based on the Content-Type header
Only `application/json` is supported right now, but this opens up the
easier possibility of additional content types to be returned from the
server.
2016-07-29 13:11:51 -05:00
Cory LaNou ac0228614d Merge pull request #7077 from influxdata/cjl-7065-remove-exists
Remove IF EXISTS/IF NOT EXISTS from influxql language
2016-07-29 13:06:54 -05:00
Cory LaNou 1117526873 remove IF EXISTS/IF NOT EXISTS from influxql language 2016-07-29 12:58:05 -05:00
Jason Wilder 37674d2c4a Merge pull request #7084 from influxdata/jw-tombstones
Tombstone memory improvements
2016-07-28 20:45:35 -06:00
Jason Wilder 5576e7fedb Simplifications 2016-07-28 20:25:37 -06:00
Jason Wilder c1840be88d Update changelog 2016-07-28 20:25:37 -06:00
Jason Wilder 8367771d35 Fix go vet 2016-07-28 20:25:37 -06:00
Jason Wilder 030f1ef622 Include full for tombstone files
The path info only contained the file name which caused tombstone
files to not be removed if there were queries running against
a file that was compacted.

This is now consistent with the TSMReader.Path which returns the
full path info.
2016-07-28 20:25:37 -06:00
Jason Wilder c3fda24cf9 Make sure all in-use files are tracked
break cause the first one to be tracked and all others would
leak as temp files that would not be removed until the server
restarted.
2016-07-28 20:25:37 -06:00
Jason Wilder c1a94e8861 Remove temp TSM files when disabling compactions
If they were left around, re-enabling them again could cause
future compactions to continuously fail.  A restart of the
server would clean them up correctly though.
2016-07-28 20:25:37 -06:00
Jason Wilder 602a2e80ce Ensure aux and cond cursors are closed when iterator is closed 2016-07-28 20:25:37 -06:00
Jason Wilder 0b60862248 Close drained iterators
Aux and condition iterators where not closed which could
cause TSM files to leak if they were queried against while
a compaction was running.
2016-07-28 20:25:37 -06:00
Jason Wilder 5764a730d5 Prevent tombstoning series keys more than once
If there were multiple TSM files and a delete/drop was run,
we would write the delete series to the tombstone file N
times for each file.  This occurred because FileStore.WalkKeys walks
every key in every TSM file which can return duplicate keys.

This issue caused TSM files to be much larger than they should be
and also cause large memory usage during the delete.
2016-07-28 20:25:36 -06:00
Jason Wilder ef8ecf0e90 Apply reload tombstones in batches
This keeps some memory bounds when reloading a TSM files tombstones
so that the heap does not grow exceedintly fast and stay there
after the deletes are applied.
2016-07-28 20:25:36 -06:00
Jason Wilder 4436e65fb9 Apply deletes to TSM files concurrently 2016-07-28 20:25:36 -06:00
Jason Wilder a8c69e222a Use scanner for reading v1 tombstones
Use a bufio.Scanner to read v1 tombstones instead of reading in
the whole file and parsing it from memory.
2016-07-28 20:25:36 -06:00
Jason Wilder 7b8959f6f2 Apply tombstones iteratively at startup
Tombstone were read fully into memory at startup which could consume
a lot of RAM and OOM the process if there were a lot of deleted
series and many TSM files.

This now walks the tombstone file and iteratively applies the tombstone
which uses significantly less RAM.  This may be slightly slower in the
generate cause, but should scale better.
2016-07-28 20:25:36 -06:00
Jason Wilder 644693ad4f Merge pull request #7086 from influxdata/jw-6990
Fix panic with parsing empty key
2016-07-28 19:29:32 -06:00
Jason Wilder d432aaa84d Fix panic with parsing empty key
Fixes #6990
2016-07-28 18:38:17 -06:00
Cory LaNou ae4a1a4d00 Merge pull request #7049 from influxdata/cjl-influx-inspect-doc-update
Update influx inspect readme with DDL information
2016-07-28 18:19:12 -05:00
Jonathan A. Sternberg 94e07158d5 Merge pull request #7009 from influxdata/js-5750-wildcard-support-for-aggregates
Support wildcards in aggregate functions
2016-07-28 18:13:59 -05:00
Jonathan A. Sternberg 86364a85b4 Merge pull request #7090 from influxdata/js-show-measurements-and-tag-values-refactor
Switch SHOW MEASUREMENTS and SHOW TAG VALUES to directly access the tsdb.Store
2016-07-28 18:04:13 -05:00
Jonathan A. Sternberg c657f48b06 Merge pull request #7092 from influxdata/js-update-dependency-license
Update LICENSE_OF_DEPENDENCIES.md with github.com/retailnext/hllpp
2016-07-28 18:04:03 -05:00
Jonathan A. Sternberg 23ef9484a4 Support wildcards in aggregate functions 2016-07-28 17:56:32 -05:00
Jonathan A. Sternberg 86bd97f3b9 Switch SHOW MEASUREMENTS and SHOW TAG VALUES to directly access the tsdb.Store
The `SHOW MEASUREMENTS` and `SHOW TAG VALUES` cannot go through the
query engine to get the speed they need. They also only need access to
the database index and do not need access to specific shards. This
removes the query rewriting that was done to turn these two queries into
a select statement and reimplements them inside of the coordinator as an
interface on the TSDBStore.
2016-07-28 17:38:11 -05:00
Jonathan A. Sternberg 839a567604 Update LICENSE_OF_DEPENDENCIES.md with github.com/retailnext/hllpp
Also reorganizing Godeps so it lists the dependencies in alphabetical
order.
2016-07-28 17:22:54 -05:00
Edd Robinson a1c7d9c8f2 Merge pull request #7080 from influxdata/er-cq-delimiter
Ensure CQ ID names can't clash
2016-07-28 22:00:34 +01:00
Edd Robinson 9a2efafa13 Ensure CQ id names can't clash 2016-07-28 16:26:05 +01:00
Cameron Sparr d688676266 Update UDP buffer setting doc to 25MB 2016-07-28 12:02:56 +01:00
Cameron Sparr 6afdcf9572 Fix incrementing pointsRx twice 2016-07-28 11:06:50 +01:00
Mark Rushakoff d6a726b5e8 Merge pull request #7085 from influxdata/mr-fix-series-keys-length
Fix length of (*tsdb.DatabaseIndex).SeriesKeys()
2016-07-27 16:25:40 -07:00
Mark Rushakoff f34a7430e3 Fix length of (*DatabaseIndex).SeriesKeys()
Previously, it would return as many empty strings in the first half of
the slice as valid values at the end of the slice.
2016-07-27 16:07:39 -07:00
David Norton b2725ca9d4 Merge pull request #7082 from influxdata/dgn-hardcode-rp-name
fix #7081: hardcode auto gen RP name
2016-07-27 17:25:34 -04:00
David Norton e3328117a9 fix #7081: hardcode auto gen RP name 2016-07-27 16:49:46 -04:00
Joe LeGasse b3341e8f66 Stablize the sorting of meta.ShardGroupInfos 2016-07-27 16:17:52 -04:00
Ben Johnson 19ff8a1b67 Merge pull request #7033 from benbjohnson/create-iterator-check
Check for nil iterator creation
2016-07-27 16:15:48 -04:00
Ben Johnson 5df6f75545
check for nil iterator creation
This commit checks if an iterator is `nil` before adding to an
iterator list during creation.
2016-07-27 13:54:56 -06:00
David Norton 1579dd1e75 Merge pull request #7078 from influxdata/dgn-revert-change
revert previous DefaultRetentionPolicyName change
2016-07-27 11:50:03 -04:00
David Norton b1079c9080 revert previous DefaultRetentionPolicyName change 2016-07-27 11:38:42 -04:00
David Norton 429c4c5c4a Merge pull request #7069 from influxdata/dgn-fix-create-database
fix #7068: use DefaultRetentionPolicyName from cfg
2016-07-26 19:56:15 -04:00
David Norton 8fbfdc5616 fix #7068: use DefaultRetentionPolicyName from cfg
Use DefaultRetentionPolicyName from the config instead of passing
through meta data.
2016-07-26 18:53:55 -04:00