Commit Graph

12749 Commits (081d0649a618995ceba6551c729ffd9df2a906fa)

Author SHA1 Message Date
Jonathan A. Sternberg abf2649d3a Update influx inspect to support unsigned types 2017-10-05 10:26:14 -05:00
Jonathan A. Sternberg 2f47c3d28f Add support for uint64 in the clients 2017-10-05 09:35:06 -05:00
Jonathan A. Sternberg b5a248802e Merge pull request #8917 from influxdata/js-remove-makefile
Remove the unused Makefile
2017-10-05 09:13:03 -05:00
Jason Wilder 00a403f60e Reduce allocation in tsmKeyIterator.Next
This reuses some intermediate buffers and structs while compacting
files.
2017-10-04 17:35:56 -06:00
Mark Rushakoff b18561cd9d Merge pull request #8920 from influxdata/master_namingfix
Master namingfix
2017-10-04 13:50:24 -07:00
Adam 5f5fda5e21 fixed merge error 2017-10-04 16:29:05 -04:00
Adam 3e6b3fd173 fix naming bug 2017-10-04 16:26:03 -04:00
Adam 74729e9e97 Master namingfix (#8916)
* Modified package script to use names similar to build.py
2017-10-04 15:13:24 -04:00
Jonathan A. Sternberg 5205e182a7 Remove the unused Makefile
The Makefile has not been used by the project since before 1.0 and is
not maintained. Removing it so it doesn't send the wrong signal to
potential developers.
2017-10-04 13:32:52 -05:00
Adam bb6c4fe75e took -oss out of tar.gz names 2017-10-04 14:24:23 -04:00
Adam 135a900391 Modified package script to use names similar to build.py 2017-10-04 14:21:18 -04:00
Jason Wilder e0746762d9 Merge pull request #8914 from influxdata/jw-shutdown-panic
Wait for compaction goroutines to finish
2017-10-04 10:56:07 -06:00
Jason Wilder 6b6ccf1a40 Wait for compaction gorotuines to finish 2017-10-04 10:01:44 -06:00
Jason Wilder 7e2b22f8d4 Merge pull request #8886 from influxdata/jw-planner
Improved compaction scheduling
2017-10-04 09:03:59 -06:00
Mark Rushakoff dc9e43101f Merge pull request #8911 from influxdata/mr-releng-manpages
Include man pages in releng packaging
2017-10-04 07:47:21 -07:00
Jason Wilder 06226d6fd3 Handle orphan lower level TSM files during full planning
Some files seem to get orphan behind higher levels.  This causes
the compactions to get blocked as the lowere level files will not
get picked up by their lower level planners.  This allows the full
plan to identify them and pull them into their plans.
2017-10-04 08:13:14 -06:00
Jason Wilder a1d0b52897 Allow lower priority compactions to use excess capacity
If there is a backlog of level 3 and 4 compacitons, but few level 1
and 2 compactions, allow them to use some excess capacity.
2017-10-04 08:11:44 -06:00
Mark Rushakoff bb6cb80d45 Update changelog 2017-10-03 16:43:26 -07:00
Mark Rushakoff f8e85e6df5 Include man pages in releng packaging
The *.1.gz files are included into the source tarball, and the .deb and
.rpm files will now install the man pages when the package is installed.

Fixes #8908.
2017-10-03 16:39:30 -07:00
Mark Rushakoff 6160e369b1 Merge pull request #8909 from influxdata/mr-service-var
Use correct variable style in influxdb.service
2017-10-03 12:59:17 -07:00
Jonathan A. Sternberg 920b2bee2e Merge pull request #7949 from lfarkas/master
always force the file inside the tarball owned by root
2017-10-03 14:37:06 -05:00
Mark Rushakoff 0aaede8421 Update CHANGELOG 2017-10-03 11:28:16 -07:00
Mark Rushakoff c2841c8342 Use correct variable style in influxdb.service
From the systemd.service docs:

Use "${FOO}" as part of a word, or as a word of its own, on the command
line, in which case it will be replaced by the value of the environment
variable including all whitespace it contains, resulting in a single
argument. Use "$FOO" as a separate word on the command line, in which
case it will be replaced by the value of the environment variable split
at whitespace, resulting in zero or more arguments.
2017-10-03 11:15:22 -07:00
Edd Robinson c80c3636dd Merge pull request #8904 from influxdata/er-changelog
Update 1.3.3 release date
2017-10-03 18:43:56 +01:00
Mark Rushakoff 276a26656e Merge pull request #8903 from influxdata/mr-systemd-detection
Use `command` instead of `which`
2017-10-03 10:11:12 -07:00
Mark Rushakoff 7525d331d1 Merge branch 'master' into mr-systemd-detection 2017-10-03 09:54:45 -07:00
Jason Wilder a43e652985 Update changelog 2017-10-03 10:49:46 -06:00
Jason Wilder f2a681c4cf Unconditionally remove file when calling Remove 2017-10-03 10:49:17 -06:00
Jason Wilder 0c0505881f Remove multiple file skipping for full compaction planning
This check doesn't make sense for high cardinality data as the files
typically get big and sparse very quickly.  This causes a lot of extra
disk space to be used which is taken up by large indexes and sparse
data.
2017-10-03 10:48:14 -06:00
Jason Wilder 90df803802 Prevent infinite scheduling loop
One shard might be able to run a compaction, but could fail to
limits being hit.  This loop would continue indefinitely as the
same task would continue to be rescheduled.
2017-10-03 10:48:14 -06:00
Jason Wilder 4ff4ba0841 Use first file in generation for level
With higher cardinality or larger series keys, the files can roll
over early which causes them to take longer to be compacted by higher
levels.  This causes larger disk usage and higher numbers of tsm files
at times.
2017-10-03 10:48:14 -06:00
Jason Wilder 71071ed67a Add compaction backlog stat
This gives an indication as to whether compactions are backed up
or not.
2017-10-03 10:48:14 -06:00
Jason Wilder 16ece490ef Reduce allocation in tsmKeyIterator.Next
The chunked slice is unnecessary and we can re-use k.blocks throughout
the compaction.
2017-10-03 10:48:14 -06:00
Jason Wilder 2c5006fccc Rework snapshotting concurrency
This switches the thresholds that are used for writing snapshots
concurrently.  This scales better than the prior model.
2017-10-03 10:48:14 -06:00
Jason Wilder 3af9c7df37 Remove a defer allocation
Shows up under high cardinality compactions.
2017-10-03 10:48:14 -06:00
Jason Wilder 70817350b7 Ensure temp index files are cleaned up on error 2017-10-03 10:48:14 -06:00
Jason Wilder a5afaf7499 Fix cache mem size not including key size 2017-10-03 10:48:14 -06:00
Jason Wilder ae821f4e2d Rework compaction scheduling
This changes the compaction scheduling to better utilize the available
cores that are free.  Previously, a level was planned in its own goroutine
and would kick off a number of compactions groups.  The problem with this
model was that if there were 4 groups, and 3 completed quickly, the planning
would be blocked for that level until the last group finished.  If the compactions
at the prior level are running more quickly, a large backlog could accumlate.

This now moves the planning to a single goroutine that plans each level in
succession and starts as many groups as it can.  When one group finishes,
the planning will start the next group for the level.
2017-10-03 10:48:13 -06:00
Jason Wilder f668b0cc3f Only use O_SYNC for tsm file writing
Doing this for the WAL reduces throughput quite a bit.
2017-10-03 10:48:13 -06:00
Jason Wilder 1610ae5727 Don't return tsm files part of a compaction plan 2017-10-03 10:48:13 -06:00
Joe LeGasse 1525069213 Merge pull request #8892 from influxdata/jl-tag-values
auth: add series auth to 'show tag values'
2017-10-03 08:47:39 -04:00
Edd Robinson defb81413e Merge branch 'master' into er-changelog 2017-10-03 12:20:49 +01:00
Jonathan A. Sternberg 1566deb320 Merge pull request #8902 from influxdata/js-uint-binary-expr
Support unsigned binary math in fields
2017-10-02 20:32:14 -05:00
Jonathan A. Sternberg 79092610c8 Support unsigned binary math in fields
Field math works similar to condition evaluation, but not the exact same
because we have more information to work with in field expressions than
we do in conditional math because fields retain the information about
their source while conditions do not.

The main difference is that you cannot add an unsigned literal to the
output of an integer iterator while you can inside of a condition. You
can perform math on a positive integer literal to an unsigned iterator.
Inside of the condition, we aren't sure if an integer is because of a
literal or because of an iterator so we can't make that distinction.
2017-10-02 17:06:49 -05:00
Jonathan A. Sternberg cd84509313 Merge pull request #8901 from influxdata/revert-8840-bugfix/issues/8813
Revert "Convert duration to string without dot and zeros"
2017-10-02 16:55:32 -05:00
Jonathan A. Sternberg 06994a6585 Revert "Convert duration to string without dot and zeros" 2017-10-02 16:31:03 -05:00
Mark Rushakoff 4d32c157a9 Update changelog 2017-10-02 11:49:16 -07:00
Mark Rushakoff e623b1b16b Use `command` instead of `which`
Some systems don't have `which` installed.

Also fix some whitespace issues.

Fixes #8900.
2017-10-02 11:49:08 -07:00
Lyon Hill 9cf517bcdf Merge pull request #8896 from influxdata/lh-batcher-leak
Fix a minor memory leak when batching points for some services.
2017-10-02 12:26:58 -06:00
Lyon Hill 7e5fd14e8a add in some optimization 2017-10-02 12:02:38 -06:00