1e345aa7a1
Merge pull request #8856 from influxdata/jw-cache
...
Snapshot compaction improvements
2017-09-22 10:45:54 -06:00
ccb6f7451f
Ensure importer emits errors to stderr ( #8833 )
2017-09-22 17:17:20 +01:00
e0caa2e0fd
Merge pull request #8696 from influxdata/er-8678-tsi1-where
...
Allow tag filtering when using DELETE with tsi1
2017-09-22 17:16:09 +01:00
44691847e9
Merge branch 'master' into er-8678-tsi1-where
2017-09-22 16:54:49 +01:00
80aa328be6
small bugfix in filename check for 'static' keywork ( #8861 )
2017-09-22 10:33:12 -04:00
94aba64b88
Re-use index entries slice when writing TSM index
2017-09-21 12:48:16 -06:00
db204f3eb7
Default concurrent compactions to 50% of available cores
2017-09-21 12:48:11 -06:00
5eb79f438e
Merge pull request #8852 from influxdata/rb-8816-glyphicons
...
Remove unused glyphicons package from licenses file.
2017-09-21 12:19:36 -04:00
fba7c9fc5e
Updated packaging script to better handle static packaging ( #8859 )
2017-09-20 14:04:01 -04:00
deef0c5649
Fix 32bit alignment
2017-09-20 10:00:20 -06:00
61ca1243c7
Increase index disk writer buffer
2017-09-20 09:05:30 -06:00
796de3dcea
Reduce encoder pool checkout contention
...
With higher cardinalities, the encoder pools where become a bottleneck.
This changes the snapshot compactions ot checkout one encoder of each
type and re-use it while writing the snapshots as opposed to repeatedly
checking it out and in.
2017-09-19 15:27:26 -06:00
391a6288c6
Write parallel snapshot for higher cardinalities
2017-09-19 15:27:26 -06:00
0d52b060df
Skip onFileStoreReplace with tsi
2017-09-19 15:27:25 -06:00
4fe81aeee6
Remove manual Gosched from compactions
...
At higher cardinalities, this dramatically slows down compaction throughput.
2017-09-19 15:27:25 -06:00
31e785d676
Don't deduplicate a single value
2017-09-19 15:27:25 -06:00
2ca9ccee1f
Reset snapshot cache outside of write lock
2017-09-19 15:27:25 -06:00
ddeba2c86b
Split large snapshots and write concurrently
2017-09-19 15:27:25 -06:00
9ee305f6f5
Periodically re-allocate cache store
...
This perioically re-allocates the cache store to avoid memory
fragmentation and gradual slow down of the store after repeated
deletes and inserts into the map.
2017-09-19 15:27:25 -06:00
2885b9b310
Remove entrySizeHints map
...
There is a lot of overhead for calculating the hints for larger
cardinalities. This slows down resetting the partitions in the ring.
2017-09-19 15:27:25 -06:00
4124a8ed97
Simplify cache ring
...
The continuum slice is not needed since the number of partitions
doesn't change. This removes the slice to make the mapping simpler.
2017-09-19 15:27:25 -06:00
65bcddf6c0
Merge pull request #8855 from influxdata/sgc-fix-findrange
...
fix FindValues panic for empty array
2017-09-19 14:26:45 -07:00
ed7bc9d825
fix FindValues panic for empty array
2017-09-19 14:23:32 -07:00
cf0a1d9d99
Merge pull request #8853 from influxdata/sgc-readentries-perf
...
Reduce allocations, improve readEntries performance by simplifying loop
2017-09-19 13:28:43 -07:00
976d214a5e
Merge remote-tracking branch 'origin/master' into sgc-readentries-perf
...
# Conflicts:
# CHANGELOG.md
2017-09-19 13:28:23 -07:00
6b1c804815
Merge pull request #8854 from influxdata/js-report-task-killed-status
...
Report the task status for a query
2017-09-19 15:22:47 -05:00
107feebb81
Report the task status for a query
...
This more accurately shows whether or not a query has been killed.
Instead of automatically removing it from the query table when it's
killed even though goroutines and iterators may still be open, it now
marks the process as killed. This should allow us to more accurately
determine if a query has been stalled and is still using resources on
the server.
This is related to #8848 , but not directly connected.
2017-09-19 14:48:56 -05:00
f8688c5588
update CHANGELOG
2017-09-19 11:57:15 -07:00
92756ec0ad
Reduce allocations, improve readEntries performance by simplifying loop
...
* callers of ReadEntries and Key API can cache allocated slice
2017-09-19 11:57:10 -07:00
baa05de3f8
add benchmarks
2017-09-19 11:47:48 -07:00
4272295f07
Remove unused glyphicons package from licenses file.
...
We are no longer using this dependency however we failed
to remove the license file when we stopped using this
with removal of the admin interface.
2017-09-19 13:21:58 -04:00
2ec4f55165
Merge pull request #8851 from influxdata/sgc-values-perf
...
Improve performance of Include and Exclude functions
2017-09-19 10:19:20 -07:00
ec852dbafe
update CHANGELOG
2017-09-19 10:00:11 -07:00
cfc6a1cd9f
implement optimization for Include function
...
```
benchmark old ns/op new ns/op delta
BenchmarkIntegerValues_IncludeNone_1000-8 651 6.69 -98.97%
BenchmarkIntegerValues_IncludeMiddleHalf_1000-8 1131 114 -89.92%
BenchmarkIntegerValues_IncludeFirst_1000-8 638 33.9 -94.69%
BenchmarkIntegerValues_IncludeLast_1000-8 1269 32.2 -97.46%
BenchmarkIntegerValues_IncludeNone_10000-8 7751 6.76 -99.91%
BenchmarkIntegerValues_IncludeMiddleHalf_10000-8 11582 1378 -88.10%
BenchmarkIntegerValues_IncludeFirst_10000-8 7911 43.8 -99.45%
BenchmarkIntegerValues_IncludeLast_10000-8 12442 38.4 -99.69%
```
(cherry picked from commit fb93ad5)
2017-09-19 09:53:28 -07:00
ca40c1ad3c
<type>Values.Exclude function uses binary search and copy builtin
...
```
± benchcmp old.txt new.txt
benchmark old ns/op new ns/op delta
BenchmarkIntegerValues_ExcludeNone_1000-8 1285 7.34 -99.43%
BenchmarkIntegerValues_ExcludeMiddleHalf_1000-8 1258 148 -88.24%
BenchmarkIntegerValues_ExcludeFirst_1000-8 1268 7.51 -99.41%
BenchmarkIntegerValues_ExcludeLast_1000-8 1125 27.7 -97.54%
BenchmarkIntegerValues_ExcludeNone_10000-8 12665 7.31 -99.94%
BenchmarkIntegerValues_ExcludeMiddleHalf_10000-8 12039 976 -91.89%
BenchmarkIntegerValues_ExcludeFirst_10000-8 12663 7.29 -99.94%
BenchmarkIntegerValues_ExcludeLast_10000-8 10990 34.9 -99.68%
```
(cherry picked from commit d7a3c23)
2017-09-19 09:53:26 -07:00
ff7b576389
Add uint support into the write protocol
...
This is currently protected behind a conditional compilation flag. Use
`-tags uint` or `-tags uint64` to enable this.
2017-09-19 10:44:26 -05:00
57b0e8efb0
Merge pull request #8844 from influxdata/js-unsigned-iterators
...
Add unsigned iterators for all types
2017-09-18 16:17:48 -05:00
0ef94e0cf0
Add unsigned iterators for all types
...
This allows unsigned data to be queried from the storage engine.
Binary math is not yet implemented for unsigned types.
2017-09-18 15:09:10 -05:00
940da04a34
Merge pull request #8829 from influxdata/jw-mmap
...
Release mmap pages when shard is cold
2017-09-18 12:08:37 -06:00
b8d0aa16c5
Merge pull request #8843 from influxdata/js-remove-unused-casting-code
...
Remove unused casting code from the query engine
2017-09-18 12:53:29 -05:00
31646aae3a
Release mmap pages when shard is cold
...
This instructs the kernel that it can release memory used by mmap'd
TSM files when they are not actively being used. It the mappings are
use, the kernel will fault the pages back in. On linux, this causes
RES memory to drop immediately when run.
2017-09-18 11:51:51 -06:00
5a9553b2c4
Remove unused casting code from the query engine
...
Originally, casting was performed inside of the query engine especially
for call iterators. Currently, the engine takes care of all casting so
we just need to normalize the iterators types for type safety reasons
rather than actual functional reasons.
Removing this code. Cover coverage showed that it was not hit when run
against the actual server. I ran the tests package and got code coverage
of the query package while running the tests in that package.
2017-09-18 12:33:34 -05:00
e39de3e427
Merge pull request #8782 from oiooj/pr-shard-fix
...
Correctly check if the Shard is ready for queries or writes
2017-09-18 18:17:19 +01:00
6bd2ae05d4
Merge pull request #8838 from influxdata/js-uint-literal-support
...
Support uint64 literals in the parser
2017-09-15 12:42:43 -05:00
3c7e62bcd5
Merge pull request #8836 from influxdata/js-csv-support-uint64
...
uint64 support in the csv handler
2017-09-15 12:42:17 -05:00
6e636264f2
Merge pull request #8837 from influxdata/js-unsigned-type-data-type
...
Unsigned data type parsing and prioritization
2017-09-15 12:42:05 -05:00
aa7ae36880
Merge pull request #8826 from influxdata/js-iterator-options-marshaling
...
Fix group by marshaling in the IteratorOptions
2017-09-14 16:46:11 -05:00
5b3b7a5102
Fix group by marshaling in the IteratorOptions
...
Ensure that the marshaling and unmarshaling an IteratorOptions returns
the same thing.
2017-09-14 15:40:54 -05:00
da743321d8
Update CHANGELOG
2017-09-14 14:24:49 -04:00
14f3c1e3d1
auth: fix required permissions for SHOW MEASUREMENTS
...
When using `SHOW MEASUREMENTS ON <db>`, the required permissions didn't
take into account `<db>` and would only use the `db` parameters from the
HTTP parameters.
2017-09-14 14:22:37 -04:00
Jason Wilder
Edd Robinson
Adam
Jason Wilder
Ryan Betts
Stuart Carnie
Jonathan A. Sternberg
Joe LeGasse