9ee305f6f5
Periodically re-allocate cache store
...
This perioically re-allocates the cache store to avoid memory
fragmentation and gradual slow down of the store after repeated
deletes and inserts into the map.
2017-09-19 15:27:25 -06:00
2885b9b310
Remove entrySizeHints map
...
There is a lot of overhead for calculating the hints for larger
cardinalities. This slows down resetting the partitions in the ring.
2017-09-19 15:27:25 -06:00
4124a8ed97
Simplify cache ring
...
The continuum slice is not needed since the number of partitions
doesn't change. This removes the slice to make the mapping simpler.
2017-09-19 15:27:25 -06:00
65bcddf6c0
Merge pull request #8855 from influxdata/sgc-fix-findrange
...
fix FindValues panic for empty array
2017-09-19 14:26:45 -07:00
ed7bc9d825
fix FindValues panic for empty array
2017-09-19 14:23:32 -07:00
cf0a1d9d99
Merge pull request #8853 from influxdata/sgc-readentries-perf
...
Reduce allocations, improve readEntries performance by simplifying loop
2017-09-19 13:28:43 -07:00
976d214a5e
Merge remote-tracking branch 'origin/master' into sgc-readentries-perf
...
# Conflicts:
# CHANGELOG.md
2017-09-19 13:28:23 -07:00
6b1c804815
Merge pull request #8854 from influxdata/js-report-task-killed-status
...
Report the task status for a query
2017-09-19 15:22:47 -05:00
107feebb81
Report the task status for a query
...
This more accurately shows whether or not a query has been killed.
Instead of automatically removing it from the query table when it's
killed even though goroutines and iterators may still be open, it now
marks the process as killed. This should allow us to more accurately
determine if a query has been stalled and is still using resources on
the server.
This is related to #8848 , but not directly connected.
2017-09-19 14:48:56 -05:00
f8688c5588
update CHANGELOG
2017-09-19 11:57:15 -07:00
92756ec0ad
Reduce allocations, improve readEntries performance by simplifying loop
...
* callers of ReadEntries and Key API can cache allocated slice
2017-09-19 11:57:10 -07:00
baa05de3f8
add benchmarks
2017-09-19 11:47:48 -07:00
4272295f07
Remove unused glyphicons package from licenses file.
...
We are no longer using this dependency however we failed
to remove the license file when we stopped using this
with removal of the admin interface.
2017-09-19 13:21:58 -04:00
2ec4f55165
Merge pull request #8851 from influxdata/sgc-values-perf
...
Improve performance of Include and Exclude functions
2017-09-19 10:19:20 -07:00
ec852dbafe
update CHANGELOG
2017-09-19 10:00:11 -07:00
cfc6a1cd9f
implement optimization for Include function
...
```
benchmark old ns/op new ns/op delta
BenchmarkIntegerValues_IncludeNone_1000-8 651 6.69 -98.97%
BenchmarkIntegerValues_IncludeMiddleHalf_1000-8 1131 114 -89.92%
BenchmarkIntegerValues_IncludeFirst_1000-8 638 33.9 -94.69%
BenchmarkIntegerValues_IncludeLast_1000-8 1269 32.2 -97.46%
BenchmarkIntegerValues_IncludeNone_10000-8 7751 6.76 -99.91%
BenchmarkIntegerValues_IncludeMiddleHalf_10000-8 11582 1378 -88.10%
BenchmarkIntegerValues_IncludeFirst_10000-8 7911 43.8 -99.45%
BenchmarkIntegerValues_IncludeLast_10000-8 12442 38.4 -99.69%
```
(cherry picked from commit fb93ad5)
2017-09-19 09:53:28 -07:00
ca40c1ad3c
<type>Values.Exclude function uses binary search and copy builtin
...
```
± benchcmp old.txt new.txt
benchmark old ns/op new ns/op delta
BenchmarkIntegerValues_ExcludeNone_1000-8 1285 7.34 -99.43%
BenchmarkIntegerValues_ExcludeMiddleHalf_1000-8 1258 148 -88.24%
BenchmarkIntegerValues_ExcludeFirst_1000-8 1268 7.51 -99.41%
BenchmarkIntegerValues_ExcludeLast_1000-8 1125 27.7 -97.54%
BenchmarkIntegerValues_ExcludeNone_10000-8 12665 7.31 -99.94%
BenchmarkIntegerValues_ExcludeMiddleHalf_10000-8 12039 976 -91.89%
BenchmarkIntegerValues_ExcludeFirst_10000-8 12663 7.29 -99.94%
BenchmarkIntegerValues_ExcludeLast_10000-8 10990 34.9 -99.68%
```
(cherry picked from commit d7a3c23)
2017-09-19 09:53:26 -07:00
57b0e8efb0
Merge pull request #8844 from influxdata/js-unsigned-iterators
...
Add unsigned iterators for all types
2017-09-18 16:17:48 -05:00
0ef94e0cf0
Add unsigned iterators for all types
...
This allows unsigned data to be queried from the storage engine.
Binary math is not yet implemented for unsigned types.
2017-09-18 15:09:10 -05:00
940da04a34
Merge pull request #8829 from influxdata/jw-mmap
...
Release mmap pages when shard is cold
2017-09-18 12:08:37 -06:00
b8d0aa16c5
Merge pull request #8843 from influxdata/js-remove-unused-casting-code
...
Remove unused casting code from the query engine
2017-09-18 12:53:29 -05:00
31646aae3a
Release mmap pages when shard is cold
...
This instructs the kernel that it can release memory used by mmap'd
TSM files when they are not actively being used. It the mappings are
use, the kernel will fault the pages back in. On linux, this causes
RES memory to drop immediately when run.
2017-09-18 11:51:51 -06:00
5a9553b2c4
Remove unused casting code from the query engine
...
Originally, casting was performed inside of the query engine especially
for call iterators. Currently, the engine takes care of all casting so
we just need to normalize the iterators types for type safety reasons
rather than actual functional reasons.
Removing this code. Cover coverage showed that it was not hit when run
against the actual server. I ran the tests package and got code coverage
of the query package while running the tests in that package.
2017-09-18 12:33:34 -05:00
e39de3e427
Merge pull request #8782 from oiooj/pr-shard-fix
...
Correctly check if the Shard is ready for queries or writes
2017-09-18 18:17:19 +01:00
6bd2ae05d4
Merge pull request #8838 from influxdata/js-uint-literal-support
...
Support uint64 literals in the parser
2017-09-15 12:42:43 -05:00
3c7e62bcd5
Merge pull request #8836 from influxdata/js-csv-support-uint64
...
uint64 support in the csv handler
2017-09-15 12:42:17 -05:00
6e636264f2
Merge pull request #8837 from influxdata/js-unsigned-type-data-type
...
Unsigned data type parsing and prioritization
2017-09-15 12:42:05 -05:00
aa7ae36880
Merge pull request #8826 from influxdata/js-iterator-options-marshaling
...
Fix group by marshaling in the IteratorOptions
2017-09-14 16:46:11 -05:00
5b3b7a5102
Fix group by marshaling in the IteratorOptions
...
Ensure that the marshaling and unmarshaling an IteratorOptions returns
the same thing.
2017-09-14 15:40:54 -05:00
365866ca30
Master 1548: add releng build scripts ( #8803 )
...
Added several docker-based scripts that can be used to build the OSS binaries and Debian and CentOS packages in a clean, standardized environment. Supports Windows, OSX, and Linux on AMD64 as well as multiple chip architectures for linux including i386, arm5, arm6 and arm8. Will also package source code with all necessary dependencies, and run basic unit tests in the same environment.
2017-09-14 13:51:53 -04:00
2228b91b0d
Unsigned data type parsing and prioritization
2017-09-14 12:28:13 -05:00
6e60edc4bd
Merge pull request #8771 from influxdata/js-restore-old-timerange-behavior
...
This restores the old time range behavior
2017-09-14 12:00:50 -05:00
a978b4fd3e
Support uint64 literals in the parser
...
When an integer cannot be parsed, we attempt to parse it as a uint64. If
that succeeds, we then have an unsigned literal that can then be used to
compare unsigned values.
This method allows us not to introduce new syntax to the language and
continue just doing the right thing at the right time. But, it also
delegates a lot of the heavy lifting to implicit casting in Reduce and
Eval.
2017-09-14 11:16:40 -05:00
b1e1a30c16
uint64 support in the csv handler
2017-09-14 10:38:30 -05:00
c8440b4778
This restores the old time range behavior
...
All time ranges are combined with AND regardless of context and
regardless of whether it makes any logical sense.
That was the previous behavior and, unfortunately, a lot of people rely
on it.
2017-09-14 09:10:37 -05:00
253b8c2b35
Merge pull request #8809 from influxdata/js-8788-fix-time-constraints-in-subqueries
...
Fix time constraints in subqueries from the refactor
2017-09-14 09:08:08 -05:00
f3f30726d8
Merge pull request #8796 from influxdata/jw-tsm-index
...
High Cardinality Fixes
2017-09-12 10:48:34 -06:00
7d467c2047
Fix windows unmapping of anonymous index slice
2017-09-12 10:30:10 -06:00
b4b3c159cc
Fixup rebase
2017-09-11 17:04:10 -06:00
d5d9f9acfe
Remove debug line
2017-09-11 15:31:28 -06:00
26f92ce6ac
Remove commented out code
2017-09-11 15:30:05 -06:00
820856347c
Don't use disk temp file for snapshots
2017-09-11 15:29:26 -06:00
4ed9c75896
Fix unmapping anonymous memory slice
2017-09-11 15:29:26 -06:00
97f7857715
Remove mutex on TSMWriter
...
This isn't used by more than one goroutine so locks are unnecessary.
2017-09-11 15:29:26 -06:00
a93a5e9bdf
Include the size of the key in the cache size
2017-09-11 15:29:26 -06:00
38460ec37e
Re-enable compactions during writes
...
A cold shard that suddenly receives a lot of writes could get a very
big cache that takes a long time to snapshot or causes the cache
max memory limit to be hit more quickly. This re-enables the compactions
if necessary during writes so we don't have to wait for the shard monitor
goroutine to re-enable them.
2017-09-11 15:29:26 -06:00
ee4d3c7b3d
Invalidate all bloom filters.
2017-09-11 15:29:26 -06:00
3c2487b97a
Clean up tsi bloom filter invalidation.
2017-09-11 15:29:26 -06:00
6af936ee61
Fix bloom filter invalidation.
2017-09-11 15:29:26 -06:00
a40b2bb210
Simplify bloom filter invalidation.
2017-09-11 15:29:26 -06:00
Jason Wilder
Stuart Carnie
Jonathan A. Sternberg
Ryan Betts
Jason Wilder
Edd Robinson
Adam
Ben Johnson