cf6d4b6e71
Fix building on solaris/illumos
2017-06-21 09:16:45 -06:00
1975940f76
intermediate compaction commit
2017-05-23 08:42:25 -06:00
48a06432df
Add tsi1 bloom filter.
2017-05-23 08:41:31 -06:00
623ff67221
Fix HLL variableLengthList size decoding.
2017-05-19 11:44:25 -06:00
503d41a08f
Add LimitedBytePool for wal buffers
...
This pool was previously a pool.Bytes to avoid repetitive allocations.
It was recently switchted to a sync.Pool because pool.Bytes held onto
very larger buffers at times which were never released. sync.Pool is
showing up in allocation profiles quite frequently.
This switches the pool to a new pool that limits how many buffers are
in the pool as well as the max size of each buffer in the pool. This
provides better bounds on allocations.
2017-05-11 11:27:00 -06:00
0d74497abe
Reset rhh map elements to reuse allocations.
2017-04-04 11:57:37 -06:00
95d4016ff2
Merge branch 'tsi' of https://github.com/influxdata/influxdb into tsi-series-block-partitioning
2017-04-04 10:14:03 -06:00
bf49b176f5
Partition tsi1 series index.
2017-04-04 09:46:04 -06:00
8da84e6144
Merge branch 'master' into tsi
2017-04-03 11:21:02 -06:00
1c4ecb12c1
Don't panic on nil engine
2017-03-22 10:07:29 -06:00
1e9fa7bc2c
Fix 32-bit rhh implementation.
2017-03-21 11:44:13 -06:00
70efc70abe
Reduce lock contention, fix rhh lookup.
2017-03-17 09:44:11 -06:00
0ebbfb8f77
hll: skip recalc of sparseSet if tmpSet is empty
...
```
benchmark old ns/op new ns/op delta
BenchmarkSet_Count/set_size_1000-8 38095 28.3 -99.93%
BenchmarkSet_Count/set_size_5000-8 152052 30.1 -99.98%
BenchmarkSet_Count/set_size_10000-8 50953 54978 +7.90%
BenchmarkSet_Count/set_size_50000-8 32495 31222 -3.92%
BenchmarkSet_Count/set_size_1000000-8 32603 30800 -5.53%
benchmark old allocs new allocs delta
BenchmarkSet_Count/set_size_1000-8 4 0 -100.00%
BenchmarkSet_Count/set_size_5000-8 4 0 -100.00%
BenchmarkSet_Count/set_size_10000-8 0 0 +0.00%
BenchmarkSet_Count/set_size_50000-8 0 0 +0.00%
BenchmarkSet_Count/set_size_1000000-8 0 0 +0.00%
benchmark old bytes new bytes delta
BenchmarkSet_Count/set_size_1000-8 16496 0 -100.00%
BenchmarkSet_Count/set_size_5000-8 16497 0 -100.00%
BenchmarkSet_Count/set_size_10000-8 0 0 +0.00%
BenchmarkSet_Count/set_size_50000-8 0 0 +0.00%
BenchmarkSet_Count/set_size_1000000-8 0 0 +0.00%
```
2017-01-31 08:51:05 -07:00
ab94c1b743
Fixes #7882
2017-01-30 19:12:24 +00:00
695adafc00
Add measurement sketches
2017-01-05 10:15:37 -07:00
1339c7b146
Initialise HLL with error
2017-01-05 10:15:37 -07:00
9f8b206b51
Fix measurement system queries.
2017-01-05 10:15:34 -07:00
fa7239ecad
Fix windows mmap
2017-01-05 10:11:12 -07:00
cb93f10120
Remove per-shard in-memory index.
2017-01-05 10:11:09 -07:00
8863e3c0f3
Refactor tsi1 merge iterators, finish multi-file compaction.
2017-01-05 10:01:25 -07:00
e2c3b52ca4
Adds a custom HyperLogLog++ implementation
2017-01-05 10:00:14 -07:00
149b1cef1d
Fix 32bit overflow; limit capacity
2017-01-05 09:59:10 -07:00
2a5c865b44
Use xxhash
2017-01-05 09:57:35 -07:00
bd8dd9a291
Sketches working
2017-01-05 09:54:04 -07:00
d19fbf5ab4
Wire in HLL estimator
2017-01-05 09:54:03 -07:00
57d0556174
Fix 32-bit issues.
2017-01-05 09:34:37 -07:00
8d40ceb00c
TSI1 Index
2017-01-05 09:34:36 -07:00
3240af07e0
Fix RHH packing.
2017-01-05 09:34:36 -07:00
e25d61e4bd
TagSet writer & reader.
2017-01-05 09:34:36 -07:00
2c34b24f5c
Implemented SeriesList
2017-01-05 09:34:36 -07:00
6523675c20
Implemented RHH hash map.
2017-01-05 09:34:35 -07:00
0551d87ddb
Update godoc for pkg
2016-12-30 18:03:01 -08:00
fa2b78d67e
Optimize string escaping & unescaping
...
benchmarks pre-commit:
go test ./pkg/escape/... -bench=. -benchmem
BenchmarkStringEscapeNoEscapes-8 10000000 122 ns/op 0 B/op 0 allocs/op
BenchmarkStringUnescapeNoEscapes-8 300000000 5.69 ns/op 0 B/op 0 allocs/op
BenchmarkManyStringEscape-8 1000000 1073 ns/op 320 B/op 8 allocs/op
BenchmarkManyStringUnescape-8 1000000 1557 ns/op 288 B/op 8 allocs/op
PASS
ok github.com/influxdata/influxdb/pkg/escape 6.304s
benchmarks post-commit:
go test ./pkg/escape/... -bench=. -benchmem
BenchmarkStringEscapeNoEscapes-8 100000000 17.5 ns/op 0 B/op 0 allocs/op
BenchmarkStringUnescapeNoEscapes-8 300000000 5.70 ns/op 0 B/op 0 allocs/op
BenchmarkManyStringEscape-8 3000000 451 ns/op 224 B/op 6 allocs/op
BenchmarkManyStringUnescape-8 2000000 944 ns/op 304 B/op 9 allocs/op
PASS
ok github.com/influxdata/influxdb/pkg/escape 8.727s
2016-11-24 10:25:35 +00:00
3fe0ffd5ad
String escape unit tests
2016-11-18 16:02:52 +00:00
743946fafb
models: Add FieldIterator type
...
The FieldIterator is used to scan over the fields of a point, providing
information, and delaying parsing/decoding the value until it is needed.
This change uses this new type to avoid the allocation of a map for the
fields which is then thrown away as soon as the points get converted
into columns within the datastore.
2016-10-03 16:30:21 -06:00
f632b41f6a
Update godoc
2016-09-26 12:19:15 -06:00
7f96d78b79
Make encoder re-usable
...
This allows encoders to be re-used and maintained in a pool to
avoid allocating new ones on every compactions and write of an encoded
block. The pool used is not a sync.Pool to ensure that the encoders
will not be garbage collected.
2016-09-26 12:19:15 -06:00
25508fa05f
Add pool.Bytes for re-using byte slices
...
This adds a new pool type that allows byte slices
to be re-used across clients. A sync.Pool can't be used in some
cases because the the slices not in use end up getting garbage
collected due to frequent GCs.
2016-09-26 12:19:15 -06:00
b692ef4f48
Rename throttle package to limiter
2016-07-18 12:00:58 -06:00
c2370b437b
Limit in-flight wal writes/encodings
...
A slower disk can can cause excessive allocations to occur when
writing to the WAL because the slower encoding and compression occurs
before taking the write lock. The encoding/compression grabs a large
byte slice from a pool and ultimately waits until it can acquire the
write lock.
This adds a throttle to limit how many inflight WAL writes can be queued
up to prevent OOMing the processess with slower disks and heavy writes.
2016-07-17 23:53:12 -06:00
84c776bf2a
Skip allocating string in unescapeString
2016-05-27 10:30:08 -06:00
4b32760dac
Use stdlib func instead of scanning slices
2016-04-20 16:08:58 -06:00
fde5576d64
Avoid allocations when no escape chars present
2016-04-20 16:00:26 -06:00
287b7173db
include missing license for equal.go
2016-04-06 17:08:13 -04:00
d9a6a7340f
add canonical paths
2016-02-10 11:30:52 -07:00
3dd6aa17f3
Test the merge iterator for every type instead of just floats
2016-02-10 09:40:26 -07:00
cde973f409
refactor query engine
2016-02-10 09:40:24 -07:00
a8c285d265
Correctly name pkg README
...
[ci skip]
2015-12-17 11:24:58 -08:00
d36a610a06
Complete lint for pkg/slices
2015-11-09 20:21:03 +01:00
7491564f9d
Added tests for esape.Unescape
2015-10-16 10:14:47 +11:00
Jason Wilder
Ben Johnson
Edd Robinson
Stuart Carnie
Mark Rushakoff
Cameron Sparr
Joe LeGasse
aviau
Jonathan A. Sternberg
Philip O'Toole
Steffen Windoffer
David Howden