Influxdata
/
influxdb
mirror of https://github.com/influxdata/influxdb.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
31,662 Commits
234 Branches
453 Tags
344 MiB
Commit Graph

249 Commits (b50eeaf6be63e293108d8cc3057608946cc26b3c)

Author SHA1 Message Date
Edd Robinson 2471c2468c fix(storage): fixes panic when building predicates 
Fixes #15916.

If a predicate was passed in with multiple key/value matches for the
same tag key, then the value index would be incorrect. This ensures that
each tag key can only be added to the location map once.
 2019-11-15 15:07:36 +00:00
George 3804d50fbd
fix(storage): array cursor iterator should return stats of all observed cursors (#15731) 
* fix(storage): add failing test for array cursor iterator stats

* fix(storage): make arrayCursorIterator.Stats() return stats of in-focus cursor

* fix(storage): add failing test to assert arrayCursorIterator.Stats() returns accumulated result

* fix(storage): assumulate stats in arrayCursorIterator.Stats() call across all observed cursors
 2019-11-05 10:41:06 +01:00
Edd Robinson 2727ae3c25 refactor: simpify Semaphore interface 2019-10-23 19:49:48 +01:00
Edd Robinson b6e911d72c refactor: move goroutine out to function 2019-10-23 19:49:46 +01:00
Edd Robinson 8f6701d4b1 feat(storage): add full compaction semaphore 
By default this feature is disabled; the full compaction behaviour does
not change. When this feature is enabled compactions can be limited
across multiple storage engines running in multiple processes.

The mechanism by which this happens is not part of the abstraction added
here.
 2019-10-23 19:45:01 +01:00
Edd Robinson ef1e15a0ad
Merge pull request #15318 from influxdata/er-mv-comp-limiter 
feat(storage): allow compaction limiter to be injected into engine
 2019-10-09 13:11:44 +01:00
Ilya Sevostyanov 596414a3ff
fix(storage): added missing string values for CacheStatus type. 
Closes: #15284.
 2019-10-04 23:50:21 +03:00
Edd Robinson 179c57ab2e feat(storage): allow compaction limiter to be injected 2019-10-04 12:35:21 -07:00
elbehery 663d4bb901 test(tasks): skip flaky test 2019-09-25 18:17:59 +02:00
elbehery c0b87c657c fix(storage): remove level=0 from TSM disk bytes metrics. 2019-09-25 15:57:25 +02:00
Lorenzo Affetti 053836e5a5
Merge pull request #15203 from influxdata/flux-staging-v0.48.x 
build(flux): update to Flux v0.48.0
 2019-09-20 18:24:02 +02:00
Edd Robinson d714be45a4
Merge pull request #15200 from influxdata/er-retention-service 
refactor(storage): add more context to traces and logs
 2019-09-20 09:00:00 +01:00
Lorenzo Affetti ab835c8e0e
refactor(dependencies): use new dependency injection framework (#15174) 
refactor(dependencies): use new dependency injection framework
 2019-09-19 17:01:17 +02:00
Edd Robinson e2f5b2bd9d refactor(storage): add more context to traces and logs 2019-09-19 13:48:06 +01:00
Stuart Carnie 9a89900785
fix(tsm1): Fix duplicate points 
All seeks must be added to the c.current slice so the
min and max read values can be updated on each read pass.
 2019-09-18 17:44:27 -07:00
Edd Robinson 7efb73930b refactor: address PR feedback 2019-08-30 21:07:32 +01:00
Edd Robinson 2e5ebbe251 perf(storage): reduce allocations when deleting from cache 
When deleting from the cache, each cache key must be checked to
determine if it matches the prefix we're deleting. Since the keys are
stored as strings in the cache (map keys) there were a lot of allocations
happening because `applySerial` expects `[]byte` keys.

It's beneficial to reduce allocations by refacting `applySerial` to work
on strings. Whilst some allocations now have to happen the other way
(string -> []byte), they only happen if we actually need to delete the
key from the cache. Most of the keys don't get deleted so it's better
doing it this way.

Performance on the benchmark from the previous commit improved by ~40-50%.

name                                          old time/op    new time/op    delta
Engine_DeletePrefixRange_Cache/exists-24         102ms ±11%      59ms ± 3%  -41.95%  (p=0.000 n=10+8)
Engine_DeletePrefixRange_Cache/not_exists-24    97.1ms ± 4%    45.0ms ± 1%  -53.66%  (p=0.000 n=10+10)

name                                          old alloc/op   new alloc/op   delta
Engine_DeletePrefixRange_Cache/exists-24        25.5MB ± 1%     3.1MB ± 2%  -87.83%  (p=0.000 n=10+10)
Engine_DeletePrefixRange_Cache/not_exists-24    23.9MB ± 1%     0.1MB ±86%  -99.65%  (p=0.000 n=10+10)

name                                          old allocs/op  new allocs/op  delta
Engine_DeletePrefixRange_Cache/exists-24          305k ± 1%       28k ± 1%  -90.77%  (p=0.000 n=10+10)
Engine_DeletePrefixRange_Cache/not_exists-24      299k ± 1%        1k ±63%  -99.74%  (p=0.000 n=9+10)

Raw benchmarks on a 24T/32GB/NVME machine are as follows:

goos: linux
goarch: amd64
pkg: github.com/influxdata/influxdb/tsdb/tsm1
BenchmarkEngine_DeletePrefixRange_Cache/exists-24         	     300	  50379720 ns/op	 3054106 B/op	   27859 allocs/op
BenchmarkEngine_DeletePrefixRange_Cache/exists-24         	     300	  57326032 ns/op	 3124764 B/op	   28217 allocs/op
BenchmarkEngine_DeletePrefixRange_Cache/exists-24         	     300	  58943855 ns/op	 3162146 B/op	   28527 allocs/op
BenchmarkEngine_DeletePrefixRange_Cache/exists-24         	     300	  60565115 ns/op	 3138811 B/op	   28176 allocs/op
BenchmarkEngine_DeletePrefixRange_Cache/exists-24         	     200	  59775969 ns/op	 3087910 B/op	   27921 allocs/op
BenchmarkEngine_DeletePrefixRange_Cache/exists-24         	     300	  59530451 ns/op	 3120986 B/op	   28207 allocs/op
BenchmarkEngine_DeletePrefixRange_Cache/exists-24         	     300	  59185532 ns/op	 3113066 B/op	   28302 allocs/op
BenchmarkEngine_DeletePrefixRange_Cache/exists-24         	     300	  59295867 ns/op	 3100832 B/op	   28108 allocs/op
BenchmarkEngine_DeletePrefixRange_Cache/exists-24         	     300	  59599776 ns/op	 3100686 B/op	   28113 allocs/op
BenchmarkEngine_DeletePrefixRange_Cache/exists-24         	     200	  62065907 ns/op	 3048527 B/op	   27879 allocs/op
BenchmarkEngine_DeletePrefixRange_Cache/not_exists-24     	     300	  44979062 ns/op	  123026 B/op	    1244 allocs/op
BenchmarkEngine_DeletePrefixRange_Cache/not_exists-24     	     300	  44733344 ns/op	   52650 B/op	     479 allocs/op
BenchmarkEngine_DeletePrefixRange_Cache/not_exists-24     	     300	  44534180 ns/op	   35119 B/op	     398 allocs/op
BenchmarkEngine_DeletePrefixRange_Cache/not_exists-24     	     300	  45179881 ns/op	  105256 B/op	     706 allocs/op
BenchmarkEngine_DeletePrefixRange_Cache/not_exists-24     	     300	  44918964 ns/op	   47426 B/op	     621 allocs/op
BenchmarkEngine_DeletePrefixRange_Cache/not_exists-24     	     300	  45000465 ns/op	   63164 B/op	     564 allocs/op
BenchmarkEngine_DeletePrefixRange_Cache/not_exists-24     	     300	  45332999 ns/op	  117008 B/op	    1146 allocs/op
BenchmarkEngine_DeletePrefixRange_Cache/not_exists-24     	     300	  45652342 ns/op	   66221 B/op	     616 allocs/op
BenchmarkEngine_DeletePrefixRange_Cache/not_exists-24     	     300	  45083957 ns/op	  154354 B/op	    1143 allocs/op
BenchmarkEngine_DeletePrefixRange_Cache/not_exists-24     	     300	  44560228 ns/op	   65024 B/op	     724 allocs/op
PASS
ok  	github.com/influxdata/influxdb/tsdb/tsm1	1690.583s
 2019-08-30 20:35:05 +01:00
Edd Robinson eba4dec7e6 perf(storage): reduce lock contention on Cache entries 
The cache is essentially a set of maps, where a key in each map is a
series key, and the value is a slice of values associated with that key.
The cache is sharded and series keys are hashed to determine which shard
(map) they live in.

When deleting from the cache we have to check each key to see if it
matches the delete command (predicate and timestamp). If it does then
the entries for that range are removed. As part of this work we check if
the entries are already empty (already removed) and if so we don't check
if the key is valid.

This involved a lot of mutex grabbing, which has now been replaced with
atomic operations.

Benchmarking this commit against the previous commit in this branch
shows a 9% improvement:

name                                          old time/op    new time/op    delta
Engine_DeletePrefixRange_Cache/exists-24         113ms ± 8%     102ms ±11%   -9.40%  (p=0.000 n=10+10)
Engine_DeletePrefixRange_Cache/not_exists-24    95.6ms ± 2%    97.1ms ± 4%     ~     (p=0.089 n=10+10)

name                                          old alloc/op   new alloc/op   delta
Engine_DeletePrefixRange_Cache/exists-24        29.6MB ± 1%    25.5MB ± 1%  -13.71%  (p=0.000 n=10+10)
Engine_DeletePrefixRange_Cache/not_exists-24    24.3MB ± 2%    23.9MB ± 1%   -1.48%  (p=0.000 n=10+10)

name                                          old allocs/op  new allocs/op  delta
Engine_DeletePrefixRange_Cache/exists-24          334k ± 0%      305k ± 1%   -8.67%  (p=0.000 n=8+10)
Engine_DeletePrefixRange_Cache/not_exists-24      302k ± 1%      299k ± 1%   -1.25%  (p=0.000 n=10+9)

Raw benchmarks on a 24T / 32GB / NVME machine:

goos: linux
goarch: amd64
pkg: github.com/influxdata/influxdb/tsdb/tsm1
BenchmarkEngine_DeletePrefixRange_Cache/exists-24         	     200	  91035525 ns/op	25557809 B/op	  305258 allocs/op
BenchmarkEngine_DeletePrefixRange_Cache/exists-24         	     200	  99416796 ns/op	25385052 B/op	  303584 allocs/op
BenchmarkEngine_DeletePrefixRange_Cache/exists-24         	     100	 100149484 ns/op	25570062 B/op	  305761 allocs/op
BenchmarkEngine_DeletePrefixRange_Cache/exists-24         	     100	 100222516 ns/op	25474372 B/op	  303089 allocs/op
BenchmarkEngine_DeletePrefixRange_Cache/exists-24         	     200	 101868258 ns/op	25531572 B/op	  304736 allocs/op
BenchmarkEngine_DeletePrefixRange_Cache/exists-24         	     100	 106268683 ns/op	25648213 B/op	  306768 allocs/op
BenchmarkEngine_DeletePrefixRange_Cache/exists-24         	     100	 102905477 ns/op	25572314 B/op	  305798 allocs/op
BenchmarkEngine_DeletePrefixRange_Cache/exists-24         	     100	 108742857 ns/op	25483068 B/op	  304788 allocs/op
BenchmarkEngine_DeletePrefixRange_Cache/exists-24         	     100	 103292149 ns/op	25401388 B/op	  303401 allocs/op
BenchmarkEngine_DeletePrefixRange_Cache/exists-24         	     100	 107178026 ns/op	25573602 B/op	  305821 allocs/op
BenchmarkEngine_DeletePrefixRange_Cache/not_exists-24     	     200	  95082692 ns/op	23942491 B/op	  299116 allocs/op
BenchmarkEngine_DeletePrefixRange_Cache/not_exists-24     	     200	  96088487 ns/op	23957028 B/op	  298545 allocs/op
BenchmarkEngine_DeletePrefixRange_Cache/not_exists-24     	     200	  94279165 ns/op	23620981 B/op	  294536 allocs/op
BenchmarkEngine_DeletePrefixRange_Cache/not_exists-24     	     200	  94509000 ns/op	23989593 B/op	  299453 allocs/op
BenchmarkEngine_DeletePrefixRange_Cache/not_exists-24     	     200	  98530062 ns/op	23935846 B/op	  299237 allocs/op
BenchmarkEngine_DeletePrefixRange_Cache/not_exists-24     	     200	  98008093 ns/op	23821683 B/op	  297875 allocs/op
BenchmarkEngine_DeletePrefixRange_Cache/not_exists-24     	     200	  97603172 ns/op	23878336 B/op	  298350 allocs/op
BenchmarkEngine_DeletePrefixRange_Cache/not_exists-24     	     200	  96867920 ns/op	23782588 B/op	  296236 allocs/op
BenchmarkEngine_DeletePrefixRange_Cache/not_exists-24     	     200	  99148908 ns/op	23997702 B/op	  299277 allocs/op
BenchmarkEngine_DeletePrefixRange_Cache/not_exists-24     	     100	 100866840 ns/op	24019916 B/op	  300339 allocs/op
PASS
ok  	github.com/influxdata/influxdb/tsdb/tsm1	1144.213s
 2019-08-30 20:35:05 +01:00
Edd Robinson da2fb27cb9 perf(storage): reduce amount of tracing 
In a previous PR I added some tracing to help investigate delete
performance within the cache. Ironically this makes performance
significantly worse when you have a very high cardinality cache.

This keeps the main benefits of the tracing, but reduces the number of
spans created. The remaining spans are smarter with context, and include
useful information about the size of the operation being traced.

Performance on a benchmark shows a significant improvement:

name                                          old time/op    new time/op    delta
Engine_DeletePrefixRange_Cache/exists-24         262ms ± 6%     113ms ± 8%  -57.06%  (p=0.000 n=10+10)
Engine_DeletePrefixRange_Cache/not_exists-24     266ms ± 4%      96ms ± 2%  -64.09%  (p=0.000 n=8+10)

name                                          old alloc/op   new alloc/op   delta
Engine_DeletePrefixRange_Cache/exists-24        62.7MB ± 0%    29.6MB ± 1%  -52.82%  (p=0.000 n=9+10)
Engine_DeletePrefixRange_Cache/not_exists-24    59.2MB ± 0%    24.3MB ± 2%  -59.03%  (p=0.000 n=8+10)

name                                          old allocs/op  new allocs/op  delta
Engine_DeletePrefixRange_Cache/exists-24          711k ± 0%      334k ± 0%  -53.07%  (p=0.000 n=9+8)
Engine_DeletePrefixRange_Cache/not_exists-24      700k ± 0%      302k ± 1%  -56.79%  (p=0.000 n=8+10)

Raw benchmarks on a 24T/32GB/Nvme machine:

goos: linux
goarch: amd64
pkg: github.com/influxdata/influxdb/tsdb/tsm1
BenchmarkEngine_DeletePrefixRange_Cache/exists-24         	     100	 104851012 ns/op	29442514 B/op	  333599 allocs/op
BenchmarkEngine_DeletePrefixRange_Cache/exists-24         	     100	 107838824 ns/op	29485649 B/op	  334369 allocs/op
BenchmarkEngine_DeletePrefixRange_Cache/exists-24         	     100	 108020671 ns/op	29443324 B/op	  333610 allocs/op
BenchmarkEngine_DeletePrefixRange_Cache/exists-24         	     100	 106507506 ns/op	29977931 B/op	  338597 allocs/op
BenchmarkEngine_DeletePrefixRange_Cache/exists-24         	     100	 116393032 ns/op	29443516 B/op	  333614 allocs/op
BenchmarkEngine_DeletePrefixRange_Cache/exists-24         	     100	 112581877 ns/op	29691455 B/op	  334699 allocs/op
BenchmarkEngine_DeletePrefixRange_Cache/exists-24         	      50	 119833106 ns/op	29444712 B/op	  333625 allocs/op
BenchmarkEngine_DeletePrefixRange_Cache/exists-24         	     100	 113851895 ns/op	29921119 B/op	  337419 allocs/op
BenchmarkEngine_DeletePrefixRange_Cache/exists-24         	      50	 121735395 ns/op	29445551 B/op	  333634 allocs/op
BenchmarkEngine_DeletePrefixRange_Cache/exists-24         	      50	 115387319 ns/op	29444513 B/op	  333627 allocs/op
BenchmarkEngine_DeletePrefixRange_Cache/not_exists-24     	     100	  94474658 ns/op	24696698 B/op	  306702 allocs/op
BenchmarkEngine_DeletePrefixRange_Cache/not_exists-24     	     100	  94767020 ns/op	24004763 B/op	  300066 allocs/op
BenchmarkEngine_DeletePrefixRange_Cache/not_exists-24     	     100	  97869523 ns/op	24556560 B/op	  305827 allocs/op
BenchmarkEngine_DeletePrefixRange_Cache/not_exists-24     	     100	  93916119 ns/op	24172163 B/op	  301244 allocs/op
BenchmarkEngine_DeletePrefixRange_Cache/not_exists-24     	     100	  96591891 ns/op	24006021 B/op	  300081 allocs/op
BenchmarkEngine_DeletePrefixRange_Cache/not_exists-24     	     100	  93521244 ns/op	24266467 B/op	  303190 allocs/op
BenchmarkEngine_DeletePrefixRange_Cache/not_exists-24     	     100	  95419569 ns/op	24006501 B/op	  300087 allocs/op
BenchmarkEngine_DeletePrefixRange_Cache/not_exists-24     	     100	  96694570 ns/op	24521126 B/op	  306041 allocs/op
BenchmarkEngine_DeletePrefixRange_Cache/not_exists-24     	     100	  95075965 ns/op	24299409 B/op	  301649 allocs/op
BenchmarkEngine_DeletePrefixRange_Cache/not_exists-24     	     100	  97182864 ns/op	24007644 B/op	  300101 allocs/op
PASS
ok  	github.com/influxdata/influxdb/tsdb/tsm1	490.287s
 2019-08-30 20:35:05 +01:00
Edd Robinson 15ade8c162 perf(storage): remeove erroneous variable 
This commit removes an unused slice that was being built up. Comparting
to the base-line performance yields a slight improvement when deleting
from the cache.

name                                          old time/op    new time/op    delta
Engine_DeletePrefixRange_Cache/exists-24         268ms ± 5%     262ms ± 6%    ~     (p=0.218 n=10+10)
Engine_DeletePrefixRange_Cache/not_exists-24     265ms ± 5%     266ms ± 4%    ~     (p=0.965 n=10+8)

name                                          old alloc/op   new alloc/op   delta
Engine_DeletePrefixRange_Cache/exists-24        64.1MB ± 0%    62.7MB ± 0%  -2.16%  (p=0.000 n=9+9)
Engine_DeletePrefixRange_Cache/not_exists-24    59.2MB ± 0%    59.2MB ± 0%    ~     (p=0.505 n=8+8)

name                                          old allocs/op  new allocs/op  delta
Engine_DeletePrefixRange_Cache/exists-24          711k ± 0%      711k ± 0%  -0.00%  (p=0.000 n=9+9)
Engine_DeletePrefixRange_Cache/not_exists-24      700k ± 0%      700k ± 0%    ~     (p=0.687 n=8+8)

Raw benchmarks using a 24T / 32GB / NVME machine:

goos: linux
goarch: amd64
pkg: github.com/influxdata/influxdb/tsdb/tsm1
BenchmarkEngine_DeletePrefixRange_Cache/exists-24         	      30	 267664312 ns/op	62689106 B/op	  711400 allocs/op
BenchmarkEngine_DeletePrefixRange_Cache/exists-24         	      30	 255017152 ns/op	62688809 B/op	  711398 allocs/op
BenchmarkEngine_DeletePrefixRange_Cache/exists-24         	      30	 258136039 ns/op	62689626 B/op	  711404 allocs/op
BenchmarkEngine_DeletePrefixRange_Cache/exists-24         	      30	 273982453 ns/op	62688325 B/op	  711395 allocs/op
BenchmarkEngine_DeletePrefixRange_Cache/exists-24         	      30	 252670795 ns/op	62688704 B/op	  711397 allocs/op
BenchmarkEngine_DeletePrefixRange_Cache/exists-24         	      30	 277700985 ns/op	61801204 B/op	  702520 allocs/op
BenchmarkEngine_DeletePrefixRange_Cache/exists-24         	      30	 272353886 ns/op	62688767 B/op	  711403 allocs/op
BenchmarkEngine_DeletePrefixRange_Cache/exists-24         	      30	 258717468 ns/op	62689461 B/op	  711408 allocs/op
BenchmarkEngine_DeletePrefixRange_Cache/exists-24         	      30	 252909070 ns/op	62688949 B/op	  711404 allocs/op
BenchmarkEngine_DeletePrefixRange_Cache/exists-24         	      30	 255436837 ns/op	62689712 B/op	  711409 allocs/op
BenchmarkEngine_DeletePrefixRange_Cache/not_exists-24     	      30	 241173429 ns/op	59202122 B/op	  700036 allocs/op
BenchmarkEngine_DeletePrefixRange_Cache/not_exists-24     	      30	 247961098 ns/op	60507541 B/op	  714102 allocs/op
BenchmarkEngine_DeletePrefixRange_Cache/not_exists-24     	      20	 263380230 ns/op	59202750 B/op	  700044 allocs/op
BenchmarkEngine_DeletePrefixRange_Cache/not_exists-24     	      30	 266035285 ns/op	59202758 B/op	  700043 allocs/op
BenchmarkEngine_DeletePrefixRange_Cache/not_exists-24     	      30	 266512878 ns/op	59202759 B/op	  700044 allocs/op
BenchmarkEngine_DeletePrefixRange_Cache/not_exists-24     	      30	 262065769 ns/op	59202726 B/op	  700043 allocs/op
BenchmarkEngine_DeletePrefixRange_Cache/not_exists-24     	      30	 270485538 ns/op	59202733 B/op	  700043 allocs/op
BenchmarkEngine_DeletePrefixRange_Cache/not_exists-24     	      30	 263355678 ns/op	62562757 B/op	  727794 allocs/op
BenchmarkEngine_DeletePrefixRange_Cache/not_exists-24     	      30	 260440337 ns/op	59203324 B/op	  700050 allocs/op
BenchmarkEngine_DeletePrefixRange_Cache/not_exists-24     	      30	 276122362 ns/op	59203316 B/op	  700050 allocs/op
PASS
ok  	github.com/influxdata/influxdb/tsdb/tsm1	259.435s
 2019-08-30 20:35:05 +01:00
Edd Robinson f2d6c93e65 test: add benchmark to track cache deletion perf 
Benchmarks using a 24T / 32GB / NVME disk machine:

goos: linux
goarch: amd64
pkg: github.com/influxdata/influxdb/tsdb/tsm1
BenchmarkEngine_DeletePrefixRange_Cache/exists-24         	      20	 280039668 ns/op	64073374 B/op	  711421 allocs/op
BenchmarkEngine_DeletePrefixRange_Cache/exists-24         	      30	 271810284 ns/op	64073207 B/op	  711420 allocs/op
BenchmarkEngine_DeletePrefixRange_Cache/exists-24         	      30	 263464797 ns/op	64072589 B/op	  711415 allocs/op
BenchmarkEngine_DeletePrefixRange_Cache/exists-24         		      30	 269460489 ns/op	64073344 B/op	  711420 allocs/op
BenchmarkEngine_DeletePrefixRange_Cache/exists-24         	      30	 268319443 ns/op	64073947 B/op	  711425 allocs/op
BenchmarkEngine_DeletePrefixRange_Cache/exists-24         	      30	 254945449 ns/op	64073463 B/op	  711421 allocs/op
BenchmarkEngine_DeletePrefixRange_Cache/exists-24         	      30	 270202990 ns/op	65616337 B/op	  724440 allocs/op
BenchmarkEngine_DeletePrefixRange_Cache/exists-24         	      30	 274113444 ns/op	64074764 B/op	  711435 allocs/op
BenchmarkEngine_DeletePrefixRange_Cache/exists-24         	      30	 264234897 ns/op	64073748 B/op	  711428 allocs/op
BenchmarkEngine_DeletePrefixRange_Cache/exists-24         	      30	 264406196 ns/op	64073797 B/op	  711429 allocs/op
BenchmarkEngine_DeletePrefixRange_Cache/not_exists-24     	      30	 250130623 ns/op	59202124 B/op	  700036 allocs/op
BenchmarkEngine_DeletePrefixRange_Cache/not_exists-24     	      30	 255092042 ns/op	59552365 B/op	  706287 allocs/op
BenchmarkEngine_DeletePrefixRange_Cache/not_exists-24     	      30	 274121068 ns/op	59202753 B/op	  700043 allocs/op
BenchmarkEngine_DeletePrefixRange_Cache/not_exists-24     	      30	 273088065 ns/op	59202702 B/op	  700043 allocs/op
BenchmarkEngine_DeletePrefixRange_Cache/not_exists-24     	      30	 264184087 ns/op	59202724 B/op	  700043 allocs/op
BenchmarkEngine_DeletePrefixRange_Cache/not_exists-24     	      30	 268075364 ns/op	59202718 B/op	  700043 allocs/op
BenchmarkEngine_DeletePrefixRange_Cache/not_exists-24     	      30	 265067057 ns/op	59202709 B/op	  700043 allocs/op
BenchmarkEngine_DeletePrefixRange_Cache/not_exists-24     	      30	 254749976 ns/op	60118957 B/op	  701435 allocs/op
BenchmarkEngine_DeletePrefixRange_Cache/not_exists-24     	      30	 266953837 ns/op	59203376 B/op	  700051 allocs/op
BenchmarkEngine_DeletePrefixRange_Cache/not_exists-24     	      30	 275083559 ns/op	59203329 B/op	  700050 allocs/op
PASS
ok  	github.com/influxdata/influxdb/tsdb/tsm1	261.273s
 2019-08-30 20:35:05 +01:00
Edd Robinson d160585a34 refactor(storage): add deeper tracing around deletes 2019-08-22 11:08:33 +01:00
maxunt 62d1474442
Merge pull request #14246 from influxdata/mu-tsm-clean-14058 
Clears tsm files when replace fails
 2019-08-07 11:34:17 -07:00
maxunt 757fb4f80c
Merge pull request #14280 from influxdata/er-rename 
feat(fs): API for replacing os calls
 2019-08-07 11:33:57 -07:00
jlapacik 945f16ff4b docs(tsm1): TagKeys and TagValues return partial results with non-nil errors 2019-08-01 13:47:11 -07:00
jlapacik e7a0068e46 fix(tsm1): TagKeys/TagValues returns when context canceled 2019-08-01 13:47:11 -07:00
jlapacik 66662dca7f test(tsm1): TagKeys/TagValues stops scanning when context is canceled 2019-08-01 13:47:11 -07:00
Max U 550da3c5a5 fix(fs): merge branch 'master' into HEAD 2019-07-25 10:23:48 -07:00
Edd Robinson ce3f58d9f0
Merge pull request #13787 from zhulongcheng/KeyCursor-overlapping 
fix(tsm1): check if blocks are overlapping in KeyCursor.Next
 2019-07-24 15:36:39 +01:00
tmgordeeva 48ee7ada04
fix(storage): move retention snapshot out of per bucket calls (#14420) 
* fix(storage): move retention snapshot out of per bucket calls

Also adds tracking for snapshots from retention and full compactions.
 2019-07-23 11:40:05 -07:00
Max U 979c075931 Merge branch 'master' into HEAD 2019-07-22 10:21:59 -07:00
Stuart Carnie 00561d5a1b
feedback: Move verify routines to `tsm1` package for consistency 
Should have left it there to begin with 🤣
 2019-07-09 09:00:41 +10:00
Max U fe2aceb5e4 Merge branch 'master' into mu-tsm-clean-14058 2019-07-08 15:19:41 -04:00
Max U 6f14314e97 change log level from info to error 2019-07-08 14:49:14 -04:00
Max U 2202d727da fixes merge conflicts 2019-07-08 14:07:04 -04:00
maxunt ca5a599261
Merge branch 'master' into er-rename 2019-07-08 13:42:24 -04:00
Max U 39f51969e9 replaced os.Create calls w API calls to fs.CreateFile, includes unit test 2019-07-08 13:01:42 -04:00
Edd Robinson 12f2eeda1f fix(storage): ensure query range is (start, stop] 2019-07-05 17:10:56 +01:00
Ben Johnson 150165e92f
fix(tsdb): Fix tsm1 block merge. (#14254) 
fix(tsdb): Fix tsm1 block merge.
 2019-07-04 08:35:43 -06:00
Max U fe748128e3 replaces os.Rename calls w api calls to fs.RenameFile. tests now are failing 2019-07-03 13:14:43 -04:00
Ben Johnson 10a2063dcc
fix(tsdb): Fix tsm1 block merge. 
Fixes the `tsm1.BlockIterator` so that it returns the current
key if there are still additional entries remaining. This previously
caused multiple entries not to be merged together during compaction
because the iterator would check if the next key matched the current
key but the key for the next set of entries was returned.
 2019-07-03 10:08:51 -06:00
Stuart Carnie a42ff1628d
fix(influxd): --pattern flag matches specified substring 
Previously the logic was inverted so `--pattern` matched
everything but the specified value.
 2019-07-03 12:02:19 +10:00
Max U af257e93ff initial commit for clearing tsm files when replace fails 2019-07-02 13:48:31 -04:00
Ben Johnson 08e24faf4c
feat(tsdb): Add block exporter. 
Adds export tooling to `influxd inspect export-blocks` so that we
can dump out block data in SQL format for better analysis during
the debugging process.
 2019-07-01 10:10:52 -06:00
Tanya Gordeeva fe4333e8e0 fix(storage): fix tracking disk bytes in memory 2019-06-27 16:36:00 -07:00
Tanya Gordeeva 3ff15a8b41 fix(storage): fix counts for level 4+ files 
The counts wreen't adding all the level 4+ files, so the last one to be counted
would override the rest.
 2019-06-27 16:36:00 -07:00
Ben Johnson b3d7986d4b
chore(tsdb): Fix read metrics declaration. 2019-06-27 09:25:27 -06:00
Ben Johnson 12549c859e
feat(tsdb): Add basic tsdb read metrics 
Adds a total cursor counter and seek location counter to a new
`readMetrics` that is added to each `Engine`. Default labels group
by `engine_id` and `node_id`.
 2019-06-26 16:16:24 -06:00
tmgordeeva fb69c5d06c
Merge pull request #13698 from influxdata/tg-fix-metrics 
fix(storage): reduce tsm level metrics cardinality
 2019-06-20 17:57:37 -07:00
Tanya Gordeeva 6428cdbce6 fix(storage): initialize tsm file metrics, update after compaction 
These metrics weren't being properly intialized on opening the file store, and
weren't being properly updated on compaction.
 2019-06-20 14:37:53 -07:00