Edd Robinson
1e891b5fbc
Change logging level
2017-11-30 14:08:44 +00:00
Jason Wilder
5cf7d52694
Ensure series keys are sorted
...
The Measurement added series keys from a map where the iteration
order is non-deterministic. The keys should be returned in sorted
order.
2017-11-29 11:24:10 -07:00
Ben Johnson
ca09f18e65
intermediate: tsdb compile
2017-11-29 11:20:18 -07:00
Edd Robinson
81976bca59
Refactor based on new design
2017-11-28 17:54:29 +00:00
Edd Robinson
38e0dd695f
Allow concurrent access to Engine Index
2017-11-28 15:57:03 +00:00
Edd Robinson
abae36f992
Ensure all index fields set
2017-11-28 15:57:02 +00:00
Edd Robinson
368420c670
Fix test due to index changes
2017-11-28 15:57:02 +00:00
Edd Robinson
67c67aeb34
Update test for Windows
2017-11-28 15:57:02 +00:00
Edd Robinson
12a2ff7fac
Add support for TSI shard streaming and shard size
...
This commit firstly ensures that a shard's size on disk is accurately
reported when using the tsi1 index, by including the on-disk size of the
tsi1 index in the calculation.
Secondly, this commit add support for shard streaming/copying when using
the tsi1 index. Prior to this, a tsi1 index would not be correctly
restored when streaming shards.
2017-11-28 15:57:02 +00:00
Ben Johnson
cc22134d8f
Merge branch 'er-tsi-index-part' of https://github.com/influxdata/influxdb into er-tsi-index-part
2017-11-27 07:52:39 -07:00
Ben Johnson
01491ca4f4
intermediate
2017-11-27 07:52:18 -07:00
Edd Robinson
4831545830
Add PR typo/doc changes
2017-11-27 14:05:30 +00:00
Ben Johnson
fc966a1b67
Add series file backup/restore.
2017-11-22 08:55:54 -07:00
Edd Robinson
68dd5e27c8
Improve performance of TagKeys
2017-11-21 17:16:47 +00:00
Edd Robinson
5ff96d9193
Merge pull request #9127 from influxdata/er-fga
...
Implement FGA on SHOW COMMANDS
2017-11-20 14:51:58 +00:00
Jason Wilder
0551849298
Reduce calls to time.Now()
...
These were showing up in profiles during heavy write load.
2017-11-17 14:23:02 -07:00
Edd Robinson
a5af19fc06
Address PR feedback
2017-11-17 12:43:48 +00:00
Edd Robinson
bff69f7a82
Refactor inmem implementation
2017-11-17 11:06:43 +00:00
Edd Robinson
25f0fedd6f
Fix MeasurementNamesByExpr in tsi1
2017-11-17 11:06:43 +00:00
Edd Robinson
3967e78885
Consolidate tests to tsdb package
2017-11-17 11:06:43 +00:00
Edd Robinson
b3407c5d46
Correct authorisation on inmem SHOW MEASUREMENTS
2017-11-17 11:06:43 +00:00
Edd Robinson
d4cecd7cc7
Add index authorisation test coverage
2017-11-17 11:06:43 +00:00
Edd Robinson
6851db3fc9
Add FGA support to SHOW MEASUREMENTS
2017-11-17 11:06:43 +00:00
Edd Robinson
aa17ef55f9
Implement FGA on SHOW SERIES
2017-11-17 11:06:43 +00:00
Edd Robinson
8acab9b5ac
Fix existing bug where database was empty
2017-11-17 11:06:43 +00:00
Ben Johnson
ede3fcf98e
intermediate
2017-11-15 16:09:25 -07:00
Ben Johnson
ba4c9e0317
Merge remote-tracking branch 'upstream/master' into er-tsi-index-part
2017-11-14 16:14:13 -07:00
Jason Wilder
8b18cc4456
Optimize deletes in tsi
...
The DropSeries code path ended up creating a MeasurementSeriesIterator
for each dropped series, this was too expensive just to see if a
series exists.
This adds a HasSeries func and fixes and issue where TSI files were
compacted while an iterator was still in use causing a panic.
2017-11-13 12:35:38 -07:00
Jason Wilder
13692639cb
Fix create/delete series race
...
This fixes a race where writes and deletes to the same series and
measurements could sometimes leave the index in an inconsistent state.
2017-11-13 09:02:10 -07:00
Jason Wilder
80cd5e63af
Optimize DeleteSeriesRange
...
This removes more allocations and speeds up some critical sections.
2017-11-13 09:02:10 -07:00
Jason Wilder
f893beb6d8
Use MeasurementSeriesKeysByExprIterator for deletes
2017-11-13 09:02:10 -07:00
Jason Wilder
16d1f4309b
Extract MeasurementSeriesKeysByExprIterator
2017-11-13 09:02:10 -07:00
Ben Johnson
9756a29678
import fix
2017-11-13 08:54:32 -07:00
Jonathan A. Sternberg
0b7c56bcd8
Update the zap logger dependency
...
The previous sha was taken from a revision on a devel branch that I
thought would continue staying in the tree after it was merged. That
revision was rebased away and the API was changed for the logger.
This updates the usage of the logger and adds a simple package for
constructing the base logger.
The 1.0 version of zap changed the format of the default console logger
so this change moves over to this new logger instead of attempting to
retain backwards compatibility with the old format.
2017-11-10 16:27:16 -06:00
Ben Johnson
e278af2b18
intermediate
2017-11-09 09:30:19 -07:00
Ben Johnson
d3cd750509
Refactor series file tombstoning.
2017-11-09 09:30:19 -07:00
Ben Johnson
3034d3fb54
intermediate
2017-11-09 09:30:19 -07:00
Ben Johnson
919f99f34d
Fixing tests.
2017-11-09 09:30:19 -07:00
Ben Johnson
07a743cca7
Rebase fixes
2017-11-09 09:29:19 -07:00
Edd Robinson
4471341d7e
Ensure error channel has capacity for all partitions
2017-11-09 09:28:37 -07:00
Ben Johnson
b24b08a23c
Fix partition loading.
2017-11-09 09:28:37 -07:00
Ben Johnson
1f6d4ed1d1
Add series map.
2017-11-09 09:28:37 -07:00
Edd Robinson
49218fd3bd
Fix issue with series being added to log file
2017-11-09 09:28:37 -07:00
Edd Robinson
87778f3c45
Open partitions in parallel
2017-11-09 09:28:37 -07:00
Edd Robinson
3ae799b3a5
WIP Fix build
2017-11-09 09:28:37 -07:00
Edd Robinson
ebb23df1cf
Implement most merge based methods
2017-11-09 09:28:37 -07:00
Edd Robinson
6d87ff7fa2
WIP - series point iterator
2017-11-09 09:28:37 -07:00
Edd Robinson
aec607bddf
Implement Measurement sketches
2017-11-09 09:28:37 -07:00
Edd Robinson
b39aa858cf
Implement series creation
2017-11-09 09:28:37 -07:00
Edd Robinson
65c6fa747e
Implement methods that don't require merge
2017-11-09 09:28:37 -07:00
Edd Robinson
bf132004a3
Implement basic partition layout
2017-11-09 09:28:37 -07:00
Edd Robinson
7aa9de508d
Initial refactor of tsi1.Index
...
This commit carries out the initial refactor of the tsi1.Index into
tsi1.Partition. We then create a new tsi1.Index that will be an
abstraction over a collection of Partitions.
2017-11-09 09:27:56 -07:00
Edd Robinson
fb646549f4
Index files -> partition files
2017-11-09 09:26:06 -07:00
Ben Johnson
328bffd658
Convert series ids to 64-bits.
2017-11-09 09:26:06 -07:00
Ben Johnson
0ffd94a37a
Fix rebase
2017-11-09 09:25:10 -07:00
Ben Johnson
08e459357a
Fix tsi race conditions.
2017-11-09 09:18:33 -07:00
Ben Johnson
c75f1127aa
intermediate
2017-11-09 09:18:33 -07:00
Ben Johnson
f223153078
Initial working version of series file.
2017-11-09 09:18:33 -07:00
Ben Johnson
e05d4fdeeb
intermediate
2017-11-09 09:18:33 -07:00
Ben Johnson
9ad2b53881
intermediate
2017-11-09 09:18:33 -07:00
Ben Johnson
7259589241
intermediate
2017-11-09 09:18:33 -07:00
Ben Johnson
48b48a8927
intermediate
2017-11-09 09:13:46 -07:00
Ben Johnson
156f25ac23
Improve SHOW TAG KEYS performance.
2017-11-07 10:59:19 -07:00
Edd Robinson
07c4fdc1ed
Fix data race on SeriesPointIterator
2017-11-07 10:48:23 +00:00
Stuart Carnie
f3d45ba301
influxdata/influxdb/influxql -> influxdata/influxql
2017-10-30 14:40:26 -07:00
Ben Johnson
49c1fca036
Handle nil MeasurementIterator.
2017-10-26 11:25:46 -06:00
Edd Robinson
47bd069315
Fix race in Measurement index
...
Fixes #8989 and #8633 .
Previously when issuing commands involving a regex check, walking
through the tags keys/values on a measurement, using the measurement's
index, would be racy.
This commit adds a new `TagKeyValue` type that abstracts away the
multi-layer map we were using as an inverted index from tag keys and
values to series ids. With this abstraction we can also make concurrent
access to this inverted index goroutine safe.
Finally, this commit fixes a very old bug in the index which will affect
any query using a regex. Previously we would always check _every_ tag
against a regex for a measurement, even when we had found a match.
2017-10-25 13:34:21 +01:00
Ben Johnson
5a77238f30
Sort & validate TSI key value insertion.
2017-10-23 10:46:01 -06:00
Ben Johnson
62093d2641
Merge pull request #8975 from benbjohnson/tsi-copy-returned-bytes
...
Copy returned bytes from TSI meta functions.
2017-10-18 09:26:02 -06:00
Ben Johnson
8ad2048a6b
TSI byte copy usage comments.
2017-10-18 07:21:54 -06:00
Ben Johnson
d17d0f18e0
Move copyBytes() and copyByteSlices() to bytesutil.
2017-10-18 07:19:46 -06:00
Jason Wilder
a6f4069ca7
Fix max select series limit for tsi
...
TSI did not check that the max select series limit during planning
the same way that inmem did. This means that the limit could be
set but the planning of a high cardinality query would still OOM
the server. This fixes that limit as well as makes the query interruptible
during planning.
2017-10-17 15:24:41 -06:00
Ben Johnson
dceb88eb30
Copy returned bytes from TSI meta functions.
2017-10-17 14:05:35 -06:00
Jason Wilder
5033783a33
Handle deleted series when rebuilding measurment index
2017-10-16 10:50:16 -06:00
Joe LeGasse
1443b22379
auth: add series auth to 'show tag values'
2017-09-27 20:01:18 -04:00
Edd Robinson
ea104596f0
Implement TSI index versioning
...
This commit adds a basic TSI versioning scheme, by adding a Version field
to an index's MANIFEST file.
Existing TSI indexes will not have this field present in their MANIFEST
files, and thus will be deemed incomatible with the current version.
Users with existing TSI indexes will be able to remove them, and convert the
resulting inmem indexes to the current version of a TSI index using the
influx_inspect tooling.
2017-09-22 17:59:39 +01:00
Edd Robinson
44691847e9
Merge branch 'master' into er-8678-tsi1-where
2017-09-22 16:54:49 +01:00
Jason Wilder
d5d9f9acfe
Remove debug line
2017-09-11 15:31:28 -06:00
Ben Johnson
ee4d3c7b3d
Invalidate all bloom filters.
2017-09-11 15:29:26 -06:00
Ben Johnson
3c2487b97a
Clean up tsi bloom filter invalidation.
2017-09-11 15:29:26 -06:00
Ben Johnson
6af936ee61
Fix bloom filter invalidation.
2017-09-11 15:29:26 -06:00
Ben Johnson
a40b2bb210
Simplify bloom filter invalidation.
2017-09-11 15:29:26 -06:00
Edd Robinson
408a78d904
Increase size of SeriesBlock partition
2017-09-11 15:29:26 -06:00
Ben Johnson
0ec2736f23
Incrementally rebuild tsi bloom filters.
2017-09-11 15:29:25 -06:00
Jason Wilder
a5a2957567
Reduce allocation in log_file
2017-09-11 15:29:25 -06:00
Jason Wilder
a9e89ede75
Reduce lock contenton on Index
...
Stat and Size are read-only and can take an RLock.
2017-09-11 15:26:25 -06:00
Jason Wilder
94e229ff59
Merge branch 'master' into jw-drop-series
2017-09-08 15:34:32 -06:00
Joe LeGasse
4fb35b373b
auth: apply series auth to TSI
2017-09-08 09:09:53 -04:00
Jason Wilder
a8d9eeef36
Reduce lock contention when deleting high cardinality series
...
Deleting high cardinality series could take a very long time, cause
write timeouts as well as dead lock the process. This fixes these
issue to by changing the approach for cleaning up the indexes and
reducing lock contention.
The prior approach delete each series and updated every index (inmem)
during the delete. This was very slow and cause the index to be locked
while it items in a slice were removed one by one. This has been changed
to mark series as deleted and then rebuild the index asynchronously which
speeds up the process.
There was also a dead lock that could occur when deleing the field set.
Deleting the field set held a write lock and the function it invoked under
the lock could try to take a read lock on the field set. This would then
deadlock. This approach was also very slow and caused time out for writes.
It now uses faster approach that checks for the existing of the measurment
in the cache and filestore which does not take write locks.
2017-09-07 11:36:02 -06:00
Joe LeGasse
732a0c2eaa
Merge pull request #8769 from influxdata/jl-map-cleanup
...
cleanup: remove poor usage of ',ok' with maps
2017-08-31 09:18:42 -04:00
Joe LeGasse
a95647b720
cleanup: remove poor usage of ',ok' with maps
...
There are several places in the code where comma-ok map retrieval was
being used poorly. Some were benign, like checking existence before
issuing an unconditional delete with no cleanup. Others were potentially
far more serious: assuming that if 'ok' was true, then the resulting
pointer retrieved from the map would be non-nil. `nil` is a perfectly
valid value to store in a map of pointers, and the comma-ok syntax is
meant for when membership is distinct from having a non-zero value.
There was only one or two cases that I saw that being used correctly for
maps of pointers.
2017-08-30 09:49:31 -04:00
Stuart Carnie
51eb85193c
release lock to avoid dead lock when calling WalkWhereForSeriesIDs
...
* WalkWhereForSeriesIDs may call SeriesIDs, which may attempt to
upgrade from a `RLock` to a `Lock`, causing the dead lock
2017-08-29 16:12:51 -07:00
Stuart Carnie
0ced270197
fix race condition reading map
2017-08-28 13:36:49 -07:00
Jonathan A. Sternberg
697759613c
Remove time comparisons from the inner sections of the storage engine
2017-08-16 16:51:13 -05:00
Jonathan A. Sternberg
9a2357c2c0
Separate the query engine into a separate package
...
This change provides a clear separation between the query engine
mechanics and the query language so that the language can be parsed and
dealt with separate from the query engine itself.
2017-08-16 13:38:43 -05:00
Ben Johnson
06bc3b6fbf
TSI Index Migration
2017-08-15 11:40:24 -06:00
Edd Robinson
45969ef3c6
Allow tag filtering when using DELETE with tsi1
2017-08-14 19:09:36 +01:00
Joe LeGasse
1121b69a9e
auth: apply FGA to SHOW SERIES
2017-08-09 14:56:53 -04:00
Edd Robinson
0f648e5170
Remove unsafe shenanigans
2017-08-03 16:38:05 +01:00
Edd Robinson
2d57b599e9
Remove debugging statement
2017-08-02 17:24:00 +01:00
Edd Robinson
da676a79ae
Implement TSI iterator
2017-08-02 16:29:14 +01:00
Edd Robinson
aa7095be5a
Use a merge-based approach for TagValues
2017-08-02 14:10:52 +01:00
Jason Wilder
94a48774b7
Pull in new index filter
2017-08-02 14:10:52 +01:00
Jason Wilder
c25f7b8b3f
Fix duplicate points returned after delete
...
The sortedSeriesIds slice was not getting reset to 0 which caused
the same series ids to exist in the slice more than once. Since
the size of the slice never matched the size of the seriesID map,
it kept appendending to the slice and sorting it which cause multiple
cursor to get created for the same series.
Fixes #8531
2017-07-10 10:37:01 -06:00
Stuart Carnie
2ccdda72a1
Free RLock prior to returning
2017-07-08 07:14:50 -07:00
Ben Johnson
9e64813db8
Defer unlock all write locks in inmem index.
...
Currently two write locks in `inmem` are obtained and then
manually unlocked at function exit points. However, we have
reports that the `inmem` index is hanging on a write lock and
cannot track the issue down to anything else besides a lock
that could have been left unlocked because of a panic.
This commit changes the two locks to always defer their unlocks
to prevent these hangs.
2017-06-29 10:23:13 -06:00
Ben Johnson
f9dc61928a
Fix TSI issue with spaces in tag values.
2017-06-28 11:39:48 -06:00
Jason Wilder
9bd703d597
Fix possible deadlocks in inmem index
2017-06-21 12:07:40 -06:00
Ben Johnson
b51f604030
Fix TSI non-contiguous compaction panic.
...
This fixes the case where log files are compacted out of order
and cause non-contiguous sets of index files to be compacted.
Previously, the compaction planner would fetch a list of index files
for each level and compact them in order starting with the oldest
ones. This can be a problem for level 1 because level 0 (log files)
are compacted individually and in some cases a log file can finish
compacting before older log files are finished compacting. This
causes there to be a gap in the list of level 1 files that is
ignored when fetching a list of index files.
Now, the planner reads the list of index files starting from the
oldest but stops once it hits a log file. This prevents that gap
from being ignored.
2017-06-13 10:53:26 -06:00
Summer
d17c205b54
fix typo
2017-06-12 11:20:08 +08:00
marchtea
6e6f92c99a
fix index file fd leak
2017-06-12 10:58:05 +08:00
Ben Johnson
bcc6ef769b
Check file count before attempting a TSI level compaction.
...
This check was previously in a different section of code which
was lost during a refactor to the new compaction strategy. The
compaction planning now makes a check to ensure at least two
files are available for compaction in a level.
2017-06-06 11:08:59 -06:00
Stuart Carnie
47f97ea134
use parsed measurement and models.Tags
2017-05-26 13:21:59 -07:00
Stuart Carnie
3ec9b401f7
fix benchmark test
2017-05-26 13:21:59 -07:00
Stuart Carnie
46796d932f
add database to index, engine and shard; call AuthorizeSeriesRead
2017-05-26 13:21:50 -07:00
Joe LeGasse
815f740f4c
initial fga work
...
wip
wip
fix tests / build
2017-05-26 13:16:27 -07:00
Stuart Carnie
c30c33dbcb
Merge remote-tracking branch 'origin/master' into sgc-tagsets
2017-05-26 09:10:18 -07:00
Stuart Carnie
c89d98dc02
gofmt
2017-05-25 16:00:23 -07:00
Stuart Carnie
386720b2e7
improvements to inmem/Measurement.TagSets API
...
```
benchmark old ns/op new ns/op delta
BenchmarkMeasurement_TagSetsNoDimensions_1000-8 234054 117315 -49.88%
BenchmarkMeasurement_TagSetsDimensions_1000-8 996838 313313 -68.57%
BenchmarkMeasurement_TagSetsNoDimensions_100000-8 58940464 39452117 -33.06%
BenchmarkMeasurement_TagSetsDimensions_100000-8 175612060 70195562 -60.03%
benchmark old allocs new allocs delta
BenchmarkMeasurement_TagSetsNoDimensions_1000-8 1026 26 -97.47%
BenchmarkMeasurement_TagSetsDimensions_1000-8 8026 2029 -74.72%
BenchmarkMeasurement_TagSetsNoDimensions_100000-8 100064 64 -99.94%
BenchmarkMeasurement_TagSetsDimensions_100000-8 800064 200067 -74.99%
benchmark old bytes new bytes delta
BenchmarkMeasurement_TagSetsNoDimensions_1000-8 117080 69080 -41.00%
BenchmarkMeasurement_TagSetsDimensions_1000-8 549081 117176 -78.66%
BenchmarkMeasurement_TagSetsNoDimensions_100000-8 23298264 18498265 -20.60%
BenchmarkMeasurement_TagSetsDimensions_100000-8 66498276 23298360 -64.96%
```
2017-05-25 15:52:27 -07:00
Jason Wilder
14b54e08cb
Fix compile error
2017-05-25 15:18:35 -06:00
Ben Johnson
24446a0297
Implement zap logging in TSI.
2017-05-25 08:57:50 -06:00
Ben Johnson
547db32d01
Fix tsi go vet issues.
2017-05-23 13:42:38 -06:00
Ben Johnson
48456d80ad
Remove tsi commented code.
2017-05-23 10:24:37 -06:00
Ben Johnson
2524df3405
Convert tsi1 series keys to uint32.
2017-05-23 09:48:13 -06:00
Ben Johnson
c744e2f562
TSI pull request fixes.
2017-05-23 09:01:05 -06:00
Ben Johnson
57eeae03fc
Add note about SeriesIDs() limitation.
2017-05-23 08:42:25 -06:00
Ben Johnson
e7f39c06ab
Refactor TSI1 compaction.
2017-05-23 08:42:25 -06:00
Ben Johnson
1975940f76
intermediate compaction commit
2017-05-23 08:42:25 -06:00
Ben Johnson
79edc0979c
Add temporary debugging stats for offset lookups.
2017-05-23 08:41:31 -06:00
Ben Johnson
48a06432df
Add tsi1 bloom filter.
2017-05-23 08:41:31 -06:00
Ben Johnson
f3e08c5871
Delta encode tag and measurement block series data.
2017-05-23 08:41:31 -06:00
Ben Johnson
6f58149052
Increase tsi compaction factor.
2017-05-23 08:40:26 -06:00
Stuart Carnie
5c5bea2baa
move Measurement and Series to inmem package
2017-05-19 08:17:09 -07:00
Jason Wilder
2cac46ebbc
Convert usage of strings to []byte
...
Measurement name and field were converted between []byte and string
repetively causing lots of garbage. This switches the code to use
[]byte in the write path.
2017-05-12 14:05:19 -06:00
Ben Johnson
489c89bea4
Add tsi support tooling.
2017-05-08 11:00:15 -06:00
Jason Wilder
73ddd4787b
Fix race in SeriesN and CreateSeriesIfNotExists
2017-05-04 14:40:50 -06:00
Jason Wilder
7371f1067b
Fix deadlock in Index.ForEachMeasurementTagKey
...
Index.ForEachMeasurementTagKey held an RLock while call the fn,
if the fn made another call into the index which acquired an RLock
and after another goroutine tried to acquire a Lock, it would deadlock.
2017-05-03 22:48:10 -06:00
Ben Johnson
ba7108f94e
Add TSI documentation.
2017-04-21 14:45:03 -06:00
Jason Wilder
02b663b651
Fix lock contention in Index.CreateSeriesListIfNotExists
...
There was contention on the write lock which only needs to be acquired
when checking to see if the log file should be rolled over.
2017-04-20 12:28:42 -06:00
Jason Wilder
40ec85aacd
Fix lock contention in LogFile.SeriesWithBuffer
...
Under high write load, the check for each series was done sequentially
which caused a lot of CPU time to acquire/release the RLock on LogFile.
This switches the code to check multiple series at once under an RLock
similar to the chang for inmem.
2017-04-20 12:28:42 -06:00
Jason Wilder
3c2825a851
Reduce lock thrashing when checking series
...
The inmem index would call CreateSeriesIfNotExist for each series
which takes and releases and RLock to see if a series exists. Under
high write load, the lock shows up in profiles quite a bit. This
adds a filtering step that obtains a single RLock and checks all the
series and returns the non-existent series to contine though the slow
path.
2017-04-20 12:28:41 -06:00
Jason Wilder
a19ce9c10f
Reduce index lock contention
...
Series and Measurment have their own locks and we do not need to
hold locks on the index while using those types.
2017-04-18 16:32:33 -06:00
Ben Johnson
9c97cd8601
Merge remote-tracking branch 'upstream/master' into tsi
2017-04-04 12:46:09 -06:00
Ben Johnson
0d74497abe
Reset rhh map elements to reuse allocations.
2017-04-04 11:57:37 -06:00
Ben Johnson
6ff27c95e5
Fix tsi assertions.
2017-04-04 11:29:21 -06:00
Ben Johnson
95d4016ff2
Merge branch 'tsi' of https://github.com/influxdata/influxdb into tsi-series-block-partitioning
2017-04-04 10:14:03 -06:00
Ben Johnson
bf49b176f5
Partition tsi1 series index.
2017-04-04 09:46:04 -06:00
Edd Robinson
fddaff2cc8
Merge master in
2017-03-29 18:00:28 +01:00
Edd Robinson
116230b427
Use varint for tag count
2017-03-29 16:31:13 +01:00
Ben Johnson
2edfb1c92d
Ignore series limit on database load.
2017-03-24 16:27:16 -06:00
Jason Wilder
ee03fbb164
Fix series tombstone sketch not updated when dropping measurment
2017-03-24 15:49:00 -06:00
Ben Johnson
9fb8f1ec1d
Fix database and tag limits.
2017-03-24 09:48:10 -06:00
Ben Johnson
85c1ae4dd4
Remove sort.Slice
2017-03-21 12:33:05 -06:00
Ben Johnson
1e9fa7bc2c
Fix 32-bit rhh implementation.
2017-03-21 11:44:13 -06:00
Jason Wilder
58c8736ebc
Merge pull request #8172 from influxdata/jw-dropped-points
...
Fix series not getting created
2017-03-21 09:44:31 -06:00
Jason Wilder
92c722913d
Unlock index if dropping non-existent series
2017-03-21 09:19:44 -06:00
Ben Johnson
5cf41adcb8
Optimize tsi1 write path.
...
Removes sorted series list in log, uses a buffer for HasSeries(),
and prepends a length for series key encoding.
2017-03-21 08:44:35 -06:00
Jason Wilder
7f78c6ad8e
Fix series not getting created
2017-03-20 17:19:22 -06:00
Ben Johnson
5d67c424bf
Refactor tsi1 write locking.
2017-03-17 11:20:50 -06:00
Ben Johnson
70efc70abe
Reduce lock contention, fix rhh lookup.
2017-03-17 09:44:11 -06:00
Ben Johnson
1807772388
Fix tsi tests.
2017-03-15 11:23:58 -06:00
Ben Johnson
ee2e046853
Merge remote-tracking branch 'upstream/tsi-log-compact' into tsi
2017-03-15 10:22:32 -06:00
Ben Johnson
cf7ba96377
Merge branch 'tsi-log-compact' into tsi
2017-03-15 10:18:40 -06:00
Ben Johnson
358b1e0b05
Merge remote-tracking branch 'upstream/master' into tsi
2017-03-15 10:13:32 -06:00
Edd Robinson
ddcea1c322
WHY YOU SMITE ME BEN. B. JOHNSON?
2017-03-15 12:50:03 +00:00
Ben Johnson
d23f2971c3
Refactor TagBlockEncoder.
2017-03-10 10:08:16 -07:00
Ben Johnson
7ee0b9bb28
Rewrite SeriesBlockEncoder.
2017-02-16 09:39:51 -07:00
Ben Johnson
7e6f33ef48
TSI1 Index file compaction.
2017-02-15 08:44:22 -07:00
Ben Johnson
71d13fed6d
Immediate, parallel tsl compactions.
2017-02-15 08:44:21 -07:00
Edd Robinson
7749bc1c67
Store sorted series in logFile
2017-02-13 17:03:42 +00:00
Edd Robinson
a6a2f9d5f0
Don't load meta data for tsi
2017-02-09 17:59:14 +00:00
Edd Robinson
2d59fb788c
Modify arguments to reduce allocations
2017-02-09 15:01:32 +00:00
Edd Robinson
c2bbc18e4b
Reduce repeated string allocations
2017-02-09 11:23:23 +00:00
Edd Robinson
21e821e5f9
Reduce memory footprint by pre-allocating
2017-02-08 18:49:03 +00:00
Edd Robinson
a9d58cc879
Add log file benchmarks
2017-02-08 16:50:46 +00:00
Ben Johnson
d6fedcacd5
Refactor tsi.LogFile compaction.
2017-02-08 09:00:08 -07:00
Ben Johnson
c4e9430de3
Recover after partial tsi log file write.
2017-02-07 10:02:31 -07:00
Ben Johnson
d91e6eabac
Add max-values-per-tag to inmem index.
2017-02-06 11:14:13 -07:00
Edd Robinson
908959a78a
Fix vet
2017-02-03 11:57:35 +00:00
Ben Johnson
c246f3d9b0
Use inmem index on existing shards.
2017-02-02 10:04:25 -07:00
Edd Robinson
5a37d51f02
Refactor DropSeries
2017-02-01 22:07:44 +00:00
Ben Johnson
57f44d5f0c
Include index in snapshot.
2017-02-01 14:19:42 -07:00
Edd Robinson
0a9fc910f6
Fix potential race on inmem
2017-02-01 16:29:52 +00:00
Edd Robinson
ffced9b538
Move sketch merging into File
2017-02-01 15:33:30 +00:00
Edd Robinson
ab238251f4
Refactor measurement sketches
2017-02-01 14:56:35 +00:00
Edd Robinson
d8a73a1954
Go vet
2017-02-01 14:51:15 +00:00
Ben Johnson
05bbda2c0a
Fix measurement block sketches.
2017-02-01 07:40:00 -07:00
Ben Johnson
86e908c384
Remove debug println().
2017-01-31 09:24:58 -07:00
Ben Johnson
f4a4d1e196
Fix series sketches.
2017-01-31 08:47:18 -07:00
Edd Robinson
ab94c1b743
Fixes #7882
2017-01-30 19:12:24 +00:00
Ben Johnson
42959ead1f
Fix CreateSeriesListIfNotExists() data race.
2017-01-30 10:07:07 -07:00
Jason Wilder
2ae9406c1d
Reincorporate memory leak fix
...
7832 got lost during a rebase/merge. This reincorporate the fix
for the in-memory index as well as tsi.
2017-01-24 20:13:48 -07:00
Jason Wilder
173003bc38
Fix go vet
2017-01-24 20:13:37 -07:00
Ben Johnson
047c21f4d9
Merge remote-tracking branch 'upstream/master' into tsi
2017-01-24 09:28:58 -07:00
Ben Johnson
c459d24a60
Test coverage.
2017-01-23 09:38:27 -07:00
Ben Johnson
f43b0f7ec9
Fix series & measurement deletion.
2017-01-12 09:29:40 -07:00
Ben Johnson
352817e8c4
Convert 32-bit offsets to 64-bit.
2017-01-11 08:58:10 -07:00
Ben Johnson
64c7715243
Rebase fixes.
2017-01-09 10:10:12 -07:00
Ben Johnson
d1f1e19591
Fixing rebase.
2017-01-06 09:31:25 -07:00
Ben Johnson
1003db0067
Add active log file tracking, time-based compaction.
2017-01-05 10:17:12 -07:00
Ben Johnson
c1c98223ec
Fix and optimize tsi1 FileSet.
2017-01-05 10:17:12 -07:00
Ben Johnson
31e74d809b
Add tsi FileSet.
2017-01-05 10:17:11 -07:00
Ben Johnson
dcd2a771b0
Optimizing tsi compaction.
2017-01-05 10:17:11 -07:00
Ben Johnson
1ce99e797f
Use series map in tsi1.LogFile.
2017-01-05 10:17:11 -07:00
Ben Johnson
9b1e8215e0
Remove dictionary encoding, add bulk series insertion.
2017-01-05 10:17:11 -07:00
Ben Johnson
5f7654173e
Add locking to sketch merge.
2017-01-05 10:17:11 -07:00
Ben Johnson
9bd19cdc69
Fix inmem DELETE SERIES.
2017-01-05 10:17:11 -07:00
Ben Johnson
f9efcb3365
Re-add shared in-memory index.
2017-01-05 10:17:09 -07:00
Edd Robinson
0f9b2bfe6a
Fix tests
2017-01-05 10:16:15 -07:00
Edd Robinson
4ccb8dbab1
Move series count check to shard
2017-01-05 10:16:13 -07:00
Edd Robinson
0cb74eedbf
Add log file (WAL) sketches
2017-01-05 10:15:38 -07:00
Edd Robinson
190c78c644
Add series sketches
2017-01-05 10:15:37 -07:00
Edd Robinson
695adafc00
Add measurement sketches
2017-01-05 10:15:37 -07:00
Ben Johnson
745b1973a8
tsi compaction
2017-01-05 10:15:37 -07:00
Ben Johnson
83e80f6d0b
Fix in-mem index integration tests.
2017-01-05 10:15:37 -07:00
Ben Johnson
183418dcbd
Fix tsi TAG KEYS iterator.
2017-01-05 10:15:36 -07:00
Ben Johnson
759ff4ab80
Add tsi1 term hash index.
2017-01-05 10:15:35 -07:00
Ben Johnson
75cfe244c4
Add series hash index.
2017-01-05 10:15:35 -07:00
Ben Johnson
9f8b206b51
Fix measurement system queries.
2017-01-05 10:15:34 -07:00
Ben Johnson
4aa78383d1
Fix tsi1 series deletion.
2017-01-05 10:14:48 -07:00
Ben Johnson
5965610de6
Refactoring tsi tombstoning.
2017-01-05 10:14:02 -07:00
Ben Johnson
e7940cc556
Add tsi1 series system iterator.
2017-01-05 10:14:00 -07:00
Ben Johnson
87f4e0ec0a
Add regex support in tsi1.
2017-01-05 10:12:29 -07:00
Ben Johnson
d13afa8f47
Iterator refactoring
2017-01-05 10:11:49 -07:00
Jason Wilder
f0427d180e
Fix tsi index panics
...
Hardcoded panics cause the server to crash in 10s due to stats collection.
2017-01-05 10:11:12 -07:00
Jason Wilder
4bf7b2bb19
Allow tsi to be enabled via config option
2017-01-05 10:11:12 -07:00
Jason Wilder
2b96c5d4d0
Set Tags on entry
...
These were lost when reloading from the index. Fixes queries not
returning any data.
2017-01-05 10:11:12 -07:00
Jason Wilder
a6490920fd
Fix reslicing indices
...
The slicing was backwards causing the buffer to grow indefinitely
and filling the disks on writes.
2017-01-05 10:11:12 -07:00
Jason Wilder
59864226b7
Add RWMutex to LogFile
...
Fixes concurrent map access panic
2017-01-05 10:11:12 -07:00
Ben Johnson
fbe7f464ee
Improve insert performance.
2017-01-05 10:11:12 -07:00
Ben Johnson
33412782ed
Fix go vet issue.
2017-01-05 10:11:10 -07:00
Ben Johnson
2b864c72c5
Refactor MeasurementBlockTrailer read/write.
2017-01-05 10:11:10 -07:00
Ben Johnson
cb93f10120
Remove per-shard in-memory index.
2017-01-05 10:11:09 -07:00
Ben Johnson
409b0165f5
shared in-memory index
2017-01-05 10:09:57 -07:00
Ben Johnson
a812502ea3
reintegrating in-memory index
2017-01-05 10:07:35 -07:00