75e142c41a
use of flag.Args instead of flag.NArg
2017-11-13 18:07:00 +02:00
5763d01613
Fix brtfs docker build messages
2017-11-13 09:02:10 -07:00
c0631c2b95
Fix temp tombstone files leaking
2017-11-13 09:02:10 -07:00
13692639cb
Fix create/delete series race
...
This fixes a race where writes and deletes to the same series and
measurements could sometimes leave the index in an inconsistent state.
2017-11-13 09:02:10 -07:00
04f4c3e993
Optimize bytesutil.Pack
2017-11-13 09:02:10 -07:00
80cd5e63af
Optimize DeleteSeriesRange
...
This removes more allocations and speeds up some critical sections.
2017-11-13 09:02:10 -07:00
aee395d3bd
Make DeleteSeriesRange take SeriesIterator
2017-11-13 09:02:10 -07:00
f893beb6d8
Use MeasurementSeriesKeysByExprIterator for deletes
2017-11-13 09:02:10 -07:00
16d1f4309b
Extract MeasurementSeriesKeysByExprIterator
2017-11-13 09:02:10 -07:00
000768371f
Optimized deletes in TSM index
...
This optimizes how deletes are processed to reduce memory usage
and improve efficiency.
2017-11-13 09:02:08 -07:00
9756a29678
import fix
2017-11-13 08:54:32 -07:00
eebd88f825
Don't write tombstones for keys that do not exist
...
This filters out keys that do not exist in a TSM file to avoid
writing entries that would end up being ignored when applied.
2017-11-13 08:50:07 -07:00
ca4998f29e
Fix pid test to ignore local data
...
If you have lots of data stored locally, this test takes a while to
complete since it loads it all up from the users home dir.
2017-11-13 08:50:07 -07:00
88c48ec78b
Rework Engine.DeleteSeriesRange to avoid allocations
...
This removes the containsSeries func which ends up creating a map
sized to the slice of keys passed in. This doesn't scale well to
high cardinalities and creates a lot of garbage.
2017-11-13 08:50:07 -07:00
cb658774bb
Reduce allocations when reading tombstone v4
2017-11-13 08:50:07 -07:00
1c65bb3bb1
Fix leaked goroutine in FileStore.WalkKeys
...
If fn returned and error, the goroutines sending keys from TSM files
would get blocked indefinitely and leak.
2017-11-13 08:50:07 -07:00
b0c7a44eaa
Adjust min/max time to work in the engine
...
The query language min and max times are slighly different than the
values used in the engine. This allows faster codes to be used when
the whole time range is deleted.
2017-11-13 08:50:07 -07:00
2959b8d2eb
Make BatchDeleters concurrent
2017-11-13 08:50:07 -07:00
5a775c50d9
Add DeleteRangeWith
...
This is a version of DeleteRange that take a func predicate to determine
whether a series key should be deleted or not. This avoids the large
slice allocations with higher cardinalities.
2017-11-13 08:50:07 -07:00
de5592cda9
Extract ParseKeyBytes from ParseKey
...
Allows callers to use []byte and avoid a string allocation
2017-11-13 08:48:03 -07:00
6b19d2b673
Add BatchDeleters type
2017-11-13 08:48:03 -07:00
9ac83601cf
Use BatchDeleter in FileStore
2017-11-13 08:48:03 -07:00
4ed19348fd
Add a BatchDelete capability to TSMReader
2017-11-13 08:48:03 -07:00
44e782f173
Store temporary tombstones on disk
...
This removes the in-memory tombstone buffer when writing tombstones
which eliminates one source of large memory spikes during deletes.
2017-11-13 08:48:03 -07:00
bd15d37c70
Extract commit func
2017-11-13 08:48:03 -07:00
1e56894097
Extract writeTombstone func
2017-11-13 08:48:03 -07:00
b958c68ce5
Avoid re-reading tombstones when writing new ones
...
This adds a new v4 tombstone format that extends the v3 format
by allowing multiple batches of tombstones to be written without
having to re-read all the existing tombstones. This uses gzip
multi stream to append multiple v3 files together to create a v4
format.
2017-11-13 08:48:03 -07:00
17bae05370
Allow buffering tombstones before writing to disk
2017-11-13 08:48:03 -07:00
7757b827c4
Lock the zap write output to avoid race conditions when logging
2017-11-13 09:42:53 -06:00
1c256e1441
shell shows error and usage when given extra parameters
2017-11-13 17:06:58 +02:00
7cae889b13
Merge pull request #9055 from influxdata/js-update-zap-logger
...
Update the zap logger dependency
2017-11-10 22:01:00 -06:00
0b7c56bcd8
Update the zap logger dependency
...
The previous sha was taken from a revision on a devel branch that I
thought would continue staying in the tree after it was merged. That
revision was rebased away and the API was changed for the logger.
This updates the usage of the logger and adds a simple package for
constructing the base logger.
The 1.0 version of zap changed the format of the default console logger
so this change moves over to this new logger instead of attempting to
retain backwards compatibility with the old format.
2017-11-10 16:27:16 -06:00
ef42c367f0
Merge pull request #9091 from influxdata/sgc-ifql
...
don't panic here; nil cursor is handled for now. fixes #9090
2017-11-10 11:25:45 -07:00
57677be010
don't panic here; nil cursor is handled for now. fixes #9090
2017-11-10 11:21:31 -07:00
e278af2b18
intermediate
2017-11-09 09:30:19 -07:00
d3cd750509
Refactor series file tombstoning.
2017-11-09 09:30:19 -07:00
3034d3fb54
intermediate
2017-11-09 09:30:19 -07:00
919f99f34d
Fixing tests.
2017-11-09 09:30:19 -07:00
07a743cca7
Rebase fixes
2017-11-09 09:29:19 -07:00
4471341d7e
Ensure error channel has capacity for all partitions
2017-11-09 09:28:37 -07:00
b24b08a23c
Fix partition loading.
2017-11-09 09:28:37 -07:00
1f6d4ed1d1
Add series map.
2017-11-09 09:28:37 -07:00
49218fd3bd
Fix issue with series being added to log file
2017-11-09 09:28:37 -07:00
87778f3c45
Open partitions in parallel
2017-11-09 09:28:37 -07:00
3ae799b3a5
WIP Fix build
2017-11-09 09:28:37 -07:00
ebb23df1cf
Implement most merge based methods
2017-11-09 09:28:37 -07:00
0dd97cc84a
Add utility functions for merging k collections of sorted slices
2017-11-09 09:28:37 -07:00
6d87ff7fa2
WIP - series point iterator
2017-11-09 09:28:37 -07:00
aec607bddf
Implement Measurement sketches
2017-11-09 09:28:37 -07:00
b39aa858cf
Implement series creation
2017-11-09 09:28:37 -07:00
e-wave
Jason Wilder
Ben Johnson
Jonathan A. Sternberg
Stuart Carnie
Edd Robinson