Commit Graph

13100 Commits (1abe176a68635d9a7ffe3d173090b04baf0e4852)

Author SHA1 Message Date
Jason Wilder 44e782f173 Store temporary tombstones on disk
This removes the in-memory tombstone buffer when writing tombstones
which eliminates one source of large memory spikes during deletes.
2017-11-13 08:48:03 -07:00
Jason Wilder bd15d37c70 Extract commit func 2017-11-13 08:48:03 -07:00
Jason Wilder 1e56894097 Extract writeTombstone func 2017-11-13 08:48:03 -07:00
Jason Wilder b958c68ce5 Avoid re-reading tombstones when writing new ones
This adds a new v4 tombstone format that extends the v3 format
by allowing multiple batches of tombstones to be written without
having to re-read all the existing tombstones.  This uses gzip
multi stream to append multiple v3 files together to create a v4
format.
2017-11-13 08:48:03 -07:00
Jason Wilder 17bae05370 Allow buffering tombstones before writing to disk 2017-11-13 08:48:03 -07:00
Jonathan A. Sternberg 7757b827c4 Lock the zap write output to avoid race conditions when logging 2017-11-13 09:42:53 -06:00
e-wave 1c256e1441 shell shows error and usage when given extra parameters 2017-11-13 17:06:58 +02:00
Jonathan A. Sternberg 7cae889b13
Merge pull request #9055 from influxdata/js-update-zap-logger
Update the zap logger dependency
2017-11-10 22:01:00 -06:00
Jonathan A. Sternberg 0b7c56bcd8 Update the zap logger dependency
The previous sha was taken from a revision on a devel branch that I
thought would continue staying in the tree after it was merged. That
revision was rebased away and the API was changed for the logger.

This updates the usage of the logger and adds a simple package for
constructing the base logger.

The 1.0 version of zap changed the format of the default console logger
so this change moves over to this new logger instead of attempting to
retain backwards compatibility with the old format.
2017-11-10 16:27:16 -06:00
Stuart Carnie ef42c367f0
Merge pull request #9091 from influxdata/sgc-ifql
don't panic here; nil cursor is handled for now. fixes #9090
2017-11-10 11:25:45 -07:00
Stuart Carnie 57677be010 don't panic here; nil cursor is handled for now. fixes #9090 2017-11-10 11:21:31 -07:00
Ben Johnson e278af2b18
intermediate 2017-11-09 09:30:19 -07:00
Ben Johnson d3cd750509
Refactor series file tombstoning. 2017-11-09 09:30:19 -07:00
Ben Johnson 3034d3fb54
intermediate 2017-11-09 09:30:19 -07:00
Ben Johnson 919f99f34d
Fixing tests. 2017-11-09 09:30:19 -07:00
Ben Johnson 07a743cca7
Rebase fixes 2017-11-09 09:29:19 -07:00
Edd Robinson 4471341d7e
Ensure error channel has capacity for all partitions 2017-11-09 09:28:37 -07:00
Ben Johnson b24b08a23c
Fix partition loading. 2017-11-09 09:28:37 -07:00
Ben Johnson 1f6d4ed1d1
Add series map. 2017-11-09 09:28:37 -07:00
Edd Robinson 49218fd3bd
Fix issue with series being added to log file 2017-11-09 09:28:37 -07:00
Edd Robinson 87778f3c45
Open partitions in parallel 2017-11-09 09:28:37 -07:00
Edd Robinson 3ae799b3a5
WIP Fix build 2017-11-09 09:28:37 -07:00
Edd Robinson ebb23df1cf
Implement most merge based methods 2017-11-09 09:28:37 -07:00
Edd Robinson 0dd97cc84a
Add utility functions for merging k collections of sorted slices 2017-11-09 09:28:37 -07:00
Edd Robinson 6d87ff7fa2
WIP - series point iterator 2017-11-09 09:28:37 -07:00
Edd Robinson aec607bddf
Implement Measurement sketches 2017-11-09 09:28:37 -07:00
Edd Robinson b39aa858cf
Implement series creation 2017-11-09 09:28:37 -07:00
Edd Robinson 65c6fa747e
Implement methods that don't require merge 2017-11-09 09:28:37 -07:00
Edd Robinson bf132004a3
Implement basic partition layout 2017-11-09 09:28:37 -07:00
Edd Robinson 7aa9de508d
Initial refactor of tsi1.Index
This commit carries out the initial refactor of the tsi1.Index into
tsi1.Partition. We then create a new tsi1.Index that will be an
abstraction over a collection of Partitions.
2017-11-09 09:27:56 -07:00
Edd Robinson fb646549f4
Index files -> partition files 2017-11-09 09:26:06 -07:00
Ben Johnson 328bffd658
Convert series ids to 64-bits. 2017-11-09 09:26:06 -07:00
Ben Johnson 0ffd94a37a
Fix rebase 2017-11-09 09:25:10 -07:00
Ben Johnson 08e459357a
Fix tsi race conditions. 2017-11-09 09:18:33 -07:00
Ben Johnson c75f1127aa
intermediate 2017-11-09 09:18:33 -07:00
Ben Johnson f223153078
Initial working version of series file. 2017-11-09 09:18:33 -07:00
Ben Johnson e05d4fdeeb
intermediate 2017-11-09 09:18:33 -07:00
Ben Johnson 9ad2b53881
intermediate 2017-11-09 09:18:33 -07:00
Ben Johnson 7259589241
intermediate 2017-11-09 09:18:33 -07:00
Ben Johnson 48b48a8927
intermediate 2017-11-09 09:13:46 -07:00
Jason Wilder fe35aee4c4
Merge pull request #9086 from influxdata/jw-points-panic
Fix panic: runtime error: slice bounds out of range
2017-11-09 08:50:55 -07:00
Jason Wilder ed246db55a Fix panic: runtime error: slice bounds out of range
Fixes #8538
2017-11-08 17:00:25 -07:00
Edd Robinson 0614ebb1d1
Merge pull request #9085 from influxdata/er-iterator-pb
Pass through SystemIterator in PB
2017-11-08 20:28:45 +00:00
Edd Robinson 6443355467 Pass through SystemIterator in PB 2017-11-08 19:57:16 +00:00
Edd Robinson 25d1976e95
Merge pull request #9083 from influxdata/er-show-tag-key-vals
SHOW TAG {KEYS, VALUES} across RPs
2017-11-08 16:49:57 +00:00
Edd Robinson d581aee285 Ensure all retention policies queried 2017-11-08 16:27:57 +00:00
Edd Robinson 59c4e4b1bc Skip shards we don't have 2017-11-08 13:33:52 +00:00
Jonathan A. Sternberg 8e6c208b55
Merge pull request #9080 from influxdata/js-influxql-data-type
Update influxql so DataType constants are the type DataType
2017-11-07 20:05:23 -06:00
Jonathan A. Sternberg b09396f977 Update influxql so DataType constants are the type DataType 2017-11-07 18:34:53 -06:00
Edd Robinson 081d0649a6
Merge pull request #9078 from influxdata/er-race-tests
Skip cardinality tests in race mode
2017-11-07 19:52:30 +00:00