Ben Johnson
f2d23b070b
add tsm1 wal quickcheck
...
This commit adds quickcheck testing for the tsm1 WAL.
2015-10-14 09:38:38 -06:00
Daniel Morsing
6d188d9703
Merge pull request #4409 from influxdb/intoq
...
wire up INTO queries.
2015-10-14 15:29:54 +01:00
Philip O'Toole
6d38559f24
RLock access to store's shards
...
Fix issue #4442 .
2015-10-13 20:28:19 -07:00
Sean Beckett
1d83c8c427
Update meta.go
2015-10-13 16:46:59 -07:00
Daniel Morsing
62dff895e2
wire up INTO queries.
...
Since INTO queries need to have absolute information about the database
to work, we need to create a loopback interface back to the cluster
in order to perform them.
2015-10-13 15:00:36 +00:00
Cory LaNou
6787525912
fixes multiple selectors overwriting each other. fixes #4360
2015-10-12 21:40:57 -05:00
Jason Wilder
16b3084ca9
Merge pull request #4397 from influxdb/jw-tsmdump
...
tsm1 file dump
2015-10-12 08:12:23 -06:00
Paul Dix
a99c9ec5af
Ensure rewrite index doesn't write corrupt file.
...
Fixes #4401
2015-10-10 16:58:25 -04:00
Jason Wilder
629219951a
Fix timestamp encoding not using run-length encoding when possible
...
influx_inpsect uncovered some scenarios where timestamps could be stored using
run-length encoding but were being stored using simple8 which uses more space.
2015-10-09 22:38:17 -06:00
David Norton
9627f5ab84
fix #4280 : add comments based on code review
2015-10-09 18:38:33 -04:00
David Norton
512d6ac050
fix #4280 : only drop points matching WHERE clause
2015-10-09 18:34:32 -04:00
Jason Wilder
758359accc
Prevent panic in DecodeSameTypeBlock
...
If DecodeSameTypeBlock is called on on an empty Values slice, it would
panic with an index out of bounds error. This func can actually be removed
because DecodeBlock can determine what type of values are encoded already.
This will still panic if the block cannot be decoded due to other reasons.
Fixes #4365
2015-10-09 12:52:23 -06:00
Ben Johnson
2b3bb5336d
add tsm1 quickcheck tests
2015-10-08 11:59:57 -06:00
Jason Wilder
79185fc1dc
Fix index out of bounds panic in int64Decoder
...
Code was missing a check for when we did not have anymore bytes to decode
so it panic when we tried to decode the empty slice.
2015-10-08 11:21:19 -06:00
Jason Wilder
b3343a6d2a
Fix similar float values encoding overflow
...
If similar float values were encoded, the number of leading bits would
overflow the 5 available bits to store them (e.g. store 33 in 5 bits). When
decoding, the values after the overflowed value would spike to very large and
small values.
To prevent the overflow, we clamp the value to 31 which is the maximum
number of leading zero bits we can encoded.
Fixes #4357
2015-10-07 15:05:56 -06:00
Nathaniel Cook
6f1a44bd07
Merge pull request #4345 from influxdb/remove_iterator
...
tsdb.Iterator is no longer used. Removing
2015-10-06 17:00:50 -06:00
Paul Dix
f041939a1c
Merge pull request #4308 from influxdb/pd-storage-engine
...
The TSM storage engine
2015-10-06 15:54:56 -07:00
Paul Dix
b11308133a
Only limit field count for non-tsm engines
2015-10-06 15:49:37 -07:00
Paul Dix
40ff4f4a86
Change default to bz1
2015-10-06 15:30:34 -07:00
Jason Wilder
41e3294d4a
Fix panic: assignment to entry in nil map
...
Closing the store did not properly return an error for in-flight
writes because the closing channel was set to nil when closed. A
nil channel is not selectable so writes continue on past the guard
checks and trigger panics.
2015-10-06 14:03:52 -06:00
Paul Dix
be477b2aab
Fix cursor bug on index
2015-10-06 12:26:45 -07:00
Nathaniel Cook
d380ee37a2
tsdb.Iterator is no longer used. Removing
2015-10-06 10:34:07 -06:00
Paul Dix
267f34b94e
Updates based on PR feedback
2015-10-05 20:09:56 -04:00
Paul Dix
26a93ec23e
Fix deletes not kept if shutdown before flush on tsm1
2015-10-05 20:09:56 -04:00
Paul Dix
bb398daf75
Updates based on @otoolp's PR comments
2015-10-05 20:09:56 -04:00
Jason Wilder
c6f2f9cec2
Avoid duplicating values slice when encoding
2015-10-05 20:09:56 -04:00
Jason Wilder
cb28dabf62
Make DecodeBlock panic if block size is too small
...
Should never get a block size 9 bytes since Encode always returns the min
timestampe and a 1 byte header. If we get this, the engine is confused.
2015-10-05 20:09:56 -04:00
Jason Wilder
b0449702e5
Fix comment typos
2015-10-05 20:09:56 -04:00
Paul Dix
d9f94bdeeb
Add db crash recovery
2015-10-05 20:09:56 -04:00
Jason Wilder
1d754db00b
Propogate all encoding errors to engine
...
Avoid panicing in lower level code and allow the engine to decide what
it should do.
2015-10-05 20:09:56 -04:00
Jason Wilder
4c54c78009
Move compression encoding constants to encoders
...
Will make it less error-prone to add new encodings int the future
since each encoder has it's set of constants. There are some placeholder
contants for uncompressed encodings which are not in all encoder currently.
2015-10-05 20:09:56 -04:00
Jason Wilder
b1a57e1628
Fix go vet errors
2015-10-05 20:09:56 -04:00
Jason Wilder
ab791ba913
Fix TestStoreOpenShardCreateDelete
...
Shard path can be a directory.
2015-10-05 20:09:56 -04:00
Paul Dix
d47ddb5454
Cleanup after pd1 -> tsm1 name change.
2015-10-05 20:09:55 -04:00
Paul Dix
594253cbba
Rename storage engine to tsm1, for Time Structured Merge Tree!
2015-10-05 20:09:55 -04:00
Paul Dix
0a11a2fdbc
Add deletes to new storage engine
2015-10-05 20:09:55 -04:00
Paul Dix
4beca1a245
Implement reverse cursor direction on pd1
2015-10-05 20:09:55 -04:00
Jason Wilder
dbf6228817
Fix go vet
2015-10-05 20:09:55 -04:00
Jason Wilder
d9499f0598
Remove zig zag encoding from timestamp encoder
...
Not needed since all timestamps will be sorted in ascending order. Negatives
are not possible.
2015-10-05 20:09:55 -04:00
Paul Dix
a2b139e006
Fix compaction and multi-write bugs.
...
* Fix bug with locking when the interval completely covers or is totally inside another one.
* Fix bug with full compactions running when the index is actively being written to.
2015-10-05 20:09:55 -04:00
Jason Wilder
2366baaf0b
Handle partial reads when loading WAL
...
If reading into fixed sized buffer using io.ReadFull, the func can
return io.ErrUnexpectedEOF if the read was short. This was slipping
through the error handling causing the shard to fail to load.
2015-10-05 20:09:55 -04:00
Paul Dix
3332236527
Fix bugs with writing old data and compaction.
2015-10-05 20:09:55 -04:00
Jason Wilder
5d938d0a8b
Add test with duplicate timestamps
...
Should not happen but makes sure that the same values are encoded
and decoded correctly.
2015-10-05 20:09:55 -04:00
Jason Wilder
c47d14540d
Add compressed string encoding
...
Uses snappy to compress multiple strings into a block
2015-10-05 20:09:55 -04:00
Paul Dix
861a15b3e6
Fix panic when data file has small index
2015-10-05 20:09:55 -04:00
Paul Dix
be011b8da9
Add logging to pd1
2015-10-05 20:09:54 -04:00
Paul Dix
c1213ba367
Update WAL to deduplicate values on Cursor query.
...
Added test and have failing section for single value encoding.
2015-10-05 20:09:54 -04:00
Jason Wilder
9f9692acdf
Rename float encoding tests
2015-10-05 20:09:54 -04:00
Jason Wilder
a4d92162ef
Add documentation about compression
2015-10-05 20:09:54 -04:00
Jason Wilder
2da52ec4fe
Fix deadlock in pd1_test.go
...
The defer tx.Rollback() tries to free the queryLock but the defer e.Cleanup() runs
before it and tries to take a write lock on the query lock (which blocks) and prevents
tx.Rollback() from acquring the read lock.
2015-10-05 20:09:54 -04:00
Jason Wilder
7e0df18e1a
Update simple8b api usage
2015-10-05 20:09:54 -04:00
Jason Wilder
cb23f5ac53
Add a compressed boolean encoding
...
Packs booleans into bytes using 1 bit per value.
2015-10-05 20:09:54 -04:00
Jason Wilder
1196587dc4
Keep track of the type of the block encoded
...
Allowes decode to decode an arbitrary block correctly.
2015-10-05 20:09:54 -04:00
Jason Wilder
731ae27123
Remove unnecessary allocations from int64 decoder
...
The decoder was creating a large slice and decoding all values when
instead, it could decode one packed value as needed.
2015-10-05 20:09:54 -04:00
Jason Wilder
95046c1e37
Add test assertions for time encoding type
2015-10-05 20:09:54 -04:00
Jason Wilder
e42d8660d0
Fix run length encoding check
...
Values were run length encoded even when they should not have been
2015-10-05 20:09:54 -04:00
Jason Wilder
092689c131
Reduce memory allocations
...
Converting between different encoders is wasting a lot of memory allocating different
typed slices.
2015-10-05 20:09:54 -04:00
Jason Wilder
ce1d45ecda
Use zigzag encoding for timestamp deltas
...
Previously were using a frame of reference approach where we would
transform the (possibly negative) deltas into positive values from
the minimum. That required an extra pass over the values as well
as a large slice allocation so we could encode the originals in uncompressed
form if they were too large.
This switches the encoding to use zigzag encoding for the deltas which
removes the extra slice allocation as well as the extra loops.
Improves encoding performane by ~4x.
2015-10-05 20:09:54 -04:00
Jason Wilder
4a37ba868d
Add int64 compression
...
This is using zig zag encoding to convert int64 to uint64s and then using simple8b
to compress them, falling back to uncompressed if the value exceeds 1 << 60. A
patched encoding scheme would likely be better in general but this provides decent
compression for integers that are not at the ends of the int64 range.
2015-10-05 20:09:53 -04:00
Jason Wilder
42e1babe7f
Add time and float compression
...
Time compression uses an adaptive approach using delta-encoding,
frame-of-reference, run length encoding as well as compressed integer
encoding.
Float compression uses an implementation of the Gorilla paper encoding
for timestamps based on XOR deltas and leading and trailing null suppression.
2015-10-05 20:09:53 -04:00
Jason Wilder
112a03f24c
Fix go vet errors
2015-10-05 20:08:58 -04:00
Jason Wilder
88248f3f81
Ensure we have files when iterating in cursor
...
Prevents index out of bounds panic
2015-10-05 20:08:58 -04:00
Paul Dix
db4ad33f3c
Update tests to use transactions. Add test for single series 10k points.
2015-10-05 20:06:22 -04:00
Paul Dix
0b33a71bb7
Add recover to maintenance. Change snapshot writer to not use bolt on shard.
2015-10-05 20:06:22 -04:00
Paul Dix
0fd116d1f2
Ensure data files can't be deleted while query is running.
...
Also ensure that queries don't try to use files that have been deleted.
2015-10-05 20:06:22 -04:00
Paul Dix
b1bdb4f15a
Make compaction run at most at set duration.
2015-10-05 20:06:22 -04:00
Paul Dix
1c8eac1523
Add PerformMaintenance to store for flushes and compactions.
...
Also fixed shard to work again with b1 and bz1 engines.
2015-10-05 20:06:22 -04:00
Paul Dix
d694454f47
Fix wal flushing, compacting, and write lock
2015-10-05 20:06:22 -04:00
Paul Dix
6c94e738a0
Add support for multiple fields
2015-10-05 20:06:22 -04:00
Paul Dix
667b3e6c08
Handle hash collisions on keys
2015-10-05 20:06:22 -04:00
Paul Dix
48069e782c
Add compaction and time range based write locks.
2015-10-05 20:06:22 -04:00
Paul Dix
2eb2a647d6
Add multicursor to combine wal and index
2015-10-05 20:06:22 -04:00
Paul Dix
7baba84a21
Ensure we don't have duplicate values. Fix panic in compaction.
2015-10-05 20:06:22 -04:00
Paul Dix
0770ccc87d
Make writes to historical areas possible
2015-10-05 20:06:21 -04:00
Paul Dix
982c28b947
Update to work with new cursor definitiono and Point in models
2015-10-05 20:06:21 -04:00
Paul Dix
365a631b53
Update wal to only open new segment file on flush if its not an idle flush
2015-10-05 20:06:21 -04:00
Paul Dix
7c8ab4f1d8
Add test for close and restart of engine and fix errors.
2015-10-05 20:06:21 -04:00
Paul Dix
c5f6c57d7f
Update engine to put index at the end of data files
2015-10-05 20:06:21 -04:00
Paul Dix
fe1f9a51e5
Add memory settings and WAL backpressure
2015-10-05 20:06:21 -04:00
Paul Dix
5e59cb9393
Update encoding test to work with new interface.
2015-10-05 20:06:21 -04:00
Paul Dix
2100e66437
Add full durability to WAL and flush on startup
2015-10-05 20:06:21 -04:00
Paul Dix
82e1be7527
WIP: more WAL work
2015-10-05 20:06:21 -04:00
Paul Dix
2ba032b7a8
WIP: finish basics of PD1. IT WORKS! (kind of)
2015-10-05 20:06:21 -04:00
Paul Dix
7555ccbd70
WIP: engine work
2015-10-05 20:06:21 -04:00
Paul Dix
12ea1cb26f
Add comment about encoding float
2015-10-05 20:06:21 -04:00
Paul Dix
fb2a1cb2f3
WIP: skeleton for encoding for new engine
2015-10-05 20:06:20 -04:00
David Norton
4375545064
fix #4276 : walk DropSeriesStatement
2015-10-05 19:56:30 -04:00
Philip O'Toole
2ac0357406
Support dropping non-Raft nodes
2015-10-04 00:19:52 -07:00
Philip O'Toole
d74e0690c7
Revert "Merge pull request #4233 from influxdb/drop-server"
...
This reverts commit 0bdb36f6dc
, reversing
changes made to 3085fbc138
.
2015-10-02 08:39:57 -07:00
Cory LaNou
f50813460e
protobuf update.. :-(
2015-10-01 15:39:15 -05:00
Jason Wilder
06c143c2dd
Handle missing values in aggregate derivative queries better
...
If an aggregate derivative query did not have a value in the first
time bucket, it would abort early and return a single row with value
of 0. Similarly, if either the current or previous value was nil,
it would skip the row and not append any values causing gaps and
no data to show up.
Instead, this will append a nil value if either the current or previous
valis is nil. This essentially allows nil values to carry through the
results as well as gives a more sensible value for rows where we cannot
compute a difference (instead of dropping the row as before).
Fixes #4237 #4263
2015-10-01 13:05:19 -06:00
Ben Johnson
343dd23ee7
refactor map functions to use list of values
...
This commit changes `tsdb.mapFunc` to use `tsdb.MapInput` instead
of an iterator. This will make it easier and faster to pass blocks
of values from the new storage engine into the engine.
2015-09-29 14:00:33 -06:00
Cory LaNou
842b36f30f
godoc
2015-09-23 15:52:44 -05:00
Cory LaNou
715e3d8d3c
add comment to clarify
2015-09-23 15:49:17 -05:00
Cory LaNou
d6ee542083
SelectFilterFields -> SelectWhereFields
2015-09-23 15:47:12 -05:00
Cory LaNou
b2474c9abf
reduce arguments to MapTopBottom
2015-09-23 15:44:36 -05:00
Cory LaNou
104c049a53
fix typo
2015-09-23 15:26:13 -05:00
Cory LaNou
81ad1f87a4
refactor MapTopBottom signature
2015-09-23 10:53:21 -05:00
Cory LaNou
7c3a542e13
revert signatures for mean/sum map funcs
2015-09-23 10:32:00 -05:00
Cory LaNou
6b19e3dd9c
minor refactorings
2015-09-23 09:47:24 -05:00
Cory LaNou
b1becfbcfc
more tests, simplify mapFunc signatures
2015-09-23 09:27:19 -05:00
Cory LaNou
78bc740434
full support for min/max/first/last
2015-09-23 09:03:57 -05:00
Ben Johnson
8e27cf1fd8
Merge remote-tracking branch 'upstream/master' into refactor-select-mapper
...
Conflicts:
tsdb/store.go
2015-09-22 13:58:24 -06:00
dgnorton
410eb4ece4
Merge pull request #3852 from influxdb/dmq-show-tag-keys
...
convert SHOW TAG KEYS to distributed query
2015-09-22 15:28:08 -04:00
Ben Johnson
96715d7d90
rename Cursor.Seek() to Cursor.SeekTo()
2015-09-22 13:23:16 -06:00
Ben Johnson
56cb2fae5d
fix integration tests
2015-09-22 13:10:13 -06:00
Ben Johnson
649663ca15
fix tests
2015-09-22 13:10:13 -06:00
Ben Johnson
b213ddad78
refactor cursor
2015-09-22 13:10:12 -06:00
Ben Johnson
a5269e9cc7
rename direction to ascending.
2015-09-22 13:09:26 -06:00
Ben Johnson
1b8b625787
refactor SelectMapper
2015-09-22 13:09:26 -06:00
Ben Johnson
0883182798
simplify select mapper stmt
2015-09-22 12:27:14 -06:00
Nathaniel Cook
79e6e3e07a
Merge pull request #4196 from influxdb/export_iterator
...
export tsdb.Iterator
2015-09-22 11:09:08 -06:00
Philip O'Toole
d96119cc01
Correct MapperValueAsJSON comment
...
[ci skip]
2015-09-22 09:49:57 -07:00
Nathaniel Cook
007508bde0
export tsdb.Iterator
2015-09-22 10:29:49 -06:00
Philip O'Toole
e43ad09f44
Perform custom unmarshal during unit tests
2015-09-22 00:24:43 -07:00
Philip O'Toole
56193be05c
Simple tests work
2015-09-21 22:50:29 -07:00
Philip O'Toole
07c9d40b87
Merge pull request #4179 from mark-rushakoff/sorted-measurements-by-tag-filters
...
Sort DatabaseIndex.measurementsByTagFilters result
2015-09-21 11:19:37 -07:00
David Norton
e176d2c0dd
make SHOW TAG KEYS support (S)LIMIT & (S)OFFSET
2015-09-21 13:05:53 -04:00
David Norton
8e236532e8
delete unused executeShowTagKeysStatement func
2015-09-21 11:32:13 -04:00
David Norton
8bd2408320
convert SHOW TAG KEYS to distributed query
2015-09-21 11:30:51 -04:00
Daniel Morsing
57b211a538
Merge branch 'master' into bottom
...
resolve changelog conflict
2015-09-21 12:31:29 +00:00
Mark Rushakoff
85275e7d59
Sort DatabaseIndex.measurementsByTagFilters result
...
Fixes #4118
2015-09-20 14:37:27 -07:00
Daniel Morsing
5b24841973
rename topReduceOut
...
Also, gofmt
2015-09-18 15:04:09 +00:00
Daniel Morsing
b07c36288d
Merge pull request #4130 from influxdb/topopt
...
optimize top queries
2015-09-18 14:40:57 +00:00
Daniel Morsing
b5fc2a96b1
Implement bottom
...
After the rework of the ordering for top, this was just a matter
of adding a flag telling the minheap which way the values should
be sorted.
2015-09-18 14:39:46 +00:00
Daniel Morsing
ce1dc840ef
Better documentation for typeCompare.
2015-09-18 13:11:46 +00:00
Philip O'Toole
f9bfb2fcc5
Merge pull request #4142 from influxdb/nil_partition
...
If partition is nil return on Close immediately
2015-09-17 16:37:37 -07:00
Cory LaNou
72f6f7d268
Merge pull request #4134 from influxdb/issue-3447
...
Refactor Points and Rows to dedicated packages
2015-09-17 15:27:48 -05:00
Daniel Morsing
d9e4b3b851
split comparison into 2 stages, one for type and one for value
...
Also, fix it so that heap inserts sort on tags as well.
This change makes it easier to implement bottom and also fixes the test that the previous change broke.
2015-09-17 14:59:02 +00:00
Philip O'Toole
f73bc6ff19
Merge pull request #4140 from influxdb/engine_config
...
Make engine configurable
2015-09-16 21:17:43 -07:00
Philip O'Toole
5e991f1703
If partition is nil return on Close immediately
2015-09-16 19:38:02 -07:00
Philip O'Toole
7b1a4e6700
Control whether each query should be logged
...
Fixes issue #4138
2015-09-16 19:26:23 -07:00
Philip O'Toole
e4fde993f1
Make engine configurable
2015-09-16 19:09:25 -07:00
Cory LaNou
ba830be3b9
actually move influxql.Row* -> models.Row*
2015-09-16 16:32:50 -05:00
Cory LaNou
d19a510ad2
refactor Points and Rows to dedicated packages
2015-09-16 15:33:08 -05:00
Cameron Sparr
6d4319d244
Add function to tsdb.point to get line-protocol string in the correct units
2015-09-16 10:53:59 -07:00
Daniel Morsing
59307b8b78
optimize top queries
...
Instead of rounding up the points, sorting and then slicing, keep a
heap that allows us to quickly see if the point needs to be in the
set. This cuts a top query on a dataset of 8 million points from 35
seconds to 11 seconds.
2015-09-16 17:02:44 +00:00
Cory LaNou
5395ac7634
Period in field name that matches measurement fails. Fixes #3457
2015-09-15 16:26:39 -05:00
Daniel Morsing
51b22f180f
simplify the sorting for top/distinct.
...
Instead of using closures and 2 different sort routines, have an
interface compare method that makes it easy to switch directions
for comparisons.
Note that this changes the sort order of distinct to match that of
top. While it is a change, I don't think it will break any code. The
important thing for distinct is just that the ordering is absolute,
not what the order is.
2015-09-15 15:03:31 +00:00
Philip O'Toole
93f1fcb38c
Merge pull request #4073 from influxdb/wal_stat_fixes
...
WAL statistics fixes
2015-09-14 07:10:26 -07:00
Daniel Morsing
eb65f50593
gofmt
2015-09-11 14:41:59 +00:00
Daniel Morsing
ab9ae468ca
unexport some functions
...
For good measure basically. Not needed for correctness, but it keeps
people from using these.
2015-09-11 14:39:09 +00:00
Daniel Morsing
66fc270d1e
Move aggregate funcs into tsdb
...
Pure move, No functional changes.
2015-09-11 14:27:47 +00:00
Cory LaNou
5c1ba44c9b
Merge pull request #4071 from influxdb/issue-3902
...
Issue 3902
2015-09-11 06:58:27 -05:00
Philip O'Toole
40b1068c81
Use unified statMap for WAL
...
Don't declare distinct stat map for partitions. It's more useful to see
the stats collated together per-WAL. This may need further change in the
future.
2015-09-10 14:23:40 -07:00
Philip O'Toole
13a302e533
WAL tag keys are "path" not "bind".
2015-09-10 14:10:45 -07:00
Philip O'Toole
bf55f61edd
Add stats for the WAL
2015-09-10 12:30:47 -07:00
Philip O'Toole
5086ea42fa
Update WAL comments
...
[ci skip]
2015-09-10 11:29:43 -07:00
Philip O'Toole
101a4d2a55
Merge pull request #4066 from influxdb/pd-fail-writes-on-memory-pressure
...
Update WAL to fail writes if pressure too high.
2015-09-10 11:27:32 -07:00
Cory LaNou
05f69b3d6c
Validate Dimension for proper time scenarios. Fixes #3902
2015-09-10 13:10:21 -05:00
Philip O'Toole
ed917aa7f8
Log each query before execution
...
If a query causes a server to panic, the query is nowhere in the logs,
which makes debug very difficult.
2015-09-10 10:52:51 -07:00
Ben Johnson
733fa0a109
disable bz1 recompression
...
This commit only appends new blocks of points and disables checks for
recompressing small blocks at the end of a series.
2015-09-10 11:26:29 -06:00
Paul Dix
2d67a9ea22
Update WAL to fail writes if pressure too high.
...
If the memory gets 5x above the partition size threshold, the WAL will start returning write failures to the clients. This will allow them to backoff their write volume.
Also updated the stress script to track failed requests and output messages on failure and when it returns to success.
2015-09-09 22:41:32 -07:00
Paul Dix
482e00d3e3
Merge pull request #4011 from influxdb/pd-simplify-wal
...
Simplify WAL to not compact
2015-09-08 22:32:53 -07:00
Philip O'Toole
3db9cc9b76
Basic instrumentation for shards
2015-09-08 19:56:42 -07:00
Philip O'Toole
2b0a40e262
Merge pull request #4007 from influxdb/instrument_bz1
...
Basic instrumentation for bz1 engine
2015-09-08 19:56:06 -07:00
Paul Dix
ecbc79e7e3
Fix disksize to work with new WAL
2015-09-08 19:37:33 -07:00
Philip O'Toole
5373f263a3
Add pending control to batcher
...
With this change, the generic batcher used by many inputs can now be
buffered. Testing shows that this performance of the Graphite input by
10-100%, with the biggest improvements at lower numbers of connections.
2015-09-08 19:32:00 -07:00
Paul Dix
dfd6b11dda
Fix memory compaction logic.
...
* Only fire a go routine to flush and compact if it isn't already running
* Have a sleep backoff time that scales up as the percentage of memory used goes up
2015-09-08 19:28:29 -07:00
Paul Dix
a1fb77198b
Simplify WAL to not compact since it doesn't really help the engine anyway
2015-09-08 19:28:29 -07:00
Philip O'Toole
76903f7440
Instrument bz1 engine
2015-09-08 19:09:39 -07:00
Cory LaNou
62e9c24b25
fixes #3926
2015-09-08 14:15:48 -05:00
Philip O'Toole
255fb3364d
Don't close nil channel
...
Obviously a TSDB store may be closed before it's opened.
2015-09-08 10:49:45 -07:00
Jason Wilder
6b2b29625d
Ensure the tsdb.Store is not closed before creating a shard
...
Fixes panic: assignment to entry in nil map
Fixes #3848
2015-09-08 11:04:00 -06:00
Ben Johnson
fd9be63b4e
rollback bolt tx on mapper open error
...
This commit fixes `SelectMapper.Open()` so that it properly rolls back
transactions. Previously, this caused transactions to stay open
indefinitely which caused mmap resizes to hang indefinitely.
2015-09-08 10:28:51 -06:00
Jason Wilder
13dbc8f0ba
Merge pull request #3841 from influxdb/jw-file-utils
...
Add inspect tool
2015-09-04 14:12:05 -06:00
Cory LaNou
fa4415b3a4
refactor processing top/bottom results. clarify some comments
2015-09-04 13:30:43 -05:00
Cory LaNou
b62d8c0515
expand variable names for clarity
2015-09-04 13:30:43 -05:00
Cory LaNou
65e652850a
btf -> tmin
2015-09-04 13:30:42 -05:00
Cory LaNou
08295c578f
refactor processTopBottom
2015-09-04 13:30:42 -05:00
Cory LaNou
9ab3d89c06
bucketTime* -> tMin*
2015-09-04 13:30:42 -05:00
Cory LaNou
3ca93594c3
BucketTime -> TMin
2015-09-04 13:30:42 -05:00
Cory LaNou
347ffc70b4
wire up advanced top sorting/slicing
2015-09-04 13:30:41 -05:00
Cory LaNou
8c4595b345
top is coming together. filling out fields properly
2015-09-04 13:30:41 -05:00
Cory LaNou
ba79007960
wip
2015-09-04 13:30:41 -05:00
Cory LaNou
35b9215aa9
refactor processTopBottom - wip
2015-09-04 13:30:40 -05:00
Cory LaNou
046282249a
wip remapping top output
2015-09-04 13:30:40 -05:00
Cory LaNou
97f2dc830f
comment/type fixes
2015-09-04 13:30:39 -05:00
Cory LaNou
72fd115dc2
exposing tags on cursors, top/bottom are valid funcs now
2015-09-04 13:30:39 -05:00
Jason Wilder
6b4926257a
Add inspect tool
...
Start of a lower-level file inspection tool. This currently dumps
summary statistics for the shards, index and WAL that can be used to
understand the shape of the data is in the local shards. This util
operates on the shards itself and not through the server and is intended
more for debugging/troubleshooting.
2015-09-04 10:38:59 -06:00
Jason Wilder
6f41c0fa87
Merge pull request #3986 from influxdb/jw-order-by
...
Support sorting by time desc
2015-09-04 09:42:58 -06:00
Jason Wilder
df70a1c8ce
Update tests to use Direction enum
2015-09-04 09:00:11 -06:00
Jason Wilder
e767feb8d9
Fix order by desc with aggregate function not return any values
2015-09-03 22:31:58 -06:00
Jason Wilder
7fa3d445f7
Support reverse iteration for b1 engine
2015-09-03 22:31:58 -06:00
Jason Wilder
2725757dba
Simplify WAL cursor seek movement logic
2015-09-03 22:31:58 -06:00
Jason Wilder
5a6b0afc4b
Replace cursor direction with a type
2015-09-03 22:31:48 -06:00
Jason Wilder
7c67e60c4f
Add bz1 reverse cursor test
2015-09-03 22:28:36 -06:00
Jason Wilder
5e481181bc
Add WAL reverse cursor test
2015-09-03 22:28:36 -06:00
Jason Wilder
e206021e13
Add reverse multi-cursor tests
2015-09-03 22:28:36 -06:00
Jason Wilder
266bdc1c2b
Support sort by time DESC in wal and bz1 engines
2015-09-03 22:28:36 -06:00
Philip O'Toole
e07432c59f
Implement diagnostics support
...
This change adds support for diagnostics by decomposing the existing
interface into two interfaces -- one for stats, and the other for
diags. It also adds some basic monitor of system, network, and the Go
runtime.
2015-09-03 20:50:54 -07:00
Cory LaNou
6592dcc699
EnableLogging -> LoggingEnabled
2015-09-03 16:56:07 -05:00
Ben Johnson
deff06f850
add copier service
...
This commit adds the copier service which allows one server to
copy shards from another server. This will be used for moving
shards in the cluster.
2015-09-03 13:07:35 -06:00
David Norton
816c5f5368
fix #2555 : don't normalize target names
2015-09-03 07:12:15 -04:00
David Norton
99a22c174b
fix #2555 : add backreference in CQs
...
Add new query syntax to allow the following in CQs:
INTO "1hPolicy".:MEASUREMENT
2015-09-03 07:12:15 -04:00
Ben Johnson
b63ebb72a5
limit bz1 quickcheck tests to 10 iterations on CI
...
This commit checks the `CI` environment variable in the bz1
test suite and limits the quickcheck runs if the value is `true`.
2015-09-02 11:27:11 -06:00
Philip O'Toole
e15bc6df11
Remove obsolete TSDB monitor file
...
This functionality will be superseded by the new monitor service.
2015-09-01 23:15:57 -07:00
Philip O'Toole
14c04eb4d6
Merge pull request #3916 from influxdb/new_stats_diags
...
Statistics and Diagnostics service
2015-09-01 18:30:53 -07:00
Philip O'Toole
f05dc20b58
Hook new monitor service to server
...
u
2015-09-01 15:03:52 -07:00
Jason Wilder
898ee8c399
Fix write fails for multiple points when tag starts with quote
...
Fixes #3928
2015-09-01 11:20:34 -06:00