Influxdata
/
influxdb
mirror of https://github.com/influxdata/influxdb.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
7,996 Commits
218 Branches
453 Tags
360 MiB
Commit Graph

7996 Commits (668b5b9bfb88ef475d68cd3dce3b21ef96fa4de7)

Author SHA1 Message Date
Jason Wilder cb23f5ac53 Add a compressed boolean encoding 
Packs booleans into bytes using 1 bit per value.
 2015-10-05 20:09:54 -04:00
Jason Wilder 1196587dc4 Keep track of the type of the block encoded 
Allowes decode to decode an arbitrary block correctly.
 2015-10-05 20:09:54 -04:00
Jason Wilder 731ae27123 Remove unnecessary allocations from int64 decoder 
The decoder was creating a large slice and decoding all values when
instead, it could decode one packed value as needed.
 2015-10-05 20:09:54 -04:00
Jason Wilder 95046c1e37 Add test assertions for time encoding type 2015-10-05 20:09:54 -04:00
Jason Wilder e42d8660d0 Fix run length encoding check 
Values were run length encoded even when they should not have been
 2015-10-05 20:09:54 -04:00
Jason Wilder 092689c131 Reduce memory allocations 
Converting between different encoders is wasting a lot of memory allocating different
typed slices.
 2015-10-05 20:09:54 -04:00
Jason Wilder ce1d45ecda Use zigzag encoding for timestamp deltas 
Previously were using a frame of reference approach where we would
transform the (possibly negative) deltas into positive values from
the minimum.  That required an extra pass over the values as well
as a large slice allocation so we could encode the originals in uncompressed
form if they were too large.

This switches the encoding to use zigzag encoding for the deltas which
removes the extra slice allocation as well as the extra loops.

Improves encoding performane by ~4x.
 2015-10-05 20:09:54 -04:00
Jason Wilder 4a37ba868d Add int64 compression 
This is using zig zag encoding to convert int64 to uint64s and then using simple8b
to compress them, falling back to uncompressed if the value exceeds 1 << 60.  A
patched encoding scheme would likely be better in general but this provides decent
compression for integers that are not at the ends of the int64 range.
 2015-10-05 20:09:53 -04:00
Jason Wilder 42e1babe7f Add time and float compression 
Time compression uses an adaptive approach using delta-encoding,
frame-of-reference, run length encoding as well as compressed integer
encoding.

Float compression uses an implementation of the Gorilla paper encoding
for timestamps based on XOR deltas and leading and trailing null suppression.
 2015-10-05 20:09:53 -04:00
Jason Wilder 112a03f24c Fix go vet errors 2015-10-05 20:08:58 -04:00
Jason Wilder 88248f3f81 Ensure we have files when iterating in cursor 
Prevents index out of bounds panic
 2015-10-05 20:08:58 -04:00
Paul Dix 12e14f105b Update stress to use second timestamps and less random floats. 2015-10-05 20:08:58 -04:00
Paul Dix db4ad33f3c Update tests to use transactions. Add test for single series 10k points. 2015-10-05 20:06:22 -04:00
Paul Dix 0b33a71bb7 Add recover to maintenance. Change snapshot writer to not use bolt on shard. 2015-10-05 20:06:22 -04:00
Paul Dix 0fd116d1f2 Ensure data files can't be deleted while query is running. 
Also ensure that queries don't try to use files that have been deleted.
 2015-10-05 20:06:22 -04:00
Paul Dix b1bdb4f15a Make compaction run at most at set duration. 2015-10-05 20:06:22 -04:00
Paul Dix 1c8eac1523 Add PerformMaintenance to store for flushes and compactions. 
Also fixed shard to work again with b1 and bz1 engines.
 2015-10-05 20:06:22 -04:00
Paul Dix d694454f47 Fix wal flushing, compacting, and write lock 2015-10-05 20:06:22 -04:00
Paul Dix 6c94e738a0 Add support for multiple fields 2015-10-05 20:06:22 -04:00
Paul Dix 667b3e6c08 Handle hash collisions on keys 2015-10-05 20:06:22 -04:00
Paul Dix 48069e782c Add compaction and time range based write locks. 2015-10-05 20:06:22 -04:00
Paul Dix 2eb2a647d6 Add multicursor to combine wal and index 2015-10-05 20:06:22 -04:00
Paul Dix 7baba84a21 Ensure we don't have duplicate values. Fix panic in compaction. 2015-10-05 20:06:22 -04:00
Paul Dix 0770ccc87d Make writes to historical areas possible 2015-10-05 20:06:21 -04:00
Paul Dix 982c28b947 Update to work with new cursor definitiono and Point in models 2015-10-05 20:06:21 -04:00
Paul Dix 365a631b53 Update wal to only open new segment file on flush if its not an idle flush 2015-10-05 20:06:21 -04:00
Paul Dix 7c8ab4f1d8 Add test for close and restart of engine and fix errors. 2015-10-05 20:06:21 -04:00
Paul Dix c5f6c57d7f Update engine to put index at the end of data files 2015-10-05 20:06:21 -04:00
Paul Dix fe1f9a51e5 Add memory settings and WAL backpressure 2015-10-05 20:06:21 -04:00
Paul Dix 5e59cb9393 Update encoding test to work with new interface. 2015-10-05 20:06:21 -04:00
Paul Dix 2100e66437 Add full durability to WAL and flush on startup 2015-10-05 20:06:21 -04:00
Paul Dix 82e1be7527 WIP: more WAL work 2015-10-05 20:06:21 -04:00
Paul Dix 2ba032b7a8 WIP: finish basics of PD1. IT WORKS! (kind of) 2015-10-05 20:06:21 -04:00
Paul Dix 7555ccbd70 WIP: engine work 2015-10-05 20:06:21 -04:00
Paul Dix 12ea1cb26f Add comment about encoding float 2015-10-05 20:06:21 -04:00
Paul Dix fb2a1cb2f3 WIP: skeleton for encoding for new engine 2015-10-05 20:06:20 -04:00
David Norton f5001903d0 update CHANGELOG.md 2015-10-05 19:58:51 -04:00
Paul Dix e3b30e3a95 Updates based on PR feedback 2015-10-05 19:57:49 -04:00
David Norton 4375545064 fix #4276: walk DropSeriesStatement 2015-10-05 19:56:30 -04:00
Philip O'Toole eb28817afe Don't panic when DROPing non-existent nodes 2015-10-05 16:56:19 -07:00
Michael Desa 7d5deda4af Add https support 2015-10-05 16:47:56 -07:00
Philip O'Toole a542d5509f Merge pull request #4333 from influxdb/retry_mon_create 
Retry monitor storage creation and only on Leader
 2015-10-05 15:54:07 -07:00
Philip O'Toole dba286a2e8 Update CHANGELOG 2015-10-05 15:53:46 -07:00
Philip O'Toole 899e1cc070 Only attempt monitor storage creation on leader 
Since only the leader can create it, the system might as well only
attempt creation on the leader.
 2015-10-05 15:46:35 -07:00
Philip O'Toole 62fab49205 Create monitor storage every monitor cycle 
This functionality is idempotent and is storage has been successfully
created, no attempt will be made to re-create it. This allows the system
to keep trying every cycle, but stop when successful.
 2015-10-05 15:35:21 -07:00
Philip O'Toole 22a385068d Remove post-install script 
This script needs to be re-worked with the new upcoming repo, so remove
this problematic operation for now.
 2015-10-05 14:57:34 -07:00
Philip O'Toole d5000c6500 Allow nightly build script to email failures 2015-10-05 14:33:52 -07:00
Paul Dix ae36c57110 Fix deletes not kept if shutdown before flush on tsm1 2015-10-05 17:21:20 -04:00
Paul Dix a0841c4508 Updates based on @otoolp's PR comments 2015-10-05 17:21:20 -04:00
Philip O'Toole fa5de49d2e Revert to explicit description of package 
The previous way is causing fpm issues due to quoting.
 2015-10-05 13:56:22 -07:00