Commit Graph

21 Commits (be4891c40b644400d18af562f3c846dbfb340e90)

Author SHA1 Message Date
Jason Wilder 3a8a19a99d Implement LoadMetaDataIndex for tsm1dev engine 2015-12-02 13:38:06 -07:00
Jason Wilder 4a03469662 Integrate TSM compaction into dev engine 2015-12-02 09:45:23 -07:00
Jason Wilder 1485ea7e41 Implement Size on TSMReader 2015-12-02 09:45:23 -07:00
Jason Wilder d4b1c25f8e Add CompactionPlanner type
CompactionPlanner is used to determine which files (WAL Segments, TSM
Files) to include in a given compaction run.
2015-12-02 09:45:23 -07:00
Jason Wilder acdb6bcdf6 Add TSMKeyIterator
Allows iterating of multiple TSM files in sort key and values order.
2015-12-02 09:45:23 -07:00
Jason Wilder 4b6767bf01 Add MMAP based file reader 2015-12-02 09:45:23 -07:00
Jason Wilder a6541937f8 Add dumptsmdev to influx_inspect
Allow inspecting the updated TSM format.
2015-11-24 08:50:13 -07:00
Jason Wilder 25206c729c Add compactor type 2015-11-24 08:50:07 -07:00
Jason Wilder 0d1508a7c6 Add comments for search 2015-11-17 23:24:10 -07:00
Jason Wilder a7d7c280ed Add block type to index
This will faciliate loading a block into a type specific result without
first loading the block.  This will also allow us to populate the database
index solely from the index.
2015-11-17 23:24:09 -07:00
Jason Wilder e5022a898d Support decoding into type specific slices
There is a lot of allocations performed when decoding blocks.  These
types can be re-used to reduce allocations in many cases.  This change
allows a type specific slice to be passed in to decode funcs to be re-used
if it is large enough.

The existing decode is is left for backwards compatibility but is not
very efficient right now.  It may be removed.
2015-11-17 23:24:09 -07:00
Jason Wilder c2530e93d7 Add mutexes around seeker usage
These are not goroutine safe.
2015-11-16 09:05:27 -07:00
Jason Wilder ef18f8afb2 Handle TSM key deletions
This writes a tombstone file containing a line per deleted key. This
file is read when a TSMReader is created and any keys listed in the file
are removed from the index.
2015-11-16 08:44:52 -07:00
Jason Wilder ed7cfb6df3 Add Keys function to TSMIndex
Useful for testing
2015-11-16 08:44:52 -07:00
Jason Wilder d8c0c26934 Return error if number of blocks would overflow 2015-11-16 08:44:52 -07:00
Jason Wilder 16c5e0a2e0 Add Close to TSMWriter interface 2015-11-16 08:44:52 -07:00
Jason Wilder 0ab423c7ff Initial FileStore implementation
Provides functionality to load a directory of TSM files (or add them manually)
as well as reading blocks of values for individual key and times.
2015-11-16 08:44:52 -07:00
Jason Wilder e4312d854c Fix typo 2015-11-12 09:53:38 -07:00
Jason Wilder 8e14877e98 Add diagrams/documentation about indirect indexing strategy 2015-11-11 17:01:20 -07:00
Jason Wilder 7df1c2dd31 Add an indirectIndex implementation
Implements the indirect indexing approach to allow indexes on MMAPed files
that may be larger than available RAM
2015-11-11 15:16:11 -07:00
Jason Wilder 06016882ab Add revised TSM file writer/readers
This adds some basic file reader/writers for creating the updated TSM file format.  It uses a simple
in-memory index without MMAP for now, but will be extended to use and indirect indexing approach as well as MMAPed file access as described in the design doc.

This code is not integrated into the TSM engine yet
2015-11-11 12:52:34 -07:00