Getting an intermittent test failure with this so removing it for now since compactions
are still able to keep up without it. Will need to look into this further because the
allocations is still very high and will affect compactions over longer periods of time.
MergeIterator will be used to merge multiple TSM KeyIterators and the
WAL KeyIterator using a stream based iteration approach. Each iteration
cycle returns a key and values ordered in way to write a new TSM file
optimally.
This provides and interface and type to combine multiple WAL segments
in order and then allow the values to be read in an order suitable for
writing to a TSM file.
Starting to integrate some of the components into a engine that is
usable for development purposes. This allows the code to evolve while
keeping the existing TSM engine in tact for reference.
Currently, just the WAL is wired up so writes can be tested. Other engine
functions will panic the server if called.
This is the existing WAL + cache implementation. Moving it to a separate file
so that it can remain intact while a refactoring to a independent WAL can occur.
The WAL was also named Log in the code so this names file more closely to the concept
in the code.
This will faciliate loading a block into a type specific result without
first loading the block. This will also allow us to populate the database
index solely from the index.
There is a lot of allocations performed when decoding blocks. These
types can be re-used to reduce allocations in many cases. This change
allows a type specific slice to be passed in to decode funcs to be re-used
if it is large enough.
The existing decode is is left for backwards compatibility but is not
very efficient right now. It may be removed.
This writes a tombstone file containing a line per deleted key. This
file is read when a TSMReader is created and any keys listed in the file
are removed from the index.
This prevented the encoders from using other implementations of the Value
interface because it would always cast one of the types to our specific
implementations.
This change ensures that if there are any fields in the WHERE clause of
an aggregate that are different from the fields in the SELECT clause,
that the cursors also decode those fields. Otherwise WHERE clauses of
the form 'SELECT f(w) FROM x WHERE y=z' will return incorrect results
Fixes issue #4701.
This adds some basic file reader/writers for creating the updated TSM file format. It uses a simple
in-memory index without MMAP for now, but will be extended to use and indirect indexing approach as well as MMAPed file access as described in the design doc.
This code is not integrated into the TSM engine yet
Jason Wilder
Philip O'Toole
Philip O'Toole
mengjinglei
Jason Wilder
Cory LaNou
CrazyJvm
Ross McDonald