83ccaaa656
Reload cache at startup
2015-12-02 14:16:36 -07:00
ba99dece0c
Wire up tsm1dev engine cursor
2015-12-02 14:01:10 -07:00
3a8a19a99d
Implement LoadMetaDataIndex for tsm1dev engine
2015-12-02 13:38:06 -07:00
3014d7e391
Return errors for func not implemented in tsm1dev engine
2015-12-02 11:06:01 -07:00
a7e21c2975
Don't set a cache memory limit by default
...
100mb is easy it hit even with basic stress test config. Don't set
a limit by default so that an operator can size it appropriately based
on their hardware.
2015-12-02 11:01:13 -07:00
6847a6ba0c
Fix rebase
2015-12-02 09:47:16 -07:00
751d1dd467
Don't rewrite TSM files while WAL segments exist
...
This approach is not working and needs to be reworked.
2015-12-02 09:45:24 -07:00
5744f5ba02
Add ability to filter values by time when writing TSM files
2015-12-02 09:45:24 -07:00
708266da69
Cache related compaction fixes
2015-12-02 09:45:24 -07:00
231c052003
Don't limit WAL segments during compaction
...
Since they are already loaded in the cache, this limit is not really
needed anymore.
2015-12-02 09:45:24 -07:00
7e249e0555
Use CacheKeyIterator instead of WALKeyIterator during compactions
2015-12-02 09:45:24 -07:00
4a03469662
Integrate TSM compaction into dev engine
2015-12-02 09:45:23 -07:00
78fda2b89b
Implement WAL SegmentStats for compactions
2015-12-02 09:45:23 -07:00
1485ea7e41
Implement Size on TSMReader
2015-12-02 09:45:23 -07:00
d4b1c25f8e
Add CompactionPlanner type
...
CompactionPlanner is used to determine which files (WAL Segments, TSM
Files) to include in a given compaction run.
2015-12-02 09:45:23 -07:00
5291fbcf39
Add TSM support to MergeIterator
...
Enables the ability to combine multiple TSM files into one as well
as merge existing TSM files with newer WAL segment values.
2015-12-02 09:45:23 -07:00
acdb6bcdf6
Add TSMKeyIterator
...
Allows iterating of multiple TSM files in sort key and values order.
2015-12-02 09:45:23 -07:00
4b6767bf01
Add MMAP based file reader
2015-12-02 09:45:23 -07:00
fc83968e2e
Cache values supports sorting order
2015-12-01 13:24:25 -08:00
3a72e40e3f
Implement descending cursor support
2015-12-01 13:24:25 -08:00
ec4daaccff
Test ascending tsm1dev cursor
2015-12-01 13:24:25 -08:00
59674fda21
Integrate cache query with tsm1dev engine
2015-12-01 13:24:25 -08:00
7da3fc1aeb
Merge pull request #4934 from influxdb/dedupe_sort_order
...
Deduplicate supports requesting sort order
2015-12-01 16:23:25 -05:00
bad0f657de
Deduplicate supports requesting sort order
2015-11-30 16:21:44 -08:00
6b3c6a90a1
Merge pull request #4911 from influxdb/integrate_cache
...
Integrate cache with tsm1dev write path
2015-11-30 14:58:42 -08:00
b858b11286
fix network closed error when show measurements
2015-11-30 10:44:00 +08:00
8649ce4c49
Integrate cache with tsm1dev write path
2015-11-26 06:07:19 -08:00
1bca38bb84
Cache supports writing multiple keys
...
This keeps the locking to a minimum if the data is available for
multiple keys at once.
2015-11-26 06:07:16 -08:00
41459cf687
fix flush deadlock
...
This commit fixes a deadlock that occurs during b1 flushes. It's
caused by taking locks in a different order. In the flush, b1
locks the engine and then bolt. However, in the query cursor, a
lock is obtained on bolt first (via `DB.Begin()`) and then the
engine is locked while reading from the engine's cache.
2015-11-25 15:00:06 -07:00
8e7dc3bef9
WAL returns current segment ID on write and delete
2015-11-25 12:23:10 -08:00
bce68ed43d
Merge pull request #4833 from li-ang/fix_show_measurements
...
Fix SHOW MEASUREMENTS across the cluster
2015-11-25 10:47:44 -08:00
ce8cf05b0e
Merge pull request #4889 from influxdb/mapper-execute-close
...
Implement close notifier and timeout on executors
2015-11-25 08:24:00 -06:00
d931f5dd22
Merge pull request #4900 from influxdb/jw-compact
...
WAL segment compaction
2015-11-24 21:35:13 -07:00
34bffd5e18
Code review fixes
2015-11-24 21:24:13 -07:00
be488b7d12
implement close notifier and timeout on executors
2015-11-24 21:07:18 -06:00
1ce8d6290b
Remove values pool replacement
...
Getting an intermittent test failure with this so removing it for now since compactions
are still able to keep up without it. Will need to look into this further because the
allocations is still very high and will affect compactions over longer periods of time.
2015-11-24 13:40:07 -07:00
0832a03333
Move pools to separate file
2015-11-24 09:44:37 -07:00
a6541937f8
Add dumptsmdev to influx_inspect
...
Allow inspecting the updated TSM format.
2015-11-24 08:50:13 -07:00
25206c729c
Add compactor type
2015-11-24 08:50:07 -07:00
f8b4950ea9
Enhance tsm1dev logging
2015-11-23 14:24:39 -08:00
f70323cb89
Add MergeIterator
...
MergeIterator will be used to merge multiple TSM KeyIterators and the
WAL KeyIterator using a stream based iteration approach. Each iteration
cycle returns a key and values ordered in way to write a new TSM file
optimally.
2015-11-23 14:59:15 -07:00
5334271e26
Add KeyIterator for WAL segments
...
This provides and interface and type to combine multiple WAL segments
in order and then allow the values to be read in an order suitable for
writing to a TSM file.
2015-11-23 14:59:15 -07:00
d2b045f89b
Code cleanup
2015-11-23 14:03:50 -07:00
697cfe604b
Add stubbed out dev tsm engine
...
Starting to integrate some of the components into a engine that is
usable for development purposes. This allows the code to evolve while
keeping the existing TSM engine in tact for reference.
Currently, just the WAL is wired up so writes can be tested. Other engine
functions will panic the server if called.
2015-11-23 13:55:34 -07:00
7461b61bf2
Fix race in WAL and WALSegmentWriter
...
WAL currentSegmentWriter was not accessed under a mutex. The WALSegmentWriter
also did not use a mutex to protect the underlying writer.
2015-11-23 13:55:34 -07:00
aa00ef953a
Fix typo in func names
2015-11-23 13:55:34 -07:00
e2b1a09ece
Implemment WAL write/delete functions
2015-11-23 13:55:33 -07:00
afc0d5bfb9
Add WALSegmentReader/Writer
...
Basic types for reading and writing WAL segment files.
2015-11-23 13:55:33 -07:00
151b33d000
Rename wal.go to log.go
...
This is the existing WAL + cache implementation. Moving it to a separate file
so that it can remain intact while a refactoring to a independent WAL can occur.
The WAL was also named Log in the code so this names file more closely to the concept
in the code.
2015-11-23 13:53:30 -07:00
1a77c76245
Merge pull request #4863 from influxdb/simple_wal_cache
...
Move to simpler cache
2015-11-23 12:49:58 -08:00
Jason Wilder
Philip O'Toole
Philip O'Toole
liang@qiniu.com
Ben Johnson
Cory LaNou
Jason Wilder