Commit Graph

8682 Commits (eafb703afc77a2a6b41fc041ec7eebdddc1e2f19)

Author SHA1 Message Date
Paul Dix eafb703afc Update TSM engine, WAL and encoding
* Add InfluxQLType to Values to map the TSM type to InfluxQL
* Fix bug in WAL where close wouldn't nil out the currentSegment after closing it
* Export writeSnapshot to be used in tests, add argument to run it async or not
* Update reloadCache to load temporary metadata information in the engine
* Update LoadMetadataIndex to use the temp WAL metadata information
2015-12-04 11:09:39 -05:00
Paul Dix b7bae53405 Merge pull request #4980 from influxdb/cursor_desc
Fix descending cache cursor
2015-12-04 07:02:13 -05:00
Jason Wilder 49b47d3584 Merge pull request #4978 from influxdb/jw-regressions
TSM dev regression fixes
2015-12-03 20:31:51 -07:00
Philip O'Toole 2d79d7e35f Fix descending cache cursor 2015-12-03 14:34:29 -08:00
Jason Wilder 94e5dc075b Change default stress batch size to 5000
10000 is too large and causes a lot of allocations.  5000 doubles the
throughput.
2015-12-03 14:41:46 -07:00
Jason Wilder 66c9ef862e Fix regressions
Something broke with writing to the WAL now that compactions are running
concurrently.  There was also a performance problem with Next/Prev doing
twice as many searches as necessary.
2015-12-03 14:25:03 -07:00
Cory LaNou b73b21f062 Merge pull request #4975 from influxdb/issue-4974
Fix data race in TSDB
2015-12-03 13:59:55 -06:00
Cory LaNou 7e1dc0231d fix data race in TSDB 2015-12-03 13:52:27 -06:00
Jason Wilder a8f4af36af Merge pull request #4969 from influxdb/jw-tsm-query
TSM Queries
2015-12-03 12:46:29 -07:00
Jason Wilder adf5c5b223 Replace Next/Prev with Scan 2015-12-03 12:39:13 -07:00
Jason Wilder 193a36eeb6 Fix code review comments 2015-12-03 12:39:13 -07:00
Jason Wilder 2019e70331 Fix reading string blocks
The block value to decode was 4 bytes too long so decoding string
block returned a snappy decode error.
2015-12-03 12:39:13 -07:00
Jason Wilder 2ad32af7ea Add desc quey support 2015-12-03 12:39:13 -07:00
Jason Wilder be59ba3455 Add Prev support to FileStore
Allows read the previous block of values given a timestamp and key.
2015-12-03 12:39:12 -07:00
Jason Wilder e9832d7414 Add multi-field cursor support to devtsm1 engine 2015-12-03 12:37:47 -07:00
Jason Wilder 6fba01df89 Implement single field TSM queries 2015-12-03 12:35:36 -07:00
Paul Dix 79477f5ce7 Merge pull request #4966 from influxdb/pd-walcache-perf
Update TSM cache, compact, wal, encoding
2015-12-03 14:28:59 -05:00
Paul Dix 4624fb2a78 Update cache to address PR comments 2015-12-03 14:03:11 -05:00
Cory LaNou deec874fc8 Merge pull request #4972 from influxdb/meta-config-logging-enabled
Add option to disable logging for meta service. useful for testing
2015-12-03 12:00:56 -06:00
Cory LaNou 8cffa4d9a2 changelog 2015-12-03 11:13:52 -06:00
Cory LaNou 477110f0b7 add option to disable logging for meta service. useful for testing 2015-12-03 11:11:36 -06:00
Paul Dix be4891c40b Update TSM write snapshot, Compactor
* Ensure that writing snapshots in engine are goroutine safe
* Add Clone method to Compactor
2015-12-03 11:49:47 -05:00
Paul Dix 6722e9ff14 Update TSM engine, engine_test, and wal_test
* Address jwilder's comments in #4966
2015-12-03 10:49:47 -05:00
Paul Dix bf65e967aa Add test for compacting multiple TSM files 2015-12-03 10:36:17 -05:00
Cory LaNou 67ea0b757b Merge pull request #4848 from influxdb/cluster-integration
Added framework for cluster integration testing
2015-12-03 08:56:51 -06:00
Cory LaNou edf8e31ee6 update changelog 2015-12-03 08:09:18 -06:00
Cory LaNou 0c2588ed43 cluster retention auto create test 2015-12-03 08:08:57 -06:00
Cory LaNou 52da1c76d6 cluster retention policy tests 2015-12-03 08:08:56 -06:00
Cory LaNou 96c230d968 allow for configurable cluster creation 2015-12-03 08:08:56 -06:00
Cory LaNou ab57bde115 more tests, some refactoring 2015-12-03 08:08:56 -06:00
David Norton 23435299ca disable test parallelism 2015-12-03 08:08:56 -06:00
Cory LaNou 91da96eb34 fix race in test harness 2015-12-03 08:08:56 -06:00
Cory LaNou 6e845839c8 fix flakey test harness 2015-12-03 08:08:56 -06:00
Cory LaNou 967a53cabd start sharing integration tests for cluster tests 2015-12-03 08:08:56 -06:00
Cory LaNou cf4e901078 silence registration logging for testing 2015-12-03 08:04:10 -06:00
Cory LaNou 35b438f8fa silence meta/rpc logging for testing 2015-12-03 08:04:10 -06:00
David Norton 3208564192 fix RP not found error messages 2015-12-03 08:04:09 -06:00
David Norton 657877dd94 make not found err messages more consistent 2015-12-03 08:04:09 -06:00
David Norton 343f00d1f2 fix go vet errors 2015-12-03 08:04:09 -06:00
David Norton 76d291e88e beginning of cluster test harness 2015-12-03 08:04:09 -06:00
Paul Dix b0fb8a0a27 Update TSM cache, compact, wal, encoding
* Update cache to have a single slice of values for a key (removed checkpoints)
* Changed compact.Plan to only worry about TSM files.
* Updated Plan to not return an error since there was no case in which it would.
* Update WAL to not keep stats since they're no longer needed.
* Update engine to flush the Cache/WAL to a new TSM file when the min threshold is hit.
* Split compact logic between TSM compacts and WAL/Cache writes.
* Remove unnecessary merge iterator, wal segment iterator, and other no longer necessary stuff.
* Remove the asending bool from the Dedupe method. Values should always be in ascending order. It's up to the cursor to iterate through values based on the direction. Giving the cursor responsibility makes it so we don't need to sort, dedupe or reallocate anything for different query orders.
* Updated engine to use its locks to ensure writes and cache flushes don't cause a race.
* Update all tests with new signatures. Removed a bunch of tests around TSM rewrites and WAL segment iteration that are no longer necessary.
2015-12-03 08:11:50 -05:00
Gunnar d85f3c3dc7 Merge pull request #4955 from influxdb/ga-config-tweaks
Clarify engine comment about tsm1 in the sample config
2015-12-02 13:51:45 -08:00
Jason Wilder 28b9fdc08d Merge pull request #4959 from influxdb/jw-tsm-query
TSM dev engine startup
2015-12-02 14:37:16 -07:00
Jason Wilder 83ccaaa656 Reload cache at startup 2015-12-02 14:16:36 -07:00
Jason Wilder 94604ccc07 Merge pull request #4956 from influxdb/jw-mmap
TSM File Compaction
2015-12-02 14:01:40 -07:00
Jason Wilder ba99dece0c Wire up tsm1dev engine cursor 2015-12-02 14:01:10 -07:00
Jason Wilder 3a8a19a99d Implement LoadMetaDataIndex for tsm1dev engine 2015-12-02 13:38:06 -07:00
Jason Wilder 3014d7e391 Return errors for func not implemented in tsm1dev engine 2015-12-02 11:06:01 -07:00
Jason Wilder a7e21c2975 Don't set a cache memory limit by default
100mb is easy it hit even with basic stress test config.  Don't set
a limit by default so that an operator can size it appropriately based
on their hardware.
2015-12-02 11:01:13 -07:00
Jason Wilder 6847a6ba0c Fix rebase 2015-12-02 09:47:16 -07:00