9cede5fb71
Address PR comments
2015-12-30 18:06:51 -05:00
26e1c6464a
Update backup to address PR comments
2015-12-30 18:06:51 -05:00
59fbd371fc
Implement backup/restore for TSM.
...
This changes backup and restore to work for TSM. It breaks it for b1 and bz1, but since those are getting removed it's ok.
The backup runs against any host that is specified and can backup either the metasstore, a database, specific retention policy, or a specific shard. It can also take incremental backups with the `since` flag, which will only backup TSM files that have been created since that timestamp.
The backup is safe to run online. However, for shards that are still hot for writes, they won't be able to create new TSM files while the backup for that single shard runs. If the backup isn't too large and the write throughput isn't too high this shouldn't be a problem since the writes will just go into the WAL cache.
2015-12-30 18:06:50 -05:00
b6da176a4b
Fix direct index size not calculated
2015-12-23 18:01:11 -07:00
f9ae8077da
Allow compactions to run when files have tombstones
2015-12-23 18:01:11 -07:00
a38c95ec85
Update compactions to run concurrently
...
This has a few changes in it (unfortuantely). The main change is to run compactions
concurrently. While implementing this, a few query and performance bugs showed up that
are also fixed by this commit.
2015-12-23 18:01:11 -07:00
48d4156eac
Fix blocks not sorted correctly when chunking
2015-12-23 18:01:11 -07:00
bb2562b2ab
Return CompactionGroups from planning
2015-12-23 18:01:11 -07:00
d0ec0a15e2
Fix wrong test data setup
2015-12-23 18:01:11 -07:00
5c888b3673
Merge branch 'master' of https://github.com/influxdb/influxdb into mvadu-patch-4358
...
Trying to get to latest master from influxdb
2015-12-19 01:45:07 +05:30
7e97b0eafd
Fix rename temp file on windows
2015-12-18 11:57:37 -07:00
611017f4ed
Add comments
2015-12-18 10:00:07 -07:00
930174bf4d
Handle calling WriteBlock with no data gracefully
2015-12-18 09:57:16 -07:00
6bc7765b88
Handle calling write with no values to TSMWriter gracefully
2015-12-18 09:52:53 -07:00
421a127f11
Add indirectIndex.UnmarshalBinary benchmark
2015-12-17 15:38:51 -07:00
8c7e11f4cf
Aggressively clean up KeyCursor resources
2015-12-17 12:51:51 -07:00
fd2a409ea3
Skip decoding blocks that are already full
2015-12-17 12:47:05 -07:00
825296ddd8
Add comments
2015-12-16 11:30:06 -07:00
88324bf61c
Optimize indirectIndex.UnmarshalBinary further
2015-12-16 11:28:13 -07:00
70d1f45058
Load TSM files concurrently
2015-12-16 11:28:12 -07:00
737871268b
Speed up indirectIndex.UnmarshalBinary
...
Remove a bunch of unnecessary allocations to improve startup times.
2015-12-16 11:16:17 -07:00
3893bc60e1
Speed up TSM compactor
...
Just keep the current block for each iterator in the buffers.
2015-12-16 11:16:17 -07:00
00f570441b
Convert TSMKeyIterator to return blocks
2015-12-16 11:16:17 -07:00
59a57d8f73
Convert CacheKeyIterator to return encoded blocks
2015-12-16 11:16:17 -07:00
0623648140
Add chunking support back to TSMKeyIterator
...
Was removed when MergeIterator was deleted.
2015-12-16 11:16:17 -07:00
31b97c3fe0
Add max points per block back for CacheKeyIterator
...
Was removed when MergeIterator was removeed.
2015-12-16 11:16:16 -07:00
45e87cdfe4
Strip checksum when returning block from ReadBytes
2015-12-16 11:16:16 -07:00
97435b9124
Return minTime/maxTime from BlockIterator.Read
2015-12-16 11:16:16 -07:00
ce6de9728e
Add test for BlockIterator with multiple blocks for a key
2015-12-16 11:16:16 -07:00
4a3037814f
Add WriteBlock to TSMWriter
2015-12-16 11:16:16 -07:00
d99c1f944e
Add BlockIterator for reading TSM blocks without decoding
2015-12-16 11:16:16 -07:00
928aef04cd
Split data_file.go into reader.go and writer.go
2015-12-16 11:16:16 -07:00
47317d73b4
Merge pull request #5131 from influxdb/site-fixes
...
Default data logging to on
2015-12-16 10:03:25 -08:00
ad1044dde9
typo: unkown -> unknown
2015-12-15 18:10:47 -05:00
d45048455a
Default data logging to on
2015-12-15 13:15:38 -08:00
0e4bc275d8
Merge pull request #5115 from influxdb/site-fixes
...
Log TSM initialization
2015-12-15 13:13:17 -08:00
01ac0b3f23
Tweak compaction log messages
2015-12-15 10:33:13 -08:00
d89e233567
Merge pull request #5100 from influxdb/dgn-fix-4303
...
fix #4303 : don't drop from multiple databases
2015-12-15 07:27:05 -05:00
a6cdb5229d
Log tsm initialization
2015-12-14 15:50:56 -08:00
3014fb90e4
fix #4303 : don't drop from multiple databases
2015-12-12 13:54:23 -05:00
75764517f6
Merge pull request #5082 from li-ang/fix_x
...
Fix wrong value of countCompacting in WAL
2015-12-11 10:07:56 -08:00
03f8cd3956
Add comment explaining magic number
2015-12-10 11:46:40 -08:00
631ecc23de
Fix growing destination buffer during WAL entry encoding
...
The test to see if the destination buffer for encoding and decoding a WAL
entry was broken and would cause a panic if there were large batches that
would overflow the buffer size.
Fixes #5075
2015-12-10 11:46:40 -08:00
34bdffdb00
Fix wrong value of countCompacting in wal
2015-12-10 17:47:20 +08:00
b7000c80dd
count with fill(none) will drop 0 valued intervals
2015-12-09 15:20:47 -07:00
07c0939fe1
Added logic To let the memeory mapped files to renamed by OS. Now a copy
...
is created in memory with SHARED_DELETE flag, so that OS is free to rename
or delete original file
2015-12-10 01:07:50 +05:30
da08304780
Merge pull request #4940 from li-ang/fix_aggregative_query_err
...
Fix distributed aggregative query error
2015-12-09 11:04:40 -08:00
992aea7bd3
Merge pull request #5060 from influxdb/jw-drop-db
...
Cancel writing TSM files when engine closes
2015-12-08 16:16:07 -07:00
b192136887
Merge pull request #5058 from influxdb/pd-update-compaction-logic
...
Update TSM compaction logic
2015-12-08 18:14:15 -05:00
27cc2ea0cc
Update compact.Plan
2015-12-08 18:01:31 -05:00
Paul Dix
Jason Wilder
Ady
Philip O'Toole
Alexandre Viau
Philip O'Toole
dgnorton
liang@qiniu.com
Nathaniel Cook
Jason Wilder