Commit Graph

13 Commits (master)

Author SHA1 Message Date
Stuart Carnie dee8977d2c
chore: move v2/v1/tsdb → v2/tsdb 2020-08-26 10:46:47 -07:00
Mark Rushakoff f2898d1992 Wipe out workspace in preparation for v2 merge
"Knock knock."

"Who's there?"

"InfluxDB Veet."

...
2019-01-11 10:38:50 -08:00
Edd Robinson 996bb9bfa6 Wire in mmap advise hint to TSMReader 2018-08-03 16:27:39 +01:00
Ben Johnson 2a8ca9a10e
Revert "Use MADV_WILLNEED when loading TSM files"
This reverts commit ee270e1dd2.
2018-03-21 13:26:45 -06:00
Jason Wilder ee270e1dd2 Use MADV_WILLNEED when loading TSM files
When the TSM index is large, this hints to the kernel to start
faulting in pages to avoid lots of smaller page faults.
2018-01-31 12:38:16 -07:00
Jason Wilder 31646aae3a Release mmap pages when shard is cold
This instructs the kernel that it can release memory used by mmap'd
TSM files when they are not actively being used.  It the mappings are
use, the kernel will fault the pages back in.  On linux, this causes
RES memory to drop immediately when run.
2017-09-18 11:51:51 -06:00
Jason Wilder 7388eb9499 Use disk when writing TSM index 2017-09-11 15:29:25 -06:00
runner bc992fea5e fix munmap bug in the windows
fix munmap bug in the windows

fix munmap bug in the windows

fix munmap bug in the windows

fix munmap bug in the windows
2016-01-31 10:46:46 +08:00
Ady 07c0939fe1 Added logic To let the memeory mapped files to renamed by OS. Now a copy
is created in memory with SHARED_DELETE flag, so that OS is free to rename
or delete original file
2015-12-10 01:07:50 +05:30
Adarsha 5482c6de03 Avoid closing the handle in mmap
Added mmap implementation for Windows. It uses MapViewOfFile similar to Bolt's implementation. MapViewOfFile  returns a pointer and not a byte array. Bolt changed their data structure to support it. 

Instead of changing the implementation of tsm data structure, I used a trick shown in https://groups.google.com/forum/#!topic/golang-nuts/g0nLwQI9www to use SliceHeader to convert the pointer into a slice.

Bolt's implementation also closes the file handle in mmap itself. It was resulting in a timeout, so implemented https://github.com/edsrzf/mmap-go/blob/master/mmap_windows.go logic to keep file handle open until munmap
2015-12-07 23:30:19 +05:30
Adarsha 6a0e60c67e Added mmap implementation for Windows
Added mmap implementation for Windows. It uses MapViewOfFile similar to Bolt's implementation. MapViewOfFile  returns a pointer and not a byte array. Bolt changed their data structure to support it. 

Instead of changing the implementation of tsm data structure, I used a trick shown in https://groups.google.com/forum/#!topic/golang-nuts/g0nLwQI9www to use SliceHeader to convert the pointer into a slice.
2015-12-04 10:20:43 +05:30
Adarsha c2b8a24004 Added mmap implementation for Windows
Added mmap implementation for Windows based on MapViewOfFile. Used SliceHeader trick to change the pointer returned by MapViewOfFile to a byte slice. This will not call for any change in rest of tsm.

However I am not sure where this mmap function is called, as go build is still complains about 
    
    tsdb\engine\tsm1\tsm1.go:1974: undefined: syscall.Mmap
    tsdb\engine\tsm1\tsm1.go:1974: undefined: syscall.PROT_READ
    tsdb\engine\tsm1\tsm1.go:1974: undefined: syscall.MAP_SHARED
    tsdb\engine\tsm1\tsm1.go:2033: undefined: syscall.Munmap
2015-12-03 23:43:48 +05:30
Jason Wilder 4b6767bf01 Add MMAP based file reader 2015-12-02 09:45:23 -07:00