Influxdata
/
influxdb
mirror of https://github.com/influxdata/influxdb.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
829 Commits
228 Branches
453 Tags
343 MiB
Commit Graph

829 Commits (596c987956563efbab47f5b7c43eccc2a650bf1b)

Author SHA1 Message Date
alamb 5ac0069020 Merge remote-tracking branch 'origin/master' into alamb/take-2-228-index-parse-error 2020-07-15 11:20:40 -04:00
Andrew Lamb 201ad1ae87
fix: Apply suggestions from code review 
Update comment to match new name

Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
 2020-07-15 11:19:07 -04:00
Edd Robinson c99daad5a8
Merge pull request #242 from influxdata/er/feat/merge-blocks 
feat: teach TSM converter to merge duplicate block data
 2020-07-15 15:14:19 +01:00
Carol (Nichols || Goulding) 582e18a241 refactor: Shorten matches 2020-07-15 09:04:59 -04:00
Edd Robinson c0a823f0d2
refactor: PR feedback 
Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
 2020-07-13 15:55:38 +01:00
Edd Robinson ceec4c9627
refactor: PR feedback 
Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
 2020-07-13 15:55:27 +01:00
Edd Robinson 3096d76d77
refactor: PR feedback 
Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
 2020-07-13 15:54:08 +01:00
Edd Robinson 8dcbfcdfb9
refactor: PR feedback 
Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
 2020-07-13 15:53:16 +01:00
Edd Robinson bfe83868fc
refactor: PR feedback 
Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
 2020-07-13 15:52:53 +01:00
Edd Robinson 999ba44fad
refactor: PR feedback 
Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
 2020-07-13 15:52:41 +01:00
Edd Robinson 51fcf59da9
refactor: PR feedback 
Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
 2020-07-13 15:52:31 +01:00
Andrew Lamb efa79bb61b
Merge pull request #227 from influxdata/alamb/doc-worker-design 
docs: Initial multi core tasks architecture description
 2020-07-13 10:37:54 -04:00
alamb e3a3a4add3 docs: add note about using synchronous I/O and notes from @stuartcarnie on backpressure 2020-07-13 10:35:28 -04:00
alamb 2dca85069f fix: clarify some points 2020-07-13 10:35:28 -04:00
Andrew Lamb 97cbcac370 fix: Update docs/multi_core_tasks.md 
Co-authored-by: Jake Goulding <jake.goulding@integer32.com>
 2020-07-13 10:35:28 -04:00
Andrew Lamb 5e256f3722 fix: Apply suggestions from code review 
Co-authored-by: Jake Goulding <jake.goulding@integer32.com>
 2020-07-13 10:35:28 -04:00
alamb 2ecfcbaae0 docs: Add worked example and caveats about priority scheduling 2020-07-13 10:35:28 -04:00
Andrew Lamb 646a244c08 fix: Apply suggestions from code review 
Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
 2020-07-13 10:35:28 -04:00
alamb 6a5854bbc3 docs: writeup multi core tasks architecture 2020-07-13 10:35:28 -04:00
Edd Robinson ec9ed12fcb refactor: move into function on BlockData 2020-07-13 10:48:58 +01:00
Edd Robinson 627805c567 Merge branch 'er/feat/merge-blocks' of github.com:influxdata/delorean into er/feat/merge-blocks 2020-07-13 10:41:03 +01:00
Edd Robinson b62810676d feat: add support for merging blocks 2020-07-13 10:39:36 +01:00
Edd Robinson cf31aebe40 feat: determine if a block overlaps another 2020-07-13 10:39:36 +01:00
Edd Robinson ad099fc722
refactor: PR feedback 
Co-authored-by: Andrew Lamb <alamb@influxdata.com>
 2020-07-13 10:38:47 +01:00
alamb 64616f21bf fix: allow arbitrary characters after the delimiter in a field key, including unsecaped spaces 2020-07-11 06:17:43 -04:00
Andrew Lamb 17c61da5f1
Merge pull request #238 from influxdata/alamb/fix-partition-errors 
fix: update error handling in memdb.rs and partitioned_store.rs, remove StorageError
 2020-07-10 09:23:02 -04:00
Edd Robinson e47277b967 feat: add support for merging blocks 2020-07-10 14:02:38 +01:00
alamb 6d78647602 Merge remote-tracking branch 'origin/master' into alamb/fix-partition-errors 2020-07-10 08:49:09 -04:00
Edd Robinson ccae8ad36d feat: determine if a block overlaps another 2020-07-10 12:26:04 +01:00
Andrew Lamb 6ee72e5733
Merge pull request #239 from influxdata/alamb/less-chatty 
fix: Make default output of delorean less chatty
 2020-07-10 06:59:54 -04:00
Edd Robinson 27733457bb
Merge pull request #229 from influxdata/er/multi-tsm 
feat: support combining multiple TSM files -> Parquet files
 2020-07-09 21:56:29 +01:00
Edd Robinson 4e66a48ba9
refactor: PR feedback 
Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
 2020-07-09 15:46:08 +01:00
Edd Robinson fd3f482652
refactor: PR feedback 
Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
 2020-07-09 15:45:50 +01:00
Edd Robinson 3d0d24d6fb
refactor: PR feedback 
Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
 2020-07-09 15:45:41 +01:00
alamb d1e334f3fe fix: Make default output of delorean less chatty 2020-07-08 18:02:48 -04:00
Edd Robinson cc7e8e8da0 fix: ensure tables merged correctly 2020-07-08 22:57:15 +01:00
Edd Robinson bd5d39f60c refactor: address PR feedback 2020-07-08 22:57:15 +01:00
Edd Robinson d19094d023 refactor: PR feedback 
Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
 2020-07-08 22:57:15 +01:00
Edd Robinson 4e455ff595 refactor: PR feedback 
Co-authored-by: Andrew Lamb <alamb@influxdata.com>
 2020-07-08 22:57:15 +01:00
Edd Robinson 5755949c01 refactor: PR feedback 
Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
 2020-07-08 22:57:15 +01:00
Edd Robinson da305596f9 refactor: PR feedback 
Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
 2020-07-08 22:57:15 +01:00
Edd Robinson a3a8b8273b refactor: PR feedback 
Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
 2020-07-08 22:57:15 +01:00
Edd Robinson a8f77c49e2 refactor: PR feedback 
Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
 2020-07-08 22:57:15 +01:00
Edd Robinson f3970990ac test: fix mapper benc 2020-07-08 22:57:15 +01:00
Edd Robinson 54a61b33fc refactor: remove redundant block type 2020-07-08 22:57:15 +01:00
Edd Robinson c5d6509ad5 feat: extend multi file conversion to CLI 2020-07-08 22:57:15 +01:00
Edd Robinson 50ef521e6c feat: add support for converting multiple TSM files 
This commit extends the ingest crate to support converting multiple TSM
files to a single Parquet file by merging identical measurements across
the TSM files.

This does not yet support merging blocks that overlap.
 2020-07-08 22:57:15 +01:00
Edd Robinson eed1e030df feat: add support for multi-block readers 
This commit embeds an index within each block materialised from a TSM
index, which can be used later on to identify which TSM block reader
should be used to decode the block.

This essentially lets one coalesce blocks for the same measurement from
different files lazily - that is, you don't need to materialise them
until you're ready, and when you do want to materialise them you know
which file to read from.
 2020-07-08 22:57:15 +01:00
Edd Robinson 70cdeb2d08 feat: add ability to merge measurement tables 2020-07-08 22:57:15 +01:00
Edd Robinson fff5577efb refactor: encapsulate mapping logic 
This commit moves some of the TSM mapper logic that had leaked into the
TSM->Parquer converter back into the mapper. The refactor allows us to
make some previously public APIs private, whilst still providing a
reasonably flexible API.
 2020-07-08 22:57:15 +01:00