Influxdata
/
influxdb
mirror of https://github.com/influxdata/influxdb.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
2,252 Commits
221 Branches
453 Tags
342 MiB
Commit Graph

10 Commits (bc3560af8ce090042fd39bd82959b1aa2b440347)

Author SHA1 Message Date
Edd Robinson 50ef521e6c feat: add support for converting multiple TSM files 
This commit extends the ingest crate to support converting multiple TSM
files to a single Parquet file by merging identical measurements across
the TSM files.

This does not yet support merging blocks that overlap.
 2020-07-08 22:57:15 +01:00
alamb 511d2b6436 refactor: move benchmark test data to tests/fixtures 2020-06-26 17:41:37 -04:00
Edd Robinson 99268f5260 test: add coverage for converting tsm file 2020-06-26 11:50:37 +01:00
Andrew Lamb 322a491b9d
perf: Improve line protocol --> parquet conversion performance by ~20% (#177) 
* feat: benchmark for lp->parquet performance

* feat: improve parser performance by storing contiguous EscapedStr

* fix: remove all string copies during LP-Parquet conversion

* refactor: Implement from_str as From<&str> only

* refactor: implement Deref instead of as_str

* refactor: Remove ends_with because Deref now makes it work

* refactor: Eq can be derived

* refactor: Remove unused From implementation

* refactor: Replace single-character strings with chars as requested by clippy

Co-authored-by: Carol (Nichols || Goulding) <carol.nichols@integer32.com>
 2020-06-23 05:42:19 -04:00
Andrew Lamb a106e55fa6
feat: Add parquet metadata dumping (#159) 
* feat: Add parquet metadata dumping

* fix: Update delorean_parquet/src/error.rs

Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>

Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
 2020-06-18 18:34:49 -04:00
Andrew Lamb d9278263a7
feat: write multiple measurements to multiple parquet files (#138) 
* feat: write to a directory of parquet files

* feat: change LineProtocolConverter to push style, move sampling there

* feat: full push mode, write to multiple measurements

* fix: clarify comments on finalize

* fix: Apply suggestions from code review

Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>

* fix: clippy/fmt

* fix: remove whitespace

* fix: Apply suggestions from code review

Co-authored-by: Jake Goulding <jake.goulding@integer32.com>

* fix: fmt

* fix: make it compile again

* fix: fixup comments

Co-authored-by: Jake Goulding <jake.goulding@integer32.com>

* fix: remove unecessary debug implementation

* fix: cleaner comment

Co-authored-by: Jake Goulding <jake.goulding@integer32.com>

* fix: clearer iterator name

Co-authored-by: Jake Goulding <jake.goulding@integer32.com>

* fix: Apply suggestions from code review

Co-authored-by: Jake Goulding <jake.goulding@integer32.com>

* fix: clean

* fix: make it compile

* fix: type fix

* fix: whitespace

* fix: more review comments

* fix: more review comments

* fix: code review comments + fmt

* fix: clippy

* fix: Use EscapedStr directly for performance

Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
Co-authored-by: Jake Goulding <jake.goulding@integer32.com>
 2020-06-12 17:19:35 -04:00
Andrew Lamb 81810c2faa
fix: rename measurement.txt to measurement.lp for consistency (#122) 
* fix: rename measurement.txt to measurement.lp for consistency

* fix: rename the file
 2020-06-05 17:28:06 -04:00
Andrew Lamb e0c38d0976
chore: Add test to check for tsm reading errors, update doc example (#117) 
* chore: Add a test that decodes the entire tsm index

* fix: update test and change example to not use hard coded len

* fix: comment cleanup

* fix: clippy

* fix: Apply suggestions from code review

Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
Co-authored-by: Jake Goulding <jake.goulding@integer32.com>

* fix: fmt/clippy after code review

Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
Co-authored-by: Jake Goulding <jake.goulding@integer32.com>
 2020-06-05 16:22:56 -04:00
Edd Robinson 413738a264 feat: support org and bucket ID in entries 2020-06-04 14:36:43 +01:00
Andrew Lamb fbc74348c6
fix: consistent capitalization and data location (#105) 2020-06-02 08:00:13 -04:00