Commit Graph

14 Commits (97cbcac3707436f7328efc00e3109bca3b087213)

Author SHA1 Message Date
Jake Goulding fdf44d2745 refactor: Support converting EscapedStr into a String 2020-06-26 13:12:19 -04:00
Carol (Nichols || Goulding) 4df99f1a7c style: Enable the clippy warning to use `Self` when recommended
Fixes #158.
2020-06-25 07:38:58 -04:00
Andrew Lamb 322a491b9d
perf: Improve line protocol --> parquet conversion performance by ~20% (#177)
* feat: benchmark for lp->parquet performance

* feat: improve parser performance by storing contiguous EscapedStr

* fix: remove all string copies during LP-Parquet conversion

* refactor: Implement from_str as From<&str> only

* refactor: implement Deref instead of as_str

* refactor: Remove ends_with because Deref now makes it work

* refactor: Eq can be derived

* refactor: Remove unused From implementation

* refactor: Replace single-character strings with chars as requested by clippy

Co-authored-by: Carol (Nichols || Goulding) <carol.nichols@integer32.com>
2020-06-23 05:42:19 -04:00
Andrew Lamb 86a425e5ef
feat: Add support for parsing bool values in line protocol parser (#156)
* feat: Implement boolean support for the line protcol parser

* fix: Apply suggestions from code review

Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>

* fix: fmt+clippy

Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
2020-06-22 16:58:38 -04:00
Andrew Lamb ae37548980
feat: Add support for parsing string values in line protocol parser (#155)
* feat: add debug logging on parser error

* feat: Add support for parsing string values in line protocol parser

* fix: Fix comment

* fix: Apply suggestions from code review

Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>

Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
2020-06-18 12:44:17 -04:00
Andrew Lamb 94f1968deb
feat: Improve line protocol parser error recovery, avoid infinite loop (#152)
* feat: Improve line protocol parser error recovery, avoid infinite loop

feat: port splitLines logic to rust line protocol parser

fix: consume trailing optional whitespace after timestamp

test: Add tests for same

* fix: Apply suggestions from code review

Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>

Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
2020-06-17 17:44:04 -04:00
Andrew Lamb d9278263a7
feat: write multiple measurements to multiple parquet files (#138)
* feat: write to a directory of parquet files

* feat: change LineProtocolConverter to push style, move sampling there

* feat: full push mode, write to multiple measurements

* fix: clarify comments on finalize

* fix: Apply suggestions from code review

Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>

* fix: clippy/fmt

* fix: remove whitespace

* fix: Apply suggestions from code review

Co-authored-by: Jake Goulding <jake.goulding@integer32.com>

* fix: fmt

* fix: make it compile again

* fix: fixup comments

Co-authored-by: Jake Goulding <jake.goulding@integer32.com>

* fix: remove unecessary debug implementation

* fix: cleaner comment

Co-authored-by: Jake Goulding <jake.goulding@integer32.com>

* fix: clearer iterator name

Co-authored-by: Jake Goulding <jake.goulding@integer32.com>

* fix: Apply suggestions from code review

Co-authored-by: Jake Goulding <jake.goulding@integer32.com>

* fix: clean

* fix: make it compile

* fix: type fix

* fix: whitespace

* fix: more review comments

* fix: more review comments

* fix: code review comments + fmt

* fix: clippy

* fix: Use EscapedStr directly for performance

Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
Co-authored-by: Jake Goulding <jake.goulding@integer32.com>
2020-06-12 17:19:35 -04:00
Andrew Lamb 1e2cc310d4
refactor: cleanup trim_leading in the parser, add tests (#142) 2020-06-12 11:04:40 -04:00
Andrew Lamb 2c1180b27a
test: add some additional parser tests (#137) 2020-06-12 11:02:57 -04:00
Jake Goulding df39eca043 style: Apply standard lints across all crates 2020-06-05 17:02:54 -04:00
Andrew Lamb 234b2f5752
feat: Line Protocol Schema extractor (#108)
* feat: schema inference from iterator of parsed lines

* fix: Clean up error handing even more

* fix: fmt

* fix: make a sacrifice to the clippy gods
2020-06-03 18:29:57 -04:00
Andrew Lamb 54125e35c8
chore: touch up docs + test for multi-part EscapedStr (#109)
* chore: Doc + Test for multi-part EscapedStr

* fix: Add additional test case, for completeness
2020-06-03 08:09:12 -04:00
Andrew Lamb 3e0c7e25ff
test: Add comments and some more tests to parsers (#96)
* test: Add comments and some more tests to parsers

* Update delorean_line_parser/src/lib.rs

Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>

* Update src/line_parser.rs

Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>

* Update src/line_parser.rs

Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>

* Update delorean_line_parser/src/lib.rs

Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>

* Remove end to end test as it is now covered by docs

* Update src/line_parser.rs

Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>

* one final tweak

Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
2020-05-28 05:32:40 -04:00
Jake Goulding d4af54c3de refactor: extract the line protocol parser to a separate crate
This will facilitate reusing the parser for other tasks.
2020-05-26 13:22:34 -04:00