8399d2a159
refactor: rename delorean_table to packers ( #409 )
2020-11-05 08:52:22 -05:00
075ba0d8d1
refactor: remove delorean_table_schema crate and fold it into data_types ( #408 )
2020-11-05 06:17:20 -05:00
9f36914351
chore: Upgrade version of Arrow / DataFusion (3 of 3) + update code for new interfaces ( #395 )
2020-11-02 11:20:44 -05:00
1004854403
refactor: remove uneeded dependencies, switch to tracing from log ( #388 )
2020-10-26 06:15:47 -04:00
77f58efca7
chore: update Arrow/Parquet/DataFusion versions, consolidate references into new crate ( #309 )
...
* chore: consolidate all arrow/parquet/datafusion dependencies
* chore: update datafusion version
2020-09-24 08:46:54 -04:00
39db199613
Merge pull request #264 from influxdata/er/perf/packers
...
perf: improve performance of materialising Packers
2020-08-04 18:49:18 +01:00
cc39bf3cb4
refactor: address PR feedback
2020-07-31 11:41:57 +01:00
e40fecb1ff
perf: reduce cost of materialising packer rows
2020-07-30 15:38:29 +01:00
19159138cc
fix: Turn off default features of parquet so arrow-flight doesn't repeatedly rebuild
...
Fixes #261
2020-07-30 09:43:12 -04:00
f946e84a12
chore: revert upgrade parquet dependency to 1.0.0"
...
This reverts commit 25259b4c99
2020-07-30 07:02:53 -04:00
25259b4c99
chore: upgrade parquet dependency to 1.0.0
2020-07-28 15:11:35 -04:00
e8974e8947
fix: clarify comments
2020-07-20 16:43:53 -04:00
ca1bd79902
feat: Update stats command to handle directories of files
2020-07-17 16:47:11 -04:00
a07bf1c67c
refactor: Change enum variant names to be PascalCase
...
This is general Rust convention for enum variant names.
2020-06-29 10:54:08 -04:00
6aa37dd7fe
fix: clippy
2020-06-29 10:25:43 -04:00
5fe022366b
fix: Use std::str::FromStr and add -c
2020-06-29 10:25:43 -04:00
830bac2e64
fix: avoid string construction on error
...
Co-authored-by: Jake Goulding <jake.goulding@integer32.com>
2020-06-29 10:25:43 -04:00
c35b0e835c
fix: clippy
2020-06-29 10:25:43 -04:00
e8a0233b09
feat: add configurable compression level
2020-06-29 10:25:43 -04:00
020d56e021
refactor: use SNAFU more idiomatically in delorean_parquet
2020-06-26 13:26:51 -04:00
d4a2cf1bd8
fix: rename timestamp column "timestamp" -> "time" to be consistent
2020-06-26 08:26:16 -04:00
4df99f1a7c
style: Enable the clippy warning to use `Self` when recommended
...
Fixes #158 .
2020-06-25 07:38:58 -04:00
afcd1efd1e
style: Unify lints everywhere
...
Then fix the failures, mostly by adding derives and then removing some
unneeded (cheap) clones.
Document places where we purposefully don't use the same lints.
Not unifying missing_docs.
👀 https://github.com/rust-lang/cargo/issues/5034
2020-06-25 07:28:42 -04:00
2c4a9dba53
fix: cleanup comment + code order
2020-06-23 17:21:20 -04:00
b22423621b
refactor: remove InputReaderAdapter
2020-06-23 17:15:02 -04:00
eee1e9fe77
fix: Setup parquet column encoding correctly
2020-06-23 09:54:16 -04:00
322a491b9d
perf: Improve line protocol --> parquet conversion performance by ~20% ( #177 )
...
* feat: benchmark for lp->parquet performance
* feat: improve parser performance by storing contiguous EscapedStr
* fix: remove all string copies during LP-Parquet conversion
* refactor: Implement from_str as From<&str> only
* refactor: implement Deref instead of as_str
* refactor: Remove ends_with because Deref now makes it work
* refactor: Eq can be derived
* refactor: Remove unused From implementation
* refactor: Replace single-character strings with chars as requested by clippy
Co-authored-by: Carol (Nichols || Goulding) <carol.nichols@integer32.com>
2020-06-23 05:42:19 -04:00
a5d090a8cb
Merge pull request #162 from influxdata/er/packer
...
refactor: make Packer generic
2020-06-22 18:01:07 +01:00
506a7f19d5
feat: add stats command for computing storage statistics ( #160 )
...
* feat: add stats command for computing storage statistics
* fix: Make function name less redundant
* fix: Apply suggestions from code review
Co-authored-by: Jake Goulding <jake.goulding@integer32.com>
* fix: remove changes to Cargo.lock
* fix: Cleanup to use `impl Into<String>` rather than String::from
* fix: fmt
* fix: less mut builder
* fix: fmt
* fix: clippy/fmt
* fix: fixup test
* fix: remove dstool reference
Co-authored-by: Jake Goulding <jake.goulding@integer32.com>
2020-06-22 07:33:52 -04:00
f011748345
refactor: update delorean_parquet/tests/read_write.rs
...
Co-authored-by: Jake Goulding <jake.goulding@integer32.com>
2020-06-22 11:24:29 +01:00
11132b9b30
docs: Update delorean_parquet/src/writer.rs
...
Co-authored-by: Jake Goulding <jake.goulding@integer32.com>
2020-06-22 11:24:29 +01:00
ac7bb6bf68
refactor: make Packer generic
2020-06-22 11:24:29 +01:00
a106e55fa6
feat: Add parquet metadata dumping ( #159 )
...
* feat: Add parquet metadata dumping
* fix: Update delorean_parquet/src/error.rs
Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
2020-06-18 18:34:49 -04:00
3fac49d1ba
fix: encode timestamp values properly in parquet files ( #166 )
2020-06-18 12:24:55 -04:00
cf248f2143
feat: upgrade to latest arrow / byteorder ( #154 )
2020-06-17 12:50:23 -04:00
b00f3ee977
chore: Apply common lints to recently-created crates
2020-06-12 09:26:18 -04:00
0415b233ec
refactor: Instantiate the table writer on demand ( #128 )
...
* refactor: instantiate ParquetWriter on demand, prep for multi measurements
* fix: doc test
* fix: update names
2020-06-09 16:11:42 -04:00
986e12d62a
refactor: Rename crate line_protocol_schema --> delorean_table_schema ( #129 )
...
* refactor: Rename crate line_protocol_schema --> delorean_table_schema
* fix: fmt
2020-06-09 11:56:16 -04:00
8475b6d183
feat: Add parquet writer, hook up conversion in dstool ( #124 )
...
* feat: Add parquet writer, hook up conversion in dstool
* fix: use bigger executor for test
* fix: less cloning
* fix: make unsupported messages less pejorative
* fix: fmt
* fix: Rename writer and do not require std::File, add example
* fix: clippy and fmt
* fix: remove unnecessary module in end to end tests
* fix: remove strange use of tempfile
* fix: Apply suggestions from code review
Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
* fix: Apply suggestions from code review
Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
* fix: cleanup use
* fix: Use more specific error messages
* fix: comment tweak
* fix: touchup temp path creation
* fix: clippy!
Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
2020-06-08 16:25:24 -04:00
Andrew Lamb
Edd Robinson
Carol (Nichols || Goulding)
alamb
Jake Goulding