b3a16c080f
feat: Update croaring
...
Jake dug into why the end-to-end tests fail with delorean running in the
Docker image I built, and it appears to be a crash with an illegal
instruction from CRoaring.
We think it's this issue: https://github.com/saulius/croaring-rs/pull/62
which was merged and released, so let's try updating CRoaring.
2020-07-08 08:49:28 -04:00
831f647b9d
feat: implement escaped tsm key parsing
2020-07-04 08:46:45 -04:00
06e9fae845
fix: ignore conflicting field types
...
Fixes #205 .
2020-06-30 18:08:05 +01:00
97a5eb7e19
Merge pull request #197 from influxdata/alamb/log-requests
...
feat: Log gRPC calls using trace crate, allow custom log levels
2020-06-30 10:47:11 -04:00
283d6691c6
feat: enable rpc debug tracing, tweaked logging levels, respect RUST_FMT env var
2020-06-29 09:59:22 -04:00
ad1e3d04bb
feat: Add a local filesystem implementation of the object store
2020-06-29 08:48:48 -04:00
d15256e0e7
refactor: address PR feedback
2020-06-26 12:08:42 +01:00
9d889828c3
fix: ensure all rows are emitted for each column
2020-06-26 11:50:37 +01:00
68ce351a3a
refactor: remove direct parquet dependency from delorean_ingest
2020-06-23 16:58:31 -04:00
d7dbf061cb
feat: Implement String encoding/decoding
...
Fixes #148 .
2020-06-22 15:15:34 -04:00
106bd69b5a
feat: support converting from TSM->Parquet
2020-06-22 18:56:17 +01:00
85e0b4ec16
refactor: hoist tsm reader into own crate
2020-06-22 18:56:17 +01:00
ac7bb6bf68
refactor: make Packer generic
2020-06-22 11:24:29 +01:00
bfb0213ac3
feat: Update Rusoto to allow streaming data on uploads
2020-06-19 09:18:44 -04:00
8185c80c03
fix: fix logical merge conflict ( #169 )
2020-06-18 18:51:25 -04:00
ae37548980
feat: Add support for parsing string values in line protocol parser ( #155 )
...
* feat: add debug logging on parser error
* feat: Add support for parsing string values in line protocol parser
* fix: Fix comment
* fix: Apply suggestions from code review
Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
2020-06-18 12:44:17 -04:00
cf248f2143
feat: upgrade to latest arrow / byteorder ( #154 )
2020-06-17 12:50:23 -04:00
d83c410a5c
feat: Update to the released version of cloud-storage
...
My submitted API improvements got merged in!
2020-06-10 17:23:52 -04:00
d3283b1096
feat: Object storage in S3 and GCS
2020-06-10 17:23:52 -04:00
faf3f534ac
refactor: move all dstool code into delorean binary ( #131 )
...
* refactor: move all dstool code into delorean binary
* fix: Move code/mods to make it compile and run
* fix: warn if db dir does not exist
* refactor: Match argument subcommands w/ more idomatic rust
* fix: Apply suggestions from code review
Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
fix: restore hyper logging
fix: Apply suggestions from code review
Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
* fix: update expected code
Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
2020-06-10 16:04:46 -04:00
0415b233ec
refactor: Instantiate the table writer on demand ( #128 )
...
* refactor: instantiate ParquetWriter on demand, prep for multi measurements
* fix: doc test
* fix: update names
2020-06-09 16:11:42 -04:00
986e12d62a
refactor: Rename crate line_protocol_schema --> delorean_table_schema ( #129 )
...
* refactor: Rename crate line_protocol_schema --> delorean_table_schema
* fix: fmt
2020-06-09 11:56:16 -04:00
f1a3058b24
feat: Add file / metadata inspection + dumping with dstool ( #112 )
...
* feat: Add file / metadata inspection + dumping
* fix: apply some PR review comments
* fix: apply suggestions from code review
Co-authored-by: Jake Goulding <jake.goulding@integer32.com>
* feat: Add tests, rearrange code into modules, add gzip aware interface
* fix: fix comment and test
* fix: test output and fmt
Co-authored-by: Jake Goulding <jake.goulding@integer32.com>
2020-06-09 10:10:55 -04:00
8475b6d183
feat: Add parquet writer, hook up conversion in dstool ( #124 )
...
* feat: Add parquet writer, hook up conversion in dstool
* fix: use bigger executor for test
* fix: less cloning
* fix: make unsupported messages less pejorative
* fix: fmt
* fix: Rename writer and do not require std::File, add example
* fix: clippy and fmt
* fix: remove unnecessary module in end to end tests
* fix: remove strange use of tempfile
* fix: Apply suggestions from code review
Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
* fix: Apply suggestions from code review
Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
* fix: cleanup use
* fix: Use more specific error messages
* fix: comment tweak
* fix: touchup temp path creation
* fix: clippy!
Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
2020-06-08 16:25:24 -04:00
ca9f9d4cae
feat: Add column packing code ( #114 )
...
* feat: Add column packing code
* fix: remove dependency on assert_approx_equal in favor of delorean_test_helpers
* fix: Cleanups from pr comments
* fix: Apply suggestions from code review
Co-authored-by: Jake Goulding <jake.goulding@integer32.com>
Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
* fix: more cleanup per code review
* fix: pr comments
* fix: remove explict string creation from caller
Co-authored-by: Jake Goulding <jake.goulding@integer32.com>
Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
2020-06-06 06:04:41 -04:00
2200def8ea
feat: Use rust nightly ( #123 )
2020-06-05 17:45:44 -04:00
68fb580b43
style: Re-enable the elided lifetimes lint and move generated types to their own crate ( #119 )
...
* refactor: rename the module containing generated types
The nested `delorean` was confusing anyway, and this will make more
sense when we extract a new crate.
* refactor: Move the generated types to their own crate
This allows us to have more lax warnings in that crate alone, keeping
the main crate more strict.
* style: Re-enable elided lifetimes lint in the main crate
2020-06-05 16:22:27 -04:00
887ffd5977
refactor: remove lifetime to make index re-usable
2020-06-04 14:36:43 +01:00
e3db077121
feat: add API for series key information
2020-06-04 14:36:43 +01:00
413738a264
feat: support org and bucket ID in entries
2020-06-04 14:36:43 +01:00
234b2f5752
feat: Line Protocol Schema extractor ( #108 )
...
* feat: schema inference from iterator of parsed lines
* fix: Clean up error handing even more
* fix: fmt
* fix: make a sacrifice to the clippy gods
2020-06-03 18:29:57 -04:00
5d2c5de39d
feat: Structs to represent line protocol schema ( #103 )
...
* feat: Structs to represent line protocol schema
Co-authored-by: Jake Goulding <jake.goulding@integer32.com>
2020-06-03 08:39:35 -04:00
18b05ce9ef
fix: move test of dstool to its delorean_storage_tool package ( #107 )
2020-06-02 16:10:30 -04:00
1a2efdfd71
feat: Add dstool command line tool ( #102 )
...
* feat: Add dstool command line tool
* clippy
* Update delorean_storage_tool/src/main.rs
Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
* Update delorean_storage_tool/src/main.rs
Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
* Add in tests + PR comments
* fmt
* build first then run tests
* actually build before test
Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
2020-06-02 07:33:43 -04:00
924f20fd50
build: update semver-compatible versions
2020-05-29 13:40:44 -04:00
d4af54c3de
refactor: extract the line protocol parser to a separate crate
...
This will facilitate reusing the parser for other tasks.
2020-05-26 13:22:34 -04:00
f2823ccecd
feat: WAL file rollover based on size of file
2020-05-18 14:08:24 -04:00
e25a4e1e83
feat: Integrate the WAL with delorean
2020-05-11 15:38:47 -04:00
4dd7a8cea8
feat: Introduce a WAL tailored for delorean
2020-05-11 15:38:44 -04:00
bff4d2d6d9
refactor: Move temporary directory creation to test helpers
2020-05-11 15:26:00 -04:00
22136d5431
build: update semver-compatible versions
2020-05-11 15:25:54 -04:00
e369ada35a
refactor: extract a crate with our custom assertions
...
There's probably an existing crate that we should use directly, but I
haven't found an exact match yet.
2020-05-01 13:04:24 -04:00
7f9eaf51d5
Merge pull request #74 from influxdata/cn-generate-points
2020-04-24 08:08:32 -04:00
f1d5d50b92
Merge pull request #68 from influxdata/er-block-writer
...
feat: add Block Type
2020-04-23 22:48:38 +01:00
fa69101945
refactor: Move the point utilities into a workspace crate
2020-04-23 11:26:37 -04:00
6791b9598c
feat: Utility to take line protocol and make write requests
2020-04-23 11:11:51 -04:00
93231c64e0
perf: Use a SmallVec for escaped strings and sets of tags and values
...
This increases the performance from 56.531 MiB/s to 58.194 MiB/s.
2020-04-08 14:41:42 -04:00
9e20743b2c
feat: add Block Type
...
This commit adds a new Block type, which is used to keep track of values
associated with individual block, and then serialise them.
2020-04-08 13:37:48 +01:00
974a142cc8
build: update semver-compatible versions
2020-04-05 16:35:27 -04:00
8629072508
build: Upgrade tonic to 0.2
2020-04-05 16:35:00 -04:00
Carol (Nichols || Goulding)
Edd Robinson
Andrew Lamb
alamb
Jake Goulding
Carol (Nichols || Goulding)