Commit Graph

990 Commits (9a345e226cf533a61c10f005f2a3e4f92046075c)

Author SHA1 Message Date
alamb 5f045fd9da fix: more cleanup 2020-07-03 07:01:22 -04:00
Andrew Lamb ea287abb50
fix: Apply suggestions from code review
Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
2020-07-03 07:00:38 -04:00
alamb 43b7614491 fix: simplification 2020-07-03 06:59:32 -04:00
alamb d3fe1c2751 fix: return bucket_name rather than id, and improve legibility 2020-07-03 06:50:17 -04:00
alamb 7297b41ee3 fix: return bucket_name rather than id, and improve legibility 2020-07-03 06:40:21 -04:00
alamb 598b246335 fix: implement chunk of PR review comments 2020-07-03 06:33:07 -04:00
Edd Robinson 2be6385ade perf: drain block data more efficiently
This commit reduces copying of block data by replacing an inefficient
`remove` call on vectors by with an index tracking approach, leving the
original vectors in place.

It further refactors some of the mapping code DRYing things up.

It improves performance of the `map_field_columns` function by 48%.

```
time:   [137.11 us 137.50 us 137.92 us]
change: [-49.095% -48.558% -48.033%] (p = 0.00 < 0.05)
Performance has improved.
```
2020-07-03 10:56:31 +01:00
Edd Robinson 1d3adea394 test: add benchmark for mapping fields 2020-07-03 10:56:31 +01:00
Edd Robinson 08058c8b63 refactor: move mock decoder 2020-07-03 10:56:31 +01:00
Andrew Lamb 6884f70a87
Merge pull request #212 from influxdata/alamb/storage-errors1
fix: Improve storage predicate error handling
2020-07-02 17:31:52 -04:00
alamb 5fad7db0ac feat: Add better error handling and logging for http_routes 2020-07-01 16:08:49 -04:00
Andrew Lamb a09cc65393
Merge pull request #210 from influxdata/alamb/break-up-server
refactor: Break up server into modules
2020-07-01 16:08:19 -04:00
alamb f92673a393 refactor: Break up server into modules 2020-07-01 12:46:44 -04:00
Andrew Lamb b5a91fae36
Merge pull request #207 from influxdata/alamb/faster-test-harness
fix: speed up end-to-end test harness so it polls rather than waits; Add PING route
2020-07-01 12:46:05 -04:00
alamb 6021ddd222 fix: more PR feedback 2020-07-01 12:41:59 -04:00
alamb 1f61fd820b fix: style cleanup 2020-07-01 12:31:42 -04:00
Andrew Lamb d16aec7e80
fix: Update tests/end-to-end.rs
Co-authored-by: Jake Goulding <jake.goulding@integer32.com>
2020-07-01 12:29:17 -04:00
Edd Robinson a8899511ad
Merge pull request #199 from influxdata/er/decoder-impl
test: exercise TSM converter path with unit test
2020-07-01 16:51:27 +01:00
Edd Robinson b78b00d30c refactor: update delorean_ingest/src/lib.rs
Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
2020-07-01 16:27:38 +01:00
Edd Robinson d75ee0cd4d refactor: update delorean_ingest/src/lib.rs
Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
2020-07-01 15:52:21 +01:00
Edd Robinson b2addf614b refactor: PR feedback
Co-authored-by: Andrew Lamb <alamb@influxdata.com>
2020-07-01 15:52:21 +01:00
Edd Robinson 55bf2a44be test: exercise TSM converter 2020-07-01 15:52:21 +01:00
Edd Robinson 414029b96d refactor: use BlockDecoder trait 2020-07-01 15:52:21 +01:00
Andrew Lamb 63c9d96181 feat: benchmark for lp->parquet performance (#176) 2020-07-01 15:52:21 +01:00
Edd Robinson 1d7d4a072f
Merge pull request #209 from influxdata/er/fix-field-conflict
fix: ignore conflicting field types
2020-07-01 15:52:04 +01:00
Andrew Lamb 54ebf380de
Merge pull request #208 from influxdata/alamb/more-logging
feat: improve error messages and logging
2020-07-01 08:49:50 -04:00
Andrew Lamb 8511613e92
Merge pull request #206 from influxdata/alamb/capabilities-response
feat: Implement CapabilitiesResponse
2020-07-01 08:47:45 -04:00
Edd Robinson a51cfca49a refactor: PR feedback 2020-07-01 13:47:29 +01:00
Andrew Lamb d61d4ae79a
fix: Apply suggestions from code review
Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
2020-07-01 08:46:52 -04:00
alamb d7ddcfa92f fix: fixup errors introduced during PR comments 2020-07-01 08:44:24 -04:00
Andrew Lamb f608bf4f0f
fix: Update tests/end-to-end.rs
Co-authored-by: Jake Goulding <jake.goulding@integer32.com>
2020-07-01 08:41:21 -04:00
Andrew Lamb 6b71fcacde
fix: Apply suggestions from code review
Co-authored-by: Jake Goulding <jake.goulding@integer32.com>
2020-07-01 08:39:56 -04:00
alamb 7c045b6be4 fix: Improve storage predicate error handling 2020-07-01 08:27:25 -04:00
alamb a509766e04 fix: remove whitespace for import 2020-06-30 16:23:53 -04:00
Andrew Lamb eb8aba75ec
fix: Update src/id.rs from PR review
Co-authored-by: Edd Robinson <me@edd.io>
2020-06-30 16:23:35 -04:00
Edd Robinson 06e9fae845 fix: ignore conflicting field types
Fixes #205.
2020-06-30 18:08:05 +01:00
alamb 9c11ae9869 feat: improve error messages and logging 2020-06-30 12:46:54 -04:00
alamb 8be278aadd fix: speed up end-to-end test harness so it polls rather than waits 2020-06-30 12:21:25 -04:00
alamb dc8d655486 feat: Implement CapabilitiesResponse 2020-06-30 11:17:45 -04:00
Andrew Lamb 97a5eb7e19
Merge pull request #197 from influxdata/alamb/log-requests
feat: Log gRPC calls using trace crate, allow custom log levels
2020-06-30 10:47:11 -04:00
Andrew Lamb 07e61cdf06
fix: fix comments
Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
2020-06-29 16:35:07 -04:00
Jake Goulding fef300e007
Merge pull request #196 from influxdata/file-object-store 2020-06-29 12:07:29 -04:00
Carol (Nichols || Goulding) 7d42a1db83
Merge pull request #198 from influxdata/cn/use-more-clap 2020-06-29 11:36:17 -04:00
Carol (Nichols || Goulding) 2bf9037422 refactor: Use built-in clap functionality that uses FromStr to enum
Then we can pass around an enum value and not worry about doing the
conversion anywhere else.
2020-06-29 10:54:48 -04:00
Carol (Nichols || Goulding) a07bf1c67c refactor: Change enum variant names to be PascalCase
This is general Rust convention for enum variant names.
2020-06-29 10:54:08 -04:00
Carol (Nichols || Goulding) c6f2508abe fix: Use clap's possible_values method
This gets us built-in help text and error messages, and does less work
before failing because of an unsupported value.

Before this change, the help text was:

```
OPTIONS:
        --compression-level <compression_level>
            Compression level: max or compatibility (default). [default: compatibility]
```

After this change, the help text is:

```
OPTIONS:
        --compression-level <compression_level>
            How much to compress the output data. 'max' compresses the most; 'compatibility' compresses in a manner more
            likely to be readable by other tools. [default: compatibility]  [possible values: max, compatibility]
```

Before this change, if you supplied an unsupported value, the error was:

```
[2020-06-29T14:47:42Z INFO  delorean::commands::convert] convert starting
[2020-06-29T14:47:42Z INFO  delorean::commands::convert] Preparing to convert 591 bytes from tests/fixtures/lineproto/temperature.lp
Conversion failed: Error creating a parquet table writer Unknown compression level 'foo'. Valid options 'max' or 'compatibility'
```

After this change, the error is:

```
error: 'foo' isn't a valid value for '--compression-level <compression_level>'
	[possible values: compatibility, max]
```
2020-06-29 10:53:01 -04:00
Andrew Lamb 724a4edcbf
Merge pull request #194 from influxdata/alamb/fix-parquet-decoding
fix: add --compression-level option for parquet decoding compatibility
2020-06-29 10:31:10 -04:00
alamb 6aa37dd7fe fix: clippy 2020-06-29 10:25:43 -04:00
alamb 5fe022366b fix: Use std::str::FromStr and add -c 2020-06-29 10:25:43 -04:00
Andrew Lamb 830bac2e64 fix: avoid string construction on error
Co-authored-by: Jake Goulding <jake.goulding@integer32.com>
2020-06-29 10:25:43 -04:00