Commit Graph

3761 Commits (466416016b54d603ba2aec0a6dda873088363968)

Author SHA1 Message Date
Andrew Lamb 1a2efdfd71
feat: Add dstool command line tool (#102)
* feat: Add dstool command line tool

* clippy

* Update delorean_storage_tool/src/main.rs

Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>

* Update delorean_storage_tool/src/main.rs

Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>

* Add in tests + PR comments

* fmt

* build first then run tests

* actually build before test

Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
2020-06-02 07:33:43 -04:00
Edd Robinson 11017e297f
Merge pull request #101 from influxdata/er/float-influxdb
refactor: add influxdb support to decoder
2020-06-02 12:28:48 +01:00
Edd Robinson 42bd4bf492 refactor: add influxdb support to decoder
InfluxDB's float encoder uses a different NaN value to terminate a
block. In order to support decoding float blocks that are inside of TSM
files created by InfluxDB this commit adds the option to explictly
decode those via another decode method.

In the future this API may get simplified if we decide we only need the
float encoder/decoder for encoding and decoding blocks solely
originating from InfluxDB.
2020-06-02 12:19:48 +01:00
Edd Robinson 84b3e4c463
Merge pull request #100 from influxdata/er-fix-rle
fix(encoders): RLE compatibility with influxdb
2020-06-01 16:08:22 +01:00
Edd Robinson 979e8ee2c2 fix(encoders): RLE compatibility with influxdb
The RLE encoder was implemented in a correct way, but not in a way that
was byte for byte compatible with the InfluxDB RLE encoder/decoder. This
commit changes the implementation to make this encoder compatible with
the InfluxDB implementation.
2020-05-31 10:02:06 +01:00
Jake Goulding e468b3c044
Merge pull request #99 from influxdata/updates
build: update semver-compatible versions
2020-05-29 17:36:17 -04:00
Jake Goulding 924f20fd50 build: update semver-compatible versions 2020-05-29 13:40:44 -04:00
Andrew Lamb 3e0c7e25ff
test: Add comments and some more tests to parsers (#96)
* test: Add comments and some more tests to parsers

* Update delorean_line_parser/src/lib.rs

Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>

* Update src/line_parser.rs

Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>

* Update src/line_parser.rs

Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>

* Update delorean_line_parser/src/lib.rs

Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>

* Remove end to end test as it is now covered by docs

* Update src/line_parser.rs

Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>

* one final tweak

Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
2020-05-28 05:32:40 -04:00
Carol (Nichols || Goulding) 54dfe826d7
Merge pull request #89 from influxdata/cn-wal-delete 2020-05-27 09:55:26 -04:00
Carol (Nichols || Goulding) 936e7c2a6f docs: Change some answered questions to statements 2020-05-27 09:49:05 -04:00
Carol (Nichols || Goulding) d84fdb13ec feat: Delete entries from the WAL up to some entry number 2020-05-27 09:46:02 -04:00
Carol (Nichols || Goulding) 4ac063088d
Merge pull request #91 from influxdata/cn-wal-metadata 2020-05-27 09:22:27 -04:00
Jake Goulding ac9958db30
Merge pull request #94 from influxdata/jpg/extract-parser 2020-05-27 08:36:55 -04:00
Carol (Nichols || Goulding) 0a02afe016
test: Wait for test server to stop before restarting (#92) 2020-05-26 17:34:54 -04:00
Jake Goulding d4af54c3de refactor: extract the line protocol parser to a separate crate
This will facilitate reusing the parser for other tasks.
2020-05-26 13:22:34 -04:00
Jake Goulding d394e2177f test: Rewrite line_parser tests at a lower level
This will ease extracting them to a separate crate
2020-05-25 20:57:48 -04:00
Jake Goulding e573ca0061 test: Combine duplicate tests 2020-05-25 20:57:39 -04:00
Andrew Lamb 4a0b2cd671
doc: Initial file encoding thoughts (#88) 2020-05-22 17:28:24 -04:00
Carol (Nichols || Goulding) 0d023b60a4 feat: Metadata file 2020-05-22 11:36:26 -04:00
Carol (Nichols || Goulding) 5940b4fcdf
Merge pull request #86 from influxdata/cn-wal-total-size 2020-05-21 14:41:17 -04:00
Carol (Nichols || Goulding) a312bcc9bb test: Move file size tests out of the file rollover tests 2020-05-20 11:12:07 -04:00
Carol (Nichols || Goulding) 23fd94664e feat: Keep track of the total size of all the files in the WAL 2020-05-20 11:12:07 -04:00
Carol (Nichols || Goulding) d6df717e42
Merge pull request #82 from influxdata/wal-early-sync 2020-05-20 09:50:05 -04:00
Jake Goulding 59493c9903 feat: Allow an HTTP task to sync the WAL to disk immediately 2020-05-20 09:26:13 -04:00
Carol (Nichols || Goulding) 30776f3d74
Merge pull request #84 from influxdata/cn-wal-file-rollover 2020-05-20 09:23:06 -04:00
Andrew Lamb f5d7e70db9
fix: improve build error message when flatc is not installed (#85)
* fix: improve build error message when flatc is not installed

* Apply suggestions from code review

Co-authored-by: Jake Goulding <jake.goulding@integer32.com>

* Make it more canonical rust

Co-authored-by: Jake Goulding <jake.goulding@integer32.com>
2020-05-19 11:05:18 -04:00
Carol (Nichols || Goulding) f2823ccecd feat: WAL file rollover based on size of file 2020-05-18 14:08:24 -04:00
Carol (Nichols || Goulding) e5d8271eeb fix: Remove redundant clone 2020-05-13 09:52:20 -04:00
Carol (Nichols || Goulding) f1f41d84b5 test: Verify batches of multiple entries increase sequence number appropriately 2020-05-13 09:38:07 -04:00
Jake Goulding cbf4dd96e1 feat: Track a sequence number in the WAL 2020-05-13 09:17:33 -04:00
Carol (Nichols || Goulding) c98c84c101
Merge pull request #77 from influxdata/cn-delorean-wal 2020-05-13 08:43:26 -04:00
Jake Goulding 0ea0cf6a93 ci: install the Flatbuffers compiler 2020-05-11 15:38:47 -04:00
Carol (Nichols || Goulding) e25a4e1e83 feat: Integrate the WAL with delorean 2020-05-11 15:38:47 -04:00
Jake Goulding 4dd7a8cea8 feat: Introduce a WAL tailored for delorean 2020-05-11 15:38:44 -04:00
Jake Goulding bff4d2d6d9 refactor: Move temporary directory creation to test helpers 2020-05-11 15:26:00 -04:00
Jake Goulding 22136d5431 build: update semver-compatible versions 2020-05-11 15:25:54 -04:00
Jake Goulding 436e8293c0
Merge pull request #81 from influxdata/ci-cleanup 2020-05-08 11:23:35 -04:00
Jake Goulding 083ec01d0b ci: No longer need to limit parallelism due to RocksDB 2020-05-08 10:29:48 -04:00
Carol (Nichols || Goulding) 6ca1ef6dce
Merge pull request #80 from influxdata/cn-new-grpc-endpoints 2020-05-08 09:51:24 -04:00
Carol (Nichols || Goulding) 2014a749aa fix: Update after rebasing on the switch from BTreeMap to BTreeSet 2020-05-04 16:35:50 -04:00
Carol (Nichols || Goulding) 7d568704cd feat: Implement MeasurementFields gRPC request 2020-05-04 15:52:01 -04:00
Carol (Nichols || Goulding) e615607f4a feat: Implement MeasurementTagValues gRPC request 2020-05-04 15:52:01 -04:00
Carol (Nichols || Goulding) 1ab9230654 feat: Implement measurement tag keys request 2020-05-04 15:52:00 -04:00
Carol (Nichols || Goulding) e6e911ce3c feat: Implement the Measurement Names gRPC request 2020-05-04 15:52:00 -04:00
Carol (Nichols || Goulding) e0eb895a14 test: Add more measurements to the e2e test data 2020-05-04 15:52:00 -04:00
Carol (Nichols || Goulding) fbf25a5f8b feat: Update protobuf from idpe and influxdb 2020-05-04 15:52:00 -04:00
Carol (Nichols || Goulding) 415c8d5c5c
Merge pull request #79 from influxdata/cn-btreeset 2020-05-04 14:41:47 -04:00
Jake Goulding e107ec8e2f
Merge pull request #78 from influxdata/float-slices 2020-05-01 16:53:12 -04:00
Carol (Nichols || Goulding) c6dbba2e71 refactor: Use BTreeSet instead of BTreeMap<_, bool> 2020-05-01 14:49:07 -04:00
Jake Goulding a6682467a7 fix: Use approximate floating point comparisons for slices as well 2020-05-01 13:05:00 -04:00