5d2c5de39d
feat: Structs to represent line protocol schema ( #103 )
...
* feat: Structs to represent line protocol schema
Co-authored-by: Jake Goulding <jake.goulding@integer32.com>
2020-06-03 08:39:35 -04:00
18b05ce9ef
fix: move test of dstool to its delorean_storage_tool package ( #107 )
2020-06-02 16:10:30 -04:00
1a2efdfd71
feat: Add dstool command line tool ( #102 )
...
* feat: Add dstool command line tool
* clippy
* Update delorean_storage_tool/src/main.rs
Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
* Update delorean_storage_tool/src/main.rs
Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
* Add in tests + PR comments
* fmt
* build first then run tests
* actually build before test
Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
2020-06-02 07:33:43 -04:00
d4af54c3de
refactor: extract the line protocol parser to a separate crate
...
This will facilitate reusing the parser for other tasks.
2020-05-26 13:22:34 -04:00
e25a4e1e83
feat: Integrate the WAL with delorean
2020-05-11 15:38:47 -04:00
4dd7a8cea8
feat: Introduce a WAL tailored for delorean
2020-05-11 15:38:44 -04:00
e369ada35a
refactor: extract a crate with our custom assertions
...
There's probably an existing crate that we should use directly, but I
haven't found an exact match yet.
2020-05-01 13:04:24 -04:00
7f9eaf51d5
Merge pull request #74 from influxdata/cn-generate-points
2020-04-24 08:08:32 -04:00
f1d5d50b92
Merge pull request #68 from influxdata/er-block-writer
...
feat: add Block Type
2020-04-23 22:48:38 +01:00
fa69101945
refactor: Move the point utilities into a workspace crate
2020-04-23 11:26:37 -04:00
6791b9598c
feat: Utility to take line protocol and make write requests
2020-04-23 11:11:51 -04:00
0956b665d8
feat: Add a utility to generate points in line protocol to stdout
2020-04-23 11:11:46 -04:00
93231c64e0
perf: Use a SmallVec for escaped strings and sets of tags and values
...
This increases the performance from 56.531 MiB/s to 58.194 MiB/s.
2020-04-08 14:41:42 -04:00
4aa3afb930
perf: Add a benchmark for the line parser
2020-04-08 14:41:17 -04:00
9e20743b2c
feat: add Block Type
...
This commit adds a new Block type, which is used to keep track of values
associated with individual block, and then serialise them.
2020-04-08 13:37:48 +01:00
8629072508
build: Upgrade tonic to 0.2
2020-04-05 16:35:00 -04:00
4a28abd4de
build: Upgrade assert_cmd to 1.0
...
This requires that we opt into the serde `derive` feature that is no
longer implicitly added from upstream.
2020-04-05 16:33:37 -04:00
df67b9715a
Merge remote-tracking branch 'origin/master' into pd-partiton-store
2020-04-02 11:15:26 -04:00
97d11633b8
feature: Use a unique directory per end-to-end test run
2020-04-02 11:06:36 -04:00
d9cf5c952a
fix: Remove RocksDB code
2020-04-02 09:41:30 -04:00
4fd0c6f210
feat: Error when parsing lines with duplicate tags
2020-03-11 22:43:09 -04:00
78a53aa391
refactor: Replace the hand-written parser with one built with nom
2020-03-06 10:00:29 -05:00
5d3f99da98
refactor: Remove unused failure crate
2020-02-28 16:54:28 -05:00
17051717e2
chore: remove dependency:
2020-02-28 12:55:28 +00:00
38f23ac07a
refactor: merge master in
2020-02-27 14:27:23 +00:00
c41652e45b
feature: Add the storage gRPC proto definitions
2020-02-24 08:26:28 -05:00
3438edd18b
feature: Switch from prost to tonic
2020-02-17 16:37:43 -05:00
9c7e3b2663
build: `rand` is only a test dependency
2020-02-17 10:48:33 -05:00
68970f8ff3
build: Update bytes to latest version
2020-02-17 10:48:33 -05:00
155bfcbd4f
build: Update prost to latest version
2020-02-17 10:48:33 -05:00
eb3113b820
build: Update dotenv to latest version
2020-02-17 10:48:32 -05:00
31df104996
build: Update env_logger to latest version
2020-02-17 10:48:32 -05:00
e3bfa0f835
build: Update byteorder to latest version
2020-02-17 10:48:32 -05:00
8b1255be9d
refactor: Switch to a hyper server
2020-02-14 09:59:09 -05:00
062bbc5a34
Merge remote-tracking branch 'origin/master' into er-encoder-bench
2020-02-14 09:15:24 -05:00
9cce1e4882
test: Add an end-to-end test
...
This test:
- Runs the server in a thread
- Writes some data
- Reads some data
- Shuts down the server
2020-02-13 10:40:03 -05:00
e361cded92
refactor: Move all encoder benchmarks to one file
2020-02-12 11:08:07 -05:00
5327efd926
test: add timestamp benchmarks
2020-02-07 13:26:59 +00:00
4185307d78
test: add float encoder/decoder bencmarks
...
This commit adds benchmarks for the float encoder and decoder. The
following scenarios are benchmarked:
- sequential values;
- random values;
- real CPU values (from Telegraf output).
Each scenario is benchmarked with a variety of block sizes.
2020-01-21 15:01:35 +00:00
80ff911259
feat: automatically create db dir and test dir
2020-01-06 15:55:22 +00:00
1a851f8d0b
Add basic read endpoint
...
This commit adds a basic read endpoint to pull data out of the database. In order to provide the basic functionality a few things were added:
* Time package with limited support for parsing Flux style durations
* API endpoint at /api/v2/read with query paramters of org_id, bucket_name, predicate, start, and stop
The start and stop query parameters only support relative durations.
The predicate parameter supports what is possible in the parse_predicate method and in the RocksDB implementation (only == comparisons on tags and AND or OR)
2020-01-04 19:07:54 -05:00
4265e7b11b
Update write API endpoint
...
Upates the actix-web and actix-rt versions to 2.0 and 1.0 respectively. Wires up the write endpoint to create buckets if they don't exist.
2020-01-03 17:35:18 -05:00
6cd4c5b583
Add basic tag key/value index
...
This commit brings in a Roaring Bitmap implementation to keep postings lists of tag key/value pairs to the set of series ids that have those pairs. The croaring implementation was used becasue the Treemap was required for u64 support for series ids and it was serializable (unlike the other pure Rust roaring implementation).
This doesn't shard the postings lists based on size. It also doesn't implement the time/index levels.
The predicate matching currently only works for a simple key = "value" match.
2019-12-24 13:44:30 -05:00
617c2960a8
feat(storage): Implement bucket definitiions and persistence
...
This updates to build system to use Prost to build the protobuf objects.
It adds tests for creating, storing and loading bucket definitions.
The tests use an actual on disk RocksDB implementation to ensure that its tested all the way to persistence.
2019-12-17 17:01:41 -05:00
9cadb1bb52
Add server skeleton with Actix and RocksDB
2019-12-12 10:15:16 -05:00
fb83e9c7fa
feat(encoding): add RLE encoder/decoder
2019-12-09 18:04:01 +00:00
55d711599e
tests: add tests for simple8b
2019-12-04 13:14:37 +00:00
b9b5a815b7
Initial commit with some notes and proto
2019-11-22 16:59:04 -05:00
Andrew Lamb
Jake Goulding
Carol (Nichols || Goulding)
Carol (Nichols || Goulding)
Edd Robinson
Paul Dix