05b60b8fd0
fix: Remove remaining mentions of Delorean
2020-11-10 11:47:42 -05:00
0532821ed9
chore: Make the project root neat and tidy ( #416 )
...
* chore: move example env file into docs
* chore: move dockerfiles out of root
* chore: move storectl-bench.sh into benches
2020-11-10 10:00:22 -05:00
5bb530ccc6
refactor: rename tsm --> influxdb_tsm ( #418 )
2020-11-05 14:35:38 -05:00
b745a180a4
refactor: rename delorean --> InfluDB IOx ( #417 )
2020-11-05 13:51:04 -05:00
a52e0001c5
refactor: rename all crates that start with`delorean_` in preparation for rename ( #415 )
...
* refactor: rename delorean_cluster --> cluster
* refactor: rebane delorean_generated_types --> generated_types
* refactor: rename delorean_write_buffer --> write_buffer
* refactor: rename delorean_ingest --> ingest
* refactor: rename delorean_storage --> storage
* refactor: rename delorean_tsm --> tsm
* refactor: rename delorean_test_helpers --> test_helpers
* refactor: rename delorean_arrow --> arrow_deps
* refactor: rename delorean_line_parser --> influxdb_line_protocol
2020-11-05 13:44:36 -05:00
4f348836fe
refactor: remove delorean_parquet by compining with delorean_ingest ( #412 )
2020-11-05 09:29:59 -05:00
8399d2a159
refactor: rename delorean_table to packers ( #409 )
2020-11-05 08:52:22 -05:00
075ba0d8d1
refactor: remove delorean_table_schema crate and fold it into data_types ( #408 )
2020-11-05 06:17:20 -05:00
9f36914351
chore: Upgrade version of Arrow / DataFusion (3 of 3) + update code for new interfaces ( #395 )
2020-11-02 11:20:44 -05:00
ef501871bb
feat: remove partition_store ( #387 )
2020-10-26 14:39:38 -04:00
2418ee5ab0
refactor: move partitioned_store into its own module
2020-09-18 08:12:19 -04:00
8ef76debe9
refactor: PR feedback
...
Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
refactor: PR feedback
Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
refactor: PR feedback
Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
refactor: PR feedback
Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
refactor: PR feedback
Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
refactor: PR feedback
Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
refactor: PR feedback
Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
refactor: PR feedback
Co-authored-by: Jake Goulding <jake.goulding@integer32.com>
refactor: PR feedback
2020-08-04 18:42:37 +01:00
23aa5a4981
test: add benchmarks for packers
2020-07-30 12:54:51 +01:00
bd5d39f60c
refactor: address PR feedback
2020-07-08 22:57:15 +01:00
f3970990ac
test: fix mapper benc
2020-07-08 22:57:15 +01:00
fff5577efb
refactor: encapsulate mapping logic
...
This commit moves some of the TSM mapper logic that had leaked into the
TSM->Parquer converter back into the mapper. The refactor allows us to
make some previously public APIs private, whilst still providing a
reasonably flexible API.
2020-07-08 22:57:15 +01:00
2be6385ade
perf: drain block data more efficiently
...
This commit reduces copying of block data by replacing an inefficient
`remove` call on vectors by with an index tracking approach, leving the
original vectors in place.
It further refactors some of the mapping code DRYing things up.
It improves performance of the `map_field_columns` function by 48%.
```
time: [137.11 us 137.50 us 137.92 us]
change: [-49.095% -48.558% -48.033%] (p = 0.00 < 0.05)
Performance has improved.
```
2020-07-03 10:56:31 +01:00
1d3adea394
test: add benchmark for mapping fields
2020-07-03 10:56:31 +01:00
a07bf1c67c
refactor: Change enum variant names to be PascalCase
...
This is general Rust convention for enum variant names.
2020-06-29 10:54:08 -04:00
c35b0e835c
fix: clippy
2020-06-29 10:25:43 -04:00
511d2b6436
refactor: move benchmark test data to tests/fixtures
2020-06-26 17:41:37 -04:00
322a491b9d
perf: Improve line protocol --> parquet conversion performance by ~20% ( #177 )
...
* feat: benchmark for lp->parquet performance
* feat: improve parser performance by storing contiguous EscapedStr
* fix: remove all string copies during LP-Parquet conversion
* refactor: Implement from_str as From<&str> only
* refactor: implement Deref instead of as_str
* refactor: Remove ends_with because Deref now makes it work
* refactor: Eq can be derived
* refactor: Remove unused From implementation
* refactor: Replace single-character strings with chars as requested by clippy
Co-authored-by: Carol (Nichols || Goulding) <carol.nichols@integer32.com>
2020-06-23 05:42:19 -04:00
683205ad03
refactor: Use `Vec::clear` instead of `Vec::truncate(0)`
2020-06-22 15:32:15 -04:00
621f2f91f0
refactor: hoist tsm mapper to delorean_tsm
2020-06-22 18:56:17 +01:00
2be21dab57
fix: Name benchmark group consistently ( #161 )
2020-06-17 20:01:17 -04:00
4aa3afb930
perf: Add a benchmark for the line parser
2020-04-08 14:41:17 -04:00
c2b49a7804
refactor: please clippy
2020-02-27 19:41:55 +00:00
be9b2bf695
fix: cargo fmt
2020-02-14 12:47:58 -05:00
78b008a062
fix: Use size_of T instead of hardcoded 8
2020-02-14 12:47:13 -05:00
6463601676
fix: Use TryFrom instead of as to get noisy conversion errors
2020-02-14 11:40:36 -05:00
575907814c
fix: Make batch sizes be usize and convert through i32 when needed
2020-02-14 11:39:26 -05:00
3a3a7993b2
refactor: Extract a function for benchmarking decoding
2020-02-12 13:08:11 -05:00
72ea8c09c9
fix: Move a vec allocation outside of the benchmarked code
...
This is consistent with the rest of the decode benchmarks and I think
matches the benchmark intentions best.
2020-02-12 12:45:43 -05:00
c498d1f524
fix: Remove truncate from encoding benchmark
...
The first thing the `encode` function does is truncate the `dst` buffer,
so this should never be necessary inside the code being benchmarked for
testing encoders.
2020-02-12 12:42:54 -05:00
1fc46c33f3
refactor: Call the general encoding benchmarking fn for CPU values
2020-02-12 11:46:49 -05:00
b36c4b9672
refactor: Extract shared benchmarking of encoding
...
Benchmarking random values was more general than sequential since it
takes an arbitrary function to create the decoded values; express
sequential in terms of random and change the name of random to be
general benchmarking of encoding.
2020-02-12 11:44:08 -05:00
2080bfc5c4
refactor: Extract a fn for benchmarking encoding of random values
2020-02-12 11:41:25 -05:00
532329f83e
refactor: Extract a fn for benchmarking encoding of sequential values
2020-02-12 11:30:21 -05:00
85b5d339a9
refactor: Extract batch sizes into constants
...
Exposes which tests use which batch sizes more clearly; names of
constants could be improved.
2020-02-12 11:14:57 -05:00
e361cded92
refactor: Move all encoder benchmarks to one file
2020-02-12 11:08:07 -05:00
6fbe9167ae
refactor: Extract large constant to a separate module
2020-02-12 10:28:00 -05:00
7a9f69c921
refactor: address PR feedback
2020-02-07 22:28:11 +00:00
5327efd926
test: add timestamp benchmarks
2020-02-07 13:26:59 +00:00
bd8246d561
test: add integer encoder/decoder benchmarks
2020-02-07 13:11:38 +00:00
4185307d78
test: add float encoder/decoder bencmarks
...
This commit adds benchmarks for the float encoder and decoder. The
following scenarios are benchmarked:
- sequential values;
- random values;
- real CPU values (from Telegraf output).
Each scenario is benchmarked with a variety of block sizes.
2020-01-21 15:01:35 +00:00
Carol (Nichols || Goulding)
Andrew Lamb
alamb
Edd Robinson
Jake Goulding