Influxdata
/
influxdb
mirror of https://github.com/influxdata/influxdb.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
5,837 Commits
188 Branches
453 Tags
378 MiB
Commit Graph

424 Commits (053afac5e921d0786c9000bc7761e388f4af29b2)

Author SHA1 Message Date
Edd Robinson b1096d2a36
Merge branch 'main' into er/feat/read_buffer/meta_delete 2021-11-01 10:26:10 +00:00
Marco Neumann bc7244c48e chore: use Rust edition 2021 2021-10-25 10:58:20 +02:00
Edd Robinson 8d267e5840
Merge branch 'main' into er/test/read_buffer/null_column 2021-10-22 15:28:39 +01:00
Edd Robinson 7a5263d709 refactor: tidy up column summary to stats 2021-10-22 13:04:42 +01:00
Edd Robinson 09a2d805b9 refactor: restore concept of NULL value 2021-10-22 13:03:16 +01:00
Edd Robinson 8b406b4f91 test: update table_summaries test 2021-10-22 12:06:56 +01:00
Edd Robinson a3e750c9fc test: add null column to test 2021-10-22 11:27:06 +01:00
Carol (Nichols || Goulding) 654d33684a fix: Move code shared between if/else blocks out 2021-10-21 20:47:35 -04:00
Andrew Lamb a82dc6f5f0
chore: Update datafusion + arrow (#2903) 
* chore: Update datafusion to latest, arrow to 6.0.0

* fix: Update tests

* fix: bubble internal error

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2021-10-19 17:14:08 +00:00
Edd Robinson 7c3b0a00e5 feat: add delete support to column_names 2021-10-14 13:08:57 +01:00
Edd Robinson 7c97cd68bf test: add test for column_names with deletes 2021-10-14 13:08:57 +01:00
Edd Robinson 96e05726ee refactor: expose negated_predicate API for columns_names 2021-10-14 13:08:56 +01:00
Raphael Taylor-Davies 8a82f92c5d
refactor: add TimeProvider abstraction (#2722) (#2815) 
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2021-10-12 21:19:03 +00:00
Raphael Taylor-Davies afe34751e7
refactor: split out schema crate (#2781) 
* refactor: split out schema crate

* chore: fix doc
 2021-10-11 09:45:08 +00:00
Edd Robinson 04a7ddd34a fix: ensure correct PredicateMatch for columns 2021-10-08 12:46:24 +01:00
dependabot[bot] d1f5209869
chore(deps): bump arrow from 5.4.0 to 5.5.0 
Bumps [arrow](https://github.com/apache/arrow-rs) from 5.4.0 to 5.5.0.
- [Release notes](https://github.com/apache/arrow-rs/releases)
- [Changelog](https://github.com/apache/arrow-rs/blob/5.5.0/CHANGELOG.md)
- [Commits](https://github.com/apache/arrow-rs/compare/5.4.0...5.5.0)

---
updated-dependencies:
- dependency-name: arrow
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
 2021-10-04 08:55:38 +00:00
Edd Robinson 8b458e2c2b feat: add API for validating a predicate against a chunk: 2021-09-29 14:42:42 +01:00
Edd Robinson 0a3ca90809 refactor: only validate same types 2021-09-29 14:39:43 +01:00
Edd Robinson a70a55cb3d test: update benchmarks 2021-09-24 15:00:17 +01:00
Edd Robinson 5c7459f488 feat: validate predicates on satisfies_predicate 2021-09-24 14:52:19 +01:00
Edd Robinson a69e46efc6 feat: validate predicates on column_values 2021-09-24 14:52:19 +01:00
Edd Robinson f618aa1b76 feat: validate predicates on column_names 2021-09-24 14:52:19 +01:00
Edd Robinson c107434d20 feat: validate predicates on read_aggregate 2021-09-24 14:52:19 +01:00
Edd Robinson 621b26166c feat: validate predicates on read_filter 2021-09-24 14:52:16 +01:00
Edd Robinson 053186ab29 feat: add ability validate predicate compatible with schema 2021-09-24 13:05:46 +01:00
dependabot[bot] 876bb10cf8
chore(deps): bump rand_distr from 0.4.1 to 0.4.2 
Bumps [rand_distr](https://github.com/rust-random/rand) from 0.4.1 to 0.4.2.
- [Release notes](https://github.com/rust-random/rand/releases)
- [Changelog](https://github.com/rust-random/rand/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-random/rand/compare/rand_distr-0.4.1...rand_distr-0.4.2)

---
updated-dependencies:
- dependency-name: rand_distr
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2021-09-20 08:39:39 +00:00
Edd Robinson e51dd0365a refactor: PR feedback 
Co-authored-by: Marko Mikulicic <mkm@influxdata.com>
 2021-09-16 10:01:44 +01:00
Edd Robinson f7228ddd60 test: add test for byte trimmed floats 2021-09-16 10:01:44 +01:00
Edd Robinson d387108dab fix: float byte trimmer filter range 2021-09-16 10:01:44 +01:00
Edd Robinson 0250bd1337 fix: ensure range filter works with null 2021-09-16 10:01:44 +01:00
Edd Robinson 1a70865a03 fix: ensure float byte trimmed predicate pushdown works for unencodable values 2021-09-16 10:01:44 +01:00
Edd Robinson 483508e3c6 feat: add rle method for identifying all non-null row IDs 2021-09-16 10:01:44 +01:00
Edd Robinson d04a0d1137 feat: add method for identifying all non-null row IDs 2021-09-16 10:01:44 +01:00
Edd Robinson 70b0ba44b3 test: failing filter test 2021-09-16 10:01:44 +01:00
Raphael Taylor-Davies c66095cad1
feat: remove metrics crate (#2552) 2021-09-15 19:43:33 +00:00
Raphael Taylor-Davies b8f7319704
feat: migrate read buffer metrics to metric crate (#2510) 
* feat: migrate read buffer metrics to metric crate

* chore: review feedback

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2021-09-10 19:51:43 +00:00
Marco Neumann 79ad48ac3a chore: rename "labels" to "attributes" 2021-08-31 11:31:15 +02:00
Edd Robinson 8dd0576070 refactor: address PR feedback 2021-08-27 12:36:19 +01:00
Edd Robinson 6c49ac5bd4 refactor: update read_buffer/src/chunk.rs 2021-08-27 12:30:20 +01:00
Edd Robinson 6c7f8d6630 feat: add delete to crate Read Buffer API 2021-08-27 12:30:20 +01:00
Edd Robinson dbbfd2a9f8 feat: add delete support to row_group: 2021-08-27 12:30:20 +01:00
Edd Robinson 95548dcec9 feat: add relative complement to RowIDs(bitmap) 2021-08-27 12:30:20 +01:00
Edd Robinson 69329b0b38
Merge branch 'main' into er/refactor/read_buffer/rle_entries 2021-08-25 12:08:44 +01:00
Edd Robinson 11e88877f4 fix: correct size estimation of RLE encoding 2021-08-25 12:03:04 +01:00
Edd Robinson d18e835b4f refactor: remove next_id generation 2021-08-25 11:31:51 +01:00
Edd Robinson 833a410e4a refactor: replace btreeset for vec 
Benchmarks are roughly the same depending on the workload

 critcmp master_string pr_string
group                                                                master_string                             pr_string
-----                                                                -------------                             ---------
_select/enc_"plain encoder"/rows_100000/loc_End/card_100             1.12     43.9±0.41µs  2.1 GElem/sec       1.00     39.4±0.40µs  2.4 GElem/sec
_select/enc_"plain encoder"/rows_100000/loc_End/card_1000            1.00     32.9±0.43µs  2.8 GElem/sec       1.00     33.0±0.48µs  2.8 GElem/sec
_select/enc_"plain encoder"/rows_100000/loc_End/card_10000           1.00     32.1±0.37µs  2.9 GElem/sec       1.00     32.2±0.43µs  2.9 GElem/sec
_select/enc_"plain encoder"/rows_100000/loc_Middle/card_100          1.02     40.2±0.79µs  2.3 GElem/sec       1.00     39.5±0.56µs  2.4 GElem/sec
_select/enc_"plain encoder"/rows_100000/loc_Middle/card_1000         1.00     33.0±0.42µs  2.8 GElem/sec       1.00     33.0±0.38µs  2.8 GElem/sec
_select/enc_"plain encoder"/rows_100000/loc_Middle/card_10000        1.00     32.3±0.41µs  2.9 GElem/sec       1.00     32.4±0.53µs  2.9 GElem/sec
_select/enc_"plain encoder"/rows_100000/loc_Start/card_100           1.04     41.2±1.45µs  2.3 GElem/sec       1.00     39.5±0.54µs  2.4 GElem/sec
_select/enc_"plain encoder"/rows_100000/loc_Start/card_1000          1.01     33.4±0.87µs  2.8 GElem/sec       1.00     32.9±0.43µs  2.8 GElem/sec
_select/enc_"plain encoder"/rows_100000/loc_Start/card_10000         1.01     32.5±0.44µs  2.9 GElem/sec       1.00     32.3±0.51µs  2.9 GElem/sec
_select/enc_"plain encoder"/rows_1000000/loc_End/card_1000           1.00    382.0±3.43µs  2.4 GElem/sec       1.00    382.0±4.04µs  2.4 GElem/sec
_select/enc_"plain encoder"/rows_1000000/loc_End/card_10000          1.00    376.7±4.67µs  2.5 GElem/sec       1.00   377.2±12.83µs  2.5 GElem/sec
_select/enc_"plain encoder"/rows_1000000/loc_End/card_100000         1.00    374.4±3.08µs  2.5 GElem/sec       1.00    375.0±4.09µs  2.5 GElem/sec
_select/enc_"plain encoder"/rows_1000000/loc_Middle/card_1000        1.00    382.4±4.68µs  2.4 GElem/sec       1.00    382.8±4.61µs  2.4 GElem/sec
_select/enc_"plain encoder"/rows_1000000/loc_Middle/card_10000       1.00    375.8±3.55µs  2.5 GElem/sec       1.00    376.0±4.17µs  2.5 GElem/sec
_select/enc_"plain encoder"/rows_1000000/loc_Middle/card_100000      1.00    374.7±3.76µs  2.5 GElem/sec       1.00    375.1±4.44µs  2.5 GElem/sec
_select/enc_"plain encoder"/rows_1000000/loc_Start/card_1000         1.00    382.1±3.80µs  2.4 GElem/sec       1.00    382.2±3.44µs  2.4 GElem/sec
_select/enc_"plain encoder"/rows_1000000/loc_Start/card_10000        1.00    376.5±4.85µs  2.5 GElem/sec       1.00    376.5±4.76µs  2.5 GElem/sec
_select/enc_"plain encoder"/rows_1000000/loc_Start/card_100000       1.00    375.0±3.41µs  2.5 GElem/sec       1.00    375.3±4.28µs  2.5 GElem/sec
_select/enc_"plain encoder"/rows_10000000/loc_End/card_10000         1.00      3.7±0.02ms  2.5 GElem/sec       1.01      3.8±0.06ms  2.5 GElem/sec
_select/enc_"plain encoder"/rows_10000000/loc_End/card_100000        1.00      3.7±0.01ms  2.5 GElem/sec       1.01      3.8±0.06ms  2.5 GElem/sec
_select/enc_"plain encoder"/rows_10000000/loc_End/card_1000000       1.00      3.7±0.01ms  2.5 GElem/sec       1.01      3.8±0.10ms  2.5 GElem/sec
_select/enc_"plain encoder"/rows_10000000/loc_Middle/card_10000      1.00      3.8±0.03ms  2.5 GElem/sec       1.00      3.8±0.04ms  2.5 GElem/sec
_select/enc_"plain encoder"/rows_10000000/loc_Middle/card_100000     1.00      3.8±0.03ms  2.5 GElem/sec       1.07      4.0±0.73ms  2.3 GElem/sec
_select/enc_"plain encoder"/rows_10000000/loc_Middle/card_1000000    1.02      3.8±0.06ms  2.4 GElem/sec       1.00      3.8±0.03ms  2.5 GElem/sec
_select/enc_"plain encoder"/rows_10000000/loc_Start/card_10000       1.00      3.8±0.03ms  2.5 GElem/sec       1.00      3.8±0.03ms  2.5 GElem/sec
_select/enc_"plain encoder"/rows_10000000/loc_Start/card_100000      1.00      3.8±0.04ms  2.5 GElem/sec       1.00      3.8±0.04ms  2.5 GElem/sec
_select/enc_"plain encoder"/rows_10000000/loc_Start/card_1000000     1.00      3.8±0.05ms  2.5 GElem/sec       1.00      3.8±0.03ms  2.5 GElem/sec
select/enc_"RLE encoder"/rows_100000/loc_End/card_100                1.00      2.9±0.03µs 32.0 GElem/sec       1.01      2.9±0.09µs 31.6 GElem/sec
select/enc_"RLE encoder"/rows_100000/loc_End/card_1000               1.06  1002.0±13.75ns 93.0 GElem/sec       1.00    948.3±9.63ns 98.2 GElem/sec
select/enc_"RLE encoder"/rows_100000/loc_End/card_10000              1.02      4.6±0.05µs 20.3 GElem/sec       1.00      4.5±0.17µs 20.7 GElem/sec
select/enc_"RLE encoder"/rows_100000/loc_Middle/card_100             1.00      3.0±0.03µs 31.5 GElem/sec       1.00      2.9±0.04µs 31.6 GElem/sec
select/enc_"RLE encoder"/rows_100000/loc_Middle/card_1000            1.04   788.9±12.39ns 118.1 GElem/sec      1.00   755.7±20.50ns 123.2 GElem/sec
select/enc_"RLE encoder"/rows_100000/loc_Middle/card_10000           1.00      2.8±0.43µs 33.5 GElem/sec       1.02      2.8±0.03µs 32.8 GElem/sec
select/enc_"RLE encoder"/rows_100000/loc_Start/card_100              1.00      2.9±0.04µs 32.3 GElem/sec       1.02      2.9±0.10µs 31.7 GElem/sec
select/enc_"RLE encoder"/rows_100000/loc_Start/card_1000             1.03   597.4±14.85ns 155.9 GElem/sec      1.00   581.1±13.60ns 160.3 GElem/sec
select/enc_"RLE encoder"/rows_100000/loc_Start/card_10000            1.42   606.6±13.37ns 153.5 GElem/sec      1.00    426.0±6.32ns 218.6 GElem/sec
select/enc_"RLE encoder"/rows_1000000/loc_End/card_1000              1.00      3.3±0.03µs 280.9 GElem/sec      1.03      3.4±0.47µs 273.5 GElem/sec
select/enc_"RLE encoder"/rows_1000000/loc_End/card_10000             1.00      4.6±0.09µs 200.6 GElem/sec      1.03      4.8±0.06µs 194.8 GElem/sec
select/enc_"RLE encoder"/rows_1000000/loc_End/card_100000            1.01     41.5±0.44µs 22.4 GElem/sec       1.00     41.1±0.57µs 22.6 GElem/sec
select/enc_"RLE encoder"/rows_1000000/loc_Middle/card_1000           1.02      3.1±0.04µs 296.8 GElem/sec      1.00      3.1±0.05µs 301.8 GElem/sec
select/enc_"RLE encoder"/rows_1000000/loc_Middle/card_10000          1.00      2.8±0.05µs 332.6 GElem/sec      1.12      3.1±0.46µs 297.2 GElem/sec
select/enc_"RLE encoder"/rows_1000000/loc_Middle/card_100000         1.10     23.7±0.30µs 39.2 GElem/sec       1.00     21.5±0.25µs 43.3 GElem/sec
select/enc_"RLE encoder"/rows_1000000/loc_Start/card_1000            1.00      2.9±0.03µs 321.1 GElem/sec      1.00      2.9±0.04µs 320.5 GElem/sec
select/enc_"RLE encoder"/rows_1000000/loc_Start/card_10000           1.00    623.6±7.76ns 1493.6 GElem/sec     1.06   661.5±44.34ns 1408.0 GElem/sec
select/enc_"RLE encoder"/rows_1000000/loc_Start/card_100000          1.00   954.4±18.68ns 975.9 GElem/sec      2.94      2.8±0.89µs 331.9 GElem/sec
select/enc_"RLE encoder"/rows_10000000/loc_End/card_10000            1.01      7.0±0.09µs 1335.5 GElem/sec     1.00      6.9±0.10µs 1353.8 GElem/sec
select/enc_"RLE encoder"/rows_10000000/loc_End/card_100000           1.06     42.8±0.78µs 217.6 GElem/sec      1.00     40.4±0.49µs 230.7 GElem/sec
select/enc_"RLE encoder"/rows_10000000/loc_End/card_1000000          1.00    397.9±6.26µs 23.4 GElem/sec       1.09    433.3±5.78µs 21.5 GElem/sec
select/enc_"RLE encoder"/rows_10000000/loc_Middle/card_10000         1.03      5.2±0.05µs 1779.4 GElem/sec     1.00      5.1±0.17µs 1840.2 GElem/sec
select/enc_"RLE encoder"/rows_10000000/loc_Middle/card_100000        1.00     20.3±0.21µs 458.9 GElem/sec      1.15     23.4±0.30µs 397.9 GElem/sec
select/enc_"RLE encoder"/rows_10000000/loc_Middle/card_1000000       1.18    211.4±3.28µs 44.1 GElem/sec       1.00    178.5±2.56µs 52.2 GElem/sec
select/enc_"RLE encoder"/rows_10000000/loc_Start/card_10000          1.00      3.0±0.04µs 3091.2 GElem/sec     1.00      3.0±0.08µs 3079.4 GElem/sec
select/enc_"RLE encoder"/rows_10000000/loc_Start/card_100000         1.00   785.1±10.39ns 11862.8 GElem/sec    2.48  1948.8±44.72ns 4778.9 GElem/sec
select/enc_"RLE encoder"/rows_10000000/loc_Start/card_1000000        1.00      6.5±0.07µs 1433.0 GElem/sec     2.07     13.5±0.16µs 692.3 GElem/sec
 2021-08-25 11:19:58 +01:00
Edd Robinson f3c57c47fa
Merge branch 'main' into er/refactor/read_buffer/table_arg 2021-08-25 10:30:12 +01:00
Marco Neumann fac79a2ae7
refactor: simplify RLE allocation code 
Co-authored-by: Edd Robinson <me@edd.io>
 2021-08-25 10:54:18 +02:00
Marco Neumann 2ad9843e5f feat: make `RLE` a bit smaller by capacity-based allocation 
For some demo data this reduced the overall chunk size from

195049367 bytes
to
191088095 bytes
 2021-08-25 10:22:43 +02:00
Edd Robinson 5648817285 refactor: remove redunant argument 2021-08-24 22:26:17 +01:00