Commit Graph

330 Commits (2f1efcf517f7504d728f6faddbf296bdc0e5ea26)

Author SHA1 Message Date
Andrew Lamb 4da8a16c18
chore: update to arrow 5.0 and master datafusion (#2049)
* chore: update to arrow 5.0 and master datafusion

* fix: Update test for change in object size
2021-07-19 12:49:51 +00:00
Edd Robinson 54ad69ed86 fix: ensure correct table meta size used 2021-07-16 10:48:45 -04:00
Carol (Nichols || Goulding) f3175ed291 test: use of different size values 2021-07-16 09:47:56 -04:00
Carol (Nichols || Goulding) abe2fe7262 test: MetaData new with a row group vs default then update_with should have the same size 2021-07-16 09:47:56 -04:00
Andrew Lamb 3fd6430fb6
fix: rename `estimated_bytes` to `memory_bytes` and expose `object_store_bytes` in ChunkSummary and system.chunks (#2017)
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-07-15 16:00:24 +00:00
kodiakhq[bot] 833debd5b5
Merge branch 'main' into cn/exploration 2021-07-14 17:30:55 +00:00
Raphael Taylor-Davies 1d00fa2fd8
refactor: track memory metrics in catalog (#1995)
* refactor: track memory metrics in catalog

* chore: update comment
2021-07-14 16:23:00 +00:00
Carol (Nichols || Goulding) 8070065e2f fix: Change RUB chunk table_summaries to table_summary
Because chunks now have only one table.

Connects to #1718, #1613, #1295
2021-07-14 11:18:02 -04:00
Andrew Lamb 97c727a2c2 fix: update read_buffer tests 2021-07-13 15:44:57 -04:00
Marco Neumann 2e391deb34 chore: update croaring to 0.5.0
Upstreame changelog:

- CRoaring updated to 0.3.1
- `-march=native` is not a default for croaring-sys anymore
- Impl Default for `Bitmap` and `Treemap`
2021-07-13 15:15:41 +02:00
Andrew Lamb d35b74c226
fix: Fix doc build warnings (#1945)
* fix: Fix doc build warnings

* refactor: add deny bare_urls to crates

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-07-13 08:03:42 +00:00
Edd Robinson f811bf1e5e refactor: log compaction activity 2021-07-08 12:48:41 +01:00
Andrew Lamb 7602bde850
chore: Update datafusion deps (#1799)
* chore: Update datafusion deps + rework code

* refactor: remove workaround as it has been contributed upstream

* fix: Update query/src/exec/split.rs

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-07-08 10:58:32 +00:00
Andrew Lamb e6d995cbd8
chore: Update to Rust 1.53.0 (#1922)
* chore: Update to Rust 1.53.0

* fix: Update to latest clippy standards

* fix: bad refactor

* fix: Update escaping

* test: update test output

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-07-07 18:02:03 +00:00
Edd Robinson 2ec9151b32
Merge branch 'main' into er/fix/read_buffer/predicate 2021-07-06 13:35:04 +01:00
Raphael Taylor-Davies b4534883fe
refactor: remove table name from upsert_table (#1882)
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-07-02 15:22:41 +00:00
Edd Robinson 8fc07cf4f0 fix: correctly evaluate exprs matching disjoint rows 2021-07-01 16:05:09 +01:00
Edd Robinson 2e430ac7f0 refactor: remove table name from read_filter schema 2021-06-30 09:50:53 +01:00
Edd Robinson 62f274cc1b refactor: remove table name from column_values 2021-06-30 09:46:54 +01:00
Edd Robinson 5737c9d962 refactor: remove table name from column_names 2021-06-30 09:43:41 +01:00
Edd Robinson 86e2fe4138 refactor: satisfies predicate 2021-06-29 11:58:28 +01:00
Edd Robinson ed98812c2a refactor: logging 2021-06-25 17:37:04 +01:00
Raphael Taylor-Davies 84e63bb1b9
feat: temporary log information on record batch data (#1807)
* feat: temporary log information on record batch data

* chore: add buffer size
2021-06-25 14:46:51 +00:00
Edd Robinson 7e3df17896 test: update benchmarks 2021-06-21 15:29:23 +01:00
Edd Robinson 2ff80162d3 refactor: one table per chunk 2021-06-21 15:08:38 +01:00
Edd Robinson 15416ca223 refactor: enable empty table init 2021-06-21 15:08:38 +01:00
Andrew Lamb ec43a87909
chore: Update itertools deps (#1750) 2021-06-17 17:56:44 +00:00
Edd Robinson ff19beb0ad refactor: export rb chunk as RBChunk 2021-06-11 18:33:10 +01:00
Raphael Taylor-Davies 1e7ef193a6
refactor: use field metadata to store influx types (#1642)
* refactor: use field metadata to store influx types

make SchemaBuilder non-consuming

* chore: remove unused variants

* chore: fix lints
2021-06-07 13:26:39 +00:00
Edd Robinson 418cc4cf0e
refactor: update read_buffer/src/column/encoding/scalar/transcoders.rs
Co-authored-by: Dom <dom@itsallbroken.com>
2021-06-03 14:02:16 +01:00
Edd Robinson 22c7592e3b refactor: DRY RLE check 2021-06-03 12:32:40 +01:00
Edd Robinson 9a45c0d05b feat: implement float byte trimming for arrow array 2021-06-03 12:32:40 +01:00
Edd Robinson 5d02a71e6f feat: implement byte trimming on float slice 2021-06-03 12:32:40 +01:00
Edd Robinson 32e5f8c715 feat: add float byte trimmer encoding 2021-06-03 12:32:40 +01:00
Edd Robinson 728476f2e1 refactor: add encoding name to float encodings 2021-06-03 12:32:40 +01:00
Edd Robinson fa729fd6b0 refactor: address PR feedback 2021-06-02 11:21:10 +01:00
Edd Robinson a5b554d2c3 feat: add RLE support to integer encodings 2021-06-02 10:57:17 +01:00
Edd Robinson 71598d9b3e refactor: move rle heuristics to rle module 2021-06-02 10:57:17 +01:00
Dom aca00a505f refactor: avoid copying encoding name for stats
Swaps the encoding name String type for a Cow in the column Statistics,
avoiding having to copy the encoding name where it is a static string
already.
2021-06-01 11:08:07 +01:00
Andrew Lamb 00e735ef0d
chore: remove unused dependencies (#1583) 2021-05-29 10:31:57 +00:00
Raphael Taylor-Davies db432de137
feat: add distinct count to StatValues (#1568) 2021-05-28 17:41:34 +00:00
Edd Robinson e94be15296
refactor: update read_buffer/src/column/encoding/scalar/fixed.rs 2021-05-27 21:18:10 +01:00
Edd Robinson 26fe4167f7 refactor: erase some types 2021-05-27 14:57:40 +01:00
Edd Robinson bba387d6ff refactor: fix benchmarks 2021-05-27 14:35:34 +01:00
Edd Robinson 7ab27c4468 refactor: get build 2021-05-27 14:35:34 +01:00
Edd Robinson 04513e737d refactor: define encodings wrt to scalar encoding trait 2021-05-27 14:35:34 +01:00
Edd Robinson 22f8a8a4a1 refactor: define scalar encoding in terms of trait 2021-05-27 14:35:34 +01:00
Edd Robinson c84d50447c feat: define a Transcoder trait 2021-05-27 14:35:34 +01:00
Edd Robinson a81ada6140 feat: add transcoder trait 2021-05-27 14:35:34 +01:00
Raphael Taylor-Davies 4fcc04e6c9
chore: enable arrow prettyprint feature (#1566) 2021-05-27 10:28:14 +00:00