Commit Graph

44 Commits (36b2cb021d554c3d2a7ba12242d59b353f00c95b)

Author SHA1 Message Date
Raphael Taylor-Davies afe34751e7
refactor: split out schema crate (#2781)
* refactor: split out schema crate

* chore: fix doc
2021-10-11 09:45:08 +00:00
Edd Robinson a70a55cb3d test: update benchmarks 2021-09-24 15:00:17 +01:00
Edd Robinson 6c7f8d6630 feat: add delete to crate Read Buffer API 2021-08-27 12:30:20 +01:00
Edd Robinson 5648817285 refactor: remove redunant argument 2021-08-24 22:26:17 +01:00
Carol (Nichols || Goulding) 9d15798288 fix: Address or allow Clippy warnings new with Rust 1.54 2021-07-30 09:59:59 -04:00
Carol (Nichols || Goulding) 11b7755325 refactor: Remove first/last write times from RUB chunks 2021-07-28 11:22:22 -04:00
Carol (Nichols || Goulding) 37f24ebfc7 feat: Record first/last write times for creation of read_buffer::Chunk 2021-07-22 11:35:23 -04:00
Carol (Nichols || Goulding) 6feea3b2d5 feat: Require at least one RecordBatch to create a read_buffer::Chunk::new
In the signature only for the moment.
2021-07-22 11:15:18 -04:00
Carol (Nichols || Goulding) 0a724878e6 refactor: Organize uses 2021-07-22 11:15:18 -04:00
Raphael Taylor-Davies b4534883fe
refactor: remove table name from upsert_table (#1882)
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-07-02 15:22:41 +00:00
Edd Robinson 86e2fe4138 refactor: satisfies predicate 2021-06-29 11:58:28 +01:00
Edd Robinson 7e3df17896 test: update benchmarks 2021-06-21 15:29:23 +01:00
Edd Robinson ff19beb0ad refactor: export rb chunk as RBChunk 2021-06-11 18:33:10 +01:00
Edd Robinson bba387d6ff refactor: fix benchmarks 2021-05-27 14:35:34 +01:00
Edd Robinson 4cb76e367b refactor: fix change to Chunk API 2021-05-20 11:11:18 +01:00
Edd Robinson c901fe1023 perf: improve values_as_dictionary with predicates 2021-05-20 10:49:49 +01:00
Edd Robinson b7b87c1c96 test: add read_filter benchmark 2021-05-20 10:49:49 +01:00
Edd Robinson 4e766d7085 refactor: reorganise benchmarks 2021-05-20 10:49:49 +01:00
Edd Robinson c8e2c9224e chore: rename benchmark 2021-05-20 10:49:49 +01:00
Raphael Taylor-Davies 37880ee89a
refactor: store chunk IDs only in catalog (#1521)
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-05-20 04:07:14 +00:00
Edd Robinson 6a72274517 feat: extend implementations to more Arrow arrays 2021-05-17 14:32:55 +01:00
Raphael Taylor-Davies f9178dbb5f
feat: push metrics into catalog (#1488)
* feat: push metrics into catalog

* chore: minor cleanup

* fix: include db labels in chunk metric domains

* chore: fmt

* fix: don't allow dropping moving chunks

* chore: further tweaks

* chore: review feedback

* feat: use new_unregistered() for metric instruments instead of default

* chore: use &[KeyValue] instead of &Vec<KeyValue>

* refactor: make GauageValue non default constructible
2021-05-14 17:37:39 +00:00
Edd Robinson 1416097a35
Merge branch 'main' into er/feat/read_buffer/num_rle 2021-05-11 23:30:55 +01:00
Edd Robinson 482e4dab86 refactor: shuffle string encodings 2021-05-11 22:47:42 +01:00
Edd Robinson f86e0641fd refactor: clarify benchmark 2021-05-11 22:47:42 +01:00
Edd Robinson f5fe270e43 refactor: move benchmark 2021-05-11 22:47:36 +01:00
Edd Robinson 696e4e0cfd fix: ensure metrics not overwriting 2021-05-11 20:57:31 +01:00
Edd Robinson 32abe2e777 feat: wire up stats to metrics 2021-05-11 13:38:32 +01:00
Raphael Taylor-Davies 411cf134e9
refactor: explode arrow_deps (#1425)
* refactor: explode arrow_deps

* chore: workaround doctest bug
2021-05-05 16:59:12 +00:00
Edd Robinson 2784f89e6e refactor: sigh 2021-04-20 17:30:50 +00:00
Edd Robinson dae9f12593 refactor: remove deprecated API 2021-04-08 10:24:19 +00:00
Edd Robinson ea2c882635 refactor: tidy up 2021-04-07 10:46:08 +00:00
Edd Robinson d88e20c7fe fix: fix bad expected cardinality 2021-03-22 16:15:49 +00:00
Andrew Lamb 6e1795fda0
refactor: Move some types (not yet exposed to clients) into internal_types (#1015)
* refactor: Move some types (not yet exposed to clients) into internal_types

* docs: Add README.md explaining the rationale

* refactor: remove some stragglers

* fix: fix benches

* fix: Apply suggestions from code review

Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>

* fix: add clippy lints

* fix: fmt

* docs: Apply suggestions from code review

fix typos

Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-03-19 16:27:57 +00:00
Raphael Taylor-Davies 65f7a1ac5b
fix: use consistent crate versions (#989)
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-03-15 15:42:19 +00:00
Andrew Lamb a316b16960
feat: Change table_names to return either Some(set) or None, rather than a plan (try 2) (#776)
* feat: Change table_names to return either Some(set) or None, rather than a plan

* docs: improve comments

* docs: Apply suggestions from code review

Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>

* fix: merge conflict

* fix: don't clone a string unless needed

Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
2021-02-09 12:20:59 -05:00
Andrew Lamb 8399c56587
feat: remove RwLock on entire ReadBuffer (#761) 2021-02-05 16:58:17 -05:00
Edd Robinson c8ce27ce5e perf: add benchmark for table_names
This commit adds some benchmarks for `table_names` against the read
buffer's Database implementation. On my laptop these look like:

database_table_names_all_tables
                        time:   [2.2104 us 2.2242 us 2.2381 us]
Found 2 outliers among 100 measurements (2.00%)
  1 (1.00%) high mild
  1 (1.00%) high severe

database_table_names_meta_pred_no_match
                        time:   [1.8389 us 1.8488 us 1.8593 us]
Found 3 outliers among 100 measurements (3.00%)
  1 (1.00%) high mild
  2 (2.00%) high severe

database_table_names_single_pred_match
                        time:   [5.5457 us 5.5694 us 5.5919 us]
Found 5 outliers among 100 measurements (5.00%)
  3 (3.00%) high mild
  2 (2.00%) high severe

database_table_names_multi_pred_match
                        time:   [478.85 us 480.32 us 481.83 us]
Found 4 outliers among 100 measurements (4.00%)
  2 (2.00%) high mild
  2 (2.00%) high severe

database_table_names_multi_pred_match_multi_tables
                        time:   [476.47 us 478.93 us 482.25 us]
Found 11 outliers among 100 measurements (11.00%)
  4 (4.00%) high mild
  7 (7.00%) high severe
2021-01-26 17:00:53 +00:00
Edd Robinson 42c0ccf274 refactor: rename method to 2021-01-18 12:05:11 +00:00
Edd Robinson c6ff633afd test: expose internals for benchmarking
This commit is a bit of a hack. The first thing I could think of. The
problem is that I want to be able to benchmark various modules in the
read buffer but I don't want to expose those internals via the external
API.

Becuase criterion only lets you exercise the exported API I needed to
expose some internals. I did this by creating a documented module
`benchmarks` in the `read_buffer` crate, which re-exports identifiers
that can be used by a criterion crate.

The idea is that it will be clear that this module is not part of the
public API.
2021-01-14 22:46:27 +00:00
Edd Robinson d536f82879 fix: get benchmarks compiling 2021-01-14 14:06:17 +00:00
Edd Robinson fa8afe845d refactor: fix benchmarks 2020-12-22 21:26:05 +00:00
Edd Robinson 0af935d123 refactor: rename segment module to row_group 2020-12-22 21:26:04 +00:00
Edd Robinson 199ba68769 refactor: rename segment_store crate to read_buffer 2020-12-22 21:26:04 +00:00