d44099d242
refactor: update read_buffer/src/column/bool.rs
...
Co-authored-by: Andrew Lamb <alamb@influxdata.com>
2021-02-01 15:17:51 +00:00
7e1ac34906
refactor: recordbatch boolean -> Bool
2021-02-01 12:29:04 +00:00
172c8d146c
refactor: wire up bool encoding to read buffer column
2021-02-01 12:02:14 +00:00
36d9541cbc
feat: an arrow-backed boolean encoding
2021-02-01 12:02:14 +00:00
679e04fdb3
refactor: remove redundant From implementations
2021-02-01 10:52:27 +00:00
42e2178110
refactor: simpify arrow -> read buffer column
2021-01-31 21:19:18 +00:00
28b596b883
refactor: wire up fixed null encoding
2021-01-31 21:12:47 +00:00
0195dfc03a
test: add coverage for NULL values in queries
2021-01-31 21:12:06 +00:00
4d107334dd
test: adjust number of rows
2021-01-31 12:01:26 +00:00
0bed5e2290
refactor: update read_buffer/src/row_group.rs
...
Co-authored-by: Andrew Lamb <alamb@influxdata.com>
2021-01-31 11:55:04 +00:00
02c154c746
refactor: min/max support
2021-01-30 22:58:39 +00:00
9cc9c714ed
refactor: wire up sum
2021-01-30 22:42:58 +00:00
fd25b6a9e2
refactor: wire up aggregate count
2021-01-30 21:01:25 +00:00
75a2e1caff
refactor: results can just contain aggregates
2021-01-30 10:02:03 +00:00
a71d3eea86
refactor: address PR feedback
2021-01-29 22:01:51 +00:00
3bb58fe971
refactor: tidy up commented code
2021-01-29 22:01:51 +00:00
46f20df089
refactor: change Rc -> Arc
2021-01-29 22:01:51 +00:00
ec10c81041
refactor: simplify chunk locking implementation
2021-01-29 22:01:51 +00:00
6b6c1476f6
refactor: implement table meta-data rebuilding
2021-01-29 22:01:51 +00:00
13cbc12298
feat: make database support concurrent access
2021-01-29 22:01:51 +00:00
bc08c6b404
feat: support concurrent access to Chunk
2021-01-29 22:01:51 +00:00
30b90943bc
feat: make Table concurrent-safe
2021-01-29 22:01:51 +00:00
050185ad92
refactor: ensure meta updated when rowgroup add/removed
2021-01-29 22:01:51 +00:00
338bbb9b55
refactor: materialise rb for read_aggregate at table
2021-01-29 22:01:51 +00:00
e3afab12a7
refactor: rb from table read_filter
2021-01-29 22:01:51 +00:00
9d3c623a14
refactor: baseline Rc
2021-01-29 22:01:51 +00:00
2282a68e65
refactor: Move selection to the data_types crate and remove redundant implemenation ( #704 )
2021-01-29 13:35:07 -05:00
c8ce27ce5e
perf: add benchmark for table_names
...
This commit adds some benchmarks for `table_names` against the read
buffer's Database implementation. On my laptop these look like:
database_table_names_all_tables
time: [2.2104 us 2.2242 us 2.2381 us]
Found 2 outliers among 100 measurements (2.00%)
1 (1.00%) high mild
1 (1.00%) high severe
database_table_names_meta_pred_no_match
time: [1.8389 us 1.8488 us 1.8593 us]
Found 3 outliers among 100 measurements (3.00%)
1 (1.00%) high mild
2 (2.00%) high severe
database_table_names_single_pred_match
time: [5.5457 us 5.5694 us 5.5919 us]
Found 5 outliers among 100 measurements (5.00%)
3 (3.00%) high mild
2 (2.00%) high severe
database_table_names_multi_pred_match
time: [478.85 us 480.32 us 481.83 us]
Found 4 outliers among 100 measurements (4.00%)
2 (2.00%) high mild
2 (2.00%) high severe
database_table_names_multi_pred_match_multi_tables
time: [476.47 us 478.93 us 482.25 us]
Found 11 outliers among 100 measurements (11.00%)
4 (4.00%) high mild
7 (7.00%) high severe
2021-01-26 17:00:53 +00:00
42d629ac32
feat: wire up predicate support to external API
2021-01-26 17:00:53 +00:00
c89a569e03
feat: add per-chunk pred support in table_names
2021-01-26 17:00:53 +00:00
8a23e22957
feat: determine if row group satisfies predicate
2021-01-26 17:00:53 +00:00
c3b0371c84
feat: Initial RPC Query Frontend ( #692 )
...
* feat: Initial RPC Query Frontend
* docs: s/immutable buffer/mutable buffer
* docs: Correct type in docstring
2021-01-25 08:33:39 -05:00
3da9b73464
refactor: add assertion
2021-01-25 11:26:15 +00:00
5fe5ed0569
test: more aggregate coverage
2021-01-25 11:21:20 +00:00
c60cfbd2bb
refactor: update read_buffer/src/lib.rs
...
Co-authored-by: Andrew Lamb <alamb@influxdata.com>
2021-01-25 11:21:20 +00:00
a4bde5c252
refactor: error on unsupported aggregates
2021-01-25 11:21:20 +00:00
d848526124
refactor: tidy up
2021-01-25 11:21:20 +00:00
9270874760
feat: wire up read_aggregate to external API
2021-01-25 11:21:20 +00:00
e6b8d0e072
feat: add support for converting to record batch:
2021-01-25 11:21:20 +00:00
09ec6b78d3
feat: add ability to merge ReadAggregateResults
2021-01-25 11:21:20 +00:00
7969808f09
feat: Chunk Migration APIs and query data in the read buffer via SQL ( #668 )
...
* feat: Chunk Migration APIs and query data in the read buffer via SQL
* fix: Make code more consistent
* fix: fmt / clippy
* chore: Apply suggestions from code review
Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
* refactor: Remove unecessary Result and make chunks() infallable
* chore: Apply more suggestions from code review
Co-authored-by: Edd Robinson <me@edd.io>
Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
Co-authored-by: Edd Robinson <me@edd.io>
2021-01-19 13:28:26 -05:00
221ed86853
refactor: apply suggestions from code review
...
Co-authored-by: Andrew Lamb <alamb@influxdata.com>
2021-01-19 15:33:44 +00:00
57365b082c
refactor: clean up filter_map
2021-01-19 15:33:44 +00:00
dbdd885e58
refactor: follow snafu style guide
2021-01-19 15:33:44 +00:00
5f6335573b
fix: ensure table missing handled
2021-01-19 15:33:44 +00:00
9f65c4b6ef
refactor: encapsulate column meta data
2021-01-19 15:33:44 +00:00
0c7424465d
refactor: use schema type for read_filter
2021-01-19 15:33:44 +00:00
93f4f8aa41
feat: teach read_buffer schema -> data_types schema
2021-01-19 15:33:44 +00:00
864e9e4dac
refactor: tidy up columns in row group
2021-01-19 15:33:44 +00:00
71fce96b3b
feat: encapsulate semantic column type in result schema
2021-01-19 15:33:44 +00:00
17358589ed
refactor: move AggregateType to schema
2021-01-19 15:33:44 +00:00
d805ce6189
refactor: move LogicalDataType into Schema
2021-01-19 15:33:44 +00:00
e34979532d
refactor: fix Display implementation
2021-01-18 12:05:11 +00:00
bdeacdcf37
docs: define merging
2021-01-18 12:05:11 +00:00
e2a16d7d4c
refactor: fix error handling
2021-01-18 12:05:11 +00:00
3c61fdb773
refactor: update read_buffer/src/row_group.rs
...
Co-authored-by: Andrew Lamb <alamb@influxdata.com>
2021-01-18 12:05:11 +00:00
a82bf8de78
refactor: Update read_buffer/src/chunk.rs
...
Co-authored-by: Andrew Lamb <alamb@influxdata.com>
2021-01-18 12:05:11 +00:00
26d11be09c
refactor: wire up read_aggregate to chunk
2021-01-18 12:05:11 +00:00
eaabb82404
refactor: wire up read_aggregate in table
2021-01-18 12:05:11 +00:00
0a4d890f31
refactor: move schema information for results into type
2021-01-18 12:05:11 +00:00
6dbb00715e
refactor: rename method
2021-01-18 12:05:11 +00:00
42c0ccf274
refactor: rename method to
2021-01-18 12:05:11 +00:00
71627120b9
refactor: consolidate line protocol schema creation into data_types and port code to use it ( #663 )
...
* refactor: consolidate line protocol schema creation into data_types, and port code to use it
refactor: Port mutable buffer to use SchemaBuilder
* fix: doctest
* refactor: remove unecessary clippyisms
* docs: Improve comments via suggestions from code review
Co-authored-by: Edd Robinson <me@edd.io>
* refactor: use more idomatic try_ naming and TryInto trait
* docs: Change from line protocol data model to InfluxDB data model
* refactor: rename LP --> Influx in code
* feat: add support for UInteger type
Co-authored-by: Edd Robinson <me@edd.io>
2021-01-15 17:29:30 -05:00
7f4f44211f
test: fix broken test
2021-01-15 12:54:24 +00:00
c6ff633afd
test: expose internals for benchmarking
...
This commit is a bit of a hack. The first thing I could think of. The
problem is that I want to be able to benchmark various modules in the
read buffer but I don't want to expose those internals via the external
API.
Becuase criterion only lets you exercise the exported API I needed to
expose some internals. I did this by creating a documented module
`benchmarks` in the `read_buffer` crate, which re-exports identifiers
that can be used by a criterion crate.
The idea is that it will be clear that this module is not part of the
public API.
2021-01-14 22:46:27 +00:00
d55bc4835a
refactor: rename method
2021-01-14 21:45:54 +00:00
d82293ba23
refactor: add API method
2021-01-14 21:26:50 +00:00
1d5bc6f345
refactor: expose column_names for getting tag keys
2021-01-14 21:26:50 +00:00
676b58dc2c
fix: ensure correct chunks used
2021-01-14 21:26:50 +00:00
4c4d2e8e67
refactor: add API method for read_aggregate_window
2021-01-14 21:26:50 +00:00
3026668878
refactor: add API method for read_aggregate
2021-01-14 21:26:50 +00:00
bdb8a78569
refactor: reduce expose API
2021-01-14 21:26:50 +00:00
d536f82879
fix: get benchmarks compiling
2021-01-14 14:06:17 +00:00
6c5fdc0fae
refactor: clean up comment
2021-01-14 13:46:20 +00:00
2ba438cf4f
refactor: update read_buffer/src/row_group.rs
...
Co-authored-by: Andrew Lamb <alamb@influxdata.com>
2021-01-14 13:46:20 +00:00
05e083d0e2
feat: support converting from DF types
2021-01-14 13:14:08 +00:00
728d556b7d
refactor: encapsulate read buffer predicats in type
2021-01-14 13:14:08 +00:00
a5240af080
docs: Document desired crate dependencies in comments ( #638 )
...
* docs: Document the desire for read buffer and mutable buffer to be independent of query layer
* docs: Document desire for the query layer to not depend on storage systems
* fix: Apply suggestions from code review
Co-authored-by: Edd Robinson <me@edd.io>
Co-authored-by: Edd Robinson <me@edd.io>
2021-01-12 17:49:03 -05:00
9ec0ae26e1
refactor: implement From<ArrayRef>
2021-01-11 16:11:21 +00:00
9eef7b4d7f
feat: add enum for selecting 'all' columns
2021-01-11 16:11:21 +00:00
61466fed44
refactor: add outline for read_aggregate
2021-01-11 16:11:21 +00:00
5a15a11a5c
feat: lazily return record batches for read_filter
2021-01-11 16:11:21 +00:00
c3019a91bd
feat: add support for determining logical column types
2021-01-11 16:11:20 +00:00
1d972e01c8
refactor: Update read_buffer/src/lib.rs
...
Co-authored-by: Andrew Lamb <alamb@influxdata.com>
2021-01-08 21:03:38 +00:00
23f27e10fa
feat: add partition concept to ReadBuffer
2021-01-08 21:03:38 +00:00
46f85bb6a6
refactor: adrress PR comments
...
Co-authored-by: Andrew Lamb <alamb@influxdata.com>
Co-authored-by: Dom <dom@itsallbroken.com>
2021-01-08 16:19:19 +00:00
4ce6821d90
feat: implement table_names on
2021-01-08 16:19:19 +00:00
590b74e386
refactor: add test for adding chunk to database
2021-01-08 16:19:19 +00:00
6df2de62bb
refactor: provide an API for table row groups
2021-01-08 16:19:19 +00:00
954da31e83
test: fix tests
2021-01-08 16:19:19 +00:00
2178a6eae4
feat: hook up record batch -> chunk to store
2021-01-08 16:19:19 +00:00
b1ab6a189d
feat: record batch -> read buffer column
2021-01-08 16:19:19 +00:00
8382501440
refactor: validate column types
2021-01-08 16:19:19 +00:00
8219403fab
feat: Instantiate ReadBuffer as part of server creation ( #620 )
...
* feat: Instantiate ReadBuffer as part of server creation
* refactor: remove Store from read_buffer
2021-01-07 13:25:42 -05:00
937442cfa0
refactor: update partition refs to chunk
2020-12-28 21:08:56 +00:00
c46cf6fdcf
refactor: rename partition to chunk
2020-12-28 21:08:56 +00:00
c0dc93a8cb
refactor: change partition to chunk
2020-12-28 21:08:56 +00:00
fa8afe845d
refactor: fix benchmarks
2020-12-22 21:26:05 +00:00
b1aabc14b2
refactor: rename Segment to RowGroup
2020-12-22 21:26:04 +00:00
0af935d123
refactor: rename segment module to row_group
2020-12-22 21:26:04 +00:00
199ba68769
refactor: rename segment_store crate to read_buffer
2020-12-22 21:26:04 +00:00
Edd Robinson
Andrew Lamb