Commit Graph

2176 Commits (bbfc49bacda5765841726ddc930f26c6abf1cdc8)

Author SHA1 Message Date
Edd Robinson 0dae29f9ea
Merge pull request #714 from influxdata/er/fix/read_buffer/arrow_fixed_null
refactor: wire up nullable column encodings
2021-02-01 12:01:38 +00:00
Edd Robinson 679e04fdb3 refactor: remove redundant From implementations 2021-02-01 10:52:27 +00:00
Edd Robinson 42e2178110 refactor: simpify arrow -> read buffer column 2021-01-31 21:19:18 +00:00
Edd Robinson 28b596b883 refactor: wire up fixed null encoding 2021-01-31 21:12:47 +00:00
Edd Robinson 0195dfc03a test: add coverage for NULL values in queries 2021-01-31 21:12:06 +00:00
Edd Robinson 172d95a5b2
Merge pull request #713 from influxdata/er/refactor/read_buffer/enable-simple-aggs
feat: aggregates without group keys
2021-01-31 20:36:20 +00:00
Edd Robinson 4d107334dd test: adjust number of rows 2021-01-31 12:01:26 +00:00
Edd Robinson 0bed5e2290
refactor: update read_buffer/src/row_group.rs
Co-authored-by: Andrew Lamb <alamb@influxdata.com>
2021-01-31 11:55:04 +00:00
Edd Robinson 02c154c746 refactor: min/max support 2021-01-30 22:58:39 +00:00
Edd Robinson 9cc9c714ed refactor: wire up sum 2021-01-30 22:42:58 +00:00
Edd Robinson fd25b6a9e2 refactor: wire up aggregate count 2021-01-30 21:01:25 +00:00
Edd Robinson 75a2e1caff refactor: results can just contain aggregates 2021-01-30 10:02:03 +00:00
Edd Robinson 07ac73a9d6
Merge pull request #710 from influxdata/er/feat/read_buffer/concurrent
feat: Read Buffer safe for concurrent access
2021-01-29 22:19:17 +00:00
Edd Robinson a71d3eea86 refactor: address PR feedback 2021-01-29 22:01:51 +00:00
Edd Robinson 3bb58fe971 refactor: tidy up commented code 2021-01-29 22:01:51 +00:00
Edd Robinson 46f20df089 refactor: change Rc -> Arc 2021-01-29 22:01:51 +00:00
Edd Robinson ec10c81041 refactor: simplify chunk locking implementation 2021-01-29 22:01:51 +00:00
Edd Robinson 6b6c1476f6 refactor: implement table meta-data rebuilding 2021-01-29 22:01:51 +00:00
Edd Robinson 13cbc12298 feat: make database support concurrent access 2021-01-29 22:01:51 +00:00
Edd Robinson bc08c6b404 feat: support concurrent access to Chunk 2021-01-29 22:01:51 +00:00
Edd Robinson 30b90943bc feat: make Table concurrent-safe 2021-01-29 22:01:51 +00:00
Edd Robinson 050185ad92 refactor: ensure meta updated when rowgroup add/removed 2021-01-29 22:01:51 +00:00
Edd Robinson 338bbb9b55 refactor: materialise rb for read_aggregate at table 2021-01-29 22:01:51 +00:00
Edd Robinson e3afab12a7 refactor: rb from table read_filter 2021-01-29 22:01:51 +00:00
Edd Robinson 9d3c623a14 refactor: baseline Rc 2021-01-29 22:01:51 +00:00
Andrew Lamb f3bd8bd0e3
chore: update deps (tokio 1.0 and ecosystem) (#707)
* chore: Update arrow + tokio deps

* chore: Use bleeding edge azure

* chore: Update aws + other deps

* fix: fmt

* fix: Switch to in-house version of routerify

* fix: Upgrade to hyper 0.14

The hyper::error module is now private; hyper::Error is the public
re-export

* fix: Upgrade cloud storage to get tokio upgrade

* fix: Upgrade open_telemetry

* fix: Do not call `panic::set_hook` during another panic

Doing so leads to a double panic which aborts the process.

* fix: new h2 error who dis

Co-authored-by: Carol (Nichols || Goulding) <carol.nichols@integer32.com>
Co-authored-by: Jake Goulding <jake.goulding@integer32.com>
2021-01-29 16:11:55 -05:00
Andrew Lamb 2282a68e65
refactor: Move selection to the data_types crate and remove redundant implemenation (#704) 2021-01-29 13:35:07 -05:00
Carol (Nichols || Goulding) 81494e351e
Merge pull request #706 from influxdata/cn/end-to-end-refactor 2021-01-29 08:50:36 -05:00
Carol (Nichols || Goulding) deaeff0173 refactor: Extract smaller functions for grpc tests
This makes it easier to see what's being tested and zero in on what
you're interested in rather than needing to scroll and scan through a
lot of code in the main test.
2021-01-29 08:34:28 -05:00
Carol (Nichols || Goulding) 8ade506a0c refactor: Extract independent test helper functions to separate files 2021-01-29 08:34:28 -05:00
Carol (Nichols || Goulding) 41e0fff039 refactor: Extract gRPC read group and read window agg tests because they manage their own data 2021-01-29 08:34:28 -05:00
Carol (Nichols || Goulding) cfe18e3e61 refactor: Extract a method to create gRPC ReadSource to the Scenario 2021-01-29 08:34:28 -05:00
Carol (Nichols || Goulding) bfe779d7b8 refactor: Incorporate ns_since_epoch into the Scenario
I want this to be an implementation detail of the tests, not something
you have to worry about when wondering what this test is hoping to test.
2021-01-29 08:34:28 -05:00
Carol (Nichols || Goulding) 05e79ed343 refactor: Make getter methods on Scenario for dependent values 2021-01-29 08:34:28 -05:00
Carol (Nichols || Goulding) 6ba27152e3 refactor: Change Scenario to use the builder pattern 2021-01-29 08:34:28 -05:00
Carol (Nichols || Goulding) 352b7ec829 refactor: Extract a Scenario struct to group related setup together 2021-01-29 08:34:28 -05:00
Carol (Nichols || Goulding) daa8b7ef0a refactor: Extract more helper functions 2021-01-29 08:34:28 -05:00
Carol (Nichols || Goulding) c4be42324b test: Remove code testing unused disk-based WAL restoration
Restarting the test server and testing for WAL restoration on startup
(and possible error scenarios) was disabled in 70ff426, which also noted
that significant refactoring will likely be needed when loading info on
startup is re-enabled. With that in mind, remove this code creating an
invalid dir in the WAL dir.
2021-01-29 08:34:28 -05:00
Carol (Nichols || Goulding) 15389d5665 refactor: Improve the names of client and client2 2021-01-29 08:34:28 -05:00
Carol (Nichols || Goulding) 422cbdb3b2 refactor: Move some helper functions after the main test 2021-01-29 08:34:28 -05:00
Carol (Nichols || Goulding) 0d174a2e1f refactor: Use data_types fn to create database name 2021-01-29 08:34:28 -05:00
Andrew Lamb 047f19fe41
feat: Implement schema merging (#703)
* feat: implement schema merging

* docs: Update data_types/src/schema.rs

Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>

* test: add test for merging schemas with no measurement names

* test: add test for merging and different measurements

* docs: clarify comments about merging

* fix: fmt

* fix: update comments to retrigger semantic pull request

Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
2021-01-28 13:15:47 -05:00
Andrew Lamb 16f2fbcb6d
docs: Improve style guide about using Snafu (#695) 2021-01-27 14:57:59 -05:00
Andrew Lamb efb1e0f8ae
feat: Add selection interface to mutable buffer and query interface (#700)
* feat: Add selection interface to mutable buffer and query interface

* docs: Update mutable_buffer/src/table.rs

* refactor: rename for consistency

* refactor: use map and filter_map  rather than fold
2021-01-27 14:31:10 -05:00
Andrew Lamb 0753d1603d
fix: influx_fields are nullable (#702) 2021-01-27 11:47:08 -05:00
Andrew Lamb df1367bcd1
feat: gRPC measurement_names support for read buffer (#697)
* test: Add tests for reading data from read buffer in different scenarios

* feat: gRPC measurement_names support for read buffer
2021-01-26 13:15:56 -05:00
Edd Robinson da4f8f30bf
Merge pull request #698 from influxdata/er/feat/read_buffer/table-names-pred
feat: add predicate support to `table_names` in the Read Buffer
2021-01-26 17:23:28 +00:00
Edd Robinson c8ce27ce5e perf: add benchmark for table_names
This commit adds some benchmarks for `table_names` against the read
buffer's Database implementation. On my laptop these look like:

database_table_names_all_tables
                        time:   [2.2104 us 2.2242 us 2.2381 us]
Found 2 outliers among 100 measurements (2.00%)
  1 (1.00%) high mild
  1 (1.00%) high severe

database_table_names_meta_pred_no_match
                        time:   [1.8389 us 1.8488 us 1.8593 us]
Found 3 outliers among 100 measurements (3.00%)
  1 (1.00%) high mild
  2 (2.00%) high severe

database_table_names_single_pred_match
                        time:   [5.5457 us 5.5694 us 5.5919 us]
Found 5 outliers among 100 measurements (5.00%)
  3 (3.00%) high mild
  2 (2.00%) high severe

database_table_names_multi_pred_match
                        time:   [478.85 us 480.32 us 481.83 us]
Found 4 outliers among 100 measurements (4.00%)
  2 (2.00%) high mild
  2 (2.00%) high severe

database_table_names_multi_pred_match_multi_tables
                        time:   [476.47 us 478.93 us 482.25 us]
Found 11 outliers among 100 measurements (11.00%)
  4 (4.00%) high mild
  7 (7.00%) high severe
2021-01-26 17:00:53 +00:00
Edd Robinson 42d629ac32 feat: wire up predicate support to external API 2021-01-26 17:00:53 +00:00
Edd Robinson c89a569e03 feat: add per-chunk pred support in table_names 2021-01-26 17:00:53 +00:00