Commit Graph

1633 Commits (efb1e0f8ae3ad59f83db088cd6585934649772fa)

Author SHA1 Message Date
Andrew Lamb efb1e0f8ae
feat: Add selection interface to mutable buffer and query interface (#700)
* feat: Add selection interface to mutable buffer and query interface

* docs: Update mutable_buffer/src/table.rs

* refactor: rename for consistency

* refactor: use map and filter_map  rather than fold
2021-01-27 14:31:10 -05:00
Andrew Lamb 0753d1603d
fix: influx_fields are nullable (#702) 2021-01-27 11:47:08 -05:00
Andrew Lamb df1367bcd1
feat: gRPC measurement_names support for read buffer (#697)
* test: Add tests for reading data from read buffer in different scenarios

* feat: gRPC measurement_names support for read buffer
2021-01-26 13:15:56 -05:00
Edd Robinson da4f8f30bf
Merge pull request #698 from influxdata/er/feat/read_buffer/table-names-pred
feat: add predicate support to `table_names` in the Read Buffer
2021-01-26 17:23:28 +00:00
Edd Robinson c8ce27ce5e perf: add benchmark for table_names
This commit adds some benchmarks for `table_names` against the read
buffer's Database implementation. On my laptop these look like:

database_table_names_all_tables
                        time:   [2.2104 us 2.2242 us 2.2381 us]
Found 2 outliers among 100 measurements (2.00%)
  1 (1.00%) high mild
  1 (1.00%) high severe

database_table_names_meta_pred_no_match
                        time:   [1.8389 us 1.8488 us 1.8593 us]
Found 3 outliers among 100 measurements (3.00%)
  1 (1.00%) high mild
  2 (2.00%) high severe

database_table_names_single_pred_match
                        time:   [5.5457 us 5.5694 us 5.5919 us]
Found 5 outliers among 100 measurements (5.00%)
  3 (3.00%) high mild
  2 (2.00%) high severe

database_table_names_multi_pred_match
                        time:   [478.85 us 480.32 us 481.83 us]
Found 4 outliers among 100 measurements (4.00%)
  2 (2.00%) high mild
  2 (2.00%) high severe

database_table_names_multi_pred_match_multi_tables
                        time:   [476.47 us 478.93 us 482.25 us]
Found 11 outliers among 100 measurements (11.00%)
  4 (4.00%) high mild
  7 (7.00%) high severe
2021-01-26 17:00:53 +00:00
Edd Robinson 42d629ac32 feat: wire up predicate support to external API 2021-01-26 17:00:53 +00:00
Edd Robinson c89a569e03 feat: add per-chunk pred support in table_names 2021-01-26 17:00:53 +00:00
Edd Robinson 8a23e22957 feat: determine if row group satisfies predicate 2021-01-26 17:00:53 +00:00
Andrew Lamb 8308fad188
chore: update arrow deps again (#699) 2021-01-26 07:55:30 -05:00
Andrew Lamb 504ca67532
test: revamp rpc query testing so it works in multiple chunk scenarios (#696)
* test: revamp testing so it works in multiple scenarios, fix bug found by same

* fix: Update docs in server/src/db.rs

Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>

* refactor: use tsp rather than different functions

Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
2021-01-25 16:34:19 -05:00
Carol (Nichols || Goulding) b7663970f3
Merge pull request #689 from influxdata/pd-load-database-configs 2021-01-25 10:02:02 -05:00
Carol (Nichols || Goulding) f7fdadc3b3
Merge branch 'main' into pd-load-database-configs 2021-01-25 09:50:16 -05:00
Andrew Lamb c3b0371c84
feat: Initial RPC Query Frontend (#692)
* feat: Initial RPC Query Frontend

* docs: s/immutable buffer/mutable buffer

* docs: Correct type in docstring
2021-01-25 08:33:39 -05:00
Edd Robinson 69f17ec310
Merge pull request #693 from influxdata/er/feat/read_buffer/read_aggregate_merge
feat: Read Buffer read_aggregate support
2021-01-25 11:46:48 +00:00
Edd Robinson 3da9b73464 refactor: add assertion 2021-01-25 11:26:15 +00:00
Edd Robinson 5fe5ed0569 test: more aggregate coverage 2021-01-25 11:21:20 +00:00
Edd Robinson c60cfbd2bb refactor: update read_buffer/src/lib.rs
Co-authored-by: Andrew Lamb <alamb@influxdata.com>
2021-01-25 11:21:20 +00:00
Edd Robinson a4bde5c252 refactor: error on unsupported aggregates 2021-01-25 11:21:20 +00:00
Edd Robinson d848526124 refactor: tidy up 2021-01-25 11:21:20 +00:00
Edd Robinson 9270874760 feat: wire up read_aggregate to external API 2021-01-25 11:21:20 +00:00
Edd Robinson e6b8d0e072 feat: add support for converting to record batch: 2021-01-25 11:21:20 +00:00
Edd Robinson 09ec6b78d3 feat: add ability to merge ReadAggregateResults 2021-01-25 11:21:20 +00:00
Dom 0f3f9abc31
Merge pull request #694 from influxdata/alamb/arrow_deps_again
chore: Bump arrow deps
2021-01-25 10:59:35 +00:00
Andrew Lamb 9b6fbae7f5 chore: Bump arrow deps 2021-01-23 08:09:46 -05:00
Paul Dix b3454daf70 chore: pr fixup 2021-01-22 16:06:10 -05:00
Paul Dix 45d25fcbcd feat: add loading of database rules on startup
This adds functionality to the server to load database rules on startup. Follow on work will update the rules to store additional data (the catalog) and ensure that updates to the catalog can occur as outlined in #651. This work also updated the configuration to not require a database directory so the server can run entirely in memory. I needed this to get the end-to-end test passing since the file object store API doesn't yet have the functionality needed. I've logged #688 to track adding that in.
2021-01-22 16:06:10 -05:00
Andrew Lamb 124d603745
feat: change default log level to INFO (#691) 2021-01-22 07:38:23 -05:00
Dom eeccea43b9
Merge pull request #690 from influxdata/dom/ci-deploy/perf
ci: trigger image promotion for deployment
2021-01-22 11:25:12 +00:00
Dom 348aaab6dd ci: trigger image promotion for deployment
Once an image is built and ready, exec the binary defined in the env var
IMAGE_PROMOTION_COMMAND to promote it (if set).
2021-01-22 10:35:10 +00:00
Andrew Lamb 7cb512ef67
refactor: Put tests in same module that defines struct (#685) 2021-01-21 19:33:15 -05:00
Andrew Lamb 75b0a62fa5
refactor: Delete remove dead code (#686) 2021-01-21 19:20:39 -05:00
Andrew Lamb 96fbf3aa10
refactor: Remove unused error (#687) 2021-01-21 19:06:45 -05:00
Andrew Lamb a967e2f1dd
fix: disallow control characters in Database names (#684) 2021-01-21 17:55:55 -05:00
Dom 0cfd543db2
Merge pull request #678 from influxdata/alamb/underscore_in_bucket_names_2
fix: allow arbitrary characters in org/bucket names
2021-01-21 21:14:32 +00:00
Andrew Lamb c50f9b1baf
Merge branch 'main' into alamb/underscore_in_bucket_names_2 2021-01-21 15:53:27 -05:00
Andrew Lamb 747b96d801
chore: Upgrade arrow dependencies, reduce duplication with upstream (#676) 2021-01-21 08:58:11 -05:00
Dom 0dae62d023
Merge pull request #682 from influxdata/fix/add-images-toplevel-to-deploy-tags
fix: add Images toplevel object to JSON in get-deploy-tags.sh
2021-01-21 13:17:20 +00:00
Dom bea906e267
Merge branch 'main' into fix/add-images-toplevel-to-deploy-tags 2021-01-21 13:03:04 +00:00
Carol (Nichols || Goulding) 6bb91653c1
refactor: Some tiny cleanups (#680)
* refactor: Remove import of unimplemented macro that's in the prelude

* refactor: Remove allowing of dead code that isn't dead anymore

Co-authored-by: Andrew Lamb <alamb@influxdata.com>
2021-01-21 07:27:17 -05:00
Wojciech Kocjan 2dc5eec3c7 fix: add Images toplevel object to JSON in get-deploy-tags.sh 2021-01-21 13:22:29 +01:00
Dom 85840cd6f8
Merge pull request #681 from influxdata/chore/add-published-at-to-deploy-tags
chore: add PublishedAt to get-deploy-tags.sh script
2021-01-21 10:55:12 +00:00
Wojciech Kocjan 316b8bfec7 chore: add PublishedAt to get-deploy-tags.sh script 2021-01-21 09:25:49 +01:00
Andrew Lamb a6b9ff9c91 fix: allow arbitrary characters in org/bucket names 2021-01-20 17:58:15 -05:00
Andrew Lamb 4b5204cc8f
fix: Update end-to-end.rs to have moe specific error messages (#674) 2021-01-19 15:54:13 -05:00
Andrew Lamb 7969808f09
feat: Chunk Migration APIs and query data in the read buffer via SQL (#668)
* feat: Chunk Migration APIs and query data in the read buffer via SQL

* fix: Make code more consistent

* fix: fmt / clippy

* chore: Apply suggestions from code review

Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>

* refactor: Remove unecessary Result and make chunks() infallable

* chore: Apply more suggestions from code review

Co-authored-by: Edd Robinson <me@edd.io>
Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>

Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
Co-authored-by: Edd Robinson <me@edd.io>
2021-01-19 13:28:26 -05:00
Edd Robinson 5f30d4b2fc
Merge pull request #673 from influxdata/er/refactor/read_buffer/read_filter_schema
refactor: use schema/metadata type in `read_filter`
2021-01-19 15:51:53 +00:00
Edd Robinson 221ed86853 refactor: apply suggestions from code review
Co-authored-by: Andrew Lamb <alamb@influxdata.com>
2021-01-19 15:33:44 +00:00
Edd Robinson 57365b082c refactor: clean up filter_map 2021-01-19 15:33:44 +00:00
Edd Robinson dbdd885e58 refactor: follow snafu style guide 2021-01-19 15:33:44 +00:00
Edd Robinson 5f6335573b fix: ensure table missing handled 2021-01-19 15:33:44 +00:00