5d342d7779
feat: associate tracker with lifecycle action ( #1099 ) ( #1556 )
...
* feat: associate tracker with lifecycle action (#1099 )
* chore: docs
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-05-27 10:47:35 +00:00
4fcc04e6c9
chore: enable arrow prettyprint feature ( #1566 )
2021-05-27 10:28:14 +00:00
c2fd85209c
feat: wait for task shutdown on DedicatedExecutor ( #1537 )
2021-05-25 11:33:55 +00:00
14ba25f86d
chore: Update datafusion and use released version of arrow crates ( #1546 )
...
* chore: Update datafusion and use released version of arrow crate
* fix: Update for change in API
2021-05-24 15:37:22 +00:00
0563005aac
chore: remove leftover comments
2021-05-21 17:01:49 -04:00
f113abacb5
feat: more unit & e2e tests plus cleanup and addressing review comments of Andrew and Edd
2021-05-21 16:48:43 -04:00
e44a3a87db
feat: fnow predicate is actuallu pushed down to RUB but there are bugs and not working yet
2021-05-20 16:56:15 -04:00
51de37e752
chore: run fmt
2021-05-19 15:28:44 -04:00
11561111d5
chore: merge main to branch
2021-05-19 15:11:15 -04:00
1f13842550
chore: modify comments
2021-05-19 14:49:48 -04:00
087d61f229
feat: Part 1 of predicate push down - Send predicates to MUB, RUB, and Parquet File. Note that MUB has not handled predicates yet
2021-05-19 13:59:51 -04:00
7e223780f3
feat: Implement Display for query::predicate to improve debug printing of plans ( #1519 )
...
* feat: Implement Display for query::predicate to improve debug printing of plans
* fix: clippy
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-05-19 12:38:34 +00:00
0680a5167f
chore: Improve DataFusion plan logging ( #1508 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-05-18 11:08:06 +00:00
07db4932ee
refactor: rename data_types/src/chunk.rs -> data_types/src/chunk_metadata.rs ( #1500 )
2021-05-15 10:18:01 +00:00
8ccc359cab
refactor: address PR feedback
2021-05-07 13:48:44 +01:00
4c4bd2f164
refactor: update query/src/func/regex.rs
2021-05-07 13:44:51 +01:00
4cc7a99854
refactor: include not match in support check
2021-05-07 13:44:51 +01:00
beee3115f4
feat: expose regex =\~ and to gRPC API
2021-05-07 13:44:51 +01:00
eae3fec571
feat: wire up regex UDF as predicate filter expr
2021-05-07 13:44:51 +01:00
3fc2c9fc04
feat: add DataFusion regex match operator
...
This commit adds a new custom UDF to IOx that provide a regex operator to Datafusion plans.
Effectively it allows predicates to contain regex operators that are applied as filters, only allowing rows that satisfy the regex to be returned.
I did not use the Arrow regex kernel for this work because that does not return a boolean array indicating which rows matched a regex, but instead returns a new string array of results. This doesn't work well with DF's approach to filtering.
2021-05-07 13:44:51 +01:00
febc1538ff
chore: Update Rust version ( #1445 )
...
* chore: Update Rust version
* refactor: Make struct constructor field orderings consistent
Sometimes I changed the struct definition, sometimes changed the struct
construction instance, depending on consistency with code around each
(other similar structs, function argument orders, etc)
More info: https://rust-lang.github.io/rust-clippy/master/index.html#inconsistent_struct_constructor
* refactor: Use flatten where appropriate
One instance is a false positive with a clippy bug.
More info:
- https://rust-lang.github.io/rust-clippy/master/index.html#filter_map_identity
- https://rust-lang.github.io/rust-clippy/master/index.html#manual_flatten
* refactor: Use Option map instead of match
More info: https://rust-lang.github.io/rust-clippy/master/index.html#manual_map
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-05-06 22:07:10 +00:00
44de42906f
refactor: use Arc<str> instead of Arc<String> ( #1442 )
2021-05-06 17:05:08 +00:00
411cf134e9
refactor: explode arrow_deps ( #1425 )
...
* refactor: explode arrow_deps
* chore: workaround doctest bug
2021-05-05 16:59:12 +00:00
2f789485e6
refactor: fix spelling
2021-05-05 11:06:04 +01:00
3b7c5ac350
fix(storage rpc): do not send back tags with empty values ( #1403 )
2021-05-04 10:35:24 +00:00
40b9b09cdc
refactor: rename assert_table_eq to assert_batches_eq ( #1368 )
2021-04-30 10:51:08 +00:00
eb8d91cf1c
refactor: remove additional uses of RecordBatch::try_new ( #1378 )
...
* refactor: remove additional uses of RecordBatch::try_new
* fix: fix accidental change
* fix: clippy
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-04-30 10:24:47 +00:00
13fbf2e68d
refactor: plumb registry to gRPC server
2021-04-29 14:00:05 +01:00
4acbdcf1c9
refactor: address PR feedback
2021-04-28 16:11:57 +00:00
a9ef604ef6
perf: avoid using channels for query execution
...
Pre-sized channels get full when the results to send over them are larger than the capacities. This causes significant runtime overhead and slows down query performance.
This commit removes the intermediate channels. The potential downside to this approach is there may be more buffering which could increase memory usage during query and also block a thread for longer periods of time.
2021-04-28 16:11:57 +00:00
7ca1da3fcd
feat: pushdown table and partition key predicates to catalog ( #736 ) ( #1327 )
...
* feat: catalog predicate pushdown (#736 )
* chore: fix lints
* chore: review comments
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-04-27 15:31:47 +00:00
91bccdfca3
ci: pass `--document-private-items` to `cargo doc`
2021-04-27 15:42:07 +02:00
eddc9319ff
docs: deny broken intradoc links
2021-04-27 13:22:28 +02:00
20117de078
feat: string dictionary encoding ( #1220 ) ( #1262 )
...
* feat: string dictionary encoding (#1220 )
* chore: review comments
* chore: fix lint
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-04-27 09:36:58 +00:00
a322d05838
refactor: rust fmt
2021-04-20 17:30:50 +00:00
554b3b4662
refactor: satisfy new clippy lints
2021-04-20 17:30:50 +00:00
51041ba2d9
fix: Prefer implementing From over Into
2021-04-19 08:48:11 -04:00
757933afc4
fix: use Self when possible
2021-04-19 08:48:11 -04:00
f136931225
fix: Inconsistent ordering lints
2021-04-19 08:48:11 -04:00
3e87ce5232
fix: Make this trait and methods more idiomatically named
...
"into" usually takes ownership and does a conversion; "as" takes
references and provides a different view.
2021-04-19 08:45:34 -04:00
529c99c93f
fix: don't clone arrays to make TimestampNanosecondArrays ( #1241 )
...
* fix: avoid clone
* fix: remove another clone
2021-04-16 18:40:22 +00:00
e226b5a820
feat: Use TimestampNanosecondArray for timestamps in IOx ( #1230 )
...
* refactor: Create Arrow arrays using iterators
* feat: use Timestamp64(TimeUnit::Nanosecond) for timestamps
* feat: add support for timestamp array
* fix: update more tests
* fix: remove unecessary code
Co-authored-by: Edd Robinson <me@edd.io>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-04-16 15:55:33 +00:00
f092294da3
fix: Use MAX (window end) for timestamps in read group ( #1228 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-04-16 10:51:38 +00:00
5aeeccb97c
feat: Run query plans on the database wide executor as well ( #1210 )
...
* feat: route all query planning through executor
* fix: Rename JoinError -> TaskJoinError and make message clearer
* fix: remove dangling comment
* fix: remove confusing comments
2021-04-15 11:57:20 +00:00
59ca090aef
feat: Use single db-wide executor for running queries ( #1198 )
...
* refactor: plumb executor into all Db instances
* refactor: Route all query executions through worker pool
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-04-14 16:46:02 +00:00
8f1bf8a960
fix: Remove mutex acquisition in impl `std::fmt::Debug` for DedicatedExecutor ( #1205 )
2021-04-14 12:09:40 +00:00
f5f768d750
feat: Add a dedicated threadpool for running queries ( #1191 )
...
* feat: use a dedicated tokio threadpool for running queries
* feat: plumb number of executor threads through to command line
thread through command line
* fix: Logical merge conflict
* fix: another logical conflict
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-04-14 10:48:09 +00:00
150ed4e1d9
refactor: Remove async from `InfluxRPCPlanner` ( #1200 )
...
* refactor: Remove async from InfluxRPCPlanner
* fix: make it compile
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-04-13 22:17:19 +00:00
7e28f8ef66
feat: Implement Entry writing to Db
...
This removes the old ReplicatedWrite structure and implements the writing of an Entry to the Db. I also call out in `server/lib.rs` and in the `Db` where sharding and replication might happen.
I've also added helpers in various places to write line protocol to chunks, tables, and databases. That enabled removing a good amount of code from the test helpers crate.
2021-04-13 12:52:14 +00:00
1997324344
feat: mutable buffer snapshotting ( #1179 )
...
* feat: mutable buffer snapshotting
* chore: review feedback
2021-04-13 12:14:54 +00:00
Raphael Taylor-Davies
Andrew Lamb
Nga Tran
Edd Robinson
Carol (Nichols || Goulding)
Marco Neumann
Carol (Nichols || Goulding)
Paul Dix