Commit Graph

20 Commits (e69b2c40c6747868af5eba2c4229e159593fed18)

Author SHA1 Message Date
Andrew Lamb fd8855fc98
chore: use upstream FlightSQL metadata implementation (#7949)
* chore: use upstream FlightSQL metadata implementation

* fix: update doc strings

* fix: Remove incorrect comment about table_type
2023-06-08 18:39:11 +00:00
Dom Dwyer bc95c70144
refactor(lints): add missing lints to service_common
Adds the standard lints to service_common and fixes any lint failures.

Note this doesn't include the normal "document things" lint, because
there's a load of missing docs
2023-05-23 14:55:41 +02:00
Andrew Lamb 96c2094302
refactor(iox_query): extract influxrpc planner to its own crate (#7241)
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-03-17 18:48:55 +00:00
Stuart Carnie fe48a685ec
refactor: Move InfluxQL behaviour from iox_query to new crate (#7156)
* refactor: Break unnecessary dependencies from `iox_query` crate

In the process, the test code has been simplified.

* refactor: Move InfluxQL plan module to iox_query_influxql crate

* refactor: Move remaining behaviour from iox_query to iox_query_influxql

* chore: rustfmt 🙄

I was under the impression `clippy` would catch formatting
2023-03-08 22:29:20 +00:00
Stuart Carnie 6fb93a7679
refactor: Make InfluxQL planning sync (#7038)
* refactor: Move statement parsing to separate fn

* refactor: Remove async from `InfluxQLToLogicalPlan`

Closes #6607

* chore: Remove async functions and tokio::test

* chore: Remove redundant attribute

* chore: Feedback, switch to dynamic dispatch vs generic implementation
2023-02-22 19:33:49 +00:00
Andrew Lamb fa15a48c31
refactor: use types rather than `Any` in `flightsql` interface (#6705)
* refactor: use types rather than Any in flightsql interface

* fix: Update service_grpc_flight/src/lib.rs

Co-authored-by: Raphael Taylor-Davies <1781103+tustvold@users.noreply.github.com>

* chore: review feedback

* fix: fmt

* fix: error conversion

---------

Co-authored-by: Raphael Taylor-Davies <1781103+tustvold@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-01-31 14:45:46 +00:00
Andrew Lamb 0c55a0f257
feat: Implement basic prepared statement support in IOx (#6667)
* feat: allow override of flightsql namespace

* feat: Implement DoAction endpoint

* refactor: Remove try_unpack

* fix: remove unused code / more clone
2023-01-25 12:00:43 +00:00
Andrew Lamb c3bc61f10e
refactor: Move `flightsql` code into its own module, add docs and tests (#6640)
* refactor: Move `flightsql`  code into its own module

* fix: get schema from LogicalPlan

* refactor: use arrow_flight::sql::Any instead of prost_types::any

* fix: cleanup docs and avoid as_ref

* fix: Use Bytes

* fix: use Any::pack

* fix: doclink
2023-01-24 18:24:32 +00:00
Stuart Carnie 2306c383f3
feat: Introduce InfluxQL to Flight (#6166)
* feat: Introduce InfluxQL to Flight

All InfluxQL queries will fail with an error

* chore: Temper protobuf lint

* chore: Finalize flight.proto changes; fix tests

* chore: Add tests for InfluxQL planner

* chore: Update docs

* chore: Update docs

* chore: Rename back to original

* chore: Use .into() rather than cast

* chore: Use function rather than field

* chore: Improved InfluxQL planner name

* chore: Restore `impl Into<String>` argument

* chore: Add a comment that Go clients are unable to execute InfluxQL

* chore: Add a test for the `--lang` argument and InfluxQL
2022-11-23 00:33:49 +00:00
Carol (Nichols || Goulding) 0657ad9600
fix: Rename QueryDatabase to QueryNamespace 2022-11-11 16:14:12 -05:00
Marco Neumann c4c83e0840
fix: query error propagation (#5801)
- treat OOM protection as "resource exhausted"
- use `DataFusionError` in more places instead of opaque `Box<dyn Error>`
- improve conversion from/into `DataFusionError` to preserve more
  semantics

Overall, this improves our error handling. DF can now return errors like
"resource exhausted" and gRPC should now automatically generate a
sensible status code for it.

Fixes #5799.
2022-10-06 08:54:01 +00:00
Marco Neumann 7e00426d49
refactor: concurrent table scan for "tag values" (#5671)
Ref #5668.
2022-09-19 14:11:51 +00:00
Marco Neumann 274bd80ecd
refactor: concurrent table scan for "tag keys" (#5670)
* refactor: concurrent table scan for "tag keys"

Ref #5668.

* feat: add table name to context metadata
2022-09-19 13:27:18 +00:00
Marco Neumann ef09573255
refactor: concurrent table scan in "field columns" (#5651)
* refactor: concurrent table scan in "field columns"

Similar to #5647 and #5649.

* docs: improve

Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>

Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-09-19 10:50:25 +00:00
Marco Neumann e346433914
refactor: concurrent table scan for "table names" (#5649)
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-09-15 15:39:00 +00:00
Marco Neumann 159250e776
refactor: concurrent table planning in InfluxRPC (#5647)
* refactor: concurrent table planning in InfluxRPC

Some InfluxRPC can scan multiple tables. Prior to this PR we were always
scanning the tables in sequence, adding up potential latencies (catalog,
ingester, object store). There is no reason we need to do this,
"ordinary" SQL queries would not serialize this way either.

So let's scan tables concurrently. This add concurrency to:

- read filter
- read group
- read window aggregate

There are other query types that could benefit from a similar treatment.
They will be changed in a follow-up.

* docs: improve

Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>

* test: explain `Send` assertion

* refactor: change `CONCURRENT_TABLE_JOBS` to 10

Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
2022-09-15 13:55:22 +00:00
Marco Neumann 9c2b6cd96c
fix: always pass proper context to `InfluxRpcPlanner` (#5144)
There were some instances were we forgot to pass context (and therefore
tracing) information to `InfluxRpcPlanner`. This removes the `Default`
implementation requires to always pass a context when creating
`InfluxRpcPlanner` to prevent this type of bug.

Ref #5129.

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-07-18 14:45:22 +00:00
Marco Neumann 52346642a0
ci: fix cargo deny (#4629)
* ci: fix cargo deny

* chore: downgrade `socket2`, version 0.4.5 was yanked

* chore: rename `query` to `iox_query`

`query` is already taken on crates.io and yanked and I am getting tired
of working around that.
2022-05-18 09:38:35 +00:00
Andrew Lamb 9e91af4501
refactor: Move IOx UDfs into a Function Registry (1/3) (#4428)
* refactor: Move all UDF implementations to query_function crate

* refactor: Move regex udf to query_functions

* refactor: Move functions out of query

* fix: lints, imports

* chore: Run cargo hakari tasks

* fix: clipy + benches

* fix: reduce borrowing and fix clippy

* fix: moar clippy

Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-04-26 17:30:27 +00:00
Andrew Lamb 29b89aaca7
refactor: extract influxrpc, flight and testing gRPC out of influxdb_ioxd (#4106)
* refactor: extract grpc service implementations out of influxdb_ioxd

* chore: Run cargo hakari tasks

* refactor: rename server_common to service_common

* refactor: rename server_grpc_influxrpc to service_grpc_influxrpc

* refactor: rename server_grpc_flight to service_grpc_flight

* refactor: rename server_grpc_testing to service_grpc_testing

* fix: Cargo.toml

Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
2022-03-23 20:14:45 +00:00