Commit Graph

47 Commits (d56217a8f64a61cdaf56200bfbe847789dda0268)

Author SHA1 Message Date
dependabot[bot] 7f20b0faa0
chore(deps): Bump bytes from 1.4.0 to 1.5.0 (#8692)
Bumps [bytes](https://github.com/tokio-rs/bytes) from 1.4.0 to 1.5.0.
- [Release notes](https://github.com/tokio-rs/bytes/releases)
- [Changelog](https://github.com/tokio-rs/bytes/blob/master/CHANGELOG.md)
- [Commits](https://github.com/tokio-rs/bytes/compare/v1.4.0...v1.5.0)

---
updated-dependencies:
- dependency-name: bytes
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-09-08 12:17:12 +00:00
Carol (Nichols || Goulding) 12b8095c46
feat: Upgrade to Rust 1.72.0 (#8589)
* feat: Upgrade to Rust 1.72.0

* fix: Allow a warning about an error we're intentionally creating

This is a test for an error. This lint warns that this code will cause
an error. Thanks lint, that's what we wanted!

* chore: rustfmt 1.72

* fix: Remove unnecessary hashes in raw string literals

Thanks Clippy!
https://rust-lang.github.io/rust-clippy/master/index.html#/needless_raw_string_hashes

Note that there are a number of false negatives with this lint; see
https://github.com/rust-lang/rust-clippy/issues/11420

* fix: Remove unnecessary explicit iteration

Looks like clippy::explicit_iter_loop was improved.
https://rust-lang.github.io/rust-clippy/master/index.html#/explicit_iter_loop

* fix: Allow clippy::manual_try_fold in a few places

Some of these might not be possible to rewrite with try_fold, or at
least not trivially. I don't feel confident enough to change these, in
any case. I think the lint is good to have on for future code though, so
that new code can be written with try_fold.

* fix: Remove useless creation of vectors when an array will do

Mostly in tests. Also fix some long lines.

Thanks Clippy!
https://rust-lang.github.io/rust-clippy/master/index.html#/useless_vec

* fix: Allow a single range in a vec init, which is actually what we want

Looks like Clippy's trying to catch a common mistake here, but for realz
we actually want `Vec<Range<usize>>` not `Vec<usize>`

https://rust-lang.github.io/rust-clippy/master/index.html#/single_range_in_vec_init

* fix: Remove a useless conversion

This looks like removing explicit iteration, but it's actually caught by
useless_conversion.

https://rust-lang.github.io/rust-clippy/master/index.html#/useless_conversion

* fix: Remove redundant pattern matching

Thanks Clippy!
https://rust-lang.github.io/rust-clippy/master/index.html#/redundant_pat

* fix: Allow an unwrap on a literal None in a test

This matches with the other tests better, and also when I tried to
remove the `unwrap_or_default` it changed the JSON sent from something
with an empty value to `null`, so I think the `or_default` part is
actually changing from one `None` to another `None`.

https://rust-lang.github.io/rust-clippy/master/index.html#/unnecessary_literal_unwrap
2023-08-29 05:57:38 +00:00
Andrew Lamb d26850390f
refactor(flightsql): Use upstream `SqlInfoData` implementation (#8453)
* refactor(flightsql): Use upstream implementation

* fix: naming

* fix: better API
2023-08-09 17:06:43 +00:00
Andrew Lamb 25ab230898
refactor(flightsql): Use upstream implementation of XdbcTypeInfo builder, support filter by datatype (#8455)
* refactor(flightsql): Use upstream implementation of XdbyTypeInfo builder

* fix: Update test

---------

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-08-08 20:16:39 +00:00
Andrew Lamb fd8855fc98
chore: use upstream FlightSQL metadata implementation (#7949)
* chore: use upstream FlightSQL metadata implementation

* fix: update doc strings

* fix: Remove incorrect comment about table_type
2023-06-08 18:39:11 +00:00
Andrew Lamb 17c0d837b3
chore: Update DataFusion, arrow, object_store pins (#7942)
* chore: Update DataFusion, arrow, object_store pins

* chore: Update for hakari

* chore: Update for new APIs

* fix: update test

---------

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-06-07 17:08:31 +00:00
Stuart Carnie 28166006a8
chore: clippy 2023-06-04 06:56:19 +10:00
Andrew Lamb a4789b0ad3
docs: Add ticket reference for other upstream implementations of flightsql (#7891)
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-05-30 13:33:17 +00:00
Andrew Lamb 9ceb3e117a
docs: Add upstream ticket link (#7881)
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-05-29 16:31:18 +00:00
Dom Dwyer 928a4d163e
build: remove unused dependencies from crates
This commit fixes loads of crates (47!) had unused dependencies, or
mis-configured dependencies (test deps as normal deps).

I added the "unused_crate_dependencies" to all crates to help prevent
this mess from growing again!

    https://doc.rust-lang.org/beta/nightly-rustc/rustc_lint_defs/builtin/static.UNUSED_CRATE_DEPENDENCIES.html

This has the minor downside of false-positives when specifying
dev-dependencies for test/bench binaries - these are files in /test or
/benches (not normal tests). This commit includes a workaround,
importing them in lib.rs (gated by a feature flag). I think the
trade-off of better dependency management is worth it!
2023-05-23 14:55:43 +02:00
Dom Dwyer e74efae118
refactor(lints): add missing lints to flightsql
Adds the standard lints to flightsql and fixes any lint failures.

Note this doesn't include the normal "document things" lint, because
there's a load of missing docs.
2023-05-23 14:55:36 +02:00
Andrew Lamb be6bcdef45
fix: do not panic on unimplemented (#7765) 2023-05-08 15:13:54 +00:00
Chunchun Ye a98952bff6
feat(flightsql): Support `CommandGetXdbcTypeInfo` metadata endpoint with actual data type info (#7696)
* chore: add expected xdbc type info value to jdbc test

* chore: add query skeleton to kick off plan_get_xdbc_type_info()`

* chore: implement a minimun version query for type info

* chore: rewrite `plan_get_xdbc_type_info` to use a static recrod batch

* chore: construct create_params as a string list

* chore: add create_params column in e2e test result

* chore: re-define create_params list items to be non-nullable

* chore: remove comment

* chore: refactor TYPE_INFO_RECORD_BATCH using XdbcTypeInfo struct and rewrite metadata for character types

chore: lint

chore: lint doc

chore: lint doc use automatic link

* chore: add unimplemented error msg

* chore: add `INTEGER`, `FLOAT`, `TIMESTAMP`, `INTERVAL` and  remove `CHAR`, `TEXT`, `STRING`

---------

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-05-04 17:22:57 +00:00
dependabot[bot] bdf7f316d7
chore(deps): Bump tokio from 1.27.0 to 1.28.0 (#7667)
* chore(deps): Bump tokio from 1.27.0 to 1.28.0

Bumps [tokio](https://github.com/tokio-rs/tokio) from 1.27.0 to 1.28.0.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.27.0...tokio-1.28.0)

---
updated-dependencies:
- dependency-name: tokio
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* chore: Run cargo hakari tasks

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: Dom <dom@itsallbroken.com>
2023-04-26 12:53:26 +00:00
Chunchun Ye eca86a0f06
feat(flightsql): Support `CommandGetXdbcTypeInfo` metadata endpoint with an empty RecordBatch (#7576)
* feat: WIP support CommandGetXdbcTypeInfo metadata endpoint with tests

* chore: update test case and add jdbc test

* chore: uncomment jdbc getCoumns test

* chore: lint

---------

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-04-25 21:06:27 +00:00
Marco Neumann 1185ced87a
chore: deny unknown and copyleft licenses (#7556)
I just don't wanna rip out parts of our software stack because
someone gets cold feet.

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-04-17 08:08:48 +00:00
Andrew Lamb f46d06d56f
chore: Update DataFusion + arrow ecosystem to 37 (#7544)
* chore: Update datafusion and arrow/parquet to 37, tonic to 0.9.1

* refactor: Update for FieldRef and other API changes

* fix: Update field size calculation

* fix: Use `NullBuffer` directly

* fix: remove outdated comment

* chore: Update test for tonic

* chore: Run cargo hakari tasks

* chore: cargo update

---------

Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-04-14 12:43:01 +00:00
Chunchun Ye 69da3c2495
feat(flightsql): Support `GetCrossReference` metadata endpoint with an empty RecordBatch (#7548)
* feat: support CommandGetCrossReference metadata endpoint with tests

* chore: create two tables in the test for GetCrossReference endpoint

---------

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-04-13 19:53:37 +00:00
Chunchun Ye 8bf47df621
feat(flightsql): Support `GetImportedKeys` metadata endpoint with an empty RecordBatch (#7546)
* feat: support CommandGetImportedKeys metadata endpoint with tests

* chore: remove comments that is no longer valid

---------

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-04-13 18:06:47 +00:00
Chunchun Ye e86e8de1a9
feat: support CommandGetExportedKeys metadata endpoint with tests (#7532)
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-04-13 13:27:09 +00:00
Andrew Lamb 20e9c91866
refactor: Use workspace dependencies for `tonic`, `tonic-build`, etc (#7515)
* refactor: Use workspace dependencies for `tonic`, `tonic-build`, etc

* chore: Run cargo hakari tasks

---------

Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-04-12 16:07:19 +00:00
Chunchun Ye 911d57c2d4
fix: replace empty schema with GET_PRIMARY_KEYS_SCHEMA (#7508)
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-04-11 21:40:52 +00:00
Chunchun Ye b131895fc2
feat(flightsql): Support `GetPrimaryKeys` metadata endpoint with an empty RecordBatch (#7496)
* feat: add CommandGetPrimaryKeys metadata endpoint and tests

* chore: update schema for the returned record batch

---------

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-04-11 14:13:44 +00:00
Andrew Lamb 555f2a67aa
refactor(flightsql): Build `GetDBSchemas` directly from DataFusion catalog (#7386)
* refactor: Build `GetDBSchemas` directly from DataFusion catalog

* fix: Apply suggestions from code review

---------

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-03-30 13:28:34 +00:00
dependabot[bot] 9cbcdc7672
chore(deps): Bump tokio from 1.26.0 to 1.27.0 (#7373)
Bumps [tokio](https://github.com/tokio-rs/tokio) from 1.26.0 to 1.27.0.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.26.0...tokio-1.27.0)

---
updated-dependencies:
- dependency-name: tokio
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-03-30 09:36:04 +00:00
Andrew Lamb 0d66b04f68
refactor(flightsql): Build `GetTables` directly from DataFusion catalog (#7325)
* refactor: Build GetTables response directly from SessionContext

* fix: fmt

---------

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-03-24 19:15:24 +00:00
Andrew Lamb 35865e1ab3
refactor(flightsql): Build `GetCatalogs` directly from DataFusion catalog (#7327)
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-03-24 18:34:56 +00:00
Andrew Lamb ce77d3bd74
refactor(flightsql): Build `GetTableTypes` response directly from the catalog (#7324)
* refactor(flightsql): Do not use plan for GetTableTypes

* test: Add an end to end test
2023-03-24 15:20:28 +00:00
Chunchun Ye 9cd2a8816a
feat(flightsql): Support `include_schema` param in `GetTables` metadata endpoint (#7307)
* feat(flightsql): Add support for table_schema in GetTables and complies

feat(fightsql): Add support for table_schema in GetTables

support actual schema

it compiles / does not fail

* chore: resolve merge conflict

* chore: make table_schema optional

* test: update e2e test for `include_schema` = true

* chore: remove info!() and update test `flightsql_schema_matches`

* chore(deps): Bump rustix from 0.36.11 to 0.37.3 (#7308)

* chore(deps): Bump rustix from 0.36.11 to 0.37.3

Bumps [rustix](https://github.com/bytecodealliance/rustix) from 0.36.11 to 0.37.3.
- [Release notes](https://github.com/bytecodealliance/rustix/releases)
- [Commits](https://github.com/bytecodealliance/rustix/compare/v0.36.11...v0.37.3)

---
updated-dependencies:
- dependency-name: rustix
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* chore: Run cargo hakari tasks

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>

* chore: make real error for existing columns

* chore: user match instead of unwrap() on column names

* chore: use datafusion::physical_plan::collect() to get record batches

* chore: use `concat_batches` to combine multiple batches into single one and fix db schema test

* chore: add doc comment for GetTables

* chore: remove pretty print

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-03-24 14:59:30 +00:00
Chunchun Ye 87708dc64a
feat(flightsql): Add filters to `GetTables` metadata endpoint (#7289)
* feat: add optional param to GetTables

* chore: add the third param to query plan

* feat: add table_types param

* chore: clippy

* test: add test cases with filters

* chore: update query to avoid SQL injection

* refactor: update where clause and cleanup

---------

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-03-22 20:14:01 +00:00
Andrew Lamb 7f7d0b8d52
fix(flightsql): Send correct schema in GetFlightInfo for Queries (#7283)
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-03-22 13:58:44 +00:00
Andrew Lamb 777128f2b7
feat(flightsql): Avoid planning Flight queries twice (#7281)
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-03-21 20:15:53 +00:00
Andrew Lamb 184565b552
feat(flightsql): Implement FlightSQL `GetSqlInfo` endpoint (#7198)
* feat(flightsql): Implement GetSqlInfo endpoint

* chore: Add some comments to clarify the tests intent
2023-03-20 19:34:18 +00:00
Andrew Lamb 795a79a7ad
feat(flightsql): Add `GetTables` metadata endpoint without filters (#7238)
* test: Add getTables jdbc_client example

* feat: add `CommandGetTables` in FlightSqlClient

* feat: add `CommandGetTables` in flightsql cmd and planner

* test: add e2e test for `CommandGetTables`

* chore: clippy

* chore: comment out the test with filters

* test: update jdbc test expected value for tables

---------

Co-authored-by: Chunchun <14298407+appletreeisyellow@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-03-20 19:20:13 +00:00
Andrew Lamb 2bc495b29e
feat(flightsql): Support `GetTableTypes` metadata API (#7242)
* feat(flightsql): Support `GetTableTypes` metadata API

* chore: comment about possible optimization

* chore: fix logical conflict
2023-03-17 13:12:23 +00:00
Andrew Lamb 3fb4fad784
refactor(iox_query): Rename `prepare_sql` to `sql_to_physical_plan` (#7226)
* refactor(iox_query): Rename `prepare_sql` to `sql_to_physical_plan`

* fix: logical conflict
2023-03-16 19:12:15 +00:00
Andrew Lamb 0c36c60d66
refactor(flightsql): simplfy parameterized statement generation (#7232)
* docs: Add doc link to command enum

* refactor(flightsql): simplfy parameterized statement generation

---------

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-03-16 14:27:21 +00:00
Andrew Lamb c819d4f865
feat(flightsql): Implement `GetDbSchemas` metadata API (#7213)
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-03-16 10:13:51 +00:00
Andrew Lamb 6d6fd8f663
feat(flightsql): implement basic `CommandGetCatalogs` support (#7212)
* refactor: reduce redundancy in test

* chore: implement basic get_catalog support

* fix: clippy
2023-03-15 21:52:59 +00:00
dependabot[bot] c538cac4ef
chore(deps): Bump tokio from 1.25.0 to 1.26.0 (#7107)
* chore(deps): Bump tokio from 1.25.0 to 1.26.0

Bumps [tokio](https://github.com/tokio-rs/tokio) from 1.25.0 to 1.26.0.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.25.0...tokio-1.26.0)

---
updated-dependencies:
- dependency-name: tokio
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* chore: Run cargo hakari tasks

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: Dom <dom@itsallbroken.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-03-02 09:50:39 +00:00
Carol (Nichols || Goulding) faae5eb438 chore: Rerun cargo hakari manage-deps 2023-02-27 11:56:15 +01:00
dependabot[bot] d0e6b16450
chore(deps): Bump bytes from 1.3.0 to 1.4.0
Bumps [bytes](https://github.com/tokio-rs/bytes) from 1.3.0 to 1.4.0.
- [Release notes](https://github.com/tokio-rs/bytes/releases)
- [Changelog](https://github.com/tokio-rs/bytes/blob/master/CHANGELOG.md)
- [Commits](https://github.com/tokio-rs/bytes/compare/v1.3.0...v1.4.0)

---
updated-dependencies:
- dependency-name: bytes
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-02-01 00:30:56 +00:00
Andrew Lamb fa15a48c31
refactor: use types rather than `Any` in `flightsql` interface (#6705)
* refactor: use types rather than Any in flightsql interface

* fix: Update service_grpc_flight/src/lib.rs

Co-authored-by: Raphael Taylor-Davies <1781103+tustvold@users.noreply.github.com>

* chore: review feedback

* fix: fmt

* fix: error conversion

---------

Co-authored-by: Raphael Taylor-Davies <1781103+tustvold@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-01-31 14:45:46 +00:00
dependabot[bot] ed7d02a225
chore(deps): Bump tokio from 1.24.2 to 1.25.0
Bumps [tokio](https://github.com/tokio-rs/tokio) from 1.24.2 to 1.25.0.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/commits/tokio-1.25.0)

---
updated-dependencies:
- dependency-name: tokio
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-01-30 01:57:27 +00:00
Andrew Lamb 2db8443a64
refactor: split flightsql crate into smaller modules (#6703)
* refactor: split flightsql crate into smaller modules

* refactor: automatically derive from Impl
2023-01-25 21:12:48 +00:00
Andrew Lamb 0c55a0f257
feat: Implement basic prepared statement support in IOx (#6667)
* feat: allow override of flightsql namespace

* feat: Implement DoAction endpoint

* refactor: Remove try_unpack

* fix: remove unused code / more clone
2023-01-25 12:00:43 +00:00
Andrew Lamb c3bc61f10e
refactor: Move `flightsql` code into its own module, add docs and tests (#6640)
* refactor: Move `flightsql`  code into its own module

* fix: get schema from LogicalPlan

* refactor: use arrow_flight::sql::Any instead of prost_types::any

* fix: cleanup docs and avoid as_ref

* fix: Use Bytes

* fix: use Any::pack

* fix: doclink
2023-01-24 18:24:32 +00:00