Commit Graph

337 Commits (41d93aea4d8e407b772e3e935f1e9e3ddb4dbe88)

Author SHA1 Message Date
dependabot[bot] bdf7f316d7
chore(deps): Bump tokio from 1.27.0 to 1.28.0 (#7667)
* chore(deps): Bump tokio from 1.27.0 to 1.28.0

Bumps [tokio](https://github.com/tokio-rs/tokio) from 1.27.0 to 1.28.0.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.27.0...tokio-1.28.0)

---
updated-dependencies:
- dependency-name: tokio
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* chore: Run cargo hakari tasks

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: Dom <dom@itsallbroken.com>
2023-04-26 12:53:26 +00:00
dependabot[bot] 09d6b4ae50
chore(deps): Bump tokio-stream from 0.1.12 to 0.1.13 (#7666)
Bumps [tokio-stream](https://github.com/tokio-rs/tokio) from 0.1.12 to 0.1.13.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-stream-0.1.12...tokio-stream-0.1.13)

---
updated-dependencies:
- dependency-name: tokio-stream
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Dom <dom@itsallbroken.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-04-26 09:31:04 +00:00
Chunchun Ye eca86a0f06
feat(flightsql): Support `CommandGetXdbcTypeInfo` metadata endpoint with an empty RecordBatch (#7576)
* feat: WIP support CommandGetXdbcTypeInfo metadata endpoint with tests

* chore: update test case and add jdbc test

* chore: uncomment jdbc getCoumns test

* chore: lint

---------

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-04-25 21:06:27 +00:00
Chunchun Ye 69da3c2495
feat(flightsql): Support `GetCrossReference` metadata endpoint with an empty RecordBatch (#7548)
* feat: support CommandGetCrossReference metadata endpoint with tests

* chore: create two tables in the test for GetCrossReference endpoint

---------

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-04-13 19:53:37 +00:00
Chunchun Ye 8bf47df621
feat(flightsql): Support `GetImportedKeys` metadata endpoint with an empty RecordBatch (#7546)
* feat: support CommandGetImportedKeys metadata endpoint with tests

* chore: remove comments that is no longer valid

---------

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-04-13 18:06:47 +00:00
Chunchun Ye e86e8de1a9
feat: support CommandGetExportedKeys metadata endpoint with tests (#7532)
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-04-13 13:27:09 +00:00
dependabot[bot] e811a69a1e
chore(deps): Bump serde_json from 1.0.95 to 1.0.96 (#7535)
Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.95 to 1.0.96.
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/v1.0.95...v1.0.96)

---
updated-dependencies:
- dependency-name: serde_json
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-04-13 10:07:32 +00:00
Stuart Carnie 5965cee179
feat: Update InfluxQL end-to-end test output (#7512)
* feat: Specialises test output formatting for each language

* Also fixes an error uncovered in the `write_columnar` when tag
  columns are `NULL`

Closes #7145

* chore: Run cargo hakari tasks

* chore: Add sorted output until #7513 is addressed

* chore: clippy 📋

* feat: Add `options` to `write_columnar`

* Added ability to configure border rendering, including removing
  borders. This helps avoid variable width issues with EXPLAIN output,
  which tends to vary and cause flaky test failures.

* chore: rustfmt 🧹

* chore: update expected output

* chore: clarify what "this" is

---------

Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
2023-04-13 09:58:12 +00:00
kodiakhq[bot] 53ddca45d8
Merge branch 'main' into cn/remove-write-summary 2023-04-12 16:07:35 +00:00
Andrew Lamb 20e9c91866
refactor: Use workspace dependencies for `tonic`, `tonic-build`, etc (#7515)
* refactor: Use workspace dependencies for `tonic`, `tonic-build`, etc

* chore: Run cargo hakari tasks

---------

Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-04-12 16:07:19 +00:00
Carol (Nichols || Goulding) 6387a9576a
fix: Remove the write_summary crate and write info service 2023-04-12 11:31:23 -04:00
Chunchun Ye b131895fc2
feat(flightsql): Support `GetPrimaryKeys` metadata endpoint with an empty RecordBatch (#7496)
* feat: add CommandGetPrimaryKeys metadata endpoint and tests

* chore: update schema for the returned record batch

---------

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-04-11 14:13:44 +00:00
Martin Hilton 1a42155077
chore: end to end tests for authorization (#7441)
* chore: end-to-end tests for authorization

Add tests to validate the behaviour of the authorization machinery
in the write and query paths.

In order to facilitate this an authorizer implentation has been
added to the the test helpers that runs an authorizer gRPC service
for the use of tests. The gRPC service is started in the process
that is running the test and listens on a OS-assigned port number.
The authorization service cannot be shared between tests so a
non-shared cluster must be used when the authorizer is configured.

The influxdb_iox_client has been enhanced so that the user can
configure additional headers in the flight client, which is used
for SQL and InfluxQL queries. This uses the same interface as the
Flight SQL client has for the same job.

* chore: fix lint errors

* chore: review suggestion

Consolate the authorization tests into fewer tests in order to avoid
repeating set-up and tear-down unnecessarily.
2023-04-11 09:56:45 +00:00
Fraser Savage 30b292f3df
feat(cli): Update namespace service protection limits
This commit adds a client method to invoke the
UpdateNamespaceServiceProtectionLimits RPC API, providing a user
friendly way to do this through the IOx command line.
2023-03-31 17:14:20 +01:00
Phil Bracikowski 6cc2f03ea0
fix: default the write cli comment to the http default port. (#7394)
* fix: default the write cli command to the http default port.

The all-in-one write api is based on influxdb cloud's v2 http api, which
uses the 8080 http default port. This changeset will
allow 'influxdb_iox write' to work against default influxdb_iox
all-in-one without needing to use the --host option to change the port.

It should not change behavior for existing users of `--host`. It adds a
new configuartion option call `--http-host` to set the http port
separately from the gRPC one.

* fix: fmt
2023-03-31 14:59:24 +00:00
dependabot[bot] 9ddca96336
chore(deps): Bump thiserror from 1.0.39 to 1.0.40 (#7375)
Bumps [thiserror](https://github.com/dtolnay/thiserror) from 1.0.39 to 1.0.40.
- [Release notes](https://github.com/dtolnay/thiserror/releases)
- [Commits](https://github.com/dtolnay/thiserror/compare/1.0.39...1.0.40)

---
updated-dependencies:
- dependency-name: thiserror
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-03-30 10:43:28 +00:00
dependabot[bot] 9cbcdc7672
chore(deps): Bump tokio from 1.26.0 to 1.27.0 (#7373)
Bumps [tokio](https://github.com/tokio-rs/tokio) from 1.26.0 to 1.27.0.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.26.0...tokio-1.27.0)

---
updated-dependencies:
- dependency-name: tokio
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-03-30 09:36:04 +00:00
dependabot[bot] 1ad0160d30
chore(deps): Bump serde_json from 1.0.94 to 1.0.95 (#7348)
Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.94 to 1.0.95.
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/v1.0.94...v1.0.95)

---
updated-dependencies:
- dependency-name: serde_json
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-03-28 11:09:59 +00:00
Andrew Lamb ce77d3bd74
refactor(flightsql): Build `GetTableTypes` response directly from the catalog (#7324)
* refactor(flightsql): Do not use plan for GetTableTypes

* test: Add an end to end test
2023-03-24 15:20:28 +00:00
Chunchun Ye 9cd2a8816a
feat(flightsql): Support `include_schema` param in `GetTables` metadata endpoint (#7307)
* feat(flightsql): Add support for table_schema in GetTables and complies

feat(fightsql): Add support for table_schema in GetTables

support actual schema

it compiles / does not fail

* chore: resolve merge conflict

* chore: make table_schema optional

* test: update e2e test for `include_schema` = true

* chore: remove info!() and update test `flightsql_schema_matches`

* chore(deps): Bump rustix from 0.36.11 to 0.37.3 (#7308)

* chore(deps): Bump rustix from 0.36.11 to 0.37.3

Bumps [rustix](https://github.com/bytecodealliance/rustix) from 0.36.11 to 0.37.3.
- [Release notes](https://github.com/bytecodealliance/rustix/releases)
- [Commits](https://github.com/bytecodealliance/rustix/compare/v0.36.11...v0.37.3)

---
updated-dependencies:
- dependency-name: rustix
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* chore: Run cargo hakari tasks

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>

* chore: make real error for existing columns

* chore: user match instead of unwrap() on column names

* chore: use datafusion::physical_plan::collect() to get record batches

* chore: use `concat_batches` to combine multiple batches into single one and fix db schema test

* chore: add doc comment for GetTables

* chore: remove pretty print

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-03-24 14:59:30 +00:00
Chunchun Ye 87708dc64a
feat(flightsql): Add filters to `GetTables` metadata endpoint (#7289)
* feat: add optional param to GetTables

* chore: add the third param to query plan

* feat: add table_types param

* chore: clippy

* test: add test cases with filters

* chore: update query to avoid SQL injection

* refactor: update where clause and cleanup

---------

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-03-22 20:14:01 +00:00
Andrew Lamb 184565b552
feat(flightsql): Implement FlightSQL `GetSqlInfo` endpoint (#7198)
* feat(flightsql): Implement GetSqlInfo endpoint

* chore: Add some comments to clarify the tests intent
2023-03-20 19:34:18 +00:00
Andrew Lamb 795a79a7ad
feat(flightsql): Add `GetTables` metadata endpoint without filters (#7238)
* test: Add getTables jdbc_client example

* feat: add `CommandGetTables` in FlightSqlClient

* feat: add `CommandGetTables` in flightsql cmd and planner

* test: add e2e test for `CommandGetTables`

* chore: clippy

* chore: comment out the test with filters

* test: update jdbc test expected value for tables

---------

Co-authored-by: Chunchun <14298407+appletreeisyellow@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-03-20 19:20:13 +00:00
Andrew Lamb 28cb56767c
fix: Rename crate `influxdb_line_protocol` to `influxdb-line-protocol` (#7244)
* fix: Rename crate `influxdb_line_protocol` to `influxdb-line-protocol`

* fix: hakari config

---------

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-03-17 13:19:50 +00:00
Andrew Lamb 2bc495b29e
feat(flightsql): Support `GetTableTypes` metadata API (#7242)
* feat(flightsql): Support `GetTableTypes` metadata API

* chore: comment about possible optimization

* chore: fix logical conflict
2023-03-17 13:12:23 +00:00
Andrew Lamb c819d4f865
feat(flightsql): Implement `GetDbSchemas` metadata API (#7213)
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-03-16 10:13:51 +00:00
Andrew Lamb 6d6fd8f663
feat(flightsql): implement basic `CommandGetCatalogs` support (#7212)
* refactor: reduce redundancy in test

* chore: implement basic get_catalog support

* fix: clippy
2023-03-15 21:52:59 +00:00
Andrew Lamb 3bb5347d4c
chore: Remove unused dependencies found with cargo-machete @alamb (#7183)
* chore: Remove unused dependencies found with cargo-machete

* chore: Remove unused dependencies

* fix: fixup

---------

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-03-13 11:06:53 +00:00
Stuart Carnie 04202bffb6
feat: Output formatting for InfluxQL (#7133)
* feat: Output formatting for InfluxQL

* chore: Use constant for metadata key based on PR feedback

* chore: Add lock changes
2023-03-08 00:42:05 +00:00
dependabot[bot] 8771dcb645
chore(deps): Bump thiserror from 1.0.38 to 1.0.39 (#7131)
Bumps [thiserror](https://github.com/dtolnay/thiserror) from 1.0.38 to 1.0.39.
- [Release notes](https://github.com/dtolnay/thiserror/releases)
- [Commits](https://github.com/dtolnay/thiserror/compare/1.0.38...1.0.39)

---
updated-dependencies:
- dependency-name: thiserror
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-03-06 10:55:26 +00:00
dependabot[bot] c538cac4ef
chore(deps): Bump tokio from 1.25.0 to 1.26.0 (#7107)
* chore(deps): Bump tokio from 1.25.0 to 1.26.0

Bumps [tokio](https://github.com/tokio-rs/tokio) from 1.25.0 to 1.26.0.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.25.0...tokio-1.26.0)

---
updated-dependencies:
- dependency-name: tokio
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* chore: Run cargo hakari tasks

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: Dom <dom@itsallbroken.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-03-02 09:50:39 +00:00
dependabot[bot] aa7d458a81
chore(deps): Bump tokio-stream from 0.1.11 to 0.1.12 (#7035)
Bumps [tokio-stream](https://github.com/tokio-rs/tokio) from 0.1.11 to 0.1.12.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-stream-0.1.11...tokio-stream-0.1.12)

---
updated-dependencies:
- dependency-name: tokio-stream
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Dom <dom@itsallbroken.com>
2023-02-21 16:31:55 +00:00
Carol (Nichols || Goulding) b11228c72e
test: Write an e2e test of namespace soft-deletion (#7002)
Co-authored-by: Dom <dom@itsallbroken.com>
2023-02-16 09:59:14 +00:00
Carol (Nichols || Goulding) 30fea67701
fix: Move variables within format strings. Thanks clippy!
Changes made automatically using `cargo clippy --fix`.
2023-02-03 13:06:17 -05:00
Dom Dwyer 77ff4dc4b8
docs: retention period values in Namespace client
Document the 0/null and negative value behaviours for retention periods
in the namespace RPC client.
2023-02-01 14:41:10 +01:00
dependabot[bot] d0e6b16450
chore(deps): Bump bytes from 1.3.0 to 1.4.0
Bumps [bytes](https://github.com/tokio-rs/bytes) from 1.3.0 to 1.4.0.
- [Release notes](https://github.com/tokio-rs/bytes/releases)
- [Changelog](https://github.com/tokio-rs/bytes/blob/master/CHANGELOG.md)
- [Commits](https://github.com/tokio-rs/bytes/compare/v1.3.0...v1.4.0)

---
updated-dependencies:
- dependency-name: bytes
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-02-01 00:30:56 +00:00
Andrew Lamb e8e50df692
chore: Enable last SQL test, retention.sql (#6721) 2023-01-31 12:46:50 +00:00
dependabot[bot] ed7d02a225
chore(deps): Bump tokio from 1.24.2 to 1.25.0
Bumps [tokio](https://github.com/tokio-rs/tokio) from 1.24.2 to 1.25.0.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/commits/tokio-1.25.0)

---
updated-dependencies:
- dependency-name: tokio
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-01-30 01:57:27 +00:00
Andrew Lamb 6a0429584a fix: update doc example 2023-01-26 06:59:34 -05:00
Andrew Lamb c100737a81 chore: Do not send dictionary encoded data to clients 2023-01-26 06:35:15 -05:00
Carol (Nichols || Goulding) 4658510102
fix: For Ingester2, persist a particular namespace on demand and share MiniClusters
This should hopefully help CI from running out of Postgres
connections 😬

The old architecture will still need to be non-shared and persist
everything.
2023-01-25 10:36:56 -05:00
Andrew Lamb 0c55a0f257
feat: Implement basic prepared statement support in IOx (#6667)
* feat: allow override of flightsql namespace

* feat: Implement DoAction endpoint

* refactor: Remove try_unpack

* fix: remove unused code / more clone
2023-01-25 12:00:43 +00:00
Andrew Lamb c3bc61f10e
refactor: Move `flightsql` code into its own module, add docs and tests (#6640)
* refactor: Move `flightsql`  code into its own module

* fix: get schema from LogicalPlan

* refactor: use arrow_flight::sql::Any instead of prost_types::any

* fix: cleanup docs and avoid as_ref

* fix: Use Bytes

* fix: use Any::pack

* fix: doclink
2023-01-24 18:24:32 +00:00
Andrew Lamb d808c57cdc
chore: Remove `iox_arrow_flight` (#6621)
* chore: Remove iox_arrow_flight

* fix: hack around tonic status errors

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-01-20 11:16:19 +00:00
Andrew Lamb 65c020c9f2
refactor: remove iox_arrow_flight use in `influxdb_iox_client ` and `querier` (#6624)
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-01-19 18:48:23 +00:00
Andrew Lamb f639bf3e23
chore: refactor ingester to use upstream arrow-flight (#6622)
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-01-19 15:16:13 +00:00
Carol (Nichols || Goulding) 0554194923
docs: Explain persist-on-demand use case and potential limitations 2023-01-12 11:52:11 -05:00
Carol (Nichols || Goulding) 3a2544a7eb
feat: Define a new gRPC service for ingester persist 2023-01-12 11:03:12 -05:00
Carol (Nichols || Goulding) adc5c2bf06
feat: Add a gRPC API to the catalog service to get Parquet files by namespace
Tests that write line protocol (that may contain writes to multiple
tables) need to be able to see when new Parquet files are saved.
2023-01-11 11:41:09 -05:00
dependabot[bot] b49cc2e35e
chore(deps): Bump tokio from 1.24.0 to 1.24.1 (#6545)
Bumps [tokio](https://github.com/tokio-rs/tokio) from 1.24.0 to 1.24.1.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.24.0...tokio-1.24.1)

---
updated-dependencies:
- dependency-name: tokio
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-01-10 09:48:44 +00:00