Commit Graph

323 Commits (682ed14b9e5a21a2b822f9ce84c8d565b8815082)

Author SHA1 Message Date
Phil Bracikowski 6cc2f03ea0
fix: default the write cli comment to the http default port. (#7394)
* fix: default the write cli command to the http default port.

The all-in-one write api is based on influxdb cloud's v2 http api, which
uses the 8080 http default port. This changeset will
allow 'influxdb_iox write' to work against default influxdb_iox
all-in-one without needing to use the --host option to change the port.

It should not change behavior for existing users of `--host`. It adds a
new configuartion option call `--http-host` to set the http port
separately from the gRPC one.

* fix: fmt
2023-03-31 14:59:24 +00:00
dependabot[bot] 9ddca96336
chore(deps): Bump thiserror from 1.0.39 to 1.0.40 (#7375)
Bumps [thiserror](https://github.com/dtolnay/thiserror) from 1.0.39 to 1.0.40.
- [Release notes](https://github.com/dtolnay/thiserror/releases)
- [Commits](https://github.com/dtolnay/thiserror/compare/1.0.39...1.0.40)

---
updated-dependencies:
- dependency-name: thiserror
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-03-30 10:43:28 +00:00
dependabot[bot] 9cbcdc7672
chore(deps): Bump tokio from 1.26.0 to 1.27.0 (#7373)
Bumps [tokio](https://github.com/tokio-rs/tokio) from 1.26.0 to 1.27.0.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.26.0...tokio-1.27.0)

---
updated-dependencies:
- dependency-name: tokio
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-03-30 09:36:04 +00:00
dependabot[bot] 1ad0160d30
chore(deps): Bump serde_json from 1.0.94 to 1.0.95 (#7348)
Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.94 to 1.0.95.
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/v1.0.94...v1.0.95)

---
updated-dependencies:
- dependency-name: serde_json
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-03-28 11:09:59 +00:00
Andrew Lamb ce77d3bd74
refactor(flightsql): Build `GetTableTypes` response directly from the catalog (#7324)
* refactor(flightsql): Do not use plan for GetTableTypes

* test: Add an end to end test
2023-03-24 15:20:28 +00:00
Chunchun Ye 9cd2a8816a
feat(flightsql): Support `include_schema` param in `GetTables` metadata endpoint (#7307)
* feat(flightsql): Add support for table_schema in GetTables and complies

feat(fightsql): Add support for table_schema in GetTables

support actual schema

it compiles / does not fail

* chore: resolve merge conflict

* chore: make table_schema optional

* test: update e2e test for `include_schema` = true

* chore: remove info!() and update test `flightsql_schema_matches`

* chore(deps): Bump rustix from 0.36.11 to 0.37.3 (#7308)

* chore(deps): Bump rustix from 0.36.11 to 0.37.3

Bumps [rustix](https://github.com/bytecodealliance/rustix) from 0.36.11 to 0.37.3.
- [Release notes](https://github.com/bytecodealliance/rustix/releases)
- [Commits](https://github.com/bytecodealliance/rustix/compare/v0.36.11...v0.37.3)

---
updated-dependencies:
- dependency-name: rustix
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* chore: Run cargo hakari tasks

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>

* chore: make real error for existing columns

* chore: user match instead of unwrap() on column names

* chore: use datafusion::physical_plan::collect() to get record batches

* chore: use `concat_batches` to combine multiple batches into single one and fix db schema test

* chore: add doc comment for GetTables

* chore: remove pretty print

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-03-24 14:59:30 +00:00
Chunchun Ye 87708dc64a
feat(flightsql): Add filters to `GetTables` metadata endpoint (#7289)
* feat: add optional param to GetTables

* chore: add the third param to query plan

* feat: add table_types param

* chore: clippy

* test: add test cases with filters

* chore: update query to avoid SQL injection

* refactor: update where clause and cleanup

---------

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-03-22 20:14:01 +00:00
Andrew Lamb 184565b552
feat(flightsql): Implement FlightSQL `GetSqlInfo` endpoint (#7198)
* feat(flightsql): Implement GetSqlInfo endpoint

* chore: Add some comments to clarify the tests intent
2023-03-20 19:34:18 +00:00
Andrew Lamb 795a79a7ad
feat(flightsql): Add `GetTables` metadata endpoint without filters (#7238)
* test: Add getTables jdbc_client example

* feat: add `CommandGetTables` in FlightSqlClient

* feat: add `CommandGetTables` in flightsql cmd and planner

* test: add e2e test for `CommandGetTables`

* chore: clippy

* chore: comment out the test with filters

* test: update jdbc test expected value for tables

---------

Co-authored-by: Chunchun <14298407+appletreeisyellow@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-03-20 19:20:13 +00:00
Andrew Lamb 28cb56767c
fix: Rename crate `influxdb_line_protocol` to `influxdb-line-protocol` (#7244)
* fix: Rename crate `influxdb_line_protocol` to `influxdb-line-protocol`

* fix: hakari config

---------

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-03-17 13:19:50 +00:00
Andrew Lamb 2bc495b29e
feat(flightsql): Support `GetTableTypes` metadata API (#7242)
* feat(flightsql): Support `GetTableTypes` metadata API

* chore: comment about possible optimization

* chore: fix logical conflict
2023-03-17 13:12:23 +00:00
Andrew Lamb c819d4f865
feat(flightsql): Implement `GetDbSchemas` metadata API (#7213)
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-03-16 10:13:51 +00:00
Andrew Lamb 6d6fd8f663
feat(flightsql): implement basic `CommandGetCatalogs` support (#7212)
* refactor: reduce redundancy in test

* chore: implement basic get_catalog support

* fix: clippy
2023-03-15 21:52:59 +00:00
Andrew Lamb 3bb5347d4c
chore: Remove unused dependencies found with cargo-machete @alamb (#7183)
* chore: Remove unused dependencies found with cargo-machete

* chore: Remove unused dependencies

* fix: fixup

---------

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-03-13 11:06:53 +00:00
Stuart Carnie 04202bffb6
feat: Output formatting for InfluxQL (#7133)
* feat: Output formatting for InfluxQL

* chore: Use constant for metadata key based on PR feedback

* chore: Add lock changes
2023-03-08 00:42:05 +00:00
dependabot[bot] 8771dcb645
chore(deps): Bump thiserror from 1.0.38 to 1.0.39 (#7131)
Bumps [thiserror](https://github.com/dtolnay/thiserror) from 1.0.38 to 1.0.39.
- [Release notes](https://github.com/dtolnay/thiserror/releases)
- [Commits](https://github.com/dtolnay/thiserror/compare/1.0.38...1.0.39)

---
updated-dependencies:
- dependency-name: thiserror
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-03-06 10:55:26 +00:00
dependabot[bot] c538cac4ef
chore(deps): Bump tokio from 1.25.0 to 1.26.0 (#7107)
* chore(deps): Bump tokio from 1.25.0 to 1.26.0

Bumps [tokio](https://github.com/tokio-rs/tokio) from 1.25.0 to 1.26.0.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.25.0...tokio-1.26.0)

---
updated-dependencies:
- dependency-name: tokio
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* chore: Run cargo hakari tasks

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: Dom <dom@itsallbroken.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-03-02 09:50:39 +00:00
dependabot[bot] aa7d458a81
chore(deps): Bump tokio-stream from 0.1.11 to 0.1.12 (#7035)
Bumps [tokio-stream](https://github.com/tokio-rs/tokio) from 0.1.11 to 0.1.12.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-stream-0.1.11...tokio-stream-0.1.12)

---
updated-dependencies:
- dependency-name: tokio-stream
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Dom <dom@itsallbroken.com>
2023-02-21 16:31:55 +00:00
Carol (Nichols || Goulding) b11228c72e
test: Write an e2e test of namespace soft-deletion (#7002)
Co-authored-by: Dom <dom@itsallbroken.com>
2023-02-16 09:59:14 +00:00
Carol (Nichols || Goulding) 30fea67701
fix: Move variables within format strings. Thanks clippy!
Changes made automatically using `cargo clippy --fix`.
2023-02-03 13:06:17 -05:00
Dom Dwyer 77ff4dc4b8
docs: retention period values in Namespace client
Document the 0/null and negative value behaviours for retention periods
in the namespace RPC client.
2023-02-01 14:41:10 +01:00
dependabot[bot] d0e6b16450
chore(deps): Bump bytes from 1.3.0 to 1.4.0
Bumps [bytes](https://github.com/tokio-rs/bytes) from 1.3.0 to 1.4.0.
- [Release notes](https://github.com/tokio-rs/bytes/releases)
- [Changelog](https://github.com/tokio-rs/bytes/blob/master/CHANGELOG.md)
- [Commits](https://github.com/tokio-rs/bytes/compare/v1.3.0...v1.4.0)

---
updated-dependencies:
- dependency-name: bytes
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-02-01 00:30:56 +00:00
Andrew Lamb e8e50df692
chore: Enable last SQL test, retention.sql (#6721) 2023-01-31 12:46:50 +00:00
dependabot[bot] ed7d02a225
chore(deps): Bump tokio from 1.24.2 to 1.25.0
Bumps [tokio](https://github.com/tokio-rs/tokio) from 1.24.2 to 1.25.0.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/commits/tokio-1.25.0)

---
updated-dependencies:
- dependency-name: tokio
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-01-30 01:57:27 +00:00
Andrew Lamb 6a0429584a fix: update doc example 2023-01-26 06:59:34 -05:00
Andrew Lamb c100737a81 chore: Do not send dictionary encoded data to clients 2023-01-26 06:35:15 -05:00
Carol (Nichols || Goulding) 4658510102
fix: For Ingester2, persist a particular namespace on demand and share MiniClusters
This should hopefully help CI from running out of Postgres
connections 😬

The old architecture will still need to be non-shared and persist
everything.
2023-01-25 10:36:56 -05:00
Andrew Lamb 0c55a0f257
feat: Implement basic prepared statement support in IOx (#6667)
* feat: allow override of flightsql namespace

* feat: Implement DoAction endpoint

* refactor: Remove try_unpack

* fix: remove unused code / more clone
2023-01-25 12:00:43 +00:00
Andrew Lamb c3bc61f10e
refactor: Move `flightsql` code into its own module, add docs and tests (#6640)
* refactor: Move `flightsql`  code into its own module

* fix: get schema from LogicalPlan

* refactor: use arrow_flight::sql::Any instead of prost_types::any

* fix: cleanup docs and avoid as_ref

* fix: Use Bytes

* fix: use Any::pack

* fix: doclink
2023-01-24 18:24:32 +00:00
Andrew Lamb d808c57cdc
chore: Remove `iox_arrow_flight` (#6621)
* chore: Remove iox_arrow_flight

* fix: hack around tonic status errors

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-01-20 11:16:19 +00:00
Andrew Lamb 65c020c9f2
refactor: remove iox_arrow_flight use in `influxdb_iox_client ` and `querier` (#6624)
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-01-19 18:48:23 +00:00
Andrew Lamb f639bf3e23
chore: refactor ingester to use upstream arrow-flight (#6622)
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-01-19 15:16:13 +00:00
Carol (Nichols || Goulding) 0554194923
docs: Explain persist-on-demand use case and potential limitations 2023-01-12 11:52:11 -05:00
Carol (Nichols || Goulding) 3a2544a7eb
feat: Define a new gRPC service for ingester persist 2023-01-12 11:03:12 -05:00
Carol (Nichols || Goulding) adc5c2bf06
feat: Add a gRPC API to the catalog service to get Parquet files by namespace
Tests that write line protocol (that may contain writes to multiple
tables) need to be able to see when new Parquet files are saved.
2023-01-11 11:41:09 -05:00
dependabot[bot] b49cc2e35e
chore(deps): Bump tokio from 1.24.0 to 1.24.1 (#6545)
Bumps [tokio](https://github.com/tokio-rs/tokio) from 1.24.0 to 1.24.1.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.24.0...tokio-1.24.1)

---
updated-dependencies:
- dependency-name: tokio
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-01-10 09:48:44 +00:00
Andrew Lamb 9b22ede3f0
refactor: Make arrow flight client return `futures::Streams` (#6438)
* refactor: Make arrow flight client use futures::Streams

* refactor: concision
2022-12-19 17:09:26 +00:00
Andrew Lamb 94c2f94ea1
refactor: Extract common ArrowFlight client into iox_arrow_flight (#6427)
* refactor: Extract common ArrowFlight client into iox_arrow_flight

* chore: Run cargo hakari tasks

* fix: clarify intent of iox_arrow_flight crate

* refactor: Apply suggestions from code review

Co-authored-by: Raphael Taylor-Davies <1781103+tustvold@users.noreply.github.com>

* fix: loop --> while let

* fix: REmove make_tonic_error in favor of From impl

Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: Raphael Taylor-Davies <1781103+tustvold@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-12-19 11:35:20 +00:00
dependabot[bot] 299f0e99f9
chore(deps): Bump thiserror from 1.0.37 to 1.0.38
Bumps [thiserror](https://github.com/dtolnay/thiserror) from 1.0.37 to 1.0.38.
- [Release notes](https://github.com/dtolnay/thiserror/releases)
- [Commits](https://github.com/dtolnay/thiserror/compare/1.0.37...1.0.38)

---
updated-dependencies:
- dependency-name: thiserror
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-12-19 10:33:32 +00:00
Andrew Lamb 47cd6821e1
feat: Document IOx Flight API and add convenience methods (#6392)
* feat: Document IOx Flight API and add convenience methods

* fix: InfluxQL handling

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-12-13 17:32:37 +00:00
Stuart Carnie 2306c383f3
feat: Introduce InfluxQL to Flight (#6166)
* feat: Introduce InfluxQL to Flight

All InfluxQL queries will fail with an error

* chore: Temper protobuf lint

* chore: Finalize flight.proto changes; fix tests

* chore: Add tests for InfluxQL planner

* chore: Update docs

* chore: Update docs

* chore: Rename back to original

* chore: Use .into() rather than cast

* chore: Use function rather than field

* chore: Improved InfluxQL planner name

* chore: Restore `impl Into<String>` argument

* chore: Add a comment that Go clients are unable to execute InfluxQL

* chore: Add a test for the `--lang` argument and InfluxQL
2022-11-23 00:33:49 +00:00
dependabot[bot] 04c00bbb62
chore(deps): Bump bytes from 1.2.1 to 1.3.0 (#6199)
Bumps [bytes](https://github.com/tokio-rs/bytes) from 1.2.1 to 1.3.0.
- [Release notes](https://github.com/tokio-rs/bytes/releases)
- [Changelog](https://github.com/tokio-rs/bytes/blob/master/CHANGELOG.md)
- [Commits](https://github.com/tokio-rs/bytes/commits)

---
updated-dependencies:
- dependency-name: bytes
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-11-22 08:23:24 +00:00
dependabot[bot] a9db7581cd
chore(deps): Bump tokio from 1.21.2 to 1.22.0 (#6183)
Bumps [tokio](https://github.com/tokio-rs/tokio) from 1.21.2 to 1.22.0.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.21.2...tokio-1.22.0)

---
updated-dependencies:
- dependency-name: tokio
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-11-21 10:21:24 +00:00
Nga Tran 49a9565240
feat: gRPC that creates namespace (#6103)
* feat: create namespace API call in router

Co-authored-by: Nga Tran <nga-tran@live.com>

* chore: treat retention as ns except in CLI

* fix: overflow in nanosecond calc

* fix: retention test after changing it from hours to ns

* chore: comment clarification in cli; better response type for error in ns API

* fix: correct some rebase mistakes

* chore: merge namespace create & create_with_retention; renamed ns create test helper fn & const

* fix: ns autocreation test was wrong after rebase

* fix: mem catalog has default 1hr retention, accidently removed in rebase

* chore: remove mem catalogs default 1hr retention; make it settable in sets & router

Co-authored-by: Luke Bond <luke.n.bond@gmail.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-11-18 13:02:12 +00:00
Luke Bond 9365d933f1
chore: router namespace api (#6151)
* chore: move ns api from querier to router

* chore: add explanatory comment in querier about moved namespace API

* fix: add namespace service to router

* fix: querier returns unimplemented error for ns retention, not panic

* chore: reuse namespace -> proto in router ns api

* chore: grpc namespace - consume ns to avoid clone

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-11-16 15:25:49 +00:00
Carol (Nichols || Goulding) 8dbdab8754
fix: Remove db_name field from DeletePayload
Doesn't seem to be used anywhere.
2022-11-14 16:46:04 -05:00
Carol (Nichols || Goulding) bdff4e8848
fix: Consistently use 'namespace' instead of 'database' in comments and other internal text 2022-11-11 15:46:04 -05:00
kodiakhq[bot] df5ec013d1
Merge branch 'main' into dom/dml-delete-namespace-id 2022-11-07 09:07:38 +00:00
Nga Tran 9356f2a1b9
feat: grpc for updating namespace retention period (#6041)
* refactor: make namespace folder for all namesapce's commands

* feat: WIP for add command to set retention period

* feat: more on updating retention period

* feat: grpc for update namespace retention period

* test: end to end test fpr namespace retention

* fix: lint proto

* chore: cleanup

* chore: kick CI run again

* fix: command hierachy

* chore: fix comments
2022-11-04 20:58:11 +00:00
Carol (Nichols || Goulding) b0dd09e2ea
fix: Specify a namespace ID so the doc test compiles 2022-11-04 14:51:46 -04:00