Influxdata
/
influxdb
mirror of https://github.com/influxdata/influxdb.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
10,959 Commits
218 Branches
453 Tags
360 MiB
Commit Graph

85 Commits (0ffb211c54209e38a94cdf9581b3bef9e13414d6)

Author SHA1 Message Date
dependabot[bot] 6327e3d9c0
chore(deps): Bump serde_json from 1.0.92 to 1.0.93 (#6918) 
Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.92 to 1.0.93.
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/v1.0.92...v1.0.93)

---
updated-dependencies:
- dependency-name: serde_json
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2023-02-09 10:39:33 +00:00
Raphael Taylor-Davies d3601a59f8
chore: update DataFusion, upgrade `arrow` `arrow-flight` and `parquet` to `32.0.0` (#6756) 
* chore: update DataFusion

* fix: test

* chore: format

* chore: clippy

* chore: update arrow

* chore: arrow upgrade fallout

* chore: Run cargo hakari tasks

* chore: remove failing warm compaction test

* fix: flight error propagation

* chore: update parquet size

* fix: Update error message

* chore: Update parquet metadata test

---------

Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: Andrew Lamb <alamb@influxdata.com>
Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2023-02-06 11:35:39 +00:00
dependabot[bot] 6f4e287a3a
chore(deps): Bump serde_json from 1.0.91 to 1.0.92 (#6860) 
Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.91 to 1.0.92.
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/v1.0.91...v1.0.92)

---
updated-dependencies:
- dependency-name: serde_json
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-02-06 08:27:41 +00:00
Carol (Nichols || Goulding) 30fea67701
fix: Move variables within format strings. Thanks clippy! 
Changes made automatically using `cargo clippy --fix`.
 2023-02-03 13:06:17 -05:00
dependabot[bot] d0e6b16450
chore(deps): Bump bytes from 1.3.0 to 1.4.0 
Bumps [bytes](https://github.com/tokio-rs/bytes) from 1.3.0 to 1.4.0.
- [Release notes](https://github.com/tokio-rs/bytes/releases)
- [Changelog](https://github.com/tokio-rs/bytes/blob/master/CHANGELOG.md)
- [Commits](https://github.com/tokio-rs/bytes/compare/v1.3.0...v1.4.0)

---
updated-dependencies:
- dependency-name: bytes
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
 2023-02-01 00:30:56 +00:00
Andrew Lamb fa15a48c31
refactor: use types rather than `Any` in `flightsql` interface (#6705) 
* refactor: use types rather than Any in flightsql interface

* fix: Update service_grpc_flight/src/lib.rs

Co-authored-by: Raphael Taylor-Davies <1781103+tustvold@users.noreply.github.com>

* chore: review feedback

* fix: fmt

* fix: error conversion

---------

Co-authored-by: Raphael Taylor-Davies <1781103+tustvold@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2023-01-31 14:45:46 +00:00
dependabot[bot] ed7d02a225
chore(deps): Bump tokio from 1.24.2 to 1.25.0 
Bumps [tokio](https://github.com/tokio-rs/tokio) from 1.24.2 to 1.25.0.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/commits/tokio-1.25.0)

---
updated-dependencies:
- dependency-name: tokio
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
 2023-01-30 01:57:27 +00:00
Andrew Lamb c100737a81 chore: Do not send dictionary encoded data to clients 2023-01-26 06:35:15 -05:00
Andrew Lamb 7853a19953
feat: JDBC integration tests with FlightSQL (#6693) 
* feat: basic JDBC integration test

* fix: do not run test without env set

* docs: add maven link

* refactor: clean up java with switch statement
 2023-01-25 22:21:18 +00:00
Andrew Lamb 0c55a0f257
feat: Implement basic prepared statement support in IOx (#6667) 
* feat: allow override of flightsql namespace

* feat: Implement DoAction endpoint

* refactor: Remove try_unpack

* fix: remove unused code / more clone
 2023-01-25 12:00:43 +00:00
Andrew Lamb c3bc61f10e
refactor: Move `flightsql` code into its own module, add docs and tests (#6640) 
* refactor: Move `flightsql`  code into its own module

* fix: get schema from LogicalPlan

* refactor: use arrow_flight::sql::Any instead of prost_types::any

* fix: cleanup docs and avoid as_ref

* fix: Use Bytes

* fix: use Any::pack

* fix: doclink
 2023-01-24 18:24:32 +00:00
Andrew Lamb 1b882e0062
fix: `error arrow/ipc: could not read message schema: EOF` (#6668) 
* chore: Test for schema from query

* fix: Send schema even for no RecordBatches

* fix: docs
 2023-01-23 22:23:34 +00:00
Andrew Lamb d808c57cdc
chore: Remove `iox_arrow_flight` (#6621) 
* chore: Remove iox_arrow_flight

* fix: hack around tonic status errors

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2023-01-20 11:16:19 +00:00
Andrew Lamb 8410998408
chore: Update datafusion to Jan 17, 2023 (2 / 2) and arrow/parquet `30.0.1` (#6604) 
* chore: Update datafusion to Jan 9, 2023 (2 / 2) and arrow/parquet `30.0.1`

* chore: Update for changes in arrow ipc

* chore: Run cargo hakari tasks

Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
 2023-01-18 15:51:24 +00:00
dependabot[bot] b49cc2e35e
chore(deps): Bump tokio from 1.24.0 to 1.24.1 (#6545) 
Bumps [tokio](https://github.com/tokio-rs/tokio) from 1.24.0 to 1.24.1.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.24.0...tokio-1.24.1)

---
updated-dependencies:
- dependency-name: tokio
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2023-01-10 09:48:44 +00:00
Andrew Lamb 6843eee1d2
feat: Extract encoding from `RecordBatch` --> `FlightData` from flight implementations (#6460) 
* feat: Extract encoding from `RecordBatch` --> `FlightData` from flight implementations

Refactor existing flight server impl

* fix: Apply suggestions from code review

Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>

* fix: fixup code review comments

* fix: update for more details

* fix: Update names / types

Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2023-01-04 13:36:16 +00:00
Andrew Lamb e1059a9009
feat: FlightSQL Milestone 2 basic FlightSQL client and FlightSQL server implementation and plumbing (#6398) 
* feat: Add basic Flight and FlightSQL client into IOx codebase

Basic flight end to end test

* fix: Apply suggestions from code review

Co-authored-by: Raphael Taylor-Davies <1781103+tustvold@users.noreply.github.com>

Co-authored-by: Raphael Taylor-Davies <1781103+tustvold@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2022-12-20 17:34:00 +00:00
dependabot[bot] 7f2aa8b10c
chore(deps): Bump serde_json from 1.0.89 to 1.0.91 
Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.89 to 1.0.91.
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/v1.0.89...v1.0.91)

---
updated-dependencies:
- dependency-name: serde_json
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2022-12-19 01:44:18 +00:00
Andrew Lamb c4bcfff80f
feat: Implement creating Tickets in gRPC service (#6422) 
* feat: Implement creating Tickets in gRPC service

* fix: remove hard coded 1024

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2022-12-16 22:18:39 +00:00
Andrew Lamb 78aba66ca2
refactor: Improve Flight API server side code and comments (#6395) 
* refactor: Improve Flight API server side code and comments

* refactor: revert &str signature in FlightService::run_query
 2022-12-15 14:10:58 +00:00
Andrew Lamb fc520e0c0f
refactor: Remove unecessary optimize_record_batch (#6262) 
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2022-11-29 13:35:46 +00:00
Stuart Carnie 2306c383f3
feat: Introduce InfluxQL to Flight (#6166) 
* feat: Introduce InfluxQL to Flight

All InfluxQL queries will fail with an error

* chore: Temper protobuf lint

* chore: Finalize flight.proto changes; fix tests

* chore: Add tests for InfluxQL planner

* chore: Update docs

* chore: Update docs

* chore: Rename back to original

* chore: Use .into() rather than cast

* chore: Use function rather than field

* chore: Improved InfluxQL planner name

* chore: Restore `impl Into<String>` argument

* chore: Add a comment that Go clients are unable to execute InfluxQL

* chore: Add a test for the `--lang` argument and InfluxQL
 2022-11-23 00:33:49 +00:00
Marco Neumann e4c12fa6a5
fix: slice flight response batches (#6205) 
* fix: slice flight response batches

Same as #6094 but for the Apache Flight interface.

Ref https://github.com/influxdata/idpe/issues/16073.

* refactor: use `RecordBatch::slice`

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2022-11-22 12:25:23 +00:00
dependabot[bot] caa595a6fc
chore(deps): Bump serde_json from 1.0.88 to 1.0.89 (#6203) 
Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.88 to 1.0.89.
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/v1.0.88...v1.0.89)

---
updated-dependencies:
- dependency-name: serde_json
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2022-11-22 09:28:31 +00:00
dependabot[bot] 04c00bbb62
chore(deps): Bump bytes from 1.2.1 to 1.3.0 (#6199) 
Bumps [bytes](https://github.com/tokio-rs/bytes) from 1.2.1 to 1.3.0.
- [Release notes](https://github.com/tokio-rs/bytes/releases)
- [Changelog](https://github.com/tokio-rs/bytes/blob/master/CHANGELOG.md)
- [Commits](https://github.com/tokio-rs/bytes/commits)

---
updated-dependencies:
- dependency-name: bytes
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2022-11-22 08:23:24 +00:00
dependabot[bot] a9db7581cd
chore(deps): Bump tokio from 1.21.2 to 1.22.0 (#6183) 
Bumps [tokio](https://github.com/tokio-rs/tokio) from 1.21.2 to 1.22.0.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.21.2...tokio-1.22.0)

---
updated-dependencies:
- dependency-name: tokio
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2022-11-21 10:21:24 +00:00
dependabot[bot] 52c50c16e1
chore(deps): Bump serde_json from 1.0.87 to 1.0.88 
Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.87 to 1.0.88.
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/v1.0.87...v1.0.88)

---
updated-dependencies:
- dependency-name: serde_json
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2022-11-21 01:52:18 +00:00
Carol (Nichols || Goulding) b0cf10c9aa
test: Add a test for decoding JSON tickets to ReadInfo 
Enshrine in a test what the Go clients are currently sending to avoid
breakage in the future, hopefully.
 2022-11-14 12:55:03 -05:00
Carol (Nichols || Goulding) 4f081a8405
fix: Remove 'legacy' from JSON ticket format as it's still used by Go clients 2022-11-14 12:46:58 -05:00
Carol (Nichols || Goulding) 51afe53d23
fix: Rename database to namespace in service_grpc_flight types, vars, and errors 2022-11-11 16:14:13 -05:00
Carol (Nichols || Goulding) 3dde82b3b9
fix: Rename QueryDatabaseProvider to QueryNamespaceProvider 2022-11-11 16:14:12 -05:00
Carol (Nichols || Goulding) 0657ad9600
fix: Rename QueryDatabase to QueryNamespace 2022-11-11 16:14:12 -05:00
Andrew Lamb 694443bb87
chore: Rename DatabaseName to NamespaceName (#6100) 
* chore: Rename DatabaseName to NamespaceName

* fix: fmt

* chore: Updates some more references

* chore: more cleanup

* fix: adjust test

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2022-11-10 14:13:59 +00:00
Dom d9c97795fc
feat: use IDs in ingester query API (#6093) 
* refactor: NS+table ID (instead of name) in querier<>ingester

* feat(ingester): use IDs for query API

Changes the ingester to utilise the ID fields (instead of names) sent
over the query wire message wrapped within the Flight API.

BREAKING: this changes the "query-ingester" CLI command arguments which
now expects the namespace & table IDs, rather than their names.

* refactor(ingester): add more query logging context

Updates the log messages during query execution to include more context
fields.

* style: remove unused import

Co-authored-by: Marco Neumann <marco@crepererum.net>
 2022-11-09 11:25:13 +00:00
Carol (Nichols || Goulding) 88c3a1f5e7
feat: Use workspace dep inheritance for the arrow-flight crate 2022-10-26 10:34:54 -04:00
Carol (Nichols || Goulding) 3145e2c05b
feat: Use workspace dep inheritance for the arrow crate 2022-10-26 10:34:29 -04:00
Carol (Nichols || Goulding) 44936f661a
feat: Use workspace dep inheritance for datafusion instead of shim crate 2022-10-26 10:33:56 -04:00
Carol (Nichols || Goulding) 2e83e04eab
feat: Use workspace package metadata to reduce differences and repetition 2022-10-24 13:04:09 -04:00
dependabot[bot] bebb15d30f
chore(deps): Bump serde_json from 1.0.86 to 1.0.87 
Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.86 to 1.0.87.
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/v1.0.86...v1.0.87)

---
updated-dependencies:
- dependency-name: serde_json
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2022-10-20 07:52:33 +00:00
Andrew Lamb d706f8221d
chore: Update datafusion and arrow / parquet / arrow-flight 25.0.0 (#5900) 
* chore: Update datafusion and  `arrow` / `parquet` / `arrow-flight` 25.0.0

* chore: Update for structure changes

* chore: Update for new projection pushdown

* chore: Run cargo hakari tasks

* fix: fmt

Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2022-10-18 20:58:47 +00:00
Andrew Lamb e16b04891e
fix: improve tonic status codes returned for query errors (#5864) 
* fix: improve tonic status codes returned for query errors

* fix: update tests

* fix: Add rationale for categorization and avoid recurson handling DataFusionError::Context

* fix: Apply suggestions from code review

Thanks @carols10cents!

Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>

* fix: Update tests

* fix: add future TOOD

Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2022-10-18 10:47:52 +00:00
Andrew Lamb d78591f52e
feat: add more logging to SQL execution via flight (#5877) 
* refactor: Extract do_get implementation into its own function

* feat: add more logging to SQL execution

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2022-10-18 10:36:11 +00:00
Andrew Lamb 8021b8be0b
fix: Use Display rather than Debug when logging errors (#5859) 
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2022-10-14 14:43:11 +00:00
Andrew Lamb d57c99638c
chore: Update datafusion + `arrow`, `arrow-flight`, and `parquet` to 24.0.0.0 (#5792) 
* chore: Update datafusion + `arrow`, `arrow-flight`, and `parquet` to 24.0.0.0

* fix: Update for coercion, fix explain plans for change in column name display

* chore: Update datafusion lock

* fix: Update for other API changes

* chore: Update to latest datafusion pin

* chore: Run cargo hakari tasks

Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2022-10-12 16:19:14 +00:00
dependabot[bot] 2277fcf08a
chore(deps): Bump serde_json from 1.0.85 to 1.0.86 
Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.85 to 1.0.86.
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/v1.0.85...v1.0.86)

---
updated-dependencies:
- dependency-name: serde_json
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2022-10-10 01:42:37 +00:00
Marco Neumann c4c83e0840
fix: query error propagation (#5801) 
- treat OOM protection as "resource exhausted"
- use `DataFusionError` in more places instead of opaque `Box<dyn Error>`
- improve conversion from/into `DataFusionError` to preserve more
  semantics

Overall, this improves our error handling. DF can now return errors like
"resource exhausted" and gRPC should now automatically generate a
sensible status code for it.

Fixes #5799.
 2022-10-06 08:54:01 +00:00
Andrew Lamb 66dbb9541f
chore: Update datafusion and `arrow`/`parquet`/`arrow-flight` to 23.0.0, `thrift` to 0.16.0 (#5694) 
* chore: Update datafusion and `arrow`/`parquet`/`arrow-flight`  to 23.0.0

* chore: Update thrift / remove parquet_format

* fix: Update APIs

* chore: Update lock + Run cargo hakari tasks

* fix: use patched version of arrow-rs to work around https://github.com/apache/arrow-rs/issues/2779

* chore: Run cargo hakari tasks

Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2022-09-27 12:50:54 +00:00
Andrew Lamb 1fd31ee3bf
chore: Update datafusion / `arrow` / `arrow-flight` / `parquet` to version 22.0.0 (#5591) 
* chore: Update datafusion / `arrow` / `arrow-flight` / `parquet` to version 22.0.0

* fix: enable dynamic comparison flag

* chore: derive Eq for clippy

* chore: update explain plans

* chore: Update sizes for ReadBuffer encoding

* chore: update more tests

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2022-09-12 17:45:03 +00:00
Marco Neumann adeacf416c
ci: fix (#5569) 
* ci: use same feature set in `build_dev` and `build_release`

* ci: also enable unstable tokio for `build_dev`

* chore: update tokio to 1.21 (to fix console-subscriber 0.1.8

* fix: "must use"
 2022-09-06 14:13:28 +00:00
Andrew Lamb 6669d85fb4
chore: Update datafusion + arrow/parquet to `21.0.0` (#5519) 
* chore: Update arrow/arrow-flight/parquet to 21.0.0

* chore: Update datafusion pin

* chore: Fix arrow update script

* chore: Update Cargo.lock

* chore: Update for new API
 2022-08-31 13:30:47 +00:00