Influxdata
/
influxdb
mirror of https://github.com/influxdata/influxdb.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
10,650 Commits
189 Branches
453 Tags
379 MiB
Commit Graph

413 Commits (6d5cfba76fae2b640b2f658dbd46994701364eda)

Author SHA1 Message Date
Andrew Lamb c3bc61f10e
refactor: Move `flightsql` code into its own module, add docs and tests (#6640) 
* refactor: Move `flightsql`  code into its own module

* fix: get schema from LogicalPlan

* refactor: use arrow_flight::sql::Any instead of prost_types::any

* fix: cleanup docs and avoid as_ref

* fix: Use Bytes

* fix: use Any::pack

* fix: doclink
 2023-01-24 18:24:32 +00:00
Andrew Lamb 9a61f36a53
chore: Update datafusion again (#6656) 
* chore: Update datafusion pin

* chore: Run cargo hakari tasks

Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
 2023-01-23 12:13:07 +00:00
Andrew Lamb b09691dc6b
chore: Upgrade datafusion (again, I know) (#6639) 
* chore: Update datafusion

* chore: Run cargo hakari tasks

Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2023-01-23 11:24:22 +00:00
kodiakhq[bot] 24ca1e6f8c
Merge branch 'main' into cn/test-old-with-new 2023-01-20 15:20:40 +00:00
Andrew Lamb d808c57cdc
chore: Remove `iox_arrow_flight` (#6621) 
* chore: Remove iox_arrow_flight

* fix: hack around tonic status errors

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2023-01-20 11:16:19 +00:00
Carol (Nichols || Goulding) 6afd782b3f
fix: Move query_tests2 into influxdb_iox/tests so that the code rebuilds 2023-01-19 16:44:31 -05:00
Andrew Lamb f639bf3e23
chore: refactor ingester to use upstream arrow-flight (#6622) 
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2023-01-19 15:16:13 +00:00
Andrew Lamb 6a4fa11dba
chore: Update to latest datafusion / arrow `31.0.0` (#6618) 
* chore: Update datafusion and arrow/parquet/arrow-flight `31.0.0`

* chore: Update for new API

* chore: Run cargo hakari tasks

Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2023-01-18 20:21:10 +00:00
kodiakhq[bot] 33168b97f0
Merge branch 'main' into cn/query-tests-grpc 2023-01-18 19:03:51 +00:00
Marco Neumann e72173d58d
feat: very basic compactor2 skeleton (#6614) 
Sets up crate and wires up the main binary. No tests yet, no algorithm
framework, just the bare minimum.

Also I decided to not offer a gRPC server in `compactor2` at the moment
and hence did not implement any handle/delegate infrastructure. We add
this later if we need it. This also means compactor2 does NOT provide a
catalog service for now.

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2023-01-18 16:36:40 +00:00
Andrew Lamb 8410998408
chore: Update datafusion to Jan 17, 2023 (2 / 2) and arrow/parquet `30.0.1` (#6604) 
* chore: Update datafusion to Jan 9, 2023 (2 / 2) and arrow/parquet `30.0.1`

* chore: Update for changes in arrow ipc

* chore: Run cargo hakari tasks

Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
 2023-01-18 15:51:24 +00:00
Carol (Nichols || Goulding) f3b5dcaab7
feat: Reimagining query_tests 2023-01-18 10:24:17 -05:00
Andrew Lamb 57f08dbccd
chore: Update datafusion to Jan 9, 2023 (1 / 2) (#6603) 
* refactor: Update DataFusion pin to early Jan 2023

* fix: Update tests now that planning is async

* fix: Updates for API changes

* chore: Run cargo hakari tasks

* fix: Update comment

* refactor: nicer config setup

* fix: gapfill async

Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
 2023-01-18 12:19:32 +00:00
dependabot[bot] cc6d8709c9
chore(deps): Bump hashbrown from 0.13.1 to 0.13.2 (#6574) 
Bumps [hashbrown](https://github.com/rust-lang/hashbrown) from 0.13.1 to 0.13.2.
- [Release notes](https://github.com/rust-lang/hashbrown/releases)
- [Changelog](https://github.com/rust-lang/hashbrown/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/hashbrown/compare/v0.13.1...v0.13.2)

---
updated-dependencies:
- dependency-name: hashbrown
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Dom <dom@itsallbroken.com>
 2023-01-12 09:47:15 +00:00
Andrew Lamb dbe52f1ca1
chore: Upgrade datafusion (#6467) 
* chore: Update datafusion

* fix: Update for new apis

* chore: Update expected plan

* fix: Update for new config construction

* chore: update clippy

* fix: Fix error codes

* fix: update another test

* chore: Run cargo hakari tasks

Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2023-01-03 15:29:11 +00:00
Andrew Lamb d0d5906476
chore: Update datafusion pin (#6442) 
* chore: Update datafusion pin

* refactor: Update iox_query for new apis

* chore: Update some more apis

* chore: Run cargo hakari tasks

Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
 2022-12-19 20:02:42 +00:00
Andrew Lamb 94c2f94ea1
refactor: Extract common ArrowFlight client into iox_arrow_flight (#6427) 
* refactor: Extract common ArrowFlight client into iox_arrow_flight

* chore: Run cargo hakari tasks

* fix: clarify intent of iox_arrow_flight crate

* refactor: Apply suggestions from code review

Co-authored-by: Raphael Taylor-Davies <1781103+tustvold@users.noreply.github.com>

* fix: loop --> while let

* fix: REmove make_tonic_error in favor of From impl

Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: Raphael Taylor-Davies <1781103+tustvold@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2022-12-19 11:35:20 +00:00
Andrew Lamb be45889be1
chore: Upgrade datafusion (#6407) 
* chore: Update datafusion

* chore: Update for API change
 2022-12-15 06:51:35 +00:00
Andrew Lamb 8729977851
chore: Upgrade datafusion / arrow to 29.0.0 to get flightsql client (#6396) 
* chore: Update datafusion pin

* chore: Update for API change

* chore: Run cargo hakari tasks

Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
 2022-12-13 20:16:09 +00:00
Andrew Lamb 336ca761a3
chore: Update datafusion pin (to get sqlparser update) (#6378) 
* chore: Update datafusion pin

* chore: Run cargo hakari tasks

Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
 2022-12-12 14:39:42 +00:00
Andrew Lamb e0ecacf6cc
chore: Update DataFusion (get median fix and automatic string to timestamp coercion) (#6363) 
* chore: Update DataFusion pin to get median fix

* chore: Update for new Expr node

* test: add test for median

* test: add test for coercion of strings to timestamps

* chore: Run cargo hakari tasks

Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2022-12-12 12:14:00 +00:00
Raphael Taylor-Davies 061d582a9b
chore: patch object_store to get apache#3274 (#6362) 
* chore: patch object_store to get apache#3274

* chore: Run cargo hakari tasks

* fix: add issue breadcrumb

Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: Andrew Lamb <alamb@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2022-12-09 14:52:25 +00:00
Andrew Lamb 9175f4a0b5
chore: Upgrade datafusion to get correct support for multi-part identifiers (#6349) 
* test: add tests for periods in measurement names

* chore: Update Datafusion

* chore: Update for changed APIs

* chore: Update expected plan output

* chore: Run cargo hakari tasks

Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2022-12-08 11:27:13 +00:00
kodiakhq[bot] 9e3d0fcefb
Merge branch 'main' into cn/ingester2 2022-12-02 13:39:55 +00:00
Andrew Lamb d0f1f6a4fd
chore: Upgrade datafusion to get memory limits (#6297) 
* chore: Update datafusion

* fix: use correctly qualified column names

* chore: Run cargo hakari tasks

Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2022-12-01 16:40:26 +00:00
Carol (Nichols || Goulding) 99698d0f46
feat: Create an ioxd_ingester2 crate 2022-12-01 11:37:21 -05:00
Andrew Lamb 14a9bc92e9
Revert "Revert "chore: Update Datafusion and arrow/arrow-flight/parquet to `28.0.0` (#6279)" (#6294)" (#6296) 
This reverts commit b7e52c0d8d.

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2022-12-01 14:20:43 +00:00
Andrew Lamb b7e52c0d8d
Revert "chore: Update Datafusion and arrow/arrow-flight/parquet to `28.0.0` (#6279)" (#6294) 
This reverts commit 039a45ddd1.
 2022-12-01 11:38:42 +00:00
Andrew Lamb 039a45ddd1
chore: Update Datafusion and arrow/arrow-flight/parquet to `28.0.0` (#6279) 
* chore: Update Datafusion and arrow/arrow-flight/parquet to `28.0.0`

* chore: Update thrift to 0.17

* fix: use workspace arrow-flight in ingester2

* chore: Update for API changes

* fix: test

* chore: Update hakari

* chore: Update hakari again

* chore: Update trace_exporters to latest thrift

* fix: update test

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2022-11-30 14:12:30 +00:00
Andrew Lamb f22b1e1a09
chore: Update datafusion (to get memory limiting code) (#6246) 
* chore: Update datafusion

* chore: Run cargo hakari tasks

* fix: Update to newer api

Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2022-11-29 12:05:17 +00:00
Andrew Lamb fc5697b8e7
chore: Update datafusion again (N of N) (#6218) 
* chore: Update datafusion again (4 of N)

* fix: Update plans

* fix: Update for renamed API

* fix: Update more plans

* chore: Update to datafusion @ d355f69aae2cc951cfd021e5c0b690861ba0c4ac

* fix: update explain plan tests

* fix: update test after schema error

* chore: Update datafusion again

* fix: Add size() calculation to selectors

* chore: Run cargo hakari tasks

* fix: Update newly added test

Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2022-11-28 17:09:40 +00:00
Dom Dwyer a66fc0b645
feat(ingester): ingester2 init 
Adds an ingester2 crate to hold the MVP of the Kafkaless project.

This was necessary due to the tight coupling of the ingester internals
with tests in external crates, and eases the parallel development of two
version of the ingester.

This commit contains various changes from the "ingester" crate, mostly
removing the concept/references to a "shard" or "ShardId" where
possible.

This commit does not copy over all of the "ingester" crate - only those
components that are definitely needed. I will drag across more as
functionality is implemented.
 2022-11-24 15:34:02 +01:00
Paul Dix db63dc55a2
feat: Initial WAL implementation 2022-11-23 17:07:43 -05:00
Andrew Lamb 71b29ae442
chore: Update DataFusion (3 of N) and arrow/parquet to 27.0.0 (#6208) 
* chore: Update DataFusion and arrow/parquet to 27.0.0

* chore: Run cargo hakari tasks

Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
 2022-11-23 14:21:47 +00:00
Andrew Lamb 9fb1de0428
chore: Update datafusion (2 of N) right before arrow 27 upgrade (#6207) 
* chore: Update datafusion (2 of N) right before arrow 27 upgrade

* fix: Update tests for better unsigned pushdown

* chore: Run cargo hakari tasks

Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
 2022-11-23 11:04:14 +00:00
Andrew Lamb 1a1ea74cb7
chore: Upgrade datafusion again (#6160) 
* Revert "Revert "chore: Update datafusion again (#6108)""

This reverts commit 766b3bbeb440618cfe332f6ee7d4f8a8217acc48.

* fix: Respect the partition sort key

* chore: update plans

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2022-11-22 19:28:26 +00:00
Andrew Lamb 67712b595c
Revert "chore: Update datafusion again (#6108)" (#6159) 
This reverts commit fbe9f27f10.
 2022-11-16 21:14:55 +00:00
Luke Bond 9365d933f1
chore: router namespace api (#6151) 
* chore: move ns api from querier to router

* chore: add explanatory comment in querier about moved namespace API

* fix: add namespace service to router

* fix: querier returns unimplemented error for ns retention, not panic

* chore: reuse namespace -> proto in router ns api

* chore: grpc namespace - consume ns to avoid clone

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2022-11-16 15:25:49 +00:00
Andrew Lamb fbe9f27f10
chore: Update datafusion again (#6108) 
* chore: Update datafusion pin + api code

* chore: Run cargo hakari tasks

* refactor: combine_sort_key is more idomatic and add rationale comments

* refactor: satisfy borrow checker and updated comments

* fix: Add test case for combine_sort_key

* fix:  Apply suggestions from code review

Co-authored-by: Marco Neumann <marco@crepererum.net>

* fix: Add back test for deeply nested expression

* fix: Update output ordering

Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: Marco Neumann <marco@crepererum.net>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2022-11-16 14:41:52 +00:00
Jake Goulding cc17e5a54b refactor: use a workspace dependency for hashbrown 2022-11-11 13:25:39 -05:00
Andrew Lamb 034d9b371d
chore: Update datafusion and arrow/arrow-flight/parquet to `26.0.0` (#6061) 
* chore: Update datafusion and arrow/arrow-flight/parquet to `26.0.0`

* fix: Update query_functions

* fix: update for TimestampNanosecondArray API changes

* fix: update for TimestampNanosecondArray API changes

* chore: Update flatbuffers and remove rustsec warning

* chore: Update text

* fix: update more test

* fix: Lock ahash to exactly 0.8.0

* fix: Update datafusion pin

* chore: Run cargo hakari tasks

Co-authored-by: Carol (Nichols || Goulding) <carol.nichols@gmail.com>
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2022-11-07 11:01:58 +00:00
Andrew Lamb 8c8e607dca
chore: Update datafusion pin (#6054) 
* chore: Update datafusion pin

* chore: Run cargo hakari tasks

* chore: Update expected error

Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
 2022-11-03 19:41:31 +00:00
Andrew Lamb e7e9945207
chore: Update datafusion pin (#6017) 
* chore: Update datafusion pin

* chore: Run cargo hakari tasks

Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2022-11-02 11:12:21 +00:00
Andrew Lamb 00953460fb
chore: Update datafusion pin (#6010) 2022-10-31 17:14:56 +00:00
Andrew Lamb ace3c11f12
chore: Update datafusion (#6004) 
* chore: Update datafusion

* chore: change path

* chore: Run cargo hakari tasks

Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2022-10-31 16:16:28 +00:00
Carol (Nichols || Goulding) b27e2bd7d1
feat: Use workspace dep inheritance for the parquet crate 2022-10-26 10:37:51 -04:00
Carol (Nichols || Goulding) 88c3a1f5e7
feat: Use workspace dep inheritance for the arrow-flight crate 2022-10-26 10:34:54 -04:00
Carol (Nichols || Goulding) 3145e2c05b
feat: Use workspace dep inheritance for the arrow crate 2022-10-26 10:34:29 -04:00
Carol (Nichols || Goulding) 44936f661a
feat: Use workspace dep inheritance for datafusion instead of shim crate 2022-10-26 10:33:56 -04:00
Marco Neumann 99f36f8889
chore: unpatch clap V3 (#5971) 
Upstream issue was solved by a new release.

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2022-10-25 09:16:16 +00:00