Influxdata
/
influxdb
mirror of https://github.com/influxdata/influxdb.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
11,658 Commits
212 Branches
453 Tags
355 MiB
Commit Graph

443 Commits (73d44ec9a1b06118107e419a3301c3da1202e90d)

Author SHA1 Message Date
Carol (Nichols || Goulding) 6c2ce01f1e
fix: Remove old ingester and ioxd_ingester 2023-04-07 11:06:37 -04:00
Carol (Nichols || Goulding) a5769a461b
fix: Remove old compactor and ioxd_compactor 2023-04-06 16:49:52 -04:00
Andrew Lamb e8b7d69b0f
chore: Update datafusion again (#7442) 
* chore: Update datafusion

* chore: Fix up plans for datafusion API change

* chore: Run cargo hakari tasks

---------

Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2023-04-05 18:21:53 +00:00
Andrew Lamb badc8865ef
chore: Update datafusion again (#7440) 
* chore: Update DataFusion

* chore: Update for new API

* chore: Run cargo hakari tasks

* fix: cargo doc

---------

Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
 2023-04-04 15:45:46 +00:00
Andrew Lamb d8b0139ea9
chore: Update datafusion + arrow/parquet/arrow-flight to 36 (#7354) 
* chore: Update datafusion +  arrow/parquet/arrow-flight to 36

* refactor: update optimize for new API

* refactor: update parquet for new API

* chore: Update more dependencies

* chore: Update to use the new buffer creation APIs

* chore: Run cargo hakari tasks

* fix: bad len

* fix: update for API change

---------

Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2023-03-29 13:41:59 +00:00
Andrew Lamb 43e236e040
chore: Update datafusion again (#7353) 
* chore: Update DataFusion

* refactor: Update predicate crate for new transform API

* refactor: Update iox_query crate for new APIs

* refactor: Update influxql for new API

* chore: Run cargo hakari tasks

---------

Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2023-03-28 16:21:49 +00:00
Andrew Lamb 1817104236
chore: Update DataFusion again (#7329) 
* chore: update datafusion

* chore: Run cargo hakari tasks

---------

Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2023-03-24 18:45:53 +00:00
Andrew Lamb 5dd71998a1
chore: Update datafusion (#7318) 
* chore: Update datafusion

* chore: Update for API change

* chore: Run cargo hakari tasks

---------

Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2023-03-24 15:07:23 +00:00
Marko Mikulicic 6660b86f6c fix: Put authorization header in request extension 2023-03-22 04:12:07 +01:00
Andrew Lamb 8ab553180e
chore: Update datafusion again (#7276) 
* chore: Update DataFusion pin

* chore: Update for new Expr type

* fix: deprecated

* chore: Run cargo hakari tasks

* docs: Update predicate/src/lib.rs

* fix: fmt

---------

Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2023-03-21 18:25:04 +00:00
Andrew Lamb 96c2094302
refactor(iox_query): extract influxrpc planner to its own crate (#7241) 
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2023-03-17 18:48:55 +00:00
Martin Hilton 13657d5bcc
feat(authz): authorization service client and write integration (#7216) 
* feat(authz): add authorization client.

Add a new authz crate to provide the interface for making authorization
checks from within IOx. This includes the default client that uses
the influxdata.iox.authz.v1 gRPC protocol. This feature is not used
by any IOx component yet.

* feat: optional authorization on write path

Support optionally enabling authorization checks on the /api/v2/write
handler. If an authrorizer is configured then the handler will
attempt to retrieve a token from the request's Authorization header.
If no such token exists then a response with a 401 error code is
returned. If the token is not valid, or does not have write permission
for the requested namespace then a response with a 403 error is
returned.

* chore: add unit test for authz in write handler

Add unit tests that test the correct functioning of the /api/v2/write
handler when an Authorizer is configured.

* chore(authz): use lazy connection

Change the initialization of the authz client to use a lazy connection.
This allows the client to be initialised synchronously.

* chore: Run cargo hakari tasks

* fix(authz): protolint complaints

* fix: authz tests

* fix: benches and lint

* chore: Update clap_blocks/src/authz.rs

Co-authored-by: Marko Mikulicic <mkm@influxdata.com>

* chore: Update authz/src/lib.rs

Co-authored-by: Marko Mikulicic <mkm@influxdata.com>

* chore: Update clap_blocks/src/authz.rs

Co-authored-by: Marko Mikulicic <mkm@influxdata.com>

* chore: review suggestions

* chore: review suggestions

Apply a number of suggestions from review comments. The main
behavioural change is that if the authz service is configured
applictions will perform a probe request to ensure it can communicate
before continuing startup.

* chore: Update router/src/server/http.rs

Co-authored-by: Dom <dom@itsallbroken.com>

---------

Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: Marko Mikulicic <mkm@influxdata.com>
Co-authored-by: Dom <dom@itsallbroken.com>
 2023-03-17 15:20:14 +00:00
Andrew Lamb 7dfaa05e8a
chore: Update datafusion again (#7208) 
* chore: update datafusion again

* fix: update test

* fix: use table_reference

* fix: clean up import

* chore: Run cargo hakari tasks

---------

Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2023-03-16 14:34:40 +00:00
Andrew Lamb 35bd5ccd13
chore: Update datafusion pin (#7207) 
* chore: Update datafusion pin

* chore: Run cargo hakari tasks

---------

Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2023-03-15 11:35:15 +00:00
Andrew Lamb 0eb858c70d
chore: Update datafusion (#7167) 
* chore: Update datafusion

* chore: Update datafusion

* refactor: use UserDefinedLogicalNodeCore

* fix: remove stray comment

* fix: clippy

* chore: Run cargo hakari tasks

---------

Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2023-03-13 16:41:32 +00:00
Stuart Carnie fe48a685ec
refactor: Move InfluxQL behaviour from iox_query to new crate (#7156) 
* refactor: Break unnecessary dependencies from `iox_query` crate

In the process, the test code has been simplified.

* refactor: Move InfluxQL plan module to iox_query_influxql crate

* refactor: Move remaining behaviour from iox_query to iox_query_influxql

* chore: rustfmt 🙄

I was under the impression `clippy` would catch formatting
 2023-03-08 22:29:20 +00:00
Marco Neumann 3828d2a50e
chore: update DataFusion to `deeaa5632ed99a58b91767261570756db736d158` (#7158) 
* chore: update DataFusion to `deeaa5632ed99a58b91767261570756db736d158`

I want to get pull:

- https://github.com/apache/arrow-datafusion/pull/5495

Changes in the IOx code base due to:

- https://github.com/apache/arrow-datafusion/pull/5423
- https://github.com/apache/arrow-datafusion/pull/5421
- https://github.com/apache/arrow-datafusion/pull/5450

* refactor: simplify expression simplifcation

Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>

* refactor: remove upstreamed code

* test: update snapshots

---------

Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
 2023-03-08 13:05:31 +00:00
Andrew Lamb ed0704ac8d
chore: Update datafusion (#7100) 
* chore: Update datafusion

* chore: iox_query to compile for API changes + update tests

* chore: Run cargo hakari tasks

---------

Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
 2023-03-06 17:59:24 +00:00
Andrew Lamb e19ce98407
chore: Update datafusion + arrow/arrow-flight/parquet to 34.0.0 (#7084) 
* chore: Update datafusion + arrow/arrow-flight/parquet to 34.0.0

* chore: Run cargo hakari tasks

* chore: Update plans

* chore: Update querier expected output

* chore: Update querier tests to use insta

* fix: sort output too

---------

Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2023-03-01 11:25:01 +00:00
Andrew Lamb f93baf7693
chore: Update DataFusion and `arrow` / `arrow-flight` / `parquet` to `33.0.0` (#7045) 
* chore: Update DataFusion and arrow/arrow-flight/parquet to 33.0.0

* fix: Update test output

* fix: update more test output

* fix: Update querier test output

* chore: Run cargo hakari tasks

* test: fix formatting

Fix formatting of batch pretty printing.

* test: fix formatting

Fix formatting of batch pretty printing.

* test: fix formatting for selector tests

---------

Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: Dom Dwyer <dom@itsallbroken.com>
Co-authored-by: Christopher Wolff <chris.wolff@influxdata.com>
 2023-02-22 21:24:20 +00:00
Andrew Lamb 27890b313f
chore: Update datafusion (#6997) 
* chore: Update datafusion

* chore: update the plans

* fix: update some plans

* chore: Update plans and port some explain plans to use insta snapshots

* fix: another plan

* chore: Run cargo hakari tasks

---------

Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2023-02-16 17:03:25 +00:00
Andrew Lamb 4d7aa1e48b
refactor: extract compactor2 test utils into `compactor2_test_utils` and integration test (#6960) 
* refactor: extract compactor2 test utils into `compactor2_test_utils` and integration test

* fix: Update compactor2/src/components/mod.rs

Co-authored-by: Marco Neumann <marco@crepererum.net>

---------

Co-authored-by: Marco Neumann <marco@crepererum.net>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2023-02-13 12:06:42 +00:00
Andrew Lamb 143cb26e87
chore: Update datafusion again (#6958) 
* chore: Update datafusion pin

* chore: Update plans

* chore: Run cargo hakari tasks

---------

Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2023-02-13 10:28:44 +00:00
Andrew Lamb 501032c2fb
chore: Update datafusion again (#6929) 
* chore: Update DataFusion

* fix: disable repartitioned sorting

* chore: Run cargo hakari tasks

---------

Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2023-02-10 11:51:47 +00:00
Andrew Lamb 2f4d901fbe
chore: Update datafusion (#6893) 
* chore: Update datafusion

* chore: Run cargo hakari tasks

* fix: update for api change

---------

Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2023-02-08 18:27:18 +00:00
Andrew Lamb ae10b33f0c
chore: Update datafusion (#6868) 
* chore: Update datafusion

* chore: Run cargo hakari tasks

* fix: Update error message

---------

Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
 2023-02-06 14:22:45 +00:00
Raphael Taylor-Davies d3601a59f8
chore: update DataFusion, upgrade `arrow` `arrow-flight` and `parquet` to `32.0.0` (#6756) 
* chore: update DataFusion

* fix: test

* chore: format

* chore: clippy

* chore: update arrow

* chore: arrow upgrade fallout

* chore: Run cargo hakari tasks

* chore: remove failing warm compaction test

* fix: flight error propagation

* chore: update parquet size

* fix: Update error message

* chore: Update parquet metadata test

---------

Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: Andrew Lamb <alamb@influxdata.com>
Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2023-02-06 11:35:39 +00:00
Carol (Nichols || Goulding) b55e4ffa21
test: Port table_schema tests to end-to-end and get rid of query_tests (#6828) 
* test: Port table_schema query_tests to end-to-end

* fix: Remove now outdated query_tests crate

---------

Co-authored-by: Andrew Lamb <alamb@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2023-02-03 17:41:08 +00:00
Andrew Lamb 5b14caa780
chore: Update DataFusion (#6753) 
* chore: Update datafusion

* fix: Update for changes

* chore: Run cargo hakari tasks

---------

Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2023-01-30 14:48:52 +00:00
Andrew Lamb 0d32662eea
chore: Update datafusion again (#6722) 
* chore: Update datafusion

* fix: Update for API

* chore: Run cargo hakari tasks

---------

Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2023-01-27 18:59:27 +00:00
Andrew Lamb c3bc61f10e
refactor: Move `flightsql` code into its own module, add docs and tests (#6640) 
* refactor: Move `flightsql`  code into its own module

* fix: get schema from LogicalPlan

* refactor: use arrow_flight::sql::Any instead of prost_types::any

* fix: cleanup docs and avoid as_ref

* fix: Use Bytes

* fix: use Any::pack

* fix: doclink
 2023-01-24 18:24:32 +00:00
Andrew Lamb 9a61f36a53
chore: Update datafusion again (#6656) 
* chore: Update datafusion pin

* chore: Run cargo hakari tasks

Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
 2023-01-23 12:13:07 +00:00
Andrew Lamb b09691dc6b
chore: Upgrade datafusion (again, I know) (#6639) 
* chore: Update datafusion

* chore: Run cargo hakari tasks

Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2023-01-23 11:24:22 +00:00
kodiakhq[bot] 24ca1e6f8c
Merge branch 'main' into cn/test-old-with-new 2023-01-20 15:20:40 +00:00
Andrew Lamb d808c57cdc
chore: Remove `iox_arrow_flight` (#6621) 
* chore: Remove iox_arrow_flight

* fix: hack around tonic status errors

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2023-01-20 11:16:19 +00:00
Carol (Nichols || Goulding) 6afd782b3f
fix: Move query_tests2 into influxdb_iox/tests so that the code rebuilds 2023-01-19 16:44:31 -05:00
Andrew Lamb f639bf3e23
chore: refactor ingester to use upstream arrow-flight (#6622) 
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2023-01-19 15:16:13 +00:00
Andrew Lamb 6a4fa11dba
chore: Update to latest datafusion / arrow `31.0.0` (#6618) 
* chore: Update datafusion and arrow/parquet/arrow-flight `31.0.0`

* chore: Update for new API

* chore: Run cargo hakari tasks

Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2023-01-18 20:21:10 +00:00
kodiakhq[bot] 33168b97f0
Merge branch 'main' into cn/query-tests-grpc 2023-01-18 19:03:51 +00:00
Marco Neumann e72173d58d
feat: very basic compactor2 skeleton (#6614) 
Sets up crate and wires up the main binary. No tests yet, no algorithm
framework, just the bare minimum.

Also I decided to not offer a gRPC server in `compactor2` at the moment
and hence did not implement any handle/delegate infrastructure. We add
this later if we need it. This also means compactor2 does NOT provide a
catalog service for now.

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2023-01-18 16:36:40 +00:00
Andrew Lamb 8410998408
chore: Update datafusion to Jan 17, 2023 (2 / 2) and arrow/parquet `30.0.1` (#6604) 
* chore: Update datafusion to Jan 9, 2023 (2 / 2) and arrow/parquet `30.0.1`

* chore: Update for changes in arrow ipc

* chore: Run cargo hakari tasks

Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
 2023-01-18 15:51:24 +00:00
Carol (Nichols || Goulding) f3b5dcaab7
feat: Reimagining query_tests 2023-01-18 10:24:17 -05:00
Andrew Lamb 57f08dbccd
chore: Update datafusion to Jan 9, 2023 (1 / 2) (#6603) 
* refactor: Update DataFusion pin to early Jan 2023

* fix: Update tests now that planning is async

* fix: Updates for API changes

* chore: Run cargo hakari tasks

* fix: Update comment

* refactor: nicer config setup

* fix: gapfill async

Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
 2023-01-18 12:19:32 +00:00
dependabot[bot] cc6d8709c9
chore(deps): Bump hashbrown from 0.13.1 to 0.13.2 (#6574) 
Bumps [hashbrown](https://github.com/rust-lang/hashbrown) from 0.13.1 to 0.13.2.
- [Release notes](https://github.com/rust-lang/hashbrown/releases)
- [Changelog](https://github.com/rust-lang/hashbrown/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/hashbrown/compare/v0.13.1...v0.13.2)

---
updated-dependencies:
- dependency-name: hashbrown
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Dom <dom@itsallbroken.com>
 2023-01-12 09:47:15 +00:00
Andrew Lamb dbe52f1ca1
chore: Upgrade datafusion (#6467) 
* chore: Update datafusion

* fix: Update for new apis

* chore: Update expected plan

* fix: Update for new config construction

* chore: update clippy

* fix: Fix error codes

* fix: update another test

* chore: Run cargo hakari tasks

Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2023-01-03 15:29:11 +00:00
Andrew Lamb d0d5906476
chore: Update datafusion pin (#6442) 
* chore: Update datafusion pin

* refactor: Update iox_query for new apis

* chore: Update some more apis

* chore: Run cargo hakari tasks

Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
 2022-12-19 20:02:42 +00:00
Andrew Lamb 94c2f94ea1
refactor: Extract common ArrowFlight client into iox_arrow_flight (#6427) 
* refactor: Extract common ArrowFlight client into iox_arrow_flight

* chore: Run cargo hakari tasks

* fix: clarify intent of iox_arrow_flight crate

* refactor: Apply suggestions from code review

Co-authored-by: Raphael Taylor-Davies <1781103+tustvold@users.noreply.github.com>

* fix: loop --> while let

* fix: REmove make_tonic_error in favor of From impl

Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: Raphael Taylor-Davies <1781103+tustvold@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2022-12-19 11:35:20 +00:00
Andrew Lamb be45889be1
chore: Upgrade datafusion (#6407) 
* chore: Update datafusion

* chore: Update for API change
 2022-12-15 06:51:35 +00:00
Andrew Lamb 8729977851
chore: Upgrade datafusion / arrow to 29.0.0 to get flightsql client (#6396) 
* chore: Update datafusion pin

* chore: Update for API change

* chore: Run cargo hakari tasks

Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
 2022-12-13 20:16:09 +00:00
Andrew Lamb 336ca761a3
chore: Update datafusion pin (to get sqlparser update) (#6378) 
* chore: Update datafusion pin

* chore: Run cargo hakari tasks

Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
 2022-12-12 14:39:42 +00:00