Commit Graph

154 Commits (f9cc6dcc443f1e1e92472e326b29080e697e75ac)

Author SHA1 Message Date
Marco Neumann 9730c52c8a
fix: InfluxQL output sorting (#7633)
* test: remove #7513 workarounds from `issue_6112.influxql`

* fix: `SHOW TAG VALUES` output sorting

* fix: `SHOW MEASUREMENTS` output sorting

* fix: `SHOW TAG KEYS` output sorting
2023-04-24 13:37:25 +00:00
Stuart Carnie 0cc6147d6f
fix: Need to handle Regex operators
Ignoring LIKE for now as they aren't used by the InfluxQL planner
2023-04-21 13:39:34 +10:00
Stuart Carnie 7a49c3484e
feat: Documentation to describe time range behaviour 2023-04-21 09:31:54 +10:00
Stuart Carnie 79fcb9bfb9
chore: Use idiomatic initialisation
Co-authored-by: Christopher M. Wolff <chris.wolff@influxdata.com>
2023-04-21 09:25:14 +10:00
Stuart Carnie 9416fcfb88
Merge branch 'main' into sgc/issue/time_range_7610 2023-04-21 07:53:34 +10:00
Stuart Carnie b2279abb59
feat: Rewrite time range expressions to apply to entire result set 2023-04-21 07:53:19 +10:00
Marco Neumann 370413b2d2
feat: `SHOW TAG KEYS ... WHERE <condition>` (#7601)
Ref https://github.com/influxdata/idpe/issues/17474 .
2023-04-20 09:16:35 +00:00
Marco Neumann 9eac89ebfb
fix: also include metadata in empty InfluxQL select results (#7604)
At least the schema metdata and the measurements column should be there.
2023-04-20 08:12:07 +00:00
Christopher M. Wolff 7a6862ee3a
refactor: let date_bin_gapfill allow omitted origin (#7595)
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-04-19 14:56:52 +00:00
Marco Neumann ce273b6006
feat: `SHOW MEASUREMENTS ... WHERE <condition>` (#7590)
Ref https://github.com/influxdata/idpe/issues/17474 .

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-04-19 08:59:54 +00:00
Marco Neumann 53abef0447
feat: `SHOW TAG VALUES ... WHERE <condition>` (#7589)
Ref https://github.com/influxdata/idpe/issues/17474 .

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-04-19 08:10:56 +00:00
Stuart Carnie fc9b89da44
feat: `SELECT DISTINCT` support 2023-04-18 16:51:02 +10:00
Stuart Carnie b2dfe30a42
chore: use `error` module 🧹 2023-04-18 16:50:48 +10:00
Stuart Carnie dbf7e73d3e
feat: COUNT(DISTINCT) 2023-04-18 16:50:24 +10:00
Stuart Carnie bf392eeb4c
feat: identify `SELECT DISTINCT <ident>` projections
These are special, so they have a new projection type
2023-04-18 16:49:11 +10:00
Stuart Carnie 8f78fe7b08
chore: rewrite `DISTINCT <ident>` to `Call` expression
This simplifies downstream processing of the AST if DISTINCT is
consistent.
2023-04-18 16:47:52 +10:00
Stuart Carnie c506d88381
chore: Use functions from `error` module 2023-04-18 06:24:24 +10:00
Stuart Carnie ad3e6aadea
Merge branch 'main' into sgc/issue/last_7538
# Conflicts:
#	iox_query_influxql/src/plan/planner.rs
2023-04-18 06:18:02 +10:00
Stuart Carnie 753f6aa85e
chore: Grammar 2023-04-18 06:02:06 +10:00
Marco Neumann 87ecdc5eaa
feat: InfluxQl `SHOW TAG KEYS` planner+exec (#7451)
Closes https://github.com/influxdata/idpe/issues/17363 .
2023-04-17 13:31:33 +00:00
Marco Neumann e7511c0f33
feat: basic InfluxQL `SHOW MEASUREMENTS` (#7504)
No WHERE-clauses hat result in actual data checks yet, just pure metdata
queries.

Ref https://github.com/influxdata/idpe/issues/17358 .
2023-04-17 11:28:01 +00:00
Marco Neumann bda9c07c0b
feat: basic InfluxQL `SHOW TAG KEYS` support (#7526)
- No `ON` clause
- No `WHERE` clause
- No time restriction yet
- No `FROM <db>.<retention>`

Ref https://github.com/influxdata/idpe/issues/17360 .

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-04-17 10:01:41 +00:00
Stuart Carnie 4ef8daf498
chore: clippy 🔨 2023-04-17 12:09:04 +10:00
Stuart Carnie 3f4e38aa78
refactor: tidy up the implementation of the function_to_df_expr function
* Also fixes an issue that `FILL(<number>)` would fail when using the
  `struct_selector_<name>` functions
2023-04-16 10:04:53 +10:00
Stuart Carnie 252da2b75f
chore: add tests for the selector functions 2023-04-16 08:42:37 +10:00
Stuart Carnie 8f5f3b2057
chore: clarify comment 2023-04-16 08:36:59 +10:00
Stuart Carnie acd6cff631
chore: validate single-selector with tags or fields is not implemented 2023-04-16 08:21:19 +10:00
Stuart Carnie 8274d584f5
chore: update all remaining code to use `error` and `error::map` module 2023-04-16 08:00:12 +10:00
Stuart Carnie 69d75745cc
feat: add limited `last`, `first`, `min` and `max` selector functions
Returns a `NotImplemented` error when attempting to execute a
selector query, which projects a single selector function and additional
tags or fields until #7533 is implemented.

Introduced `error` module to simplify error handling and ensure
consistency of error messages.
2023-04-16 07:59:28 +10:00
Stuart Carnie 03ea8ea2b8
feat: add `last` selector function
This does not complete the implementation, as we must still use the
timestamp of the `struct_selector_last` when the projection semantics
are selector
2023-04-15 13:54:42 +10:00
Stuart Carnie 007d5b90f3
chore: add APIs to find selector user-defined aggregate functions
This will be used to complete queries that have selector semantics,
meaning they project a single selector function and therefore
use the timestamp for the time column.
2023-04-15 13:54:42 +10:00
Stuart Carnie d11097cf18
chore: add APIs to find selector user-defined aggregate functions
This will be used to complete queries that have selector semantics,
meaning they project a single selector function and therefore
use the timestamp for the time column.
2023-04-15 13:54:42 +10:00
Stuart Carnie 42074e7a9d
chore: refactor and rename `validate_select`
This will be used to determine the semantics of the projection clause
2023-04-15 13:54:42 +10:00
Stuart Carnie 3529762726
chore: remove `println` 2023-04-15 13:54:42 +10:00
Andrew Lamb 3ebd07358b
chore: Update DataFusion pin, upgrade `date_bin` and `InfluxQL` to use `Interval(MonthDayNano)` (#7516)
* chore: Update datafusion

* chore: Update for change in PhysicalSortExpr

* refactor: Update date_bin_gapfill to take IntervalMonthDayNano, fix FlightSQL

* chore: Run cargo hakari tasks

---------

Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-04-13 10:43:32 +00:00
dependabot[bot] e811a69a1e
chore(deps): Bump serde_json from 1.0.95 to 1.0.96 (#7535)
Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.95 to 1.0.96.
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/v1.0.95...v1.0.96)

---
updated-dependencies:
- dependency-name: serde_json
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-04-13 10:07:32 +00:00
Stuart Carnie 5965cee179
feat: Update InfluxQL end-to-end test output (#7512)
* feat: Specialises test output formatting for each language

* Also fixes an error uncovered in the `write_columnar` when tag
  columns are `NULL`

Closes #7145

* chore: Run cargo hakari tasks

* chore: Add sorted output until #7513 is addressed

* chore: clippy 📋

* feat: Add `options` to `write_columnar`

* Added ability to configure border rendering, including removing
  borders. This helps avoid variable width issues with EXPLAIN output,
  which tends to vary and cause flaky test failures.

* chore: rustfmt 🧹

* chore: update expected output

* chore: clarify what "this" is

---------

Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
2023-04-13 09:58:12 +00:00
Marco Neumann c7197a289e
fix: gRPC error code for invalid InfluQL (#7519)
This was "internal". The mapping works like this: we take the
`DataFusionError` and call `find_root` which should traverse the
`External(...)` chain (even through Arrow) to find the last error that
is not within the Arrow/DataFusion land. This is then mapped by us.

`DataFusionError::External(...)` is no further inspected and mapped
straight to "internal". I think this if fine because in the end we're
mostly dealing w/ DataFusion stuff anyways.

I've slightly changed the error mapping in the planner to emit
`DataFusionError::Plan(...)` instead which we map to "invalid argument".
I think this is way better for the user.

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-04-12 16:00:38 +00:00
Marco Neumann e7fcb25904
feat: `SHOW FIELD KEYS` InfluxQL plan+exec (#7405)
Closes https://github.com/influxdata/idpe/issues/17362
2023-04-12 09:09:29 +00:00
Stuart Carnie 772b79b86b
feat: Validate the semantics of the projection clause of a SELECT (#7500)
* feat: Validate the semantics of the projection clause of a SELECT

Closes #7499

* chore: Complete sentences are helpful 😂
2023-04-11 22:43:23 +00:00
Stuart Carnie b629023f45
feat: Add `FILL(linear)` support (#7466)
* feat: Add FILL(linear) support

Closes #6916

* chore: Remove old test output

* chore: Fix `FILL(linear)` test
2023-04-08 21:34:30 +00:00
Stuart Carnie b9479a2b3c
refactor: `limit` function supports multiple measurements and tag sets (#7464)
* fix: Add sort operator after window aggregate operator

Closes #7460

* fix: Refactor `LIMIT` and `OFFSET` implementation

These changes should allow the `limit` function to be used
generically with any plan following the same conventions.

* chore: No need to reorder this

* chore: Add documentation to the `limit` function
2023-04-08 21:22:54 +00:00
Stuart Carnie 9f5fec42b8
feat: Teach InfluxQL how to plan `LIMIT` and `OFFSET` with `GROUP BY` (#7447)
* feat: Support LIMIT and OFFSET with GROUP BY

* fix: Compile error

* chore: Improve function name and comment

* chore: rustfmt

* chore: fix clippy warnings

Allowing the too-many-arguments warning for project_select,
as it will require some refactoring after this PR has already
been reviewed. It may be refactored in the future when subqueries are
implemented
2023-04-05 23:01:56 +00:00
Christopher M. Wolff d57a4f8947
refactor: make null-as-missing default behavior for LOCF (#7443)
* refactor: make null-as-missing default behavior for LOCF

* test: update InfluxQL test

---------

Co-authored-by: Christopher Wolff <cwolff@athena.tail244ec.ts.net>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-04-04 18:03:09 +00:00
Stuart Carnie a1b29b3ebb
fix: Incorrect results when planning aggregate expressions (#7431)
* fix: Incorrect results when mixing non-existent fields in aggregates

* chore: Improve comments around aggregates
2023-04-04 00:19:50 +00:00
Stuart Carnie 2dddda7c33
refactor: InfluxQL parser improvements (#7403)
* chore: Simplify insta snapshots

* chore: Extract struct-like enums to structs

This is in line with DataFusion, which also represents many of its
expression types as struct. The change permits explicit visit
methods for these new types.

These changes will be used by rewriting and visitors to treat the
types, such as Call as an atomic unit that can be replaced.
2023-04-02 21:41:49 +00:00
Stuart Carnie 19a0c7fe9c
feat: Teach InfluxQL how to process `FILL(null|previous|<value>)` (#7359)
* chore: Publicise gap-filling APIs

Helps #6916

* feat: IOx learns `FILL(null|previous|<value>)`

Helps #6916

* chore: More test cases

* chore: Revert change to TreeNodeVisitor

* chore: Update snapshot with expected gap-filling changes
2023-03-29 23:11:20 +00:00
Stuart Carnie a26cb6032a
feat: InfluxQL learns LOG function (arity 2) (#7333) 2023-03-28 20:22:27 +00:00
Andrew Lamb 43e236e040
chore: Update datafusion again (#7353)
* chore: Update DataFusion

* refactor: Update predicate crate for new transform API

* refactor: Update iox_query crate for new APIs

* refactor: Update influxql for new API

* chore: Run cargo hakari tasks

---------

Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-03-28 16:21:49 +00:00
dependabot[bot] 1ad0160d30
chore(deps): Bump serde_json from 1.0.94 to 1.0.95 (#7348)
Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.94 to 1.0.95.
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/v1.0.94...v1.0.95)

---
updated-dependencies:
- dependency-name: serde_json
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-03-28 11:09:59 +00:00
Stuart Carnie 08ef689d21
feat: Teach InfluxQL how to plan an aggregate query (#7230)
* feat: Display failed query

Allows a user to immediately identify the failed query.

* feat: API improvements to InfluxQL parser

* feat: Extend `SchemaProvider` trait to query for UDFs

* fix: We don't want the parser to panic on overflows

* fix: ensure `map_type` maps the timestamp data type

* feat: API to map a InfluxQL duration expression to a DataFusion interval

* chore: Copied APIs from DataFusion SQL planner

These APIs are private but useful for InfluxQL planning.

* feat: Initial aggregate query support

* feat: Add an API to fetch a field by name

* chore: Fixes to handling NULLs in aggregates

* chore: Add ability to test expected failures for InfluxQL

* chore: appease rustfmt and clippy 😬

* chore: produce same error as InfluxQL

* chore: appease clippy

* chore: Improve docs

* chore: Simplify aggregate and raw planning

* feat: Add support for GROUP BY TIME(stride, offset)

* chore: Update docs

* chore: remove redundant `is_empty` check

Co-authored-by: Christopher M. Wolff <chris.wolff@influxdata.com>

* chore: PR feedback to clarify purpose of function

* chore: The series_sort can't be empty, as `time` is always added

This was originally intended as an optimisation when executing an
aggregate query that did not group by time or tags, as it will produce
N rows, where N is the number of measurements queried.

* chore: update comment for clarity

---------

Co-authored-by: Christopher M. Wolff <chris.wolff@influxdata.com>
2023-03-23 01:13:15 +00:00
Andrew Lamb 8ab553180e
chore: Update datafusion again (#7276)
* chore: Update DataFusion pin

* chore: Update for new Expr type

* fix: deprecated

* chore: Run cargo hakari tasks

* docs: Update predicate/src/lib.rs

* fix: fmt

---------

Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-03-21 18:25:04 +00:00
Andrew Lamb 7dfaa05e8a
chore: Update datafusion again (#7208)
* chore: update datafusion again

* fix: update test

* fix: use table_reference

* fix: clean up import

* chore: Run cargo hakari tasks

---------

Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-03-16 14:34:40 +00:00
Stuart Carnie fe48a685ec
refactor: Move InfluxQL behaviour from iox_query to new crate (#7156)
* refactor: Break unnecessary dependencies from `iox_query` crate

In the process, the test code has been simplified.

* refactor: Move InfluxQL plan module to iox_query_influxql crate

* refactor: Move remaining behaviour from iox_query to iox_query_influxql

* chore: rustfmt 🙄

I was under the impression `clippy` would catch formatting
2023-03-08 22:29:20 +00:00