* chore: Update DataFusion pin
* chore: Update for new API
* fix: Update for API
* fix: update compactor test
* fix: Update to patched version of arrow 46.0.0
* fix: map `DataFusionError::Configuration` to an internal error
* fix: do not use deprecated API
---------
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
* chore(deps): Bump chrono from 0.4.26 to 0.4.27
Bumps [chrono](https://github.com/chronotope/chrono) from 0.4.26 to 0.4.27.
- [Release notes](https://github.com/chronotope/chrono/releases)
- [Changelog](https://github.com/chronotope/chrono/blob/main/CHANGELOG.md)
- [Commits](https://github.com/chronotope/chrono/compare/v0.4.26...v0.4.27)
---
updated-dependencies:
- dependency-name: chrono
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* chore: Run cargo hakari tasks
* fix: Update deprecated chrono methods to their now-recommended versions
`chrono::DateTime::<Tz>::from_utc` has been deprecated and it's now
recommended to use `chrono::DateTime::from_naive_utc_and_offset`
instead.
<https://github.com/chronotope/chrono/pull/1175>
Note that the `Timestamp` type in `influxdb_influxql_parser` is an alias
for `chrono::DateTime<FixedOffset>`.
---------
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: Carol (Nichols || Goulding) <carol.nichols@gmail.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
* refactor(influxql): remove `copy_with_replacement` and use upstream version
* fix: clippy
---------
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
For backwards compatibility with version 1 clients support a minimal
implementation of SHOW RETENTION POLICIES. This advertises a single,
default, retention policy for any database. This is suffiecient for
compatibility with the grafana plugin and avoids the need for
cross-database catalog queries in the querier.
The values used for the "duration", "shardGroupDuration", and
"resourceN" are the same canned values used in the cloud 2
implementation.
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
Update the implementation of the MOVING_AVERAGE function to be a
user-defined window function allowing the values to be calculated
for the entire window in one go.
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
* chore: Update datafusion pin
* fix: Update for change in API
* chore: Update plan
---------
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
* chore: update expected output for `COUNT` aggregates with `FILL(null)`
See #8232
* fix(influxql): fill count aggregates with 0 by default
When gap-filling a COUNT aggregate any missing rows should be filled
with 0, unless otherwise directed by a FILL clause. To do this the
projection on the aggregate plan is modiefied to coalesce any COUNT
fields with 0 unless a FILL value has been specified in the query.
* chore: add more tests
* chore: add explanation of COUNT gap filling with multiple measurements
* fix: update test introduced with merge
---------
Co-authored-by: Stuart Carnie <stuart.carnie@gmail.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
Ensure that advanced syntax window functions that contain a selector,
rather than an aggregate, function are considered valid and generate
a correct plan.
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
* chore: Update datafusion to get new grouping
* chore: Update for new API
* chore: update tests
* fix: new API
* fix: state type
---------
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
Now that user-defined window functions are available, change the
DERIVATIVE and NON_NEGATIVE_DERIVATIVE function implementations to
use user-defined windows functions. This should improve performance
by allowing the entire window to be processed in one go, rather
than processing one row at a time.
The implementation is also moved out of the planner module alongside
the other user-defined window functions.
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
* refactor: refactor `Schemas` type in InfluxQL planner
* chore: update after merging main
---------
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
Change the implementation of DIFFERENCE, and NON_NEGATIVE_DIFFERENCE,
to use a user-defined window function, rather than a user-defined
aggregator function. This slightly simplifies the implementation.
The window function calculates difference operation on the full
window in one operation rather than a row at a time. Where ther are
no NULL values in the input this function uses arrow compute kernels.
As part of this refactor the window function's implementation is
moved out of the planner module.
* feat(influxql): CUMULATIVE_SUM window function
Implement the InfluxQL CUMULATIVE_SUM window function. This is
implemented as described in
https://docs.influxdata.com/influxdb/v1.8/query_language/functions/#cumulative_sum.
* chore: Add a test demonstrating NULL handling of CUMULATIVE_SUM
---------
Co-authored-by: Stuart Carnie <stuart.carnie@gmail.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
* feat(influxql): support TOP and BOTTOM functions
Add support for the TOP and BOTTOM functions which return the first
n rows in some ordered data set.
* fix: clippy
* refactor(influxql): use window aggregates for selectors
Change the implentation of ProjectionType::Selector to use a window
aggregate, rather than an aggregate with a custom selector function.
This is in preparation for implementing PERCENTILE.
* feat(influxql): PERCENTILE selector
Add a selector for the row containing the nth percentile of a
partition. This is the behaviour used when a single selector function
is used in an influxql query.
* feat(influxql): PERCENTILE aggregator
Add the PERCENTILE aggregation function for when the PERCENTILE
function is used in an aggregating projection. This implementation
buffers all non-null field values in memory in order to perform the
operation and therefore could be an expensive operation. This is
necessary for compatibility with earlier influxdb versions.
* refactor(influxql): move PERCENTILE implementation out of plan
The plan module is getting rather full of user-defined function
implementations. This breaks the new functions used to implement
percentile into some new top-level modules for aggregate and window
UDFs.
* fix: doc-lint
* chore: refactor `find_enumerated`
* chore: use `s` in format string
* chore: include the unexpected selector function in the error
* chore(influxql): review suggestions
Added some addition comments to help understanding.
Changed the handling os slector functions such that FIRST, LAST,
MAX & MIN behave the same as they did before PERCENTILE was added.
* chore(influxql): make percent_row_number a window UDF
Now that user-defined window functions are available make the
percent_row_number function be one of those. this allows the values
to be calculated for the entire window partition in one go.
For some reason the user-defined window function cannot return NULL
values. This function uses 0 where it would otherwise use NULL, as
row numbering starts at 1.
---------
Co-authored-by: Stuart Carnie <stuart.carnie@gmail.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
* refactor(iox_query_influxql): expand select projection
Change the SELECT projection in the planner to make it clearer how
each projection type works.
* feat(influxql): support TOP and BOTTOM functions
Add support for the TOP and BOTTOM functions which return the first
n rows in some ordered data set.
* fix: clippy
* chore: Use array / slice destructuring
* chore: review suggestion in iox_query_influxql/src/plan/planner.rs
Co-authored-by: Stuart Carnie <stuart.carnie@gmail.com>
---------
Co-authored-by: Stuart Carnie <stuart.carnie@gmail.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
Add the DERIVATIVE and NON_NEGATIVE_DERIVATIVE functions to influxql.
These are used to calculate derivatives over arbitrary time units.
The implementation is modeled after the DIFFERENCE and
NON_NEGATIVE_DIFFERENCE functions, with a difference that the unit
parameters is a configuration of the user-defined aggregator function
and therefore there cannot be a single shared definition of the
function.
The NON_NEGATIVE_DIFFERENCE function implementation has been
refactored to be an arbitrary NON_NEGATIVE wrapper for any Accumulator
function.
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
* test: ensure that selectors check arg count
* feat: basic non-aggregates w/ InfluxQL selector functions
See #7533.
* refactor: clean up code
* feat: get more advanced cases to work
* docs: remove stale comments
---------
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
dependabot[bot]
Andrew Lamb
Carol (Nichols || Goulding)
Martin Hilton
Christopher M. Wolff
Carol (Nichols || Goulding)
Marko Mikulicic
Stuart Carnie
Marco Neumann