83a5037e61
feat: query support for custom partitioning ( #8025 )
...
* feat: querier-specific stat creation routine
* feat: prune querier chunks using partition col ranges
* feat: add table client
* test: custom partitioning
* fix: correctly set up stats for chunks with col subsets
* fix: flaky test
* refactor: remove obsolete dead_code markers
* feat: add partition template to `create_namespace`
* test: extend custom partitioning end2end tests
* fix: explain shuffling, make it actual deterministic
2023-06-21 09:03:19 +00:00
5889c96501
chore: Update `datafusion` and other dependencies ( #7981 )
...
* chore: Update DatFaFusion pin
* chore: Update other dependencies
* chore: Update hakari
* fix: Update for API changes
* fix: Update explain plan
* fix: Update influxql plans
* fix: rustdoc links
---------
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-06-16 09:48:55 +00:00
17c0d837b3
chore: Update DataFusion, arrow, object_store pins ( #7942 )
...
* chore: Update DataFusion, arrow, object_store pins
* chore: Update for hakari
* chore: Update for new APIs
* fix: update test
---------
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-06-07 17:08:31 +00:00
c18902b05e
Merge branch 'main' into sgc/issue/7829_time_bounds_3
2023-06-07 08:51:38 +10:00
a2f5f37b2e
test: turn interval 0 test on after upgrading DF with the fix ( #7938 )
...
* test: turn interval 0 test on after upgrading DF with the fix
* chore: remove obsolete comments
2023-06-06 15:50:54 +00:00
f114842711
feat: Push outer query time-range to subqueries
...
Added additional end-to-end tests to validate time-range behaviour
2023-06-06 16:33:01 +10:00
9e2550c933
Merge branch 'main' into sgc/issue/7829_time_bounds_3
...
# Conflicts:
# iox_query_influxql/src/plan/planner.rs
2023-06-06 12:55:43 +10:00
f571aeb445
chore: Update DataFusion pin ( #7916 )
...
* chore: Update DataFusion pin
* chore: Update cargo
* fix: update for API changes
* fix: Update plans
* chore: Update for new api
* fix: Update plans
* chore: Update for API changes more
---------
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-06-05 18:38:59 +00:00
d8c2f2c679
refactor: Simplify `TimeRange` to match InfluxQL OG behaviour explicitly
2023-06-05 15:14:13 +10:00
efbaf455a0
feat: `selector_first` with additional args ( #7898 )
...
* feat: `selector_first` with additional args
Foundation for #7533 .
* test: `selector_first` malformed args
* docs: explain type handling
2023-06-02 10:08:21 +00:00
21752cfb69
test: reproducer for panic bug attempt to calculate the remainder with a divisor of zero ( #7903 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-06-01 15:43:24 +00:00
600ed6652c
refactor: rewrite time-range expressions to a single range
...
Fixes gap filling, which was confused by multiple lower or upper
time bounds.
2023-05-30 15:46:45 +10:00
2a07b53879
feat: add more tag predicate rewrite logic for InfluxQL ( #7869 )
...
* feat: add more tag predicate rewrite logic for InfluxQL
* chore: cargo fmt
* chore: fmt
* test: add more tests
---------
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-05-26 21:53:52 +00:00
ed9a16c4ad
chore: Add test to validate compatibility
2023-05-22 16:23:21 +10:00
90a25a3ff0
chore: update DataFusion ( #7825 )
...
* chore: update DataFusion
* chore: Run cargo hakari tasks
---------
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
2023-05-18 17:51:16 +00:00
9cc2169ce2
fix: Rename Step::WaitForPersisted2 to Step::WaitForPersisted
2023-05-17 17:02:59 -04:00
6785dcfd37
fix: Correct invalid test setups that the detector now detects
2023-05-17 17:00:17 -04:00
45e47af974
test: Add an invalid test configuration checker
...
If the test setup calls `Step::Persist` to persist on-demand, that
means it shouldn't be used with `ChunkStage::Parquet`, which tries to
persist as fast as possible. This will fail the test with a hopefully
helpful message to prevent this.
2023-05-17 16:58:50 -04:00
62fed73bcd
refactor: upgrade DataFusion to `19b03240920ad63cac916b42951754c0337bdac8#19b03240920ad63cac916b42951754c0337bdac8` ( #7813 )
...
I need:
- https://github.com/apache/arrow-datafusion/pull/6226 .
Changes in code due to:
- https://github.com/apache/arrow-datafusion/pull/6332
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-05-17 13:57:12 +00:00
7e64264eef
refactor: remove `RedudantSort` optimizer pass ( #7809 )
...
* test: add dedup test for multiple partitions and ranges
* refactor: remove `RedudantSort` optimizer pass
Similar to #7807 this is now covered by DataFusion, as demonstrated by
the fact that all query tests (incl. explain tests) still pass.
The good thing is: passes that are no longer required don't require any
upstreaming, so this also closes #7411 .
2023-05-17 09:30:04 +00:00
ca12f1c03d
fix: correctly recurse in `ParquetSortness` ( #7778 )
...
* test: reproducer for idpe_17556
* fix: `ParquetSortness` and partial opt
1. correctly handle cases where `ParquetSortness` would optimize one
child branch but not the other
2. handle cases where `ParquetSortness` recusion should stop a bit
clearer (using `TreeNodeRewriter`)
3. rename query tests to be a bit clearer
4. add test case with many (but not too many) duplicate files and an
ingester (basically a prod use case where the compactor is slightly
behind)
---------
Co-authored-by: Marco Neumann <marco@crepererum.net>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-05-17 06:09:23 +00:00
d7ab96c879
Merge branch 'main' into sgc/issue/6879_subquery_01
2023-05-17 07:20:08 +10:00
a4ad4fe69e
fix(4895): handle measurement missing, null bytes, and `=` in measurement names ( #7759 )
...
* test: add tests for the desired contract for parsing measurements from line protocol
* fix: restrict null chars in measurement
* chore: make an explicit Measurement type
* refactor: have iox lp parser match influxdb contract, for acceptance of eq in measurements
* test: create end_to_end test to confirm same write-then-read behavior with `=` in measurements, is the same as influxdb
2023-05-16 10:48:39 -07:00
5a813fb61f
chore: Simplify first queries
2023-05-16 10:18:34 +10:00
d2fe92f71e
chore: Add additional queries to be fixed by #7794
2023-05-16 10:05:04 +10:00
7ba619a32b
feat: outer GROUP BY pushed down to subqueries; more Cloud 2 examples
2023-05-15 15:31:20 +10:00
62a4c02836
feat: Handle default FILL behaviour for subqueries
2023-05-15 11:22:26 +10:00
c77c4b3d23
feat: support nested aggregate subqueries
2023-05-15 09:31:06 +10:00
4e96f814db
chore: Improve docs
2023-05-15 07:21:36 +10:00
f4a19fc6c1
fix: Aggregate subqueries with push-down `GROUP BY tags`
2023-05-12 16:53:16 +10:00
87dca540a7
feat: More subqueries succeeding
2023-05-12 13:55:42 +10:00
0c391de410
Merge branch 'main' into sgc/issue/6879_subquery_01
2023-05-11 06:02:09 +10:00
3cafa9ace3
fix: Rename query_tests2 to query_tests
2023-05-09 22:10:46 -04:00
Marco Neumann
Andrew Lamb
Stuart Carnie
Nga Tran
Christopher M. Wolff
Carol (Nichols || Goulding)
wiedld