ee91a91700
fix: Remove needless returns
...
As now caught by clippy. https://rust-lang.github.io/rust-clippy/master/index.html#needless_return
2022-08-11 15:21:01 -04:00
b982bdaf2f
fix: Derive Eq when we derive PartialEq and members can derive Eq
...
Allow this in generated code that we don't control, though.
Recommended by clippy now. https://rust-lang.github.io/rust-clippy/master/index.html#derive_partial_eq_without_eq
2022-08-11 15:04:06 -04:00
c46e1c6347
chore: Update datafusion + arrow/parquet/arrow-flight to `17.0.0` ( #5021 )
...
* fix: correct nullability declaration of system tables
* chore: Update datafusion and arrow/parquet/arrow-flight
* chore: Run cargo hakari tasks
* fix: Update tests
* fix: Update tests
* fix: predicate pruning
* fix: add some tests
* fix: query_functions
* fix: fix read_buffer test
* fix: fix clippy
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-07-07 19:22:15 +00:00
69a3ad95b6
fix: Remove let bindings to the unit value found by clippy
...
The union and intersect methods on RowIDs don't return any value, so
these let bindings were always getting the unit value. Then the if/else
block that the bindings were returned from wasn't being assigned to
anything, so removing the binding and the return of the unit value from
these blocks is fine.
2022-07-01 13:31:32 -04:00
69c123031a
fix: Remove unused lifetime declarations clippy found
2022-07-01 13:31:32 -04:00
db24838221
refactor: remove table name from read buffer ( #4910 )
...
The low-level chunk storage shouldn't care about the table name (this is
also true for parquet chunks btw). In fact, the table name is already
only a partial information since it misses the namespace.
If we need a table name, then the high-level chunk/data management is
responsible for that.
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-06-21 11:57:28 +00:00
5af0cc6acf
fix: Handle read buffer column not existing for column_names in QueryChunk impl
2022-06-03 12:45:16 -04:00
7b080ca9e1
fix: Revert read_buffer table handling of column not existing
...
And change the test to a case that might be incorrect, but it at least
captures the current behavior
2022-06-03 11:41:33 -04:00
6b882db671
fix: Don't return any column names if predicate references a column that doesn't exist
2022-06-03 09:16:04 -04:00
c7f52ce362
fix: Return a different error type to distinguish different situations
2022-06-03 09:16:04 -04:00
3592aa52d8
chore: Update datafusion + `arrow`/`parquet`/`arrow-flight` to `15.0.0` ( #4743 )
...
* chore: Update datafusion + `arrow`/`parquet`/`arrow-flight` to `15.0.0`
* chore: Update APIs
* chore: Run cargo hakari tasks
* feat: normalize parquet file metadata
* chore: update size tests
* chore: add docs on metadata stripping
* chore: TEMP UPDATE TO DF BRANCH
* chore: Update for new API
* fix: Update to latest DF
* fix: cargo hakari
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: Raphael Taylor-Davies <r.taylordavies@googlemail.com>
2022-06-03 10:32:26 +00:00
068096e7e1
fix: Rename data_types2 to data_types
2022-05-06 14:45:39 -04:00
bf1d08333d
fix: Move ChunkColumnSummary to read_buffer, the only place it's used
2022-05-06 14:45:39 -04:00
d2671355c3
fix: Move partition metadata types to data_types2
2022-05-06 14:45:37 -04:00
4c94e46642
chore(deps): Bump croaring from 0.5.2 to 0.6.0 ( #4408 )
...
* chore(deps): Bump croaring from 0.5.2 to 0.6.0
Bumps [croaring](https://github.com/saulius/croaring-rs ) from 0.5.2 to 0.6.0.
- [Release notes](https://github.com/saulius/croaring-rs/releases )
- [Commits](https://github.com/saulius/croaring-rs/compare/0.5.2...0.6.0 )
---
updated-dependencies:
- dependency-name: croaring
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* fix: croaring 0.6.0 compat
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Marco Neumann <marco@crepererum.net>
2022-04-25 16:41:08 +00:00
d7370de615
test: add property tests to RLE encoder
2022-04-14 11:40:33 +01:00
aa4ad56939
test: add test case
2022-04-14 11:35:19 +01:00
5ce11a37f8
test: add property tests for float byte trimmer
2022-04-14 11:35:19 +01:00
e548ba7b17
test: add property tests for byte trimmer
2022-04-14 11:35:19 +01:00
cf0d048037
refactor: rename helper
2022-04-14 11:35:19 +01:00
20bbb88dc5
refactor: remove table name from `TableSummary` ( #4170 )
...
This allows us to remove the table name from the low-level chunk
representations (like `ParquetFile`, RUB, ...) since table names are
already tracked by the higher-level data structures (e.g. catalog,
catalog chunk) that manage the low-level chunk representations.
This is similar to #4167 .
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-03-30 13:24:00 +00:00
2b76c31157
refactor: make statistics null counts optional ( #4160 )
...
Min/max values and distinct counts are already optional, so let's make
the null counts optional as well. This will be helpful for NG to deal w/
partial statistics (e.g. we only populate stats for the time column).
Note that the total count is still mandatory, but we normally have the
chunk/file-level row count at hand.
2022-03-29 17:47:57 +00:00
204dd7c8e9
refactor: Fix some random clippy lints from the future ( #4118 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-03-24 09:21:29 +00:00
33851be3a5
chore: upgrade Rust to 1.59 ( #3875 )
...
Mostly a few new clippy crates around `flat_map`, `and_then`, and
"underscore locks" (!!!):
https://rust-lang.github.io/rust-clippy/master/index.html#let_underscore_lock
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-02-28 15:14:19 +00:00
37c65fc24f
fix: Use a string literal with unreachable ( #3768 )
...
This should have been taken care of with the update to Rust 2021
Edition, but rustc had a bug [1]. The bug will be fixed in Rust 1.59, which
means this code will stop compiling.
[1] https://github.com/rust-lang/rust/pull/93179
2022-02-16 16:44:48 +00:00
04a85515d3
Merge branch 'main' into er/refactor/read_buffer_clippy
2022-02-01 15:11:00 +00:00
04a59f825f
refactor: fix clippy lints
2022-02-01 14:56:49 +00:00
fa546047fb
refactor: update dep with API change ( #3596 )
2022-02-01 14:53:10 +00:00
2f3cc046f6
chore: enable to_string restrictive lints
2022-02-01 14:47:30 +00:00
87af662d3b
refactor: include comments in test cases ( #3550 )
2022-01-27 20:35:14 +00:00
5befa7922b
refactor: move chunk metrics to module ( #3548 )
...
* refactor: move chunk metrics into module
* refactor: use Metrics as internal name
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-01-27 15:24:31 +00:00
0a0b8b2150
feat: decouple read buffer row group size from Datafusion batch size ( #3538 )
...
* feat: add chunk builder
* test: test coverage for chunk builder
* refactor: apply suggestions from code review
* refactor: address PR feedback
2022-01-26 12:39:29 +00:00
d1816b662f
chore: update rust to 1.58 ( #3461 )
...
* chore: update rust to 1.58
* fix: clippy
Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
2022-01-13 21:12:46 +00:00
f3f6f335a9
chore: upgrade to snafu 0.7 ( #3440 )
2022-01-11 19:22:36 +00:00
80c048528c
refactor: Clean up code by using `lit` and `lit_timestamp_nano` rather than `Expr::Literal(ScalarValue(Some(...)))` ( #3427 )
2022-01-07 17:23:14 +00:00
527885f7f8
chore: Update datafusion ( #3413 )
...
* chore: Update datafusion and update code to handle timezone aware timestamps
* fix: cargo hakari
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-12-23 14:52:12 +00:00
a08246e3c7
Merge branch 'main' into er/refactor/debug_rub_query
2021-12-02 19:42:46 +00:00
083ba1daf4
refactor: apply PR feedback
2021-12-02 19:34:47 +00:00
0683f851e9
fix: Derive Default instead of doing an equivalent manual impl
...
Identified by clippy. The impl did have `#[cfg(test)]` on it, but the
entire struct does too, so this is fine.
https://rust-lang.github.io/rust-clippy/master/index.html#derivable_impls
2021-12-02 11:52:02 -05:00
1a899b939e
fix: Remove redundant closures identified by clippy
...
https://rust-lang.github.io/rust-clippy/master/index.html#redundant_closure
2021-12-02 11:52:02 -05:00
de25c8e1a4
refactor: add some trace timings
2021-12-02 12:23:44 +00:00
197634ed50
feat: reload chunk back into read buffer ( #3209 ) ( #3216 )
...
* feat: reload chunk back into read buffer (#3209 )
* chore: fix logical conflict
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-11-29 11:34:55 +00:00
b1096d2a36
Merge branch 'main' into er/feat/read_buffer/meta_delete
2021-11-01 10:26:10 +00:00
8d267e5840
Merge branch 'main' into er/test/read_buffer/null_column
2021-10-22 15:28:39 +01:00
7a5263d709
refactor: tidy up column summary to stats
2021-10-22 13:04:42 +01:00
09a2d805b9
refactor: restore concept of NULL value
2021-10-22 13:03:16 +01:00
8b406b4f91
test: update table_summaries test
2021-10-22 12:06:56 +01:00
a3e750c9fc
test: add null column to test
2021-10-22 11:27:06 +01:00
654d33684a
fix: Move code shared between if/else blocks out
2021-10-21 20:47:35 -04:00
a82dc6f5f0
chore: Update datafusion + arrow ( #2903 )
...
* chore: Update datafusion to latest, arrow to 6.0.0
* fix: Update tests
* fix: bubble internal error
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-10-19 17:14:08 +00:00
Carol (Nichols || Goulding)
Andrew Lamb
Marco Neumann
dependabot[bot]
Edd Robinson
Carol (Nichols || Goulding)
Andrew Lamb
Raphael Taylor-Davies
Carol (Nichols || Goulding)