Dom Dwyer
5a5f9d104b
refactor(read_buffer): todo!() -> unimplemented!()
...
todo!() is useful for incremental work (telling the compiler to shut up
during dev work).
unimplemented!() should be used for prod code that is not yet
implemented but expected to be deployed. It signifies "this isn't
implemented but doesn't need doing now".
2022-09-29 13:10:07 +02:00
Dom Dwyer
cd4087e00d
style: add no todo!() or dbg!() lints
...
Some crates had theme, some not - lets be consistent and have the
compiler spot dbg!() and todo!() macro calls - they should never be in
prod code!
2022-09-29 13:10:07 +02:00
Andrew Lamb
66dbb9541f
chore: Update datafusion and `arrow`/`parquet`/`arrow-flight` to 23.0.0, `thrift` to 0.16.0 ( #5694 )
...
* chore: Update datafusion and `arrow`/`parquet`/`arrow-flight` to 23.0.0
* chore: Update thrift / remove parquet_format
* fix: Update APIs
* chore: Update lock + Run cargo hakari tasks
* fix: use patched version of arrow-rs to work around https://github.com/apache/arrow-rs/issues/2779
* chore: Run cargo hakari tasks
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-09-27 12:50:54 +00:00
Carol (Nichols || Goulding)
c8108f01e7
chore: Upgrade to Rust 1.64 ( #5727 )
...
* chore: Upgrade to Rust 1.64
* fix: Use iter find instead of a for loop, thanks clippy
* fix: Remove some needless borrows, thanks clippy
* fix: Use then_some rather than then with a closure, thanks clippy
* fix: Use iter retain rather than filter collect, thanks clippy
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-09-22 18:04:00 +00:00
dependabot[bot]
ea1e822e3b
chore(deps): Bump itertools from 0.10.4 to 0.10.5 ( #5707 )
...
Bumps [itertools](https://github.com/rust-itertools/itertools ) from 0.10.4 to 0.10.5.
- [Release notes](https://github.com/rust-itertools/itertools/releases )
- [Changelog](https://github.com/rust-itertools/itertools/blob/master/CHANGELOG.md )
- [Commits](https://github.com/rust-itertools/itertools/commits )
---
updated-dependencies:
- dependency-name: itertools
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-09-21 08:15:59 +00:00
dependabot[bot]
7e1f013346
chore(deps): Bump itertools from 0.10.3 to 0.10.4 ( #5631 )
...
Bumps [itertools](https://github.com/rust-itertools/itertools ) from 0.10.3 to 0.10.4.
- [Release notes](https://github.com/rust-itertools/itertools/releases )
- [Changelog](https://github.com/rust-itertools/itertools/blob/master/CHANGELOG.md )
- [Commits](https://github.com/rust-itertools/itertools/compare/v0.10.3...v0.10.4 )
---
updated-dependencies:
- dependency-name: itertools
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-09-14 14:02:14 +00:00
Andrew Lamb
1fd31ee3bf
chore: Update datafusion / `arrow` / `arrow-flight` / `parquet` to version 22.0.0 ( #5591 )
...
* chore: Update datafusion / `arrow` / `arrow-flight` / `parquet` to version 22.0.0
* fix: enable dynamic comparison flag
* chore: derive Eq for clippy
* chore: update explain plans
* chore: Update sizes for ReadBuffer encoding
* chore: update more tests
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-09-12 17:45:03 +00:00
Marco Neumann
5936941784
chore: slim down `criterion` ( #5611 )
...
Criterion comes with some extra cargo tooling called `cargo criterion`
which can be used instead of `cargo bench`. The advantage is that we
don't need to compile the entire reporting infrastructure into our
benchmarks. So let's embrace this separation of concerns.
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-09-12 08:47:35 +00:00
dependabot[bot]
c62c7d32b1
chore(deps): Bump criterion from 0.3.6 to 0.4.0 ( #5608 )
...
Bumps [criterion](https://github.com/bheisler/criterion.rs ) from 0.3.6 to 0.4.0.
- [Release notes](https://github.com/bheisler/criterion.rs/releases )
- [Changelog](https://github.com/bheisler/criterion.rs/blob/master/CHANGELOG.md )
- [Commits](https://github.com/bheisler/criterion.rs/compare/0.3.6...0.4.0 )
---
updated-dependencies:
- dependency-name: criterion
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-09-12 08:26:10 +00:00
Andrew Lamb
6669d85fb4
chore: Update datafusion + arrow/parquet to `21.0.0` ( #5519 )
...
* chore: Update arrow/arrow-flight/parquet to 21.0.0
* chore: Update datafusion pin
* chore: Fix arrow update script
* chore: Update Cargo.lock
* chore: Update for new API
2022-08-31 13:30:47 +00:00
Andrew Lamb
b60e0beee2
fix: read_buffer benchmarks array type creation ( #5401 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-08-16 10:28:27 +00:00
Andrew Lamb
0a7e6919f2
chore: do not build benchmark binary for lib targets ( #5400 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-08-16 08:11:23 +00:00
Carol (Nichols || Goulding)
ee91a91700
fix: Remove needless returns
...
As now caught by clippy. https://rust-lang.github.io/rust-clippy/master/index.html#needless_return
2022-08-11 15:21:01 -04:00
Carol (Nichols || Goulding)
b982bdaf2f
fix: Derive Eq when we derive PartialEq and members can derive Eq
...
Allow this in generated code that we don't control, though.
Recommended by clippy now. https://rust-lang.github.io/rust-clippy/master/index.html#derive_partial_eq_without_eq
2022-08-11 15:04:06 -04:00
Andrew Lamb
16ddc5efc6
chore: Update datafusion / arrow/parquet/arrow-flight and prost/tonic ecosystem ( #5360 )
...
* chore: Update datafusion and arrow
* chore: Update Cargo.lock
* chore: update to Decimal128
* chore: Update tonic/prost/pbjson/etc
* chore: Run cargo hakari tasks
* fix: doctest in generated types
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
2022-08-09 17:30:44 +00:00
Andrew Lamb
9215a534d0
chore: Update datafusion and `arrow`/`parquet`/`arrow-flight` to `19.0.0` ( #5229 )
...
* chore: Update datafusion and `arrow`/`parquet`/`arrow-flight` to `19.0.0`
* chore: Run cargo hakari tasks
* fix: Update for API changes
* fix: clippy
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-07-28 08:10:47 +00:00
dependabot[bot]
e6d68f90b4
chore(deps): Bump permutation from 0.4.0 to 0.4.1 ( #5182 )
...
Bumps [permutation](https://github.com/jeremysalwen/rust-permutations ) from 0.4.0 to 0.4.1.
- [Release notes](https://github.com/jeremysalwen/rust-permutations/releases )
- [Commits](https://github.com/jeremysalwen/rust-permutations/commits )
---
updated-dependencies:
- dependency-name: permutation
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-07-22 08:38:23 +00:00
Andrew Lamb
e2d871b00b
chore: Update datafusion and arrow/parquet/arrow-flight to `18.0.0` ( #5079 )
...
* chore: Update datafusion to 10.0.0, arrow/parquet/arrow-flight to 18
* chore: Run cargo hakari tasks
* fix: update cargo pin
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-07-18 15:01:03 +00:00
Andrew Lamb
c46e1c6347
chore: Update datafusion + arrow/parquet/arrow-flight to `17.0.0` ( #5021 )
...
* fix: correct nullability declaration of system tables
* chore: Update datafusion and arrow/parquet/arrow-flight
* chore: Run cargo hakari tasks
* fix: Update tests
* fix: Update tests
* fix: predicate pruning
* fix: add some tests
* fix: query_functions
* fix: fix read_buffer test
* fix: fix clippy
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-07-07 19:22:15 +00:00
dependabot[bot]
c443d07a5c
chore(deps): Bump criterion from 0.3.5 to 0.3.6 ( #5059 )
...
Bumps [criterion](https://github.com/bheisler/criterion.rs ) from 0.3.5 to 0.3.6.
- [Release notes](https://github.com/bheisler/criterion.rs/releases )
- [Changelog](https://github.com/bheisler/criterion.rs/blob/master/CHANGELOG.md )
- [Commits](https://github.com/bheisler/criterion.rs/compare/0.3.5...0.3.6 )
---
updated-dependencies:
- dependency-name: criterion
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-07-07 07:09:27 +00:00
Carol (Nichols || Goulding)
69a3ad95b6
fix: Remove let bindings to the unit value found by clippy
...
The union and intersect methods on RowIDs don't return any value, so
these let bindings were always getting the unit value. Then the if/else
block that the bindings were returned from wasn't being assigned to
anything, so removing the binding and the return of the unit value from
these blocks is fine.
2022-07-01 13:31:32 -04:00
Carol (Nichols || Goulding)
69c123031a
fix: Remove unused lifetime declarations clippy found
2022-07-01 13:31:32 -04:00
dependabot[bot]
65a6ba7d45
chore(deps): Bump either from 1.6.1 to 1.7.0 ( #4990 )
...
Bumps [either](https://github.com/bluss/either ) from 1.6.1 to 1.7.0.
- [Release notes](https://github.com/bluss/either/releases )
- [Commits](https://github.com/bluss/either/compare/1.6.1...1.7.0 )
---
updated-dependencies:
- dependency-name: either
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-06-30 08:21:20 +00:00
Marco Neumann
db24838221
refactor: remove table name from read buffer ( #4910 )
...
The low-level chunk storage shouldn't care about the table name (this is
also true for parquet chunks btw). In fact, the table name is already
only a partial information since it misses the namespace.
If we need a table name, then the high-level chunk/data management is
responsible for that.
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-06-21 11:57:28 +00:00
Marco Neumann
1962fcc229
chore: reduce dependencies and run `cargo update` ( #4906 )
...
* chore: reduce proptest features
* chore: remove `grpc-router`
This crate is currently unused and we don't have immediate plans to use
it. And there's GIT, so it can always be restored.
* chore: `cargo update`
2022-06-20 12:18:28 +00:00
Andrew Lamb
e91d00b10c
chore: Update datafusion + `arrow`/`parquet`/`arrow-flight` to `16.0.0 ( #4851 )
...
* chore: TEMP Update DataFusion to pre-release
* chore: update arrow et al to 16.0.0
* chore: Run cargo hakari tasks
* fix: update reader read_dictionary API
* chore: Update to real Datafusion release
* fix: Update parquet API
* fix: update test
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
2022-06-14 16:31:40 +00:00
Carol (Nichols || Goulding)
5af0cc6acf
fix: Handle read buffer column not existing for column_names in QueryChunk impl
2022-06-03 12:45:16 -04:00
Carol (Nichols || Goulding)
7b080ca9e1
fix: Revert read_buffer table handling of column not existing
...
And change the test to a case that might be incorrect, but it at least
captures the current behavior
2022-06-03 11:41:33 -04:00
Carol (Nichols || Goulding)
6b882db671
fix: Don't return any column names if predicate references a column that doesn't exist
2022-06-03 09:16:04 -04:00
Carol (Nichols || Goulding)
c7f52ce362
fix: Return a different error type to distinguish different situations
2022-06-03 09:16:04 -04:00
Andrew Lamb
3592aa52d8
chore: Update datafusion + `arrow`/`parquet`/`arrow-flight` to `15.0.0` ( #4743 )
...
* chore: Update datafusion + `arrow`/`parquet`/`arrow-flight` to `15.0.0`
* chore: Update APIs
* chore: Run cargo hakari tasks
* feat: normalize parquet file metadata
* chore: update size tests
* chore: add docs on metadata stripping
* chore: TEMP UPDATE TO DF BRANCH
* chore: Update for new API
* fix: Update to latest DF
* fix: cargo hakari
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: Raphael Taylor-Davies <r.taylordavies@googlemail.com>
2022-06-03 10:32:26 +00:00
Andrew Lamb
3a33e806c7
chore: Update datafusion + `arrow`/`parquet`/`arrow-flight` to `14.0.0` ( #4619 )
...
* chore: Update datafusion deps
* chore: update arrow/parquet/arrow flight deps
* chore: Run cargo hakari tasks
* chore: Update location of utils
* chore: Update some more APIs
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
2022-05-17 14:13:03 +00:00
Jake Goulding
e07bcd40c2
refactor: Remove unused dependencies
...
These were found by iterating over all of the dependencies of each
Cargo.toml, then grepping that crate for the dependency's name. If it
didn't show up, I attempted to remove it.
I left a few dependencies that this process flagged:
* generated_types
- `pbjson`,`serde`. Apparently used by the generated code.
* grpc-router-test-gen
- `prost`. Apparently used by the generated code.
* influxdb_iox
- `heappy`. Doesn't appear used, but is behind enough feature
flags that I don't care to reason about and it's already optional.
- `tikv_jemalloc_sys`. Appears to be setting a feature flag of an
indirect dependency.
* iox_gitops_adapter
- `k8s_openapi`. Appears to be setting a feature flag of an indirect
dependency.
2022-05-06 15:57:58 -04:00
Carol (Nichols || Goulding)
068096e7e1
fix: Rename data_types2 to data_types
2022-05-06 14:45:39 -04:00
Carol (Nichols || Goulding)
bf1d08333d
fix: Move ChunkColumnSummary to read_buffer, the only place it's used
2022-05-06 14:45:39 -04:00
Carol (Nichols || Goulding)
d2671355c3
fix: Move partition metadata types to data_types2
2022-05-06 14:45:37 -04:00
Andrew Lamb
02893e598c
chore: Update datafusion and upgrade arrow/parquet/arrow-flight to 13 ( #4516 )
...
* chore: Tool for automating arrow version update
* chore: Update datafusion and arrow/parquet/arrow-flight
* fix: update for changes in Arrow API
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-05-05 00:21:02 +00:00
dependabot[bot]
4c94e46642
chore(deps): Bump croaring from 0.5.2 to 0.6.0 ( #4408 )
...
* chore(deps): Bump croaring from 0.5.2 to 0.6.0
Bumps [croaring](https://github.com/saulius/croaring-rs ) from 0.5.2 to 0.6.0.
- [Release notes](https://github.com/saulius/croaring-rs/releases )
- [Commits](https://github.com/saulius/croaring-rs/compare/0.5.2...0.6.0 )
---
updated-dependencies:
- dependency-name: croaring
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* fix: croaring 0.6.0 compat
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Marco Neumann <marco@crepererum.net>
2022-04-25 16:41:08 +00:00
Andrew Lamb
73bed810da
chore: Update arrow, arrow-flight, parquet, tonic, prost, etc ( #4357 )
...
* chore: Update datafusion
* chore: Update arrow/arrow-flight/parquet to 12
* chore: update datafusion correctly
* chore: Update prost, tonic, and dependents
* fix: Fixup some api changes
* fix: Update test output in db
* fix: Update test output in parquet_file
* fix: remove old pbjson types
* fix: Add "--experimental_allow_proto3_optional" flag
* chore: Run cargo hakari tasks
* fix: compile error
* chore: Update heappy
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-04-20 11:12:17 +00:00
Edd Robinson
d7370de615
test: add property tests to RLE encoder
2022-04-14 11:40:33 +01:00
Edd Robinson
aa4ad56939
test: add test case
2022-04-14 11:35:19 +01:00
Edd Robinson
5ce11a37f8
test: add property tests for float byte trimmer
2022-04-14 11:35:19 +01:00
Edd Robinson
e548ba7b17
test: add property tests for byte trimmer
2022-04-14 11:35:19 +01:00
Edd Robinson
cf0d048037
refactor: rename helper
2022-04-14 11:35:19 +01:00
Marco Neumann
20bbb88dc5
refactor: remove table name from `TableSummary` ( #4170 )
...
This allows us to remove the table name from the low-level chunk
representations (like `ParquetFile`, RUB, ...) since table names are
already tracked by the higher-level data structures (e.g. catalog,
catalog chunk) that manage the low-level chunk representations.
This is similar to #4167 .
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-03-30 13:24:00 +00:00
Marco Neumann
2b76c31157
refactor: make statistics null counts optional ( #4160 )
...
Min/max values and distinct counts are already optional, so let's make
the null counts optional as well. This will be helpful for NG to deal w/
partial statistics (e.g. we only populate stats for the time column).
Note that the total count is still mandatory, but we normally have the
chunk/file-level row count at hand.
2022-03-29 17:47:57 +00:00
Andrew Lamb
5c69a3f43b
chore: Update deps: datafusion, arrow/arrow-flight/parquet to 11, zstd to 0.11 ( #4119 )
...
* chore: update datafusion
* chore(deps): Bump arrow from 10.0.0 to 11.0.0
Bumps [arrow](https://github.com/apache/arrow-rs ) from 10.0.0 to 11.0.0.
- [Release notes](https://github.com/apache/arrow-rs/releases )
- [Changelog](https://github.com/apache/arrow-rs/blob/master/CHANGELOG.md )
- [Commits](https://github.com/apache/arrow-rs/compare/10.0.0...11.0.0 )
---
updated-dependencies:
- dependency-name: arrow
dependency-type: direct:production
update-type: version-update:semver-major
...
Signed-off-by: dependabot[bot] <support@github.com>
* chore(deps): Bump arrow-flight from 10.0.0 to 11.0.0
Bumps [arrow-flight](https://github.com/apache/arrow-rs ) from 10.0.0 to 11.0.0.
- [Release notes](https://github.com/apache/arrow-rs/releases )
- [Changelog](https://github.com/apache/arrow-rs/blob/master/CHANGELOG.md )
- [Commits](https://github.com/apache/arrow-rs/compare/10.0.0...11.0.0 )
---
updated-dependencies:
- dependency-name: arrow-flight
dependency-type: direct:production
update-type: version-update:semver-major
...
Signed-off-by: dependabot[bot] <support@github.com>
* chore: update parquet to 11.0.0
* fix: error on create schema, test for same
* fix: upgrade zstd
* chore: Run cargo hakari tasks
* fix: fix logical merge conflict
* fix: hakari
* fix: hakari
* fix: update newly introduced dep
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-03-24 15:27:36 +00:00
Andrew Lamb
204dd7c8e9
refactor: Fix some random clippy lints from the future ( #4118 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-03-24 09:21:29 +00:00
Andrew Lamb
2c3d30ca32
chore: Update datafusion, arrow, flight and parquet ( #4000 )
...
* chore: Update datafusion, arrow, flight and parquet
* fix: api change
* fix: fmt
* fix: update test metadata size
* fix: Update sizes in parquet test
* fix: more metadata size update
2022-03-10 12:24:47 +00:00
Marco Neumann
33851be3a5
chore: upgrade Rust to 1.59 ( #3875 )
...
Mostly a few new clippy crates around `flat_map`, `and_then`, and
"underscore locks" (!!!):
https://rust-lang.github.io/rust-clippy/master/index.html#let_underscore_lock
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-02-28 15:14:19 +00:00