3b7d31c88a
chore(deps): Bump arrow from 9.0.2 to 9.1.0 ( #3826 )
...
Bumps [arrow](https://github.com/apache/arrow-rs ) from 9.0.2 to 9.1.0.
- [Release notes](https://github.com/apache/arrow-rs/releases )
- [Changelog](https://github.com/apache/arrow-rs/blob/master/CHANGELOG.md )
- [Commits](https://github.com/apache/arrow-rs/compare/9.0.2...9.1.0 )
---
updated-dependencies:
- dependency-name: arrow
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-02-23 09:25:46 +00:00
a30803e692
chore: Update datafusion, update `arrow`/`parquet`/`arrow-flight` to 9.0 ( #3733 )
...
* chore: Update datafusion
* chore: Update arrow
* fix: missing updates
* chore: Update cargo.lock
* fix: update for smaller parquet size
* fix: update test for smaller parquet files
* test: ensure parquet_file tests write multiple row groups
* fix: update callsite
* fix: Update for tests
* fix: harkari
* fix: use IoxObjectStore::existing
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-02-15 12:10:24 +00:00
22778a3a80
chore: upgrade rskafka and parking_lot ( #3592 )
2022-02-01 11:50:42 +00:00
442d63e65b
feat: catalog timestamp pruning ( #3571 )
...
* feat: catalog timestamp pruning
* chore: test
2022-01-28 13:45:13 +00:00
32d7c4cbfe
refactor: remove InfluxColumnType::IOx ( #3565 )
...
* refactor: remove InfluxColumnType::IOx
Remove unused column variant - see #3554 for context.
* refactor: reserve SEMANTIC_TYPE_IOX name in proto
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-01-27 21:15:36 +00:00
5488c257d1
chore: Update datafusion, upgrade to arrow/parqet/arrow-flight 8.0.0 ( #3517 )
...
* chore: Update datafusion
* chore: update to arrow 8
* fix: update to use new DataFusion APIs
* fix: update case for sortedness
* fix: cargo hakari
2022-01-27 13:33:27 +00:00
168afb63ad
feat: add `size` methods to DML-related types
...
This will be helpful when we want to batch DML operations in memory
(e.g. when using RSKafka).
This also ensures that `MBChunk` accounts for the column names that
are stored within `MutableBatch`.
2022-01-18 13:52:31 +01:00
dd23056efd
chore: update datafusion, arrow, prost, tonic, pbjson, etc ( #3455 )
...
* chore: update datafusion, arrow, prost, tonic, etc
* fix: update pprof as well
* chore: update hakari
* fix: update pbjson
* chore: update heappy
* fix: hakari
* fix: workaround https://github.com/influxdata/influxdb_iox/issues/3458
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-01-13 17:07:15 +00:00
cdf5c21cd4
fix: Fix max timestamp value comparison in chunk metadata ( #3453 )
...
* fix: Fix max timestamp value comparison in chunk metadata
* refactor: rename contains to overlaps
Co-authored-by: Edd Robinson <me@edd.io>
2022-01-13 16:58:30 +00:00
f3f6f335a9
chore: upgrade to snafu 0.7 ( #3440 )
2022-01-11 19:22:36 +00:00
96afb9ffe9
fix: Remove unused tokio dev-dependency in mutable_buffers
2021-12-06 09:37:16 -05:00
02c297e850
fix: Always specify the parking_lot feature of tokio to get potential perf boost
2021-12-06 09:37:15 -05:00
5d0fd1c603
fix: Allow dead code on fields that are now detected as never read
2021-12-02 11:52:01 -05:00
9fd4a560f5
feat: Results of running cargo hakari manage-deps
2021-11-19 09:21:57 -05:00
60f0deaf1e
feat: remove flatbuffer entry ( #3045 )
2021-11-05 20:19:24 +00:00
e83b8691db
refactor: remove chunk module from mutable_buffer ( #3015 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-11-03 17:56:52 +00:00
c540b40f05
chore(deps): bump tokio from 1.12.0 to 1.13.0
...
Bumps [tokio](https://github.com/tokio-rs/tokio ) from 1.12.0 to 1.13.0.
- [Release notes](https://github.com/tokio-rs/tokio/releases )
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.12.0...tokio-1.13.0 )
---
updated-dependencies:
- dependency-name: tokio
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2021-11-01 11:21:59 +00:00
8a2410e161
feat: mutable batch write entry ( #2724 ) ( #2973 )
...
* feat: mutable batch write entry (#2724 )
* chore: lint
* chore: review feedback
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-10-28 20:15:28 +00:00
bc7244c48e
chore: use Rust edition 2021
2021-10-25 10:58:20 +02:00
a82dc6f5f0
chore: Update datafusion + arrow ( #2903 )
...
* chore: Update datafusion to latest, arrow to 6.0.0
* fix: Update tests
* fix: bubble internal error
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-10-19 17:14:08 +00:00
bdd6d67e7a
refactor: split out mutable_batch crate ( #2841 )
...
* refactor: split out mutable_batch crate
* refactor: restore chunk module for better diffs
* chore: fmt
* chore: review feedback
Co-authored-by: Andrew Lamb <alamb@influxdata.com>
Co-authored-by: Andrew Lamb <alamb@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-10-14 16:46:54 +00:00
8a82f92c5d
refactor: add TimeProvider abstraction ( #2722 ) ( #2815 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-10-12 21:19:03 +00:00
afe34751e7
refactor: split out schema crate ( #2781 )
...
* refactor: split out schema crate
* chore: fix doc
2021-10-11 09:45:08 +00:00
d1f5209869
chore(deps): bump arrow from 5.4.0 to 5.5.0
...
Bumps [arrow](https://github.com/apache/arrow-rs ) from 5.4.0 to 5.5.0.
- [Release notes](https://github.com/apache/arrow-rs/releases )
- [Changelog](https://github.com/apache/arrow-rs/blob/5.5.0/CHANGELOG.md )
- [Commits](https://github.com/apache/arrow-rs/compare/5.4.0...5.5.0 )
---
updated-dependencies:
- dependency-name: arrow
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2021-10-04 08:55:38 +00:00
a9662b0edb
chore: remove unused `mutable_buffer` => `influxdb_line_protocol` dep
2021-09-22 10:56:15 +02:00
c66095cad1
feat: remove metrics crate ( #2552 )
2021-09-15 19:43:33 +00:00
44918e4afc
feat: migrate chunk metrics ( #2491 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-09-09 16:02:16 +00:00
6d5fb8b53d
refactor: simplify nested code
2021-09-07 12:01:06 +02:00
31cbb646b9
feat: skip individual rows during replay based on timestamp
2021-09-07 11:44:52 +02:00
b67610d9b9
chore(deps): bump tokio from 1.10.1 to 1.11.0
...
Bumps [tokio](https://github.com/tokio-rs/tokio ) from 1.10.1 to 1.11.0.
- [Release notes](https://github.com/tokio-rs/tokio/releases )
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.10.1...tokio-1.11.0 )
---
updated-dependencies:
- dependency-name: tokio
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2021-09-06 09:11:38 +00:00
b1bb390893
chore(deps): bump parking_lot from 0.11.1 to 0.11.2
...
Bumps [parking_lot](https://github.com/Amanieu/parking_lot ) from 0.11.1 to 0.11.2.
- [Release notes](https://github.com/Amanieu/parking_lot/releases )
- [Changelog](https://github.com/Amanieu/parking_lot/blob/master/CHANGELOG.md )
- [Commits](https://github.com/Amanieu/parking_lot/compare/0.11.1...0.11.2 )
---
updated-dependencies:
- dependency-name: parking_lot
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2021-09-06 01:18:24 +00:00
3c968ac092
feat: correctly account MUB sizes
...
Fixes #1565 .
2021-09-03 09:15:49 +02:00
3de6b44e23
build: use new rustdoc lint name ( #2261 )
...
* fix: nocache feature code rot
The MBChunk::snapshot code when using the "nocache" option no longer
compiles - this commit updates it to match the not(nocache) code.
* build: use updated broken_intra_doc_links name
The broken_intra_doc_links lint was renamed
rustdoc::broken_intra_doc_links
https://doc.rust-lang.org/rustdoc/lints.html
2021-08-11 19:48:51 +00:00
98a19b5ca6
fix(mutable_buffer): Statistics incorrect when subset of columns is inserted ( #2249 )
...
* fix(mutable_buffer): Statistics incorrect when subset of columns is inserted
* docs: remove wild newline
* fix: clippy
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-08-11 13:00:14 +00:00
9d15798288
fix: Address or allow Clippy warnings new with Rust 1.54
2021-07-30 09:59:59 -04:00
4689b5e4e5
refactor: Remove first/last write times from MUB chunks
2021-07-28 11:02:57 -04:00
5fb3e00f2a
fix: Properly record total_count and null_count in statistics ( #2103 )
...
* fix: Properly record total_count and null_count in statistics
* fix: fix statistics calculation in mutable_buffer
* refactor: expose null counts in read_buffer
* refactor: expose null_count in parquet_file
* fix: update server crate tests
* fix: update query_tests tests
* docs: tweak comments
* refactor: Use storage_stats rather than adding `null_count`
* refactor: rename test data field for clarity
* fix: fixup merge conflicts
* refactor: rename initial_non_null_count to initial_total_count
* refactor: caculate null_count as row_count - to_add
2021-07-26 18:13:36 +00:00
0b88deea43
refactor: don't pass sequence to MUB ( #2107 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-07-26 14:40:39 +00:00
446af5eb15
fix: consistent write timestamps ( #2104 )
...
* fix: consistent write timestamps
* chore: fix benchmarks
2021-07-23 18:04:15 +00:00
71f3f1aba2
Merge branch 'main' into cn/refactorings
2021-07-22 19:44:18 +00:00
01c79f1a1a
fix: Print all timestamps using RFC3339 format ( #2098 )
...
* fix: Use IOx pretty printer rather than arrow pretty printer
* chore: update tests in the query crate
* chore: update influxdb_iox tests
* chore: Update end to end tests
* chore: update query_tests
* chore: update mutable_buffer tests
* refactor: update parquet_file tests
* refactor: update db tests
* chore: update kafka integration test output
* fix: merge conflict
2021-07-22 19:04:52 +00:00
799406be72
refactor: Have MUB chunk snapshots hold a TableSummaryAndTimes rather than only ColumnSummaries
...
This way we can have the first/last write times on MUB chunk snapshots
without needing to deconstruct/reconstruct TableSummaries as it was
doing.
2021-07-22 11:35:23 -04:00
956647d966
docs: Fix an outdated comment
2021-07-22 11:35:23 -04:00
0a724878e6
refactor: Organize uses
2021-07-22 11:15:18 -04:00
4da8a16c18
chore: update to arrow 5.0 and master datafusion ( #2049 )
...
* chore: update to arrow 5.0 and master datafusion
* fix: Update test for change in object size
2021-07-19 12:49:51 +00:00
1d00fa2fd8
refactor: track memory metrics in catalog ( #1995 )
...
* refactor: track memory metrics in catalog
* chore: update comment
2021-07-14 16:23:00 +00:00
7ccbab8c90
feat: Make a TableSummaryAndTimes to use to slowly replace TableSummary
...
And use TableSummaryAndTimes with the mutable buffer chunks when turning
them into catalog chunks.
It's proving too big to switch over everything using TableSummary at
once, so this will let us switch over more incrementally.
2021-07-14 10:28:12 -04:00
d35b74c226
fix: Fix doc build warnings ( #1945 )
...
* fix: Fix doc build warnings
* refactor: add deny bare_urls to crates
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-07-13 08:03:42 +00:00
b9a6a11b34
feat: Record time of first/last write on MBChunk
2021-07-08 16:39:35 -04:00
22495dd355
fix: Take a TableBatch in the MBChunk constructor
...
Thus ensuring all MBChunks will have data in them.
2021-07-08 16:39:35 -04:00
dependabot[bot]
Andrew Lamb
Marco Neumann
Raphael Taylor-Davies
Dom
Andrew Lamb
Carol (Nichols || Goulding)
Carol (Nichols || Goulding)
kodiakhq[bot]