dd23056efd
chore: update datafusion, arrow, prost, tonic, pbjson, etc ( #3455 )
...
* chore: update datafusion, arrow, prost, tonic, etc
* fix: update pprof as well
* chore: update hakari
* fix: update pbjson
* chore: update heappy
* fix: hakari
* fix: workaround https://github.com/influxdata/influxdb_iox/issues/3458
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-01-13 17:07:15 +00:00
cdf5c21cd4
fix: Fix max timestamp value comparison in chunk metadata ( #3453 )
...
* fix: Fix max timestamp value comparison in chunk metadata
* refactor: rename contains to overlaps
Co-authored-by: Edd Robinson <me@edd.io>
2022-01-13 16:58:30 +00:00
c5cf03511c
fix: parquet column count statistics ( #2124 ) ( #3444 )
...
* fix: parquet metadata total_count (#2124 )
* chore: review feedback
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-01-11 21:56:24 +00:00
f3f6f335a9
chore: upgrade to snafu 0.7 ( #3440 )
2022-01-11 19:22:36 +00:00
37bb7f2120
chore: `cargo update`
...
dependabot currently doesn't work due to
https://github.com/dependabot/dependabot-core/issues/4574
Excluded `quote` due to
https://github.com/dtolnay/quote/issues/204
2022-01-11 14:57:51 +01:00
ec8644a39a
refactor: return clearer error message
2021-12-07 12:24:28 -05:00
561c5ed8e7
refactor: make checking no data happen during reading inout stream
2021-12-07 12:03:41 -05:00
c992c82582
chore: Merge branch 'main' into ntran/compact_os_tests
2021-12-07 11:08:12 -05:00
5fdaa5b4ab
chore: don't panic with invalid parquet ( #3309 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-12-06 21:15:35 +00:00
7499eac067
fix: Disable uuid serde feature; we're not actually serializing any UUIDs
...
Connects to #3117 .
2021-12-06 09:37:31 -05:00
02c297e850
fix: Always specify the parking_lot feature of tokio to get potential perf boost
2021-12-06 09:37:15 -05:00
0b24b3c227
fix: Use a consistent version specifier when depending on the futures crate
2021-12-06 09:37:12 -05:00
bca561366b
feat: don't copy parquet files out of disk object store ( #3282 ) ( #3293 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-12-05 16:31:40 +00:00
11067bfe3f
feat: simplify parquet reader ( #3282 ) ( #3291 )
...
* feat: simplify parquet reader (#3282 )
* chore: add back log line
2021-12-03 23:21:58 +00:00
86f9fe0bcb
refactor: no longer need to create and test no-row-groups parquet files
2021-12-03 15:14:04 -05:00
152281e428
fix: Capture the right 'no data' while parquet has no data
2021-12-03 12:19:48 -05:00
2857b6a990
Merge branch 'main' into er/feat/load_chunk_cli
2021-12-02 20:20:56 +00:00
b4ea9887ba
refactor: error name
2021-12-02 20:14:02 +00:00
5d0fd1c603
fix: Allow dead code on fields that are now detected as never read
2021-12-02 11:52:01 -05:00
88aedc556e
feat: add FromStr implementation
2021-12-02 12:59:52 +00:00
bf74608dc8
docs: not persist of the input stream is empty
2021-12-01 17:53:19 -05:00
f085af034e
refactor: not persist empty chunk resulting from deleting & deduplicating ( #3274 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-12-01 20:57:30 +00:00
f53cdca010
feat: handling empty compacted stream
2021-11-30 18:13:36 -05:00
197634ed50
feat: reload chunk back into read buffer ( #3209 ) ( #3216 )
...
* feat: reload chunk back into read buffer (#3209 )
* chore: fix logical conflict
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-11-29 11:34:55 +00:00
d16a7759ca
Merge branch 'main' into cn/workspace-hack
2021-11-22 17:05:31 +00:00
73d60539ad
refactor: use ChunkGenerator in parquet_catalog ( #2209 ) ( #3167 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-11-22 10:29:33 +00:00
9fd4a560f5
feat: Results of running cargo hakari manage-deps
2021-11-19 09:21:57 -05:00
ca4e0ad13b
refactor: add parquet chunk generator ( #2209 ) ( #3163 )
...
* refactor: add parquet chunk generator (#2209 )
* fix: tests
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-11-19 12:35:18 +00:00
c8d80e5c28
fix: Change database paths to be under /dbs/ instead of under /[server id]/
2021-11-05 10:14:06 -04:00
1902c4f8a9
chore: Update DataFusion ( #3012 )
...
* chore: Update DataFusion
* fix: restore Cargo.log
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-11-02 18:06:21 +00:00
4c9570b519
refactor: move `catalog` protobuf to `preserved_catalog`
...
This makes it clearer what's going since the contained messages are
only for the preserved part, not the in-mem catalog and its management.
2021-11-01 18:07:25 +01:00
c540b40f05
chore(deps): bump tokio from 1.12.0 to 1.13.0
...
Bumps [tokio](https://github.com/tokio-rs/tokio ) from 1.12.0 to 1.13.0.
- [Release notes](https://github.com/tokio-rs/tokio/releases )
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.12.0...tokio-1.13.0 )
---
updated-dependencies:
- dependency-name: tokio
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2021-11-01 11:21:59 +00:00
990f768cda
fix: Assign a UUID when creating a database
2021-10-28 13:20:28 -04:00
8198c1ff2a
refactor: Rename IoxObjectStore constructors to better match what server does with Databases
2021-10-28 13:20:27 -04:00
bc7244c48e
chore: use Rust edition 2021
2021-10-25 10:58:20 +02:00
a82dc6f5f0
chore: Update datafusion + arrow ( #2903 )
...
* chore: Update datafusion to latest, arrow to 6.0.0
* fix: Update tests
* fix: bubble internal error
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-10-19 17:14:08 +00:00
d8f35d8ee9
chore: remove unused `parquet_file` => `chrono` dep
2021-10-19 14:45:56 +02:00
28195b9c0c
chore: new `parquet_catalog` crate
2021-10-14 14:34:59 +02:00
0568452a0c
chore: Update datafusion ( #2838 )
...
* chore: update datafusion version
* refactor: Update to use new datafusion apis
* fix: do not upgrade other packages
2021-10-13 20:51:19 +00:00
1523e0edcd
refactor: clean up preserved catalog interface
...
1. Remove `new_empty` logic. It's a leftover from the time when the
`PreservedCatalog` owned the in-memory catalog.
2. Make `db_name` a part of the `PreservedCatalogConfig`.
2021-10-13 13:58:11 +02:00
8414e6edbb
feat: migrate preserved catalog to TimeProvider ( #2722 ) ( #2808 )
...
* feat: migrate preserved catalog to TimeProvider (#2722 )
* fix: deterministic catalog prune tests
* fix: failing test
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-10-12 14:43:05 +00:00
3dfe400e6b
feat: migrate write path to TimeProvider ( #2722 ) ( #2807 )
2021-10-12 12:09:08 +00:00
b39e01f7ba
feat: migrate PersistenceWindows to TimeProvider ( #2722 ) ( #2798 )
2021-10-11 20:40:00 +00:00
06c2c23322
refactor: create PreservedCatalogConfig struct ( #2793 )
...
* refactor: create PreservedCatalogConfig struct
* chore: fmt
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-10-11 15:43:05 +00:00
5da2f7b1b0
Merge branch 'main' into cn/less-database-name
2021-10-11 10:35:42 -04:00
afe34751e7
refactor: split out schema crate ( #2781 )
...
* refactor: split out schema crate
* chore: fix doc
2021-10-11 09:45:08 +00:00
8407735e00
fix: Pass the database name into PreservedCatalog
2021-10-08 15:25:10 -04:00
276aef69c9
refactor: Move PreservedCatalog test helper functions to test helpers and use them more
2021-10-08 15:25:10 -04:00
3aff4fcb07
refactor: Extract test helper functions for common catalog operations
...
This will make the next change easier, and I think it makes the tests
easier to read.
2021-10-08 15:25:10 -04:00
559a7e0221
Merge branch 'main' into cn/chunk-addr-smaller
2021-10-08 17:26:20 +00:00
Andrew Lamb
Raphael Taylor-Davies
Marco Neumann
Nga Tran
Carol (Nichols || Goulding)
kodiakhq[bot]
Edd Robinson
dependabot[bot]
Carol (Nichols || Goulding)
Carol (Nichols || Goulding)