00e735ef0d
chore: remove unused dependencies ( #1583 )
2021-05-29 10:31:57 +00:00
db432de137
feat: add distinct count to StatValues ( #1568 )
2021-05-28 17:41:34 +00:00
d8f19348bf
feat: per-column dictionaries in MUB ( #1570 )
...
* feat: per-column dictionaries in MUB
* chore: fmt
* refactor: remove chunk-level dictionary
* chore: remove redundant sort
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-05-28 13:51:56 +00:00
b362bb9c22
Merge pull request #1577 from influxdata/crepererum/remove_invalid_chunk_state
...
refactor: remove `ChunkState::Invalid`
2021-05-28 10:24:29 +00:00
d70d7a63a2
Merge branch 'main' into crepererum/remove_invalid_chunk_state
2021-05-28 10:20:05 +00:00
c6f42cf304
refactor: Remove unnecessary code ( #1573 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-05-28 10:12:47 +00:00
5cfede51f2
refactor: remove `ChunkState::Invalid`
...
This seems to only exist to fight the borrow checker and we can actually
live without it.
2021-05-28 11:16:06 +02:00
ac9fc4dfc4
Merge pull request #1559 from influxdata/crepererum/issue1376
...
feat: implement catalog rebuilding from files
2021-05-28 07:18:21 +00:00
6098c7cd00
Merge branch 'main' into crepererum/issue1376
2021-05-28 07:13:15 +00:00
9e5721b9f3
Merge pull request #1563 from influxdata/er/refactor/encodings
...
refactor: traitify column encodings in Read Buffer
2021-05-27 21:09:36 +00:00
840e40cbf4
Merge branch 'main' into er/refactor/encodings
2021-05-27 21:18:30 +01:00
e94be15296
refactor: update read_buffer/src/column/encoding/scalar/fixed.rs
2021-05-27 21:18:10 +01:00
3ae44a0375
refactor: Chunks can have at most one object store path ( #1574 )
...
* refactor: Chunk can have at most one path
* fix: update tests
2021-05-27 19:52:09 +00:00
f3bec93ef1
feat: Cache TableSummary in Catalog rather than computing it on demand ( #1569 )
...
* feat: Cache `TableSummary` in catalog Chunks
* refactor: use consistent table summary
2021-05-27 16:03:05 +00:00
26fe4167f7
refactor: erase some types
2021-05-27 14:57:40 +01:00
bba387d6ff
refactor: fix benchmarks
2021-05-27 14:35:34 +01:00
7ab27c4468
refactor: get build
2021-05-27 14:35:34 +01:00
04513e737d
refactor: define encodings wrt to scalar encoding trait
2021-05-27 14:35:34 +01:00
22f8a8a4a1
refactor: define scalar encoding in terms of trait
2021-05-27 14:35:34 +01:00
c84d50447c
feat: define a Transcoder trait
2021-05-27 14:35:34 +01:00
a81ada6140
feat: add transcoder trait
2021-05-27 14:35:34 +01:00
dd2a976907
feat: add a flag to ignore metadata errors during catalog rebuild
2021-05-27 13:10:14 +02:00
bc7389dc38
fix: fix typo
...
Co-authored-by: Andrew Lamb <alamb@influxdata.com>
2021-05-27 12:51:01 +02:00
48307e4ab2
docs: adjust error description to reflect internal errors
...
Co-authored-by: Andrew Lamb <alamb@influxdata.com>
2021-05-27 12:51:01 +02:00
d6f0dc7059
feat: implement catalog rebuilding from files
...
Closes #1376 .
2021-05-27 12:51:01 +02:00
024323912a
docs: explain what `PreservedCatalog::wipe` offers
2021-05-27 12:48:41 +02:00
5d342d7779
feat: associate tracker with lifecycle action ( #1099 ) ( #1556 )
...
* feat: associate tracker with lifecycle action (#1099 )
* chore: docs
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-05-27 10:47:35 +00:00
d2806164b9
Merge pull request #1564 from influxdata/crepererum/extend_aws_support
...
feat: extend AWS configs for prod and test
2021-05-27 10:42:37 +00:00
11b97a25b2
Merge branch 'main' into crepererum/extend_aws_support
2021-05-27 10:37:33 +00:00
792bff07d1
feat: only store ChunkSnapshot in Closed state ( #1560 )
...
* feat: only store ChunkSnapshot in Closed state
* chore: review feedback
* feat: record MUB size as closed size
* chore: document column ordering assumption
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-05-27 10:36:47 +00:00
4fcc04e6c9
chore: enable arrow prettyprint feature ( #1566 )
2021-05-27 10:28:14 +00:00
1cbe4031d3
feat: allow passing a session token to AWS stores
2021-05-27 11:53:12 +02:00
d5df7c381a
test: accept optional `AWS_ENDPOINT` for OS tests
2021-05-26 18:14:58 +02:00
0c6277c2e9
Merge pull request #1554 from influxdata/crepererum/issue1313
...
feat: add background task to clean up OS
2021-05-26 14:51:29 +00:00
efe077da8f
Merge branch 'main' into crepererum/issue1313
2021-05-26 14:46:18 +00:00
9f451423d5
feat: log files that are deleted
2021-05-26 12:49:44 +02:00
24ec1a472e
fix: do NOT delete parquet files that are reachable by time travel
2021-05-26 12:38:54 +02:00
c03b8a3963
refactor: remove tables from ChunkSnapshot ( #1295 ) ( #1558 )
2021-05-26 10:37:40 +00:00
1fb6af2364
refactor: split DB background loop into lifecycle and cleanup
...
This should prevent one from blocking / stalling the other.
2021-05-26 11:09:30 +02:00
5983336366
refactor: rename `parquet_file::{utils => test_utils}`
2021-05-26 11:09:29 +02:00
d7e3bc569e
refactor: shorten time we hold the transaction lock during clean-up
2021-05-26 11:04:57 +02:00
18f5dd9ae1
test: ensure transaction lock exists during cleanup planning
2021-05-26 11:04:57 +02:00
b55eae98da
fix: do not delete non-parquet files during catalog-driven cleanup
2021-05-26 11:04:57 +02:00
5ed16ff294
refactor: improve error message in `parquet_file::cleanup`
2021-05-26 11:04:57 +02:00
dd6bbeec42
feat: add background task to clean up OS
...
Closes #1313 .
2021-05-26 11:04:56 +02:00
14fdf3b7c7
feat: implement object store cleanup core routine
2021-05-26 11:02:40 +02:00
cc78b5317d
feat: add method to get all parquet files from catalog state
2021-05-26 11:02:40 +02:00
953114af2e
feat: add method to abort catalog transaction
2021-05-26 11:02:40 +02:00
92fcd7e940
feat: add a way to get OS, server ID and DB name from catalog
2021-05-26 11:02:40 +02:00
9daa4d00d6
test: re-organize `parquet_file` test utils a bit
2021-05-26 11:02:39 +02:00
Andrew Lamb
Raphael Taylor-Davies
kodiakhq[bot]
Marco Neumann
Edd Robinson