influxdb/data_types/src
Dom Dwyer 03f7025211
perf: minimise partition catalog queries
This commit implements a PartitionProvider decorator that
probabilistically determines if a partition is going to be a an
"old-style" row-addressed partition created prior to #7963, or a
"new-style" hash-addressed partition created after using a fast,
space-efficient, compressed bloom filter.

If a partition is identified as a new-style, hash-addressed partition,
the PartitionData is immediately initialised using the deterministic
hash ID without performing a catalog query at all.

If a partition is identified as an old-style, row-addressed partition, a
catalog query is performed to resolve the row ID as it would without
this filter.

A new-style, hash-addressed partition may sometimes be incorrectly
identified as a row-addressed partition, causing a spurious catalog
query, which is then correctly identified as a hash-addressed partition.
This is tuned to happen ~1-0.1% of the time, eliminating 99% to 99.9% of
unnecessary catalog queries.
2023-08-03 16:40:38 +02:00
..
columns.rs feat: ColumnSchema instantiation from gossip 2023-08-02 13:36:24 +02:00
lib.rs feat: Make parquet_file.partition_id optional in the catalog (#8339) 2023-07-31 12:40:56 +00:00
namespace_name.rs chore: update term conventions to allowlist 2023-04-24 17:07:25 -07:00
partition.rs perf: minimise partition catalog queries 2023-08-03 16:40:38 +02:00
partition_template.rs refactor: accessor for table partition proto 2023-08-02 13:36:35 +02:00
sequence_number_set.rs feat(data_types): Add `Extend<SequenceNumberSet>` for `SequenceNumberSet` 2023-07-05 14:23:18 +01:00