* feat: Implement write buffer to Parquet snapshotting
This introduces snapshot to the server packages to manage snapshotting. It also introduces a new trait for representing a Partition. There is a very crude API wired up in http_routes for testing purposes. Follow on work will bring the server package into http_routes and rework the snapshot API.
Previosuly the $ORG and $BUCKET was joined as:
$ORG + "_" + $BUCKET
Which is fine unless either $ORG or $BUCKET includes a "_", such as:
$ORG = "org_a"
$BUCKET = "bucket"
and
$ORG = "org"
$BUCKET = "a_bucket"
This change continues to join $ORG and $BUCKET with an underscore, but
disallows underscores in either $ORG or $BUCKET. It appears these values
are non-zero u64s in the gRPC protocol converted to their base-10 string
representations for the DB name, so this seems safe to enforce.
In addition, this change introduces a `DatabaseName` type to avoid
passing bare strings around, and allow consuming code to ensure only
valid database names are provided at compile type. This type works with
both owned & borrowed content so doesn't force a string copy where we
can avoid it, and derefs to `str` to make it easier to use with existing
code.
I've been minimally invasive in pushing the `DatabaseName` through the
existing code and figured I'd see what the sentement is first.
Candidates for conversion from `str` to `DatabaseName` that seem to make
sense to me include:
- `DatabaseStore` trait
- `RemoteServer` trait
- Others? Basically anywhere other than the "edge" API inputs
Fixes#436 (thanks @zeebo)
Stops adding the IOx source code and performing a cargo build/test/clippy each
night. Previously this build would compile the IOx source & dependencies,
populating the incremental build cache and allowing builds that used the same
dependencies to complete quicker. This build caching was moved to
per-dependency-set caching in #496, and this pre-build is no longer used.
This should reduce the build image size substantially, making the whole CI
process a bit faster.
The `ReadFilterResults` type encapsulates results from multiple
segments. It implements `Display` to allow visualisation of results from
segments in a `select` call.
This commit also adds `Display` and `Debug` implementations for
`ReadFilterResult`. These can be used for visualising the contents of
the result of a `read_filter` call on a segment.
The former trait elides the column names.
Edd Robinson
Dom
huming
Brandon Sov
Paul Dix
Andrew Lamb