`procspawn` indirectly pulls in many outdated dependencies. It was
introduced in #7850 to work around a flaky test. Isolating the test into
its own binary has the same affect and requires less dependencies.
We don't really need the color feature. Removing it has several
benifits:
- less dependencies
- one less MPL license
- no `atty` (only used by tests but has a security bug)
* add FlightService metric to record a duration histogram across requests.
* duration per partition, per request
* make available to the FlightFrameEncodeRecorder
* update naming conventions to reflect updated functionality
The `dml` crate and its contained types simultaneously contain more and
less data than the ingester needs for writes. This type is to replace
the use of `DmlOperation` and `DmlWrite` within the ingester's internals
so that the type can be specialised with low blast-radius changes.
The key change here is to remove the ties to the `DmlMeta` construction
and allow sequencing of data on a per-partition basis
The template length should always return a value > 0 because templates
must have at least one part. Before this change, `len` would have
returned 0 if there was no override because of the `unwrap_or_default`.
Instead, use the `parts` method, which takes care of the fallback to the
hardcoded default template, whose len will always be 1.
Avoid that the querier accesses files that were flagged for deletion a
long time ago. This would happen if the following conditions hold:
- we have very long-running querier pods (e.g. over holidays)
- the table doesn't receive any writes (or the partition if we ever
change the cache granularity), hence the querier is never informed
that its state is out-of-date
- a compactor runs a cold compaction, and by doing so flags a file for
deletion
- the GC finally wants to delete it
This is mostly a safety measure to prevent weird internal server errors
that should nearly never happen. On the other hand I do not want to hunt
Heisenbugs.
This commit adds an `inspect` command to read through the sequenced
operations in a WAL file and debug pretty print their contents to
stdout, optionally filtering by a sequence number range.
Historically the authz crate didn't distinguish between an invalid
token and a valid token without the required permissions. Recently
errors were added to distinguish these cases. This means that an
invalid token now returns an "Internal" error if supplied an invalid
token. Detect this case and return a "Permission Denied" error,
which is the error type that was previously returned in this case.
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
Partition templates should not contain more than 8 parts, which when
combined with a per-part byte limit, bounds the maximum size of a
partition key.
This commit causes the router to refuse to service a write request that
contains > 8 parts in the template - this causes a panic, as it's a
broken system invariant and should be an unreachable state. Templates
are pre-validated at creation time to contain no more than 8 parts, and
are immutable:
https://github.com/influxdata/influxdb_iox/pull/7930
Allow a table template override to report the number of template parts
within it.
This ignores the lint wanting an "is_empty()" method too, because it's
misleading and redundant - a template MUST never be empty.
This commit ensures all partition key parts are less than or equal to
200 bytes long.
If a string exceeds the 200 byte limit, it is truncated (avoiding
splitting unicode code-points or graphemes) and then a single "#"
sentinel value is appended. When reversed from the string, these column
values are indicated to be suitable for prefix-matching only - a
property that is encoded into the type system.
This commit takes a conservative approach of not splitting graphemes as
outlined in the module documentation, but this could be relaxed in the
future if needed.
* docs: Remove TODO comment that's TODONE
* docs: Oops, turns out the TODO comment was this enum's documentation
---------
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
Dom Dwyer
Marco Neumann
dependabot[bot]
wiedld
kodiakhq[bot]
Fraser Savage
Joe-Blount
Carol (Nichols || Goulding)
Joe-Blount
Martin Hilton
Carol (Nichols || Goulding)
Andrew Lamb