* feat(idpe-17265): authorization should occur as part of the single_tenant specific mod
* authz service is accessed only through the single_tenant mod handler
* authz service is wrapped in auth mod
* move auth integration test into auth mod
* push down the authorize() call into the query params parser call, in order to access query params in the extract_token
* provide configuration error when authz or single_tenant mode are not co-presented
* update authz e2e fixtures
* feat(idpe-17265): extract tokens based upon preferred ordering in spec, and write tests to verify behavior.
* chore(idpe-17265): update naming conventions for a unifying parser
* test: make MockAuthorizer have default, and add a test_delegate_to_authz for CST
* chore: record authz duration metric, and include in delegation test.
* chore: use authz terminology instead of auth_service
* chore: more explicit naming
* Revert "chore: record authz duration metric, and include in delegation test."
This reverts commit 05c36888ca7247b6953343d759a5185098fae679.
* refactor: extract_header_token versus the else condition
* refactor: make single_tenant mod and move auth within
* chore: make unreachable explicitly panic in the build
* test: make token values be const, to be consumed when MockAuthorizer is used
* test: use locking for calls_counter in test
* fix: add base64 encoding as expected for Basic header
* fix: merge conflict resolution. The AuthorizationHeaderExtension is now under the authz::http mod, which is a required feature for router package.
* chore: run rustfmt nightly with preferred import handling, on files with modified imports
* chore: code cleanup, to have minimal code needed
Provide a configuration item for the router (in RPC mode) that controls
the maximum outgoing RPC message size when communicating with an
Ingester.
Raises the maximum from the default 4MiB to 100MiB. This does not
increase exposure to memory-based DOS, as writes are size-limited by the
HTTP layer to 10MiB, preventing a user from submitting a write this
large (or larger!) across the RPC boundary.
The "server_util" crate exists only to support HTTP authz operations, so
this commit moves it under the authz crate. This helper is gated by a
feature flag allowing callers to opt into this extra HTTP dependency
(disabled by default).
And return the "deletes unsupported" error sooner.
Co-authored-by: Dom <dom@itsallbroken.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
The NamespaceResolver was using its own very similar look-aside caching
to the DML handlers, this commit leverages the read-through cache
implementation to deduplicate more code and makes the read through
behavioural expectation explicit for namespace autocreation.
This removes the look-aside cache from the retention_validation
and schema_validation DML handlers, instead setting up the new
NamespaceCache decorator and using that to handle cache misses.
This commit refactors the NamespaceCache trait to return a result
instead of an option for calls to `get_schema()`, allowing callers and
decorators to differentiate between cache misses, namespaces not
existing and transient I/O errors. This allows implementations to
interact with backend catalog storage.
In order to implement a read-through NamespaceCache
decorator the `get_cache()` call will need to interact
with async catalog methods, so this allows implementations
to call await within the `get_cache()` body.
Part of the wider effort to consistently use tht term "database"
for the user-facing terminology, update the authorization system.
Whilst this system is technically user-facing, it is unlikely many
users will see it. It is however new enough that the change is
relatively little effort.
Adds tests that drive the multi-tenant HTTP write request parser.
Note that in addition to these unit tests, there's still considerable
integration converge of the HTTP write endpoint in multi-tenant mode in
http.rs (test_write_handler!) that asserts the system is unchanged in
the "default" run-mode of multi-tenant.
Adds a single-tenant mode (CST) to the IOx routers.
Single-tenancy mode differs in two main ways:
* V1 write endpoint is partially supported
* V2 write endpoint ignores "org" parameter
The "normal" mode is "multi tenant" which is the default operational
mode, and all existing behaviour remains unchanged. Single tenant mode
can be enabled by specifying INFLUXDB_IOX_SINGLE_TENANCY=true.
Request parsing is delegated to two implementations of the
WriteParamExtractor trait, one each for CST and MT - the logic of each
"mode" is defined within these files and all other functionality is
common between the two.
This commit also renames some of the error types for clarity
(NoSpecified -> NoOrgBucketSpecified, other NotSpecified ->
NoQueryParams, etc).
Note: single tenant code requires testing
This commit also cleans up the code formatting for the gRPC handler and
simplifies some of the gRPC handler tests for the new update service
limit API.
This adds additional testing coverage for updates to service protection
limits to a namespace, and how they affect subsequent writes that
exceed the limits.
There was a mix of different ways of returning errors - this commit
unifies them, adds some documentation to the returned errors, and
removes the capitalisation.
Errors should be lower-case so they compose nicely like this:
"something failed: super important error: inner error"
rather than:
"something failed: Super important error: Inner error"
Changes the org/bucket to NamespaceName calls to move the values into
the constructor, allowing it to reuse them if they do not require
encoding (the common case) instead of forcing them to be cloned to
obtain a 'static NamespaceName.
* feat(service_grpc_flight): optional query authorization
Add support for requiriing namespace-level authorization for
arrow flight based query requests. These are the flight SQL commands
as well as the IOx-specific SQL over flight and InfluxQL over flight
protocols.
Supports the optional configuration of an authorization sidecar,
in the same manner as is used in the router. If this is configured
then all arrow flight gRPC requests that are implemented will require
a valid authorization token to be supplied in the request. For a
multi-legged operation such as GetFlightInfo + DoGet required for
FlightSQL then a valid authorization is required for every request.
Ideally this support would be implemented using some sort of
interceptor, however the namespace isn't known until the request
processing has been started. The authorization check is performed
as soon as possible once the desired operation is known.
The legacy "storage" API has no authorization checks. Care should
be taken to ensure this API is never exposed to an untrusted network.
* chore(service_grpc_flight): review suggestions
Implement some suggestions from reviewers. The main change is adding
authorization checks to the handshake command.
* chore(service_grpc_flight): remove authorization of handshake
The Handshake call is used by existing clients to verify the
connection. These clients do not send a namespace header with the
request meaning there is nothing to authorize against. Remove this
authorization for now to avoid breaking existing clients.
* refactor: implement Authorizer trait on Option
Based on a suggestion from Dom implement the Authorizer trait on
Option<T: Authorizer> so that the call sites no longer need to check
if an authorizer is configured. This simplifies the code at the
call sites.
To maximise the utility the signature has changed so that a optional
token is now used. When no authorizer is configured this will not
be looked at. When a token is required a new error will be returned
if no token was supplied.
* fix: suggestions from clippy
* test: Add an e2e test for write replication
* fix: Pass through rpc_write_replicas configuration to RpcWrite handler
---------
Co-authored-by: Dom <dom@itsallbroken.com>
This commit implements replication for the router's RpcWrite handler.
The desired number of replica copies is specified at startup time, and
each user write will be fanned-out with the specified replication factor
(replicas + 1).
A failure to write to any upstreams returns the write error, but a
failure to obtain enough ACKs (enough successful writes) after at least
1 ACK will return a "partial write" error - this differentiation is
important, as the user's write will be readable after a partial write
error has occurred.
This currently writes to upstreams serially; this is clearly an
opportunity for improvement! A follow-on PR will parallelise writes
across the desired number of replicas while maintaining the "at most one
ack'd write to one host" invariant.
Note that replication is currently hard-coded as disabled.
Whenever an RPC write to an upstream ingester fails, it is retried after
an increasing delay, until the RPC_TIMEOUT is hit. Because of this, any
RPC write error would be returned as a "timeout", masking the underling
reason the write actually failed.
This commit pushes down the timeout logic, and retains the most recently
observed RPC write error, returning it to the user instead of the
timeout error.
This PR changes the RpcWrite DmlHandler to facilitate testing using
mocked RPC client circuit breaker states, and assert the error response
for no healthy upstreams.
wiedld
wiedld
Dom Dwyer
kodiakhq[bot]
Carol (Nichols || Goulding)
dependabot[bot]
Andrew Lamb
Carol (Nichols || Goulding)
Fraser Savage
Martin Hilton