Most of the test cases set up a context with or without namespace
autocreation, while few set a retention period. This removes the
requirement to pass a retention period when enabling autocreation
or vice versa.
Using the builder pattern enables a more ergonomic workflow
for setting optional namespace creation semantics when constructing
the TestContext for the router.
Changes the router's RPC balancer to return a iterator of elements
starting from the given offset, that can remove elements from the
infinite/cycling iterator to prevent them from being yielded again.
Adds proptests to assert correctness of set operations against a
SequenceNumberSet, by ensuring they match those of a known-good
implementation from the stdlib (HashSet).
Ensure (de-)serialisation correctness by asserting a round trip results
in equal sets.
Prior to this commit, the SequenceNumberSet accepted values up to
u32::MAX (approx 4.2 billion) which works out to be ~50 days of
continious ID allocation at 1k writes/s.
This commit changes the SequenceNumberSet to accept the full range of a
u64, with the caveat that outside of the u32 range, values may be
ordered incorrectly. See:
https://github.com/influxdata/influxdb_iox/issues/7260
Fortunately values from SequenceNumberSet are used as an identity, and
never for ordering. We should remove the PartialOrd bounds on
SequenceNumber once Kafka has been cleaned up to encode this in the type
system.
* feat(authz): add authorization client.
Add a new authz crate to provide the interface for making authorization
checks from within IOx. This includes the default client that uses
the influxdata.iox.authz.v1 gRPC protocol. This feature is not used
by any IOx component yet.
* feat: optional authorization on write path
Support optionally enabling authorization checks on the /api/v2/write
handler. If an authrorizer is configured then the handler will
attempt to retrieve a token from the request's Authorization header.
If no such token exists then a response with a 401 error code is
returned. If the token is not valid, or does not have write permission
for the requested namespace then a response with a 403 error is
returned.
* chore: add unit test for authz in write handler
Add unit tests that test the correct functioning of the /api/v2/write
handler when an Authorizer is configured.
* chore(authz): use lazy connection
Change the initialization of the authz client to use a lazy connection.
This allows the client to be initialised synchronously.
* chore: Run cargo hakari tasks
* fix(authz): protolint complaints
* fix: authz tests
* fix: benches and lint
* chore: Update clap_blocks/src/authz.rs
Co-authored-by: Marko Mikulicic <mkm@influxdata.com>
* chore: Update authz/src/lib.rs
Co-authored-by: Marko Mikulicic <mkm@influxdata.com>
* chore: Update clap_blocks/src/authz.rs
Co-authored-by: Marko Mikulicic <mkm@influxdata.com>
* chore: review suggestions
* chore: review suggestions
Apply a number of suggestions from review comments. The main
behavioural change is that if the authz service is configured
applictions will perform a probe request to ensure it can communicate
before continuing startup.
* chore: Update router/src/server/http.rs
Co-authored-by: Dom <dom@itsallbroken.com>
---------
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: Marko Mikulicic <mkm@influxdata.com>
Co-authored-by: Dom <dom@itsallbroken.com>
Allow a caller to await processing of a submitted WAL segment in the WAL
reference tracker.
This allows callers to (optionally) order operations that should happen
after the file has been processed.
Implement a waker primitive, allowing a caller to be woken once there
are no inactive WAL files.
This enables a caller to sequence operations to happen after deletion of
inactive WAL files.
This is helpful so that optimizer passes to forget the sort key, esp.
when the run after `DedupNullColumns` and `DedupSortOrder`.
For #6098.
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
* chore(influxdb_line_protocol): Remove ffi feature
* chore(influxdb_line_protocol): Remove workspace dependencies
* docs(influxdb_line_protocol): add initial readme and release instructions
* chore: Run cargo hakari tasks
* chore: Exclude influxdb_line_protocol from hakari checks and the workspace hack features
* feat: Add a link to nom in the docs
* docs: Copy edit documentation and comments
* fix: Mark the line protocol Error as non-exhaustive
So that we can easily add more error variants in the future.
* fix: Remove some functions from the public API
* fix: Make split_lines pub again
Oops, we're actually depending on this being pub in other parts of IOx
🙈
* docs: Add doc comments so everything on the crate docs page has a description
* chore: Add homepage, readme and repo links
* fix: use cargo-rdme
* chore: Add comment to hakari.toml
* chore: Increase version to `1.0.0`
* fix: fix LineProtocolBuilder link
---------
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: Carol (Nichols || Goulding) <carol.nichols@gmail.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
Similar to #7217 there is no need to convert the arrow schema to an IOx
schema. This also makes it easier to handle the chunk order column in #6098.
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
Breaks the reference_tracker module into sub-modules - this keeps the
code a little tidier. Soon I'll add more code, so this one file was
getting really big!
This type is exposed as an Arc-wrapped (reference counted) shared item
to avoid cloning the underlying data, and as such this type should not
be cloned directly.
Fraser Savage
Dom Dwyer
kodiakhq[bot]
dependabot[bot]
CircleCI[bot]
Christopher M. Wolff
Andrew Lamb
Martin Hilton
Marco Neumann
Carol (Nichols || Goulding)