8ab553180e
chore: Update datafusion again ( #7276 )
...
* chore: Update DataFusion pin
* chore: Update for new Expr type
* fix: deprecated
* chore: Run cargo hakari tasks
* docs: Update predicate/src/lib.rs
* fix: fmt
---------
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-03-21 18:25:04 +00:00
4b55e09d09
Merge pull request #7268 from influxdata/savage/router-test-context-builder-pattern
...
refactor(router): Utilise builder pattern for construction of `TestContext`s
2023-03-21 17:02:44 +00:00
81a886ecae
Merge branch 'main' into savage/router-test-context-builder-pattern
2023-03-21 15:53:08 +00:00
779234eb20
chore(flightsql): Add tests that schema and data matches ( #7280 )
...
* fix(flightsql): Tests for matching schema in FligthSQL
* fix: Update influxdb_iox/tests/end_to_end_cases/flightsql.rs
2023-03-21 15:40:21 +00:00
8e9d0e8e74
refactor(router): Leverage MissingNamespaceAction directly for TestContext
...
This minimises addition of unnecessary types.
2023-03-21 15:28:00 +00:00
22127453da
Merge branch 'main' into savage/router-test-context-builder-pattern
2023-03-21 13:28:06 +00:00
eb72fbf3d3
test(ingester): graceful shutdown
...
Adds a ingester e2e test asserting the graceful shutdown behaviour;
specifically that:
* Buffered data is uploaded to object storage
* WAL files are cleaned up and empty (no replay on startup)
2023-03-21 14:21:56 +01:00
a385bb2ab2
Merge pull request #7267 from influxdata/dom/rpc-upstream-set-iter
...
refactor(router): stateful upstream iter for RPC balancer
2023-03-21 13:03:33 +00:00
8272ca56e8
Merge branch 'main' into dom/rpc-upstream-set-iter
2023-03-21 12:35:44 +00:00
bf35b951ad
refactor(router): Allow test contexts to specify namespace autocreate retention using a Duration
2023-03-21 11:13:55 +00:00
37211923e8
Merge branch 'main' into savage/router-test-context-builder-pattern
2023-03-21 10:31:22 +00:00
310dcc23c5
chore(deps): Bump ordered-float from 3.4.0 to 3.6.0 ( #7277 )
...
Bumps [ordered-float](https://github.com/reem/rust-ordered-float ) from 3.4.0 to 3.6.0.
- [Release notes](https://github.com/reem/rust-ordered-float/releases )
- [Commits](https://github.com/reem/rust-ordered-float/compare/v3.4.0...v3.6.0 )
---
updated-dependencies:
- dependency-name: ordered-float
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-03-21 10:31:06 +00:00
e773fca7ec
Merge branch 'main' into savage/router-test-context-builder-pattern
2023-03-21 10:31:03 +00:00
392972a3ad
refactor(router): Simplify builder code patterns
...
Use more fluent method names and leverage defaulting
to remove the need for an option on the namespace autocreate
policy.
2023-03-21 10:26:23 +00:00
6010405b1b
chore(deps): Bump reqwest from 0.11.14 to 0.11.15 ( #7273 )
...
Bumps [reqwest](https://github.com/seanmonstar/reqwest ) from 0.11.14 to 0.11.15.
- [Release notes](https://github.com/seanmonstar/reqwest/releases )
- [Changelog](https://github.com/seanmonstar/reqwest/blob/master/CHANGELOG.md )
- [Commits](https://github.com/seanmonstar/reqwest/compare/v0.11.14...v0.11.15 )
---
updated-dependencies:
- dependency-name: reqwest
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-03-21 10:18:13 +00:00
955fab89b1
chore(deps): Bump io-lifetimes from 1.0.8 to 1.0.9 ( #7274 )
...
Bumps [io-lifetimes](https://github.com/sunfishcode/io-lifetimes ) from 1.0.8 to 1.0.9.
- [Release notes](https://github.com/sunfishcode/io-lifetimes/releases )
- [Commits](https://github.com/sunfishcode/io-lifetimes/compare/v1.0.8...v1.0.9 )
---
updated-dependencies:
- dependency-name: io-lifetimes
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-03-21 10:10:36 +00:00
8d3fabf8a1
chore(deps): Bump bitflags from 1.3.2 to 2.0.2 ( #7272 )
...
* chore(deps): Bump bitflags from 1.3.2 to 2.0.2
Bumps [bitflags](https://github.com/bitflags/bitflags ) from 1.3.2 to 2.0.2.
- [Release notes](https://github.com/bitflags/bitflags/releases )
- [Changelog](https://github.com/bitflags/bitflags/blob/main/CHANGELOG.md )
- [Commits](https://github.com/bitflags/bitflags/compare/1.3.2...2.0.2 )
---
updated-dependencies:
- dependency-name: bitflags
dependency-type: direct:production
update-type: version-update:semver-major
...
Signed-off-by: dependabot[bot] <support@github.com>
* chore: Run cargo hakari tasks
---------
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-03-21 10:02:40 +00:00
2c4908bf51
Merge branch 'main' into savage/router-test-context-builder-pattern
2023-03-21 09:55:47 +00:00
9f0dfac8ad
chore(deps): Bump rustix from 0.36.10 to 0.37.3 ( #7271 )
...
* chore(deps): Bump rustix from 0.36.10 to 0.37.3
Bumps [rustix](https://github.com/bytecodealliance/rustix ) from 0.36.10 to 0.37.3.
- [Release notes](https://github.com/bytecodealliance/rustix/releases )
- [Commits](https://github.com/bytecodealliance/rustix/compare/v0.36.10...v0.37.3 )
---
updated-dependencies:
- dependency-name: rustix
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* chore: Run cargo hakari tasks
---------
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
2023-03-21 09:55:03 +00:00
12974cf426
fix: Fail if compactor shard args are invalid before contacting the catalog
...
Fail fast if we know the shard config is invalid, without possibly
failing while contacting the catalog.
Also, fetching of the transition shard ID is going to go away soon
anyway.
2023-03-20 16:41:34 -04:00
635b22b5a9
fix: Explicitly use memory catalog for invalid compactor CLI tests
...
To avoid the command picking up the catalog from your `.env`.
2023-03-20 16:40:46 -04:00
efae5246cf
docs: add comments for split/compact ( #7269 )
...
* docs: add comments for split/compact
* chore: Apply suggestions from code review
---------
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-03-20 19:58:39 +00:00
d8bd185fa0
test: Add compactor shard e2e tests
2023-03-20 15:52:47 -04:00
184565b552
feat(flightsql): Implement FlightSQL `GetSqlInfo` endpoint ( #7198 )
...
* feat(flightsql): Implement GetSqlInfo endpoint
* chore: Add some comments to clarify the tests intent
2023-03-20 19:34:18 +00:00
795a79a7ad
feat(flightsql): Add `GetTables` metadata endpoint without filters ( #7238 )
...
* test: Add getTables jdbc_client example
* feat: add `CommandGetTables` in FlightSqlClient
* feat: add `CommandGetTables` in flightsql cmd and planner
* test: add e2e test for `CommandGetTables`
* chore: clippy
* chore: comment out the test with filters
* test: update jdbc test expected value for tables
---------
Co-authored-by: Chunchun <14298407+appletreeisyellow@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-03-20 19:20:13 +00:00
1d8c85e61c
Merge pull request #7252 from influxdata/cn/compactor2-once
...
test: Re-enable running compactor once in step tests
2023-03-20 17:28:38 +00:00
01a609af45
fix: Actually use the bind addrs when running the compactor once
2023-03-20 13:21:25 -04:00
528d83e989
test: Re-enable running compactor once in step tests
...
Putting back the test that I removed in
<https://github.com/influxdata/influxdb_iox/pull/6850 >
now that they can use `run compactor2 --compaction-process-once
--compaction-process-all-partitions`
thanks to <https://github.com/influxdata/influxdb_iox/pull/6886 >!
2023-03-20 13:20:43 -04:00
acdeee62e0
refactor(router): Remove unused catalog field on TestContextBuilder
2023-03-20 17:11:55 +00:00
1a2565b45a
refactor(router): Split TestContextBuilder namespace parameters
...
Most of the test cases set up a context with or without namespace
autocreation, while few set a retention period. This removes the
requirement to pass a retention period when enabling autocreation
or vice versa.
2023-03-20 17:04:14 +00:00
32a3c81ce9
refactor(router): Use builder pattern to construct TestContext
...
Using the builder pattern enables a more ergonomic workflow
for setting optional namespace creation semantics when constructing
the TestContext for the router.
2023-03-20 17:03:40 +00:00
93ebb42fdd
refactor: stateful upstream iter for RPC balancer
...
Changes the router's RPC balancer to return a iterator of elements
starting from the given offset, that can remove elements from the
infinite/cycling iterator to prevent them from being yielded again.
2023-03-20 17:30:32 +01:00
f28ba2381d
test: assert all upstreams are (re)-tried
...
Adds a test ensuring requests are sent to all upstream ingesters until a
request succeeds.
2023-03-20 17:23:02 +01:00
c0a53ae5a9
Merge pull request #7261 from influxdata/dom/seqnumset-range
...
fix: full u64 range in SequenceNumberSet
2023-03-20 14:10:35 +00:00
8f9e4d21c7
chore: workspace-hack update
...
Add proptest to workspace-hack.
2023-03-20 14:24:58 +01:00
0a2cb33f7d
test: property test SequenceNumberSet operations
...
Adds proptests to assert correctness of set operations against a
SequenceNumberSet, by ensuring they match those of a known-good
implementation from the stdlib (HashSet).
Ensure (de-)serialisation correctness by asserting a round trip results
in equal sets.
2023-03-20 14:23:43 +01:00
5680710e75
fix: full u64 range in SequenceNumberSet
...
Prior to this commit, the SequenceNumberSet accepted values up to
u32::MAX (approx 4.2 billion) which works out to be ~50 days of
continious ID allocation at 1k writes/s.
This commit changes the SequenceNumberSet to accept the full range of a
u64, with the caveat that outside of the u32 range, values may be
ordered incorrectly. See:
https://github.com/influxdata/influxdb_iox/issues/7260
Fortunately values from SequenceNumberSet are used as an identity, and
never for ordering. We should remove the PartialOrd bounds on
SequenceNumber once Kafka has been cleaned up to encode this in the type
system.
2023-03-20 14:15:30 +01:00
89c55981b8
Merge pull request #7251 from influxdata/dom/wal-ref-empty-waker
...
refactor(wal): file management sequencing helpers
2023-03-20 10:53:08 +00:00
67ab2e46ca
Merge branch 'main' into dom/wal-ref-empty-waker
2023-03-20 10:45:26 +00:00
36b0a2fa8e
chore(deps): Bump io-lifetimes from 1.0.7 to 1.0.8 ( #7254 )
...
Bumps [io-lifetimes](https://github.com/sunfishcode/io-lifetimes ) from 1.0.7 to 1.0.8.
- [Release notes](https://github.com/sunfishcode/io-lifetimes/releases )
- [Commits](https://github.com/sunfishcode/io-lifetimes/compare/v1.0.7...v1.0.8 )
---
updated-dependencies:
- dependency-name: io-lifetimes
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Dom <dom@itsallbroken.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-03-20 10:45:13 +00:00
c828d47863
chore(deps): Bump clap from 4.1.9 to 4.1.11 ( #7255 )
...
Bumps [clap](https://github.com/clap-rs/clap ) from 4.1.9 to 4.1.11.
- [Release notes](https://github.com/clap-rs/clap/releases )
- [Changelog](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md )
- [Commits](https://github.com/clap-rs/clap/compare/v4.1.9...v4.1.11 )
---
updated-dependencies:
- dependency-name: clap
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-03-20 10:38:09 +00:00
dfb60b9685
chore(deps): Bump rustix from 0.36.9 to 0.37.2 ( #7257 )
...
* chore(deps): Bump rustix from 0.36.9 to 0.37.2
Bumps [rustix](https://github.com/bytecodealliance/rustix ) from 0.36.9 to 0.37.2.
- [Release notes](https://github.com/bytecodealliance/rustix/releases )
- [Commits](https://github.com/bytecodealliance/rustix/compare/v0.36.9...v0.37.2 )
---
updated-dependencies:
- dependency-name: rustix
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* chore: Run cargo hakari tasks
---------
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-03-20 10:31:27 +00:00
afb38d2199
chore(deps): Bump is-terminal from 0.4.4 to 0.4.5 ( #7258 )
...
Bumps [is-terminal](https://github.com/sunfishcode/is-terminal ) from 0.4.4 to 0.4.5.
- [Release notes](https://github.com/sunfishcode/is-terminal/releases )
- [Commits](https://github.com/sunfishcode/is-terminal/compare/v0.4.4...v0.4.5 )
---
updated-dependencies:
- dependency-name: is-terminal
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-03-20 10:24:18 +00:00
626fb62009
chore(deps): Bump bitflags from 1.3.2 to 2.0.2 ( #7256 )
...
* chore(deps): Bump bitflags from 1.3.2 to 2.0.2
Bumps [bitflags](https://github.com/bitflags/bitflags ) from 1.3.2 to 2.0.2.
- [Release notes](https://github.com/bitflags/bitflags/releases )
- [Changelog](https://github.com/bitflags/bitflags/blob/main/CHANGELOG.md )
- [Commits](https://github.com/bitflags/bitflags/compare/1.3.2...2.0.2 )
---
updated-dependencies:
- dependency-name: bitflags
dependency-type: direct:production
update-type: version-update:semver-major
...
Signed-off-by: dependabot[bot] <support@github.com>
* chore: Run cargo hakari tasks
---------
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-03-20 10:16:43 +00:00
76373d69dc
Merge pull request #7259 from influxdata/dependabot/cargo/dirs-5.0.0
...
chore(deps): Bump dirs from 4.0.0 to 5.0.0
2023-03-20 10:08:34 +00:00
0ac6ddf057
chore: Run cargo hakari tasks
2023-03-20 02:02:12 +00:00
dea6ad9e93
chore(deps): Bump dirs from 4.0.0 to 5.0.0
...
Bumps [dirs](https://github.com/soc/dirs-rs ) from 4.0.0 to 5.0.0.
- [Release notes](https://github.com/soc/dirs-rs/releases )
- [Commits](https://github.com/soc/dirs-rs/commits )
---
updated-dependencies:
- dependency-name: dirs
dependency-type: direct:production
update-type: version-update:semver-major
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-03-20 02:00:36 +00:00
866f9cefa1
feat: add null-as-missing gap filling ( #7245 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-03-17 20:34:45 +00:00
96c2094302
refactor(iox_query): extract influxrpc planner to its own crate ( #7241 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-03-17 18:48:55 +00:00
13657d5bcc
feat(authz): authorization service client and write integration ( #7216 )
...
* feat(authz): add authorization client.
Add a new authz crate to provide the interface for making authorization
checks from within IOx. This includes the default client that uses
the influxdata.iox.authz.v1 gRPC protocol. This feature is not used
by any IOx component yet.
* feat: optional authorization on write path
Support optionally enabling authorization checks on the /api/v2/write
handler. If an authrorizer is configured then the handler will
attempt to retrieve a token from the request's Authorization header.
If no such token exists then a response with a 401 error code is
returned. If the token is not valid, or does not have write permission
for the requested namespace then a response with a 403 error is
returned.
* chore: add unit test for authz in write handler
Add unit tests that test the correct functioning of the /api/v2/write
handler when an Authorizer is configured.
* chore(authz): use lazy connection
Change the initialization of the authz client to use a lazy connection.
This allows the client to be initialised synchronously.
* chore: Run cargo hakari tasks
* fix(authz): protolint complaints
* fix: authz tests
* fix: benches and lint
* chore: Update clap_blocks/src/authz.rs
Co-authored-by: Marko Mikulicic <mkm@influxdata.com>
* chore: Update authz/src/lib.rs
Co-authored-by: Marko Mikulicic <mkm@influxdata.com>
* chore: Update clap_blocks/src/authz.rs
Co-authored-by: Marko Mikulicic <mkm@influxdata.com>
* chore: review suggestions
* chore: review suggestions
Apply a number of suggestions from review comments. The main
behavioural change is that if the authz service is configured
applictions will perform a probe request to ensure it can communicate
before continuing startup.
* chore: Update router/src/server/http.rs
Co-authored-by: Dom <dom@itsallbroken.com>
---------
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: Marko Mikulicic <mkm@influxdata.com>
Co-authored-by: Dom <dom@itsallbroken.com>
2023-03-17 15:20:14 +00:00
Andrew Lamb
kodiakhq[bot]
Fraser Savage
Dom Dwyer
dependabot[bot]
Carol (Nichols || Goulding)
Nga Tran
CircleCI[bot]
Christopher M. Wolff
Martin Hilton