* build(flux): update flux to v0.171.0
* chore: remove testing.loadStorage from tests
Also update skip lists in Flux test harness
* chore: remove now redundant Go end-to-end unit tests
This testing is all now provided by the `fluxtest` harness.
* fix(fluxtest): update Flux tests for new option support
The Flux test harness now allows inheriting options, this updates the
test cases with the new syntax and simplifies any tests that had to
duplicate the options.
* build(flux): update flux to v0.165.0
* chore: upgrade flux to v0.167.0
Co-authored-by: Nathaniel Cook <nvcook42@gmail.com>
Co-authored-by: Paul Hummer <paul@eventuallyanyway.com>
Flux HTTP and template fetching requests do not perform IP address
checks for local addresses. This behavior on the one hand allows SSRF
(Server Side Request Forgery) attacks via authenticated requests but on
the other hand is useful for scenarios that have legitimate requirements
to fetch from private addresses (eg, hosting templates internally or
performing flux queries to local resources during development).
To not break existing installations, the default behavior will remain
the same but a new --hardening-enabled option is added to influxd to
turn on IP address verification and limit both flux and template
fetching HTTP requests to non-private addresses. We plan to enable new
security features that aren't suitable for the default install with this
option. Put another way, this new option is intended to be used to make
it easy to turn on all security options when running in production
environments. The 'Manage security and authorization' section of the
docs will also be updated for this option.
Specifically for flux, when --hardening-enabled is specified, we now
pass in PrivateIPValidator{} to the flux dependency configuration. The
flux url validator will then tap into the http.Client 'Control'
mechanism to validate the IP address since it is called after DNS lookup
but before the connection starts.
For pkger (template fetching), when --hardening-enabled is specified,
the template parser's HTTP client will be configured to also use
PrivateIPValidator{}. Note that /api/v2/stacks POST ('init', aka create)
and PATCH ('update') only store the new url to be applied later with
/api/v2/templates/apply. While it is possible to have InitStack() and
UpdateStack() mimic net.DialContext() to setup a go routine to perform a
DNS lookup and then loop through the returned addresses to verify none
are for a private IP before storing the url, this would add considerable
complexity to the stacks implementation. Since the stack's urls are
fetched when it is applied and the IP address is verified as part of
apply (see above), for now we'll keep this simple and not validate the
IPs of the stack's urls during init or update.
Lastly, update pkger/http_server_template_test.go's Templates() test for
disabled jsonnet to also check the contents of the 422 error (since the
flux validator also returns a 422 with different message). Also, fix the
URL in one of these tests to use a valid path.
Fixes a few issues:
* flux needs to write to the replication service, instead of the engine directly.
* the replication service incorrectly had value receiver methods, I think this
was just an accident. Pointer receivers make things easier to reason about. Also
with value receivers flux was not picking up the replication config properly.
* The flux to() function previously did not receive the org properly for internal
writes. Previously this was not necessary as the write path only needs the bucket
ID at this level (after authentication). But now we need the org id to look up
replications properly.
Closes#23183
* build(flux): update flux to v0.156.0
* chore(flux/schema): update schema tests to assert planner rules
The schema test where updated in Flux, this updates them here so that we
can assert that the planner rules are applied. See note about copied
data.
Co-authored-by: Nathaniel Cook <nvcook42@gmail.com>
* feat: Add WITH KEY to show tag keys
* fix: add tests for multi measurement tag value queries
* chore: fix linter problems
* chore: revert influxql changes to keep WITH KEY disabled
* chore: add TODO for moving flux tests to flux repo
Co-authored-by: Sam Arnold <sarnold@influxdata.com>
This updates the flux integration to use the `flux/array` package rather
than directly using the `arrow/array` package.
Flux recently switched to wrapping the array types from arrow and
creating its own array package to be used for table columns instead of
directly referencing the arrow package. This allows us to keep a
consistent interface, but potentially change internal implementations
without changing downstream consumers. Most recently, the
`*array.String` type has some of its own optimizations for certain array
patterns.
This change updates the flux integration to use the new API.
* fix: field metaqueries take fast path if predicate is only on `_measurement`
* chore: update CHANGELOG
* test: add test for fields with measurement predicate
The kind for experimental-to is in conflict with the one used by native
flux. In the past, native flux didn't use the kind and expected
implementations to register their own procedure spec with that value.
In the latest Flux, this isn't the case and it is using the kind for a
native implementation. As we have not implemented the requirements for
the native implementation, we are still replacing the procedure spec but
using a different name that's unique to our implementation.
* feat(query): enable min/max pushdown
* fix(query): fix the group last pushdown to use descending cursors
* test(storage): add read group test with no agg
Co-authored-by: Jonathan A. Sternberg <jonathan@influxdata.com>
* fix(storage): Detect need for descending cursor in WindowAggregate
* chore: Format + comments
* chore: PR cosmetic feedback (#21141)
Co-authored-by: Phil Bracikowski <pbracikowski+git@influxdata.com>
* chore: rename testcase and fix comments
Co-authored-by: Phil Bracikowski <pbracikowski+git@influxdata.com>
A static initialization is not desirable in the main binaries, as it forces all
paths of code to init, but it is still useful in tests. It allows static
intialization to be performed once for all tests and eliminates the need to
always add the FluxInit call. Added a fluxinit/static package that calls
fluxinit.FluxInit() to replace the builtin package. This hides the nature of
the initialization and makes it clear that it is mandatory initialization code
getting called.
Christopher M. Wolff
Brandon Pfeifer
Jonathan A. Sternberg
Jamie Strandboge
Sam Arnold
Paul Hummer
Dane Strandboge
Daniel Moran
William Baker
Faith Chikwekwe
Sean Brickley
Paul Hummer
Adrian Thurston
Yiqun (Ethan) Zhang
jl