* fix(query): accept queue-size > 0 when concurrency = 0
* fix(influxd): revert defaults for query settings to avoid validation err
* test: lower the default query concurrency used by test launchers
I think it is a nice utility function that I would like to use
elsewhere.
So I decided to move it out to the tracing package.
Signed-off-by: Gianluca Arbezzano <gianarb92@gmail.com>
This removes the spec and updates the lang package usage to make use of
passing in the runtime as a parameter.
It removes all direct dependendencies on the flux runtime from the http
package.
* chore: Remove several instances of WithLogger
* chore: unexport Logger fields
* chore: unexport some more Logger fields
* chore: go fmt
chore: fix test
chore: s/logger/log
chore: fix test
chore: revert http.Handler.Handler constructor initialization
* refactor: integrate review feedback, fix all test nop loggers
* refactor: capitalize all log messages
* refactor: rename two logger to log
The controller now supports setting an initial memory limit and setting
a maximum amount of memory that the controller may use separately from
the memory quota per query and the concurrency quota.
This allows the controller to increase the concurrency quota to a larger
number while setting the maximum amount of memory to a lower amount than
would be required for all queries to use 100% of their allowable memory.
Functionally, this means that a query will have a soft limit for an
initial memory byte quota that a query is guaranteed to have, a shared
pool that it is allowed access to in the case it uses more, and a hard
limit that no query may exceed to prevent runaway queries from taking
over the entire pool.
This change is completely backwards compatible with older configurations
as the new options will default to values that mimic the old behavior
where a query is allocated the full amount of its memory quota and the
maximum amount of memory is based on the concurrency quota and this
maximum memory quota.
In addition to the above, this also fixes a bug in the controller that
allowed it to run more than its concurrency as executing queries. This
happened when the results had finished being sent by the executor, but
the query had not yet been read and/or serialized. The executor would be
freed up and would take the next query even though the previous query
hadn't yet been finalized with `Done()`.
The secret service is tested by creating a secret and then attempting to
use it in a flux query. There is one test where accessing the secret
should work and one where it should return that the action is forbidden.
If we handle the flux errors in the query controller, it makes it so we
are handling the errors in the location where the happen rather than at
a layer further up the stack.
This should simplify it so the errors are handled in this single
location instead.
The controller implementation is primarily used by influxdb so it
shouldn't be part of the flux repository. This copies the code from flux
to influxdb so it can be removed from the next flux release.
I did this with a dumb editor macro, so some comments changed too.
Also rename root package from platform to influxdb.
In interest of minimizing risk, anyone importing the root package has
now aliased it to "platform" so that no changes beyond imports were
necessary in those files.
Lastly, replace the old platform module to local path /dev/null so that
nobody can accidentally reintroduce a platform dependency while
migrating platform code to influxdb.
The flux query controller was updated to include a Shutdown method a
while ago. Explicitly handle query controller creation and shutdown
where applicable.
In influxd, this ensures that outstanding queries are handled before the
process dies. In tests, this ensures that query controller goroutines
aren't leaked, which drastically simplifies reading full stack traces.
This change also registers query controller metrics with the prometheus
registry in influxd.
The previous default was just to have no limit at all. This adds a
configuration option to the planner so a static value can be set for the
memory limit on each individual query.
Daniel Moran
Dan Moran
Paul Hummer
Christopher M. Wolff
Brett Buddin
Stuart Carnie
Yiqun (Ethan) Zhang
Paul Hummer
Jonathan A. Sternberg
Gianluca Arbezzano
Lorenzo Affetti
Gianluca Arbezzano
Alirie Gray
Gavin Cabbage
Jacob Marble
docmerlin (j. Emrys Landivar)
Mark Rushakoff
Mark Rushakoff
Nathaniel Cook