This updates the repl to support the new influxdb source and use it by
default in the repl. It will automatically set some default variables
for the influxdb source to make it easier to use the cli. In particular,
it will set the default organization, token, and the host. The
organization gets set to the one specified in the repl command and the
token gets filled in with the user installed one. The host defaults to
localhost but will change to whichever one was specified on the cli.
In addition, this will replace the http client with one that sets
insecure skip verify if the `--skip-verify` flag is used.
This adds support for using pkg-config to build libflux inside of the
flux dependency. The build can occur by either installing `pkg-config`
into your path or the `env` script can be used to invoke it from the go
modules.
The repl no longer takes in a querier and it will run everything
locally. The spec interface will now not be used and will be removed
from the http endpoint at some point.
Fixes an issue resolving dependencies with `go mod`.
Specifically, when starting from a clean environment, and
executing:
```sh
$ go mod download
```
the following error is produced:
```
github.com/uber/jaeger-lib@v1.5.0+incompatible: invalid version: +incompatible suffix not allowed: major version v1 is compatible
```
The storage engine isn't capable of sending back empty tables when a
series is empty. Because of this, we disable the push down and let flux
do the filtering in the case where there is a filter and it is specified
to keep the empty tables.
* feat(tracing): dont trace spans with full URL path names in ExtractFromHTTPRequest
* chore(multiple): replace all occurrences of julienschmidt/httprouter with influxdata/httprouter
does not apply to mappings, will apply mappings no matter what. we need to
remedy the uniqueness of resources from the API side. Applies to labels,
label mappings,
The controller now supports setting an initial memory limit and setting
a maximum amount of memory that the controller may use separately from
the memory quota per query and the concurrency quota.
This allows the controller to increase the concurrency quota to a larger
number while setting the maximum amount of memory to a lower amount than
would be required for all queries to use 100% of their allowable memory.
Functionally, this means that a query will have a soft limit for an
initial memory byte quota that a query is guaranteed to have, a shared
pool that it is allowed access to in the case it uses more, and a hard
limit that no query may exceed to prevent runaway queries from taking
over the entire pool.
This change is completely backwards compatible with older configurations
as the new options will default to values that mimic the old behavior
where a query is allocated the full amount of its memory quota and the
maximum amount of memory is based on the concurrency quota and this
maximum memory quota.
In addition to the above, this also fixes a bug in the controller that
allowed it to run more than its concurrency as executing queries. This
happened when the results had finished being sent by the executor, but
the query had not yet been read and/or serialized. The executor would be
freed up and would take the next query even though the previous query
hadn't yet been finalized with `Done()`.
This avoids a module error in Go 1.13 from a poorly tagged transitive
test dependency that looks like:
gopkg.in/stretchr/testify.v1/assert: go.mod has non-....v1 module path
"github.com/stretchr/testify" at revision v1.4.0
The API slightly changed: LoadSwaggerFromYAMLData is now replaced
by LoadSwaggerFromData.
See: https://github.com/getkin/kin-openapi/pull/67
Signed-off-by: Robert-André Mauchin <zebob.m@gmail.com>
This commit adds a reason label to the total compaction metric. For
snapshots, the reason will indicate why the cache was snapshotted. For
other compactions, the reason label will be blank.
* deps: update flux-staging to master of both influxdb and flux (#12698)
* test(stdlib): skip failing window test due to new trigger optimization (#12708)
* chore: update Flux to v0.22.0
This commit consists of several improvements or changes:
* migrate the influxd binary to cobra.Command
* introduce a default run sub-command to start the server
* register the run sub-command flags with viper
to maintain compatibility with the existing behavior of automatic
binding of flags to environment variables.
Closes#12602
This refactors everything to generate and use a flux AST when
transpiling an influxql query. This also updates the spectests so they
use flux instead of writing out the AST and compare the resulting
AST's.
I did this with a dumb editor macro, so some comments changed too.
Also rename root package from platform to influxdb.
In interest of minimizing risk, anyone importing the root package has
now aliased it to "platform" so that no changes beyond imports were
necessary in those files.
Lastly, replace the old platform module to local path /dev/null so that
nobody can accidentally reintroduce a platform dependency while
migrating platform code to influxdb.
all tests use a unique bucket based on the test file name. copied all tests over from flux repo
the tests are currently disabled due to engine consistency issues: https://github.com/influxdata/flux/issues/613
There were two issues.First the `chronograf` subdir was not listed.
Second the tmpl generate dependency was not listed in the tools.go list
of dependencies.
Fixes#2044
Steps taken:
- go test -short ./... repeatedly, and on each checksum failure:
- remove line for that repository from go.sum, and re-run go test
- go mod tidy repeatedly, and on each checksum failure:
- remove line for that repository from go.sum, and re-run go mod tidy
I think this fixes local development, but it is unlikely to succeed on
CI until other images also migrate to go1.11.4. Unfortunately, the
docker image for go1.11.4 has not yet been published.
Co-authored-by: Leonardo Di Donato <leodidonato@gmail.com>
Co-authored-by: Michael Desa <mjdesa@gmail.com>
feat(kv): add kv store interface for services
feat(bolt): add boltdb implementation of kv.Store
spike(platform): add kv backed user service
feat(kv): add static cursor
Note here that this operation cannot be transactionally done. This poses
a bit of issues that will need to be worked out.
fix(bolt): use error explicit error message
squash: play with interface a bit
fix(kv): remove commit and rollback from kv interface
feat(inmem): add inmem kv store
chore: add note for inmem transactions
fix(bolt): remove call to tx in kv store tests
feat(kv): add tests for static cursor
doc(kv): add comments to store and associated interfaces
doc(bolt): add comments to key value store
feat(testing): add kv store tests
test(testing): add conformance test for kv.Store
test(inmem): add kv.Store conformance tests
doc(inmem): add comments to key value store
feat(inmem): remove CreateBucketIfNotExists from Tx interface
feat(bolt): remove CreateBucketIfNotExists from Tx
feat(inmem): remove CreateBucketIfNotExists from Tx
doc(kv): add note to bucket interface about conditions methods can be called
feat(kv): add context methods to kv.Tx
feat(bolt): add context methods to bolt.Tx
feat(inmem): add context methods to inmem.Tx
test(kv): add contract tests for view/update transactions
feat(kv): ensure that static cursor is always valid
Co-authored-by: Leonardo Di Donato <leodidonato@gmail.com>
Co-authored-by: Michael Desa <mjdesa@gmail.com>
fix(kv): remove error from cursor methods
test(kv): remove want errors from cursor test
test(testing): add concurrent update test for kv.Store
feat(kv): make kv user service an example service
fix(testing): add concurrnent update test to the kv.Store contract tests
test(platform): fix example kv service tests
dep(platform): make platform tidy
BREAKING CHANGE: The API endpoint exposed internal details about the implementation of
the logical and physical plans. Those data structures need to be free to
change without breaking the API. For now since the endpoint is unused it
is being removed. A similar API may be added back in that correctly
abstracts away the internal details of plans.
The table interface was modified to expose the arrow buffers. The
storage table has now been converted to use this interface with the same
fixes so that it exposes arrow buffers.
The influxql package has also been updated to use the `DoArrow` method
from the `flux.Table` interface.
The test used `AND` instead of `and` which is not valid syntax. It also
did not include packages and imports in the generated JSON from parsing
the flux query.
test(platform): run testcontainer integration tests for nightly release
Integration tests for the vault secret service using testcontiners
should not run along with unit tests, however, they should run on some
regular schedule. This commit introduces `make test-integration` which
runs integration tests for vault using testcontainers. The command introduced
relies on docker being available on the host it is executed on.
chore(platform): make go modules tidy
chore: try to fix go mod
chore(platform): remove explicit logrus dependency
chore(platform): run go mod tidy
chore(platform): replace github.com/Sirupsen/logrus with github.com/sirupsen/logrus
chore(platform): update docker dependency
feat(vault): add vault implementation of secret service
test(platform): run testcontainer integration tests for nightly release
Integration tests for the vault secret service using testcontiners
should not run along with unit tests, however, they should run on some
regular schedule. This commit introduces `make test-integration` which
runs integration tests for vault using testcontainers. The command introduced
relies on docker being available on the host it is executed on.
chore(platform): make go modules tidy
chore: try to fix go mod
chore(platform): run go mod tidy
feat(vault): add vault implementation of secret service
chore(platform): make go modules tidy
feat(platform): add Put/Patch/Delete methods on secret service
feat(vault): add Put/Patch/Delete methods on vault secret service
feat(http): add http handler methods for secret service
feat(bolt): add Put/Delete/Patch methods to bolt secret service
feat(testing): add tests for Put/Patch/Delete methods in secret service
feat(mock): add mock secret service
feat(http): add tests for secrets endpoints
feat(http): update swagger for secrets endpoints
chore: run go mod tidy
The pb package was only referenced in cmd/influx/query.go, but in
dead code, since it uses the same machinery as the repl, which goes
through the HTTP endpoints, rather than the gRPC endpoints.
The version of goreleaser we pinned used `masterminds/semver`, but the
project itself uses `Masterminds/semver`. This caused go modules to be
unable to resolve both of them.
The most recent goreleaser on master switched to using the official
casing.
We reorganized the functions in flux to have the structure:
/functions
/inputs
/transformations
/outputs
this PR catches up platform to work with the new package layout.
As a separate refactoring issue, we should discuss:
from(bucket: ) should migrate from flux --> platform
to_http and to_kafka should migrate from platform --> flux
The `-i` flag causes a problem when a cross compilation happens because
it attempts to build files in GOROOT which is located in a directory
that is not writable by the build user. This same problem exists for
when the `-tags` are changed like adding `-tags assets` in the `all`
build that runs on the master branch.
This removes `-i` because, in the latest go release, the `-i` flag
doesn't actually do anything useful since there is another build cache
that the results are placed in regardless of whether `-i` is used.
This fixes the build on master so that `make all` and `make nightly`
will function properly.
This updates the dependency manager to use go modules instead of dep so
that we can remove dep as a dependency and we can begin using and giving
feedback on modules within this project. It should simplify dependency
management and make it faster to run the builds too.
The big change is updating the various Makefiles to stop relying on the
vendor directory because it no longer exists. This change creates a
`tools.go` file with the revisions of the tools we use that are currently
in the `required` section of `Gopkg.toml`. We are currently in other
discussions to modify how we handle tool dependency management, but this
change does not change that.
Jonathan A. Sternberg
Christopher M. Wolff
MikaelUrankar
Stuart Carnie
docmerlin (j. Emrys Landivar)
jlapacik
Johnny Steenbergen
Faith Chikwekwe
Kelvin Wang
Chris Goller
George
Lorenzo Affetti
Yiqun Zhang
Nathaniel Cook
Mark Rushakoff
Adam
Giacomo Tirabassi
Robert-André Mauchin
Ryan Betts
Jeff Wendling
Edd Robinson
Jacob Marble
Adam
Leonardo Di Donato
AJ Bond
Jacob Marble
Mark Rushakoff
Andrew Watkins
Michael Desa
Iris Scholten
Ben Johnson
Jeff Wendling