Renaming Generate in anticipation of a new method that will onboard
users other than the initial user. The intent is to simplify multi-user
setups.
Co-authored-by: Chris Goller <goller@gmail.com>
this is applied to all the resoruces that have had the spec.name field applied.
all resources that have not will work in the same way before this commit.
this work is the first of making resources ALL unique by metadata.name. The
displayName is a means to rename an existing resource. This is all to support
pkger idempotency. The metadata.name field will be the unique identifier within
a pkg.
* refactor(storage): move type ByTagKey to the only package that uses it
* refactor(tsdb): use types in tsdb/cursors
* refactor(tsdb): remove unused type SeriesIDElems
* refactor(tsdb): inline only use of tsdb.ReadAllSeriesIDIterator
* refactor(tsdb): move series file to its own package
* refactor(storage): remove platform->influxdb aliases
* fix: allow authorized label service to be called indirectly
17071 exists because pkger loads all service resources as authorized on
start, resulting in them all being authorized when referenced indirectly
(not hit directly via api by consumer). Rather than restructure pkger to
only authorize direct services, this allows proper indirect auth to
labels (the cause of 17071).
* Add orgService to tests
* Add resource types to find orgID from
* refactor(storage): add readSource field accessors
* refactor(storage): remove unused limitSeriesCursor
* refactor(storage): export IndexSeriesCursor
This allows IDPE to use the same implementation, rather than duplicate
code. Also copied unit tests from IDPE.
* chore: go fmt
The tasks subsystem will now use the flux language service to parse and
evaluate flux instead of directly interacting with the parser or
runtime. This helps break the dependency on the libflux parser for the
base influxdb package.
This includes the task notification packages which were changed at the
same time.
This updates the repl to support the new influxdb source and use it by
default in the repl. It will automatically set some default variables
for the influxdb source to make it easier to use the cli. In particular,
it will set the default organization, token, and the host. The
organization gets set to the one specified in the repl command and the
token gets filled in with the user installed one. The host defaults to
localhost but will change to whichever one was specified on the cli.
In addition, this will replace the http client with one that sets
insecure skip verify if the `--skip-verify` flag is used.
* fix(storage): simplify storage/seriesCursor
storage/seriesCursor releases series file and TSI references sooner.
Remove unhelpful request object, inherited from 1.x
* chore(storage): replace SeriesCursor interface with sole implementation
The repl no longer takes in a querier and it will run everything
locally. The spec interface will now not be used and will be removed
from the http endpoint at some point.
* fix(backup): handle backup with no credentials file
Backups and restores should work whether or not the original installation uses
a credentials file and whether or not the backup contains a credentials file.
Prior to this change influxql requests were sent to the same back end as Flux queries.
This MAY not always be the case. Now InfluxQL queries are specifically routed to the InfluxQLService.
In the case of this OSS build the FluxService and InfluxQLService are the same.
there was an issue where you could call, `influx pkg summarize`
and the influx cli would actually prescribe that to `influx pkg` cmd
and pass summarize as an arg. This removes that ambiguity
also provides an interface to mix and match everything together. you can
now provide `-f` flags for file or directories, `-u` flags for urls, and
use the | to pipe in a pkg. all of which can be done at the same time.
this also extend dry run to provide env refs to it. the refactoring was
to enable that bit. Having the ability to dry run with the env ref entries
means we can dry run the pkg with the env ref values to see the impact before
the application takes place.
this is the last step for pkger to follow the service definition pattern
that is in the works. Some bits from http were moved into kit/transport/http
for reusability. End result is to hopefully axe http pkg for resuable types in
kit. Long ways off still...
* feat(backup): `influx backup` creates data backup
* feat(backup): initial restore work
* feat(restore): initial restore impl
Adds a restore tool which does offline restore of data and metadata.
* fix(restore): pr cleanup
* fix(restore): fix data dir creation
* fix(restore): pr cleanup
* chore: amend CHANGELOG
* fix: restore to empty dir fails differently
* feat(backup): backup and restore credentials
Saves the credentials file to backups and restores it from backups.
Additionally adds some logging for errors when fetching backup files.
* fix(restore): add missed commit
* fix(restore): pr cleanup
* fix(restore): fix default credentials restore path
* fix(backup): actually copy the credentials file for the backup
* fix: dirs get 0777, files get 0666
* fix: small review feedback
Co-authored-by: tmgordeeva <tanya@influxdata.com>
also makes the yaml decoder the default. To foten we end up in application/octet-stream
which is the default for many different mime types. This provides a mechanism
around that so that when the automagical detection fails it can allow the user
to provide it via the CLI.
feat(pkger): export dashboard and variable *ToResource methods
fix(pkger): add empty selected _measurement to builder config
feat(chronograf): add note & note visibility to dashboard cell
The 1.x DashboardCell struct had migrated since we brought the code into
the InfluxDB codebase. This allows us to migrate cells that were created
since then.
feat(cmd/chronograf-migrator): add 1.x chronograf migrator tool
feat(chronograf-migrator): add function to transpile queries
fix: update spelling of todo comment pkger/models.go
Co-Authored-By: Deniz Kusefoglu <deniz@influxdata.com>
fix(chronograf): add type to DashboardQuery
The type has evolved since this code was moved over from chronograf.
Previously, we did not have access to flux as a type of query.
feat(chonograf-migrator): transpile influxql query to flux if possible
fix(chronograf): omit fields when empty on old chronograf structs
fix: make linter not mad at me
feat(chronograf-migrator): lowercase variable names
fix(pkger): add empty selected measurement to builder config
chore(chronograf-migrator): add basic readme
chore(pkger): export Variable and Dashboard ToResource methods
fix(chronograf-migrator): move flags out of init call
this work is to support pkger, but was able to add back in the
skipped tests. seeing failures upstream, and didn't catch it in
influxdb b/c the tests were being skipped.
closes: #14799
this is a blocker for anyone who hits the endpoint services internally. They
had to know that they need to also know of hte secret service then do all that
put/delete alongside the operation. This makes that unified inside the store tx.
one other thing this does is make obvious the dependencies that
notification services has. In this case it is the secrets service it
depends on.
this is a step towards providing a shared http client that manages pooling connections,
timeouts, and reducing GC for by not creating/GCing a client each req. Bring on the red!
tests failign from a data race caused in the tests setup. an incrementing
const needs something to serialzie it (atmoic in this case) to remove that
data race. This touches that up.
governs the concurrency with a simple semaphore. Defaults to 5
concurrent reqs, anything greater, could put a lot of pressure
on the system as a whole (especially OSS/bolt store).
* chore: Remove several instances of WithLogger
* chore: unexport Logger fields
* chore: unexport some more Logger fields
* chore: go fmt
chore: fix test
chore: s/logger/log
chore: fix test
chore: revert http.Handler.Handler constructor initialization
* refactor: integrate review feedback, fix all test nop loggers
* refactor: capitalize all log messages
* refactor: rename two logger to log
also renames a the commands to start with cmd followed by the args. this
is an attempt to order the commands in a slightly more autocomplete friendly
way.
one thing to note here is that new endpoint was created. there was no
endpoint for setting an initial password that worked. The existin endpoint
was a bit messy and coupled across multiple routes. Having multiple auth
schemes proved incredibly taxing to write against.
Previously we overwrote the tasks existing latestCompleted to be used for latestCompleted as well as latestScheduled.
For obvious reasons this is confusing and missleading. I believe by seperating the two fields we can have a clear seperation
of concerns.
does not apply to mappings, will apply mappings no matter what. we need to
remedy the uniqueness of resources from the API side. Applies to labels,
label mappings,
extends the influx CLI to include the pkg cmd for buckets. Included is
the user interface for confirmation and printing of summaries before and
after the package is applied.
Signed-off-by: Lorenzo Affetti <lorenzo.affetti@gmail.com>
Signed-off-by: Julius Volz <julius.volz@gmail.com>
move to internal
update flux to v0.50
Revert "move to internal"
This reverts commit bcd4caffbd44135f1dbeac4163cb2a22a751f45a.
promtests/internal --> internal/promtests
the flags have to match the flags with the exception of beign lower case and
all `_` be changed to `-` in the flag. This is a result of using the cobra
flag to env var mapping with the `-` replaced to `_`.
The secret service is tested by creating a secret and then attempting to
use it in a flux query. There is one test where accessing the secret
should work and one where it should return that the action is forbidden.
To have checks and notifications happen transactionally we need to be
able to alert the task system when a new task was created using the checks and notifications systems.
These two new middlewares allow us to inform the task system of a update
to a task that was created through the check or notification systems.
This command performs verification of TSM blocks
* expected and actual CRC-32 checksums match
* expected and actual min and max timestamps match decoded
data
Adds export tooling to `influxd inspect export-blocks` so that we
can dump out block data in SQL format for better analysis during
the debugging process.
The controller implementation is primarily used by influxdb so it
shouldn't be part of the flux repository. This copies the code from flux
to influxdb so it can be removed from the next flux release.
Co-authored-by: Jade McGough <jade@influxdata.com>
* Add session renew option to launcher and use in middlewhere
* pass session options to services
* Update SessionAutoRenew to SessionRenewDisabled
* Add test for service constructor defaults
* Update changelog
Until https://github.com/influxdata/flux/issues/1215 is fixed, the query
writer must make sure to always place a range directly after a from
operation. Otherwise the query will fail planning.
feat(http): add prometheus counters for tracking write/query usage
feat(http/metric): add metric recoder for recording http metrics
feat(prometheus): implement metric.Recorder for prometheus metrics
fix(prometheus): remove erroneous fmt.Printlns
feat(http): add prometheus registry to API backend
This was done as exposing prometheus metrics to a higher level was quite
difficult. It was much simple to simply pass the registry down to
anything that needs it.
feat(cmd/influxd/launcher): pass prom registry in on api backend
feat(http): collect metrics for write and query endpoints
This was much messier than I would have preferred. Future work is
outlined in TODOs.
review(influxdb): rename metric.Metric to metric.Event
This replaces usages of the spec compiler with the ast compiler and it
removes the error message referencing the spec compiler as an available
input.
It does not remove any of the code using the spec compiler that is
involved for proxying requests and it does not remove it from the API.
The synchronous executor was missing a call to ResultIterator.Release.
The asynchronous executor wasn't even calling Query.Statistics.
Also add a test that the scheduler records the statistics to the run
log, and that the statistics are visible from the launcher test. The
launcher test is the most likely place to catch if something goes wrong
in the full stack.
This commit consists of several improvements or changes:
* migrate the influxd binary to cobra.Command
* introduce a default run sub-command to start the server
* register the run sub-command flags with viper
to maintain compatibility with the existing behavior of automatic
binding of flags to environment variables.
Closes#12602
Marc Delagrammatikas
Johnny Steenbergen
Edd Robinson
Jacob Marble
Kelvin Wang
Ben Johnson
Gavin Cabbage
Greg
Sebastian Borza
Jonathan A. Sternberg
Jade McGough
Alirie Gray
jlapacik
tmgordeeva
Nathaniel Cook
Christopher M. Wolff
Stephanie Engel
Michael Desa
Yiqun Zhang
Deary Hudson III
pierwill
Chris Goller
Lyon Hill
Faith Chikwekwe
kelwang
docmerlin (j. Emrys Landivar)
Julius Volz
Johnny Steenbergen
Brandon Farmer
David McKay
Lorenzo Affetti
Edd Robinson
zhulongcheng
Adam
Max U
Adam Perlin
George
maxunt
Stuart Carnie
Iris Scholten
Palak Bhojani
Todd Persen
zhulongcheng
Jacob Marble
Mark Rushakoff