This new session service has the ability to work independant of other systems
it relies on having its own store type which should allow us to be more flexible
then using the built in kv system.
I have included an in mem session store.
* chore(kv): add failing test for session renewal extension
* fix(kv): ensure renew session only updates expiration if it is newer than existing
* chore: update changelog to reflect renew session expiration fix
At the moment, the default session length only gets set to the default
when no service config is specified. So if a service config is used and
it does not set a value for the session length, then the session length
will be zero.
This modifies the creation of the kv service so that it will set the
default session length if the session length is set to zero (the default
value) instead of only doing it when no service config is specified.
Renaming Generate in anticipation of a new method that will onboard
users other than the initial user. The intent is to simplify multi-user
setups.
Co-authored-by: Chris Goller <goller@gmail.com>
* feat(kv): migration types for managing kv buckets and indexes over time
chore(kv): fixup comments in migrator types
fix(kv): initialize migrator bucket on kv service initialize
chore(kv): remove currently unused auth index
chore(kv): remove currently unused urm index
* chore(kv): move migrator tests into testing package and run for inmem and bolt
* chore(changelog): update changelog to reflect kv migrator type
* fix(kv): update auto migration store to return migratable store
* chore(kv): wrap error using func instead of defer in index
* chore(kv): rename Name method to MigrationName for clarity
* chore(kv): update migration log messages to match influxdb standard
* fix: allow authorized label service to be called indirectly
17071 exists because pkger loads all service resources as authorized on
start, resulting in them all being authorized when referenced indirectly
(not hit directly via api by consumer). Rather than restructure pkger to
only authorize direct services, this allows proper indirect auth to
labels (the cause of 17071).
* Add orgService to tests
* Add resource types to find orgID from
* feat(kv): new index type for managing kv indexes
* feat(kv): index verify diff now can produce list of corrupt foreign keys
* fix(testing): refactor kv index tests into testing package and integrate with bolt
* chore(kv): fix go lint error in index
* fix(kv): add test for index walk
* fix(kv): change index cursing strategy to reduce calls to source bkt.Get
* feat(kv): add delete and populate remove dangling foreign keys to index
In the past, the default was only being set if a service config wasn't
provided. But if a service config was provided and gave a zero value, it
would not fill in the default value. This changes the code so that it
will always set the default value if the session length is set to zero.
This moves a few types and constants to the global package so it can be
used without importing the `task/backend` package. These constants are
referenced in non tasks-specific code.
This is needed to break a dependency chain where the task backend will
call into the flux runtime to perform parsing or evaluation of a script
and to prevent the http package from inheriting that dependency.
The tasks subsystem will now use the flux language service to parse and
evaluate flux instead of directly interacting with the parser or
runtime. This helps break the dependency on the libflux parser for the
base influxdb package.
This includes the task notification packages which were changed at the
same time.
* Revert "fix(kv): Don't stop when key not found from index."
This reverts commit bd9167d383.
* Revert "fix(kv): push down org ID to skip in delete URM (#16841)"
This reverts commit a5f508de77.
* Revert "fix(kv): delete authorization from correct index bucket (#16835)"
This reverts commit 7349216e94.
* Revert "feat(kv): Index Authorizations by User ID (#16818)"
This reverts commit df36fe957b.
* Revert "feat: add indexes to urm for user lookups (#16789)"
This reverts commit 9561d0a4f4.
* fix(kv): push down org ID to skip in delete URM
* fix(kv): use database key rather than resource id
We are trying to skip deletes that would remove keys
that have already been deleted. This is a rather
extreme approach and I believe we should think about how
to fix user-resource-mappings.
Co-authored-by: Lyon Hill <lyondhill@gmail.com>
Signed-off-by: Chris Goller <goller@gmail.com>
Co-authored-by: George <me@georgemac.com>
Co-authored-by: Lyon Hill <lyondhill@gmail.com>
* fix(kv): delete authorization from correct index bucket
* fix(kv): return not found code when user resource mapping indexed by not in source
* chore(kv): define failing test for URM on delete
* feat(kv): add user id index on authorizations
* chore(auths): test FindAuthorizations both with and without a populated index
* chore(kv): cleanup index skipping flag in auths service
* fix(kv): bad flag around auth by user index population
* fix(kv): auth by user index lookup use correct buckets
* chore(kv): ensure indexer is called as expected when auth user index missing
* chore(kv): add benchmarks around authorization lookup
* feat(kv): Create a indexer to allow the addition of indexes
This will allow the population of indexes to be incremental and allow
for a rolling update to the index's to be handled cleanly.
* Begin implementing retreival of telegraf plugin stats
* Implement storing/deletion of telegraf plugin stats
* Test plugin stats
* Initialize plugins bucket for tests
* Add comment
* Shorten time and frequency in bolt when providing telegraf plugins metrics
* Simplify ticker loop
* Leak underlying ticker while still satisfying linter
* fix(kv): Update scrapers to use new forward cursor
I also made a minor update to move a db lookup outside of a for loop to save
time and optimize
* fix(inmem): fix a potential race condition
* feat(backup): `influx backup` creates data backup
* feat(backup): initial restore work
* feat(restore): initial restore impl
Adds a restore tool which does offline restore of data and metadata.
* fix(restore): pr cleanup
* fix(restore): fix data dir creation
* fix(restore): pr cleanup
* chore: amend CHANGELOG
* fix: restore to empty dir fails differently
* feat(backup): backup and restore credentials
Saves the credentials file to backups and restores it from backups.
Additionally adds some logging for errors when fetching backup files.
* fix(restore): add missed commit
* fix(restore): pr cleanup
* fix(restore): fix default credentials restore path
* fix(backup): actually copy the credentials file for the backup
* fix: dirs get 0777, files get 0666
* fix: small review feedback
Co-authored-by: tmgordeeva <tanya@influxdata.com>
* feat(checks): Add custom check type
* feat(checks): Remove alert builder from custom check
* feat(checks): Add AlertBuilderAction to list of possible actions
* feat(checks): Query visualization does not make sense for custom check
* feat(check): check editor should only reexecute queries if view query changes
* Update ui/src/timeMachine/components/TimeMachineFluxEditor.tsx
Co-Authored-By: Bucky Schwarz <hoorayimhelping@users.noreply.github.com>
* Address PR review
Co-authored-by: Bucky Schwarz <hoorayimhelping@users.noreply.github.com>
issue here is that the unique by name index for variables was implemented
and has the same functionality about it that this orgs index has. The duplicative
orgs index was nuked. The migration to hydrate the org/name index never
happened. This is a stop gap until that migration is in place.
this is work moving us towards more reusable components that add some
rigidity around handling indexes and the entity bucket. The behavior
is very common across much of the kv pkg. This can be reused throughout.
adding some easy wins for tracing(eventually metrics) that enable more
insight that what is currently possible. It normalizes these concerns
across the kv store.
this work is to support pkger, but was able to add back in the
skipped tests. seeing failures upstream, and didn't catch it in
influxdb b/c the tests were being skipped.
closes: #14799
In the event that findTaskByIDWithAuth cannot find the task ID contained
in the bucket, the outer loop will never terminate.
This ensures that we are calling Next() along-side any calls to continue
while using a Cursor.
this is a blocker for anyone who hits the endpoint services internally. They
had to know that they need to also know of hte secret service then do all that
put/delete alongside the operation. This makes that unified inside the store tx.
one other thing this does is make obvious the dependencies that
notification services has. In this case it is the secrets service it
depends on.
Task runs are stored and retrieved from the `taskRunsv1` bucket, but
when they are canceled they are incorrectly placed in the `tasksv1`
bucket. Once this has been done, further look ups of the task run fail,
because it is located in the wrong bucket.
This addresses the problem by placing them back into the `taskRunsv1`
bucket. An additional test has been added to ensure we are able to
successfully read a canceled run.
note: tests are seriously borked here. Cannot reuse any existing testing
as the setup is very particular and the http layer doesn't suppport everyting.
that being said, there are goign to be implicit testing in the
`launcher/pkger_test.go` file. This feels broken, and probably needs to be
readdressed before we GA a 2.0 influxdb....
This change ensures the following behavior:
* task LatestScheduled is always set when a task is updated and
transitions from a status of inactive to active
* task LatestScheduled is non-zero when created, to set the initial
schedule time as some point after it was created
In addition, the kv.Service introduces clock.Clock that is used for
task create and task updates only. This change permits testing
in a deterministic fashion.
Alirie Gray
jlapacik
George
Lyon Hill
Johnny Steenbergen
Brett Buddin
Jonathan A. Sternberg
Yiqun (Ethan) Zhang
Lorenzo Affetti
Marc Delagrammatikas
Gavin Cabbage
Greg
Chris Goller
Christopher M. Wolff
Kelvin Wang
Gavin Cabbage
Jacob Marble
Zoe Steinkamp
Deniz Kusefoglu
Stuart Carnie