At times snowflake id generation would create org and bucket IDs with
characters that had special meaning for the storage engine.
The storage engine concats the org and bucket bytes together into a
single 128 bit value. That value is used in the old measurement
section. Measurement was transformed into the tag, _measurement.
However, certain properties of the older measurement data location
are still required for the org/bucket bytes. We cannot have
commas, spaces, nor backslashes.
This PR puts a specific ID generator in place during the creation of
orgs and buckets. The IDs are just random numbers but with each
of the restricted chars incremented by one. While this changes the
entropy distribution somewhat, it does not matter too much for our
purposes.
... because now org and bucket ids are checked for previous existence
transactionally in the key-value stores. If the ID does already exist
then we try to generate a new key up to 100 times.
We need to only update the updated at time when we recieve a external request for
an update. LatestCompleted is an internal request from the scheduler.
When using a internal function in kv we can only use internal funciton calls.
By trying to use a Public function we attempt to create a read lock after acquiring a write lock
When a task was created we used to use a token, we then decided to switch to using
A user id. To facilitate this change we pull the users auth associated with the task
and use the auth's `GetUserID` method. This only works if they have not deleted the auth.
We need a fail safe way to populate the ownerID in the circumstance that the auth associated
with the task has been deleted. To facilitate this we can pull the UserResourceMapping's for the org
resource of the type `Owner` and use a organization owner as the task owner. This makes sense because
any organization owner also own's any part of the organization including tasks.
fix(notification/check): include tags in check object in generated flux
Closes https://github.com/influxdata/influxdb/issues/14769
fix(notification/check): use selected field in threshold functions
Closes https://github.com/influxdata/influxdb/issues/14776
fix(testing): add selected field for check tests
fix(check): use real flux for threshold check
feat(notification/check): generate flux for deadman checks
chore(endpoint): rename webhook endpoint to http endpoint
fix(notification/rule): fetch url for flux script off of endpoint
fix(notification/rule): clean up slack and http rules
fix(notification/rule): change MessageTemp to MessageTemplate
fix(rules): pass endpoint in to rule during create
fix(ui): rename webhook to http
feat(notification/check): namespace deadman under alerts
fix(notification/check): nest tags under tags key in data object in flux
wip
feat(kv): log error if urm cannot be deleted for notification rule
fix(notification/rule): remove name from notify call in slack rule
chore(ui/cypress/e2e): skip rule create test
If a task is created and the auth associated is removed
lookup of tasks currently fail. This is bad behavior.
We should expect to be able to at least see and delete task's that
have old auth.
* fix(task): Remove unnecessary auth lookups
Currently we look up task authorization for all task lookups even if we are only
looking up the task for internal use's. We can keep the same interface but cutdown
on the amount of lookups we do by using seperate internal functions for task lookup
and task with id lookup
* feat(task): Remove token's from task structures
We had previously removed token's from the task api but left the token in place in several locations in the stack.
Now we can cleanly remove the extra tokens.
* feat(task): impersonate user on task execution
Passing tokens to tasks is cumbersome and we needed a way to more easily create tasks. With this change we no longer need a token on task create. We take the user that created the task and pass that in as the "owner". As far as the task is concerned the owner is the source of permissions.
This is done by adding an additional field on task create that is OwnerID. We will no longer respect the token passed in and it will be deprecated soon.
Things to do still:
Task updates need to allow for owners to be set.
Current behavior is that the first execution of a task happens based on the create time
of the task when using a 'every' schedule. If you create a task at 12:02 and want
the task to run every 15m. The first execution would happen at 12:17, and the 2nd would happen
at 12:30.
To fix this behavior I refactored the kv task to give a single source of knowledge.
We now have one function for finding exactly what the last scheduled task was.
We also now have a single method that calculates when the next schedule is due.
By unifying the logic it should always work the same way weather your asking when to run
or when creating a task.
First pass at flux AST generation from check
Co-authored-by: Michael Desa <mjdesa@gmail.com>
Co-authored-by: Deniz Kusefoglu <deniz@influxdata.com>
fix(notification/check): format call expression
Co-authored-by: Michael Desa <mjdesa@gmail.com>
Co-authored-by: Deniz Kusefoglu <deniz@influxdata.com>
fix(notification/check): cleanup CheckDefinition
Co-authored-by: Michael Desa <mjdesa@gmail.com>
Co-authored-by: Deniz Kusefoglu <deniz@influxdata.com>
fix(notification/check): clean up threshold functions
Co-authored-by: Michael Desa <mjdesa@gmail.com>
Co-authored-by: Deniz Kusefoglu <deniz@influxdata.com>
fix(notification/check): clean up message function
Co-authored-by: Michael Desa <mjdesa@gmail.com>
Co-authored-by: Deniz Kusefoglu <deniz@influxdata.com>
fix(notification/check): misc fixes
Co-authored-by: Michael Desa <mjdesa@gmail.com>
Co-authored-by: Deniz Kusefoglu <deniz@influxdata.com>
fix(notification/check): remove dead code
Co-authored-by: Michael Desa <mjdesa@gmail.com>
Co-authored-by: Deniz Kusefoglu <deniz@influxdata.com>
fix(notification/check): move threshold flux generation to check pkg
Co-authored-by: Michael Desa <mjdesa@gmail.com>
Co-authored-by: Deniz Kusefoglu <deniz@influxdata.com>
fix(notification/check): move base ast generation to its own package
Co-authored-by: Michael Desa <mjdesa@gmail.com>
Co-authored-by: Deniz Kusefoglu <deniz@influxdata.com>
fix(notification/check): add comment for GenerateFluxAST
Co-authored-by: Michael Desa <mjdesa@gmail.com>
Co-authored-by: Deniz Kusefoglu <deniz@influxdata.com>
docs(notification/flux): add comments to each exported function
Co-authored-by: Michael Desa <mjdesa@gmail.com>
Co-authored-by: Deniz Kusefoglu <deniz@influxdata.com>
feat(notification/check): add tests for GenerateFlux
Co-authored-by: Michael Desa <mjdesa@gmail.com>
Co-authored-by: Deniz Kusefoglu <deniz@influxdata.com>
feat(notification/check): add task options to generated flux
fix(notification/check): use flux compatible duration type
test(notification/check): add task option to task definition
test(http): use check Duration in checks http handlers
feat(check): add TaskID to checks base
fix(notification/check): hack around issue with formatting ast package
wtih multiple files
test(check): create task when check is created
A lot of little changes had to happen as a result of this. This change
was rather painful.
feat(checks): add update and delete of task for check
fix(notifications/check): hack around the alerts package not being
available
test(kv): temporarily skip check tests while we merge the pr above
Chris Goller
Jade McGough
Lyon Hill
Kelvin Wang
Ben Johnson
Michael Desa
kelwang
Alirie Gray
Deniz Kusefoglu
zhulongcheng
lisale0