At the moment, the default session length only gets set to the default
when no service config is specified. So if a service config is used and
it does not set a value for the session length, then the session length
will be zero.
This modifies the creation of the kv service so that it will set the
default session length if the session length is set to zero (the default
value) instead of only doing it when no service config is specified.
* feat(kv): migration types for managing kv buckets and indexes over time
chore(kv): fixup comments in migrator types
fix(kv): initialize migrator bucket on kv service initialize
chore(kv): remove currently unused auth index
chore(kv): remove currently unused urm index
* chore(kv): move migrator tests into testing package and run for inmem and bolt
* chore(changelog): update changelog to reflect kv migrator type
* fix(kv): update auto migration store to return migratable store
* chore(kv): wrap error using func instead of defer in index
* chore(kv): rename Name method to MigrationName for clarity
* chore(kv): update migration log messages to match influxdb standard
* feat(kv): new index type for managing kv indexes
* feat(kv): index verify diff now can produce list of corrupt foreign keys
* fix(testing): refactor kv index tests into testing package and integrate with bolt
* chore(kv): fix go lint error in index
* fix(kv): add test for index walk
* fix(kv): change index cursing strategy to reduce calls to source bkt.Get
* feat(kv): add delete and populate remove dangling foreign keys to index
In the past, the default was only being set if a service config wasn't
provided. But if a service config was provided and gave a zero value, it
would not fill in the default value. This changes the code so that it
will always set the default value if the session length is set to zero.
The tasks subsystem will now use the flux language service to parse and
evaluate flux instead of directly interacting with the parser or
runtime. This helps break the dependency on the libflux parser for the
base influxdb package.
This includes the task notification packages which were changed at the
same time.
* Revert "fix(kv): Don't stop when key not found from index."
This reverts commit bd9167d383.
* Revert "fix(kv): push down org ID to skip in delete URM (#16841)"
This reverts commit a5f508de77.
* Revert "fix(kv): delete authorization from correct index bucket (#16835)"
This reverts commit 7349216e94.
* Revert "feat(kv): Index Authorizations by User ID (#16818)"
This reverts commit df36fe957b.
* Revert "feat: add indexes to urm for user lookups (#16789)"
This reverts commit 9561d0a4f4.
* feat(kv): add user id index on authorizations
* chore(auths): test FindAuthorizations both with and without a populated index
* chore(kv): cleanup index skipping flag in auths service
* fix(kv): bad flag around auth by user index population
* fix(kv): auth by user index lookup use correct buckets
* chore(kv): ensure indexer is called as expected when auth user index missing
* chore(kv): add benchmarks around authorization lookup
* feat(kv): Create a indexer to allow the addition of indexes
This will allow the population of indexes to be incremental and allow
for a rolling update to the index's to be handled cleanly.
issue here is that the unique by name index for variables was implemented
and has the same functionality about it that this orgs index has. The duplicative
orgs index was nuked. The migration to hydrate the org/name index never
happened. This is a stop gap until that migration is in place.
this is work moving us towards more reusable components that add some
rigidity around handling indexes and the entity bucket. The behavior
is very common across much of the kv pkg. This can be reused throughout.
adding some easy wins for tracing(eventually metrics) that enable more
insight that what is currently possible. It normalizes these concerns
across the kv store.
This change ensures the following behavior:
* task LatestScheduled is always set when a task is updated and
transitions from a status of inactive to active
* task LatestScheduled is non-zero when created, to set the initial
schedule time as some point after it was created
In addition, the kv.Service introduces clock.Clock that is used for
task create and task updates only. This change permits testing
in a deterministic fashion.
* chore: Remove several instances of WithLogger
* chore: unexport Logger fields
* chore: unexport some more Logger fields
* chore: go fmt
chore: fix test
chore: s/logger/log
chore: fix test
chore: revert http.Handler.Handler constructor initialization
* refactor: integrate review feedback, fix all test nop loggers
* refactor: capitalize all log messages
* refactor: rename two logger to log
* feat(auth): add createdAt and updatedAt to authorization
Co-Authored-By: Ariel <ariel.salem1989@gmail.com>
* feat(auth): passing createAuth tests
* test: ensured that createdAt and updatedAt are valid on authorizations
At times snowflake id generation would create org and bucket IDs with
characters that had special meaning for the storage engine.
The storage engine concats the org and bucket bytes together into a
single 128 bit value. That value is used in the old measurement
section. Measurement was transformed into the tag, _measurement.
However, certain properties of the older measurement data location
are still required for the org/bucket bytes. We cannot have
commas, spaces, nor backslashes.
This PR puts a specific ID generator in place during the creation of
orgs and buckets. The IDs are just random numbers but with each
of the restricted chars incremented by one. While this changes the
entropy distribution somewhat, it does not matter too much for our
purposes.
... because now org and bucket ids are checked for previous existence
transactionally in the key-value stores. If the ID does already exist
then we try to generate a new key up to 100 times.
Co-authored-by: Jade McGough <jade@influxdata.com>
* Add session renew option to launcher and use in middlewhere
* pass session options to services
* Update SessionAutoRenew to SessionRenewDisabled
* Add test for service constructor defaults
* Update changelog
feat(influxdb): add generic store for documents
feat(influxdb): support authorizations in document store
feat(influxdb): support orgs in user resource mapping
feat(influxdb): add read-only included field on documents
feat(influxdb): add labels support to documents service
fix(influxdb): rename data field to content on documents
feat(influxdb): add with org id options for document store
feat(http): add templates swagger
feat(influxdb): add documentation to document options
doc(kv): add documentation for kv document store
test(kv): pull document tests in to the testing package
fix(http): fix swagger specification of templates endpoints
* feat(kv:inmem:bolt): implement user service in a kv
* refactor(kv): use consistent func receiver name
* feat(kv): add initial basic auth service
* refactor(passwords): move auth interface into own file
* refactor(passwords): rename basic auth files to passwords
* refactor(passwords): rename from BasicAuth to Passwords
* refactor(kv): copy bolt user test into kv
Co-authored-by: Michael Desa <mjdesa@gmail.com>
* feat(kv): add inmem testing to kv store
* fix(kv): remove extra user index initialization
* feat(kv): attempt at making errors nice
* fix(http): return not found error if filter is invalid
* fix(http): s/platform/influxdb/ for user service
* fix(http): s/platform/influxdb/ for user service
* feat(kv): initial port of telegraf configs to kv
* feat(kv): first pass at migrating bolt org service to kv
* feat(kv): first pass at bucket service
* feat(kv): first pass at migrating kvlog to kv package
* feat(kv): add resource op logs
* feat(kv): first pass at user resource mapping migration
* feat(kv): add urm usage to bucket and org services
* feat(kv): first pass at kv authz service
* feat(kv): add cascading auth delete for users
* feat(kv): first pass d authorizer.OrganizationService in kv
* feat(cmd/influxd/launcher): user kv services where appropriate
* fix(kv): initialize authorizations
* fix(influxdb): use same buckets while slowly migrating stuff
* fix(kv): make staticcheck pass
* feat(kv): add dashboards to kv
review: make suggestions from pr review
fix: use common bucket names for bolt/kv stores
* test(kv): add complete password test coverage
* chore(kv): fixes for staticcheck
* feat(kv): implement labels generically on kv
* feat(kv): implement macro service
* feat(kv): add source service
* feat(kv): add session service
* feat(kv): add kv secret service
* refactor(kv): update telegraf and urm with error messages
* feat(kv): add lookup service
* feat(kv): add kv onboarding service
* refactor(kv): update telegraf to avoid repetition
* feat(cmd/influxd): use kv lookup service
* feat(kv): add telegraf to lookup service
* feat(cmd/influxd): use kv telegraf service
* feat(kv): initial port of scrapers in bolt to kv
* feat(kv): update scraper error messaging
* feat(cmd/influxd): add kv scraper
* feat(kv): add inmem backend tests
* refactor(kv): copy paste errors
* refactor(kv): add code to password errors
* fix(testing): update error messages for incorrect passwords
* feat(kv:inmem:bolt): implement user service in a kv
* refactor(kv): use consistent func receiver name
* refactor(kv): copy bolt user test into kv
Co-authored-by: Michael Desa <mjdesa@gmail.com>
* feat(kv): add inmem testing to kv store
* fix(kv): remove extra user index initialization
* feat(kv): attempt at making errors nice
* fix(http): return not found error if filter is invalid
* fix(http): s/platform/influxdb/ for user service
* feat(kv): first pass at migrating bolt org service to kv
* feat(kv): first pass at bucket service
* feat(kv): first pass at migrating kvlog to kv package
* feat(kv): add resource op logs
* feat(kv): first pass at user resource mapping migration
* feat(kv): add urm usage to bucket and org services
* feat(kv): first pass at kv authz service
* feat(kv): add cascading auth delete for users
* feat(kv): first pass d authorizer.OrganizationService in kv
* feat(cmd/influxd/launcher): user kv services where appropriate
* feat(kv): add initial basic auth service
* refactor(passwords): move auth interface into own file
* refactor(passwords): rename basic auth files to passwords
* fix(http): s/platform/influxdb/ for user service
* fix(kv): initialize authorizations
* fix(influxdb): use same buckets while slowly migrating stuff
* fix(kv): make staticcheck pass
* feat(kv): add dashboards to kv
review: make suggestions from pr review
fix: use common bucket names for bolt/kv stores
* feat(kv): implement labels generically on kv
* refactor(passwords): rename from BasicAuth to Passwords
* test(kv): add complete password test coverage
* chore(kv): fixes for staticcheck
* feat(kv): implement macro service
* feat(kv): add source service
* feat(kv): add session service
* feat(kv): initial port of telegraf configs to kv
* feat(kv): initial port of scrapers in bolt to kv
* feat(kv): add kv secret service
* refactor(kv): update telegraf and urm with error messages
* feat(kv): add lookup service
* feat(kv): add kv onboarding service
* refactor(kv): update telegraf to avoid repetition
* feat(cmd/influxd): use kv lookup service
* feat(kv): add telegraf to lookup service
* feat(cmd/influxd): use kv telegraf service
* feat(kv): update scraper error messaging
* feat(cmd/influxd): add kv scraper
* feat(kv): add inmem backend tests
* refactor(kv): copy paste errors
* refactor(kv): add code to password errors
* fix(testing): update error messages for incorrect passwords
* feat(http): initial support for flushing all key/values from kv store
* feat(kv): rename macro to variable
* feat(cmd/influxd/launcher): user kv services where appropriate
* refactor(passwords): rename from BasicAuth to Passwords
* feat(kv): implement macro service
* test(ui): introduce cypress
* test(ui): introduce first typescript test
* test(ui/e2e): add ci job
* chore: update gitignore to ignore test outputs
* feat(inmem): in memory influxdb
* test(e2e): adding pinger that checks if influxdb is alive
* hackathon
* hack
* hack
* hack
* hack
* Revert "feat(inmem): in memory influxdb"
This reverts commit 30ddf032003e704643b07ce80df61c3299ea7295.
* hack
* hack
* hack
* hack
* hack
* hack
* hack
* hack
* hack
* hack
* hack
* hack
* hack
* chore: lint ignore node_modules
* hack
* hack
* hack
* add user and flush
* hack
* remove unused vars
* hack
* hack
* ci(circle): prefix e2e artifacts
* change test to testid
* update cypress
* moar testid
* fix npm warnings
* remove absolte path
* chore(ci): remove /home/circleci proto mkdir hack
* wip: crud resources e2e
* fix(inmem): use inmem kv store services
* test(dashboard): add first dashboard crud tests
* hack
* undo hack
* fix: use response from setup for orgID
* chore: wip
* add convenience getByTitle function
* test(e2e): ui can create orgs
* test(e2e): add test for org deletion and update
* test(e2e): introduce task creation test
* test(e2e): create and update of buckets on org view
* chore: move types to declaration file
* chore: use route fixture in dashboard tests
* chore(ci): hack back
* test(ui): update snapshots
* chore: package-lock
* chore: remove macros
* fix: launcher rebase issues
* fix: compile errors
* fix: compile errors
* feat(cmd/influxdb): add explicit testing, asset-path, and store flags
Co-authored-by: Andrew Watkins <watts@influxdb.com>
* fix(cmd/influxd): set default HTTP handler and flags
Co-authored-by: Andrew Watkins <watts@influxdb.com>
* build(Makefile): add run-e2e and PHONY
* feat(kv:inmem:bolt): implement user service in a kv
* refactor(kv): use consistent func receiver name
* feat(kv): add initial basic auth service
* refactor(passwords): move auth interface into own file
* refactor(passwords): rename basic auth files to passwords
* refactor(passwords): rename from BasicAuth to Passwords
* refactor(kv): copy bolt user test into kv
Co-authored-by: Michael Desa <mjdesa@gmail.com>
* feat(kv): add inmem testing to kv store
* fix(kv): remove extra user index initialization
* feat(kv): attempt at making errors nice
* fix(http): return not found error if filter is invalid
* fix(http): s/platform/influxdb/ for user service
* fix(http): s/platform/influxdb/ for user service
* feat(kv): initial port of telegraf configs to kv
* feat(kv): initial port of scrapers in bolt to kv
* feat(kv): first pass at migrating bolt org service to kv
* feat(kv): first pass at bucket service
* feat(kv): first pass at migrating kvlog to kv package
* feat(kv): add resource op logs
* feat(kv): first pass at user resource mapping migration
* feat(kv): add urm usage to bucket and org services
* feat(kv): first pass at kv authz service
* feat(kv): add cascading auth delete for users
* feat(kv): first pass d authorizer.OrganizationService in kv
* feat(cmd/influxd/launcher): user kv services where appropriate
* fix(kv): initialize authorizations
* fix(influxdb): use same buckets while slowly migrating stuff
* fix(kv): make staticcheck pass
* feat(kv): add dashboards to kv
review: make suggestions from pr review
fix: use common bucket names for bolt/kv stores
* test(kv): add complete password test coverage
* chore(kv): fixes for staticcheck
* feat(kv): implement labels generically on kv
* feat(kv): implement macro service
* feat(kv): add source service
* feat(kv): add session service
* feat(kv): add kv secret service
* refactor(kv): update telegraf and urm with error messages
* feat(kv): add lookup service
* feat(kv): add kv onboarding service
* refactor(kv): update telegraf to avoid repetition
* feat(cmd/influxd): use kv lookup service
* feat(kv): add telegraf to lookup service
* feat(cmd/influxd): use kv telegraf service
* feat(kv): update scraper error messaging
* feat(cmd/influxd): add kv scraper
* feat(kv): add inmem backend tests
* refactor(kv): copy paste errors
* refactor(kv): add code to password errors
* fix(testing): update error messages for incorrect passwords
* feat(kv): rename macro to variable
* refactor(kv): auth/bucket/org/user unique checks return errors now
* feat(inmem): add way to get all bucket names from store
* feat(inmem): Buckets to return slice of bytes rather than strings
* feat(inmem): add locks around Buckets to avoid races
* feat(cmd/influx): check for unauthorized error in wrapCheckSetup
* chore(e2e): add video and screenshot artifcats to gitignore
* docs(ci): add build instructions for e2e tests
* feat(kv): add id lookup for authorized resources
Lyon Hill
jlapacik
George
Jonathan A. Sternberg
Gavin Cabbage
Johnny Steenbergen
Stuart Carnie
Jacob Marble
Ariel Salem
Chris Goller
Ben Johnson
Kelvin Wang
Jade McGough
Alirie Gray
kelwang
Iris Scholten
Michael Desa