Commit Graph

236 Commits (68183bb15694c594f92c2c4f9aa350c6b874cb50)

Author SHA1 Message Date
Lyon Hill 4421a39639
feat(tenant): Build out the onboarding system in tenant and integrate it into launcher (#17558)
We are adding in a setup/user route this is not in swagger at the moment but will be
added once we feel it is stable.
2020-04-06 15:58:15 -06:00
Jonathan A. Sternberg 0ae8bebd75
refactor: rewrite imports to include the /v2 suffix for version 2 2020-04-03 12:39:20 -05:00
George 2840e6f26e
feat(kv): add ability to configure a max permissions using auth filter function (#17598) 2020-04-03 17:02:51 +01:00
George 0a7772ac93
fix(kv): update index to use new bucket.GetBatch (#17544)
* fix(kv): update index to use new bucket.GetBatch

* chore(kv): add index walk benchmark for inmem and bolt
2020-04-02 13:12:35 +01:00
Jonathan A. Sternberg f385840f3a
refactor(kv): set the default session length when not specified (#17547)
At the moment, the default session length only gets set to the default
when no service config is specified. So if a service config is used and
it does not set a value for the session length, then the session length
will be zero.

This modifies the creation of the kv service so that it will set the
default session length if the session length is set to zero (the default
value) instead of only doing it when no service config is specified.
2020-04-01 15:02:22 -05:00
George 9d5d63518d
feat(kv): add support for batch getting keys on bucket interface (#17531) 2020-04-01 17:51:35 +01:00
George 173ded1a10
fix(kv): make index walk use cursor instead of get (#17501) 2020-03-31 17:26:21 +01:00
Yiqun (Ethan) Zhang 48efdb52ea
build(flux): update Flux to v0.65.0 (#17484) 2020-03-27 18:42:19 -04:00
Lorenzo Affetti 1e36e7b3ad
feat(tenant): add duplicate read service 2020-03-25 12:06:28 +01:00
Marc Delagrammatikas 71ee028973 feat: export transactions to support importing kv as a library
Co-authored-by: Chris Goller <goller@gmail.com>
2020-03-20 11:59:11 -07:00
Marc Delagrammatikas ea8e6d15d0 refactor: fixes typo in OnboardInitialUser method
Fixes typo introduced by 7b99c28a99
2020-03-20 10:48:20 -07:00
Johnny Steenbergen 481ee1653d fix: add validation for existing resource for index store base
issue was that all checks were being made excdept for the initial check
in the entity store. fix added with tests here to validate.
2020-03-20 10:15:02 -07:00
George 6f0d769625
feat(kv): add user resource mapping by user index (#17349)
* feat(kv): add user resource mapping by user index

* feat(kv): add support for reading from URM user index when enabled
2020-03-20 11:03:00 +00:00
Lorenzo Affetti 413f7ad126
test: add tests for tenant service 2020-03-19 11:34:15 +01:00
Marc Delagrammatikas 7b99c28a99 refactor: rename Generate to OnboardInitialUser
Renaming Generate in anticipation of a new method that will onboard
users other than the initial user. The intent is to simplify multi-user
setups.

Co-authored-by: Chris Goller <goller@gmail.com>
2020-03-18 15:35:26 -07:00
George 1d400a4f0f
feat(kv): support for migrations (#17145)
* feat(kv): migration types for managing kv buckets and indexes over time

chore(kv): fixup comments in migrator types

fix(kv): initialize migrator bucket on kv service initialize

chore(kv): remove currently unused auth index

chore(kv): remove currently unused urm index

* chore(kv): move migrator tests into testing package and run for inmem and bolt

* chore(changelog): update changelog to reflect kv migrator type

* fix(kv): update auto migration store to return migratable store

* chore(kv): wrap error using func instead of defer in index

* chore(kv): rename Name method to MigrationName for clarity

* chore(kv): update migration log messages to match influxdb standard
2020-03-18 12:23:51 +00:00
Gavin Cabbage f5e8547482
fix(buckets): prevent returning system buckets to unauthorized users (#17117) 2020-03-11 13:06:08 -04:00
Greg f98874566c
fix: allow authorized label service to be called indirectly (#17111)
* fix: allow authorized label service to be called indirectly

17071 exists because pkger loads all service resources as authorized on
start, resulting in them all being authorized when referenced indirectly
(not hit directly via api by consumer). Rather than restructure pkger to
only authorize direct services, this allows proper indirect auth to
labels (the cause of 17071).

* Add orgService to tests

* Add resource types to find orgID from
2020-03-11 10:58:39 -06:00
George e25b8e5931
feat(kv): index utility type (#16910)
* feat(kv): new index type for managing kv indexes

* feat(kv): index verify diff now can produce list of corrupt foreign keys

* fix(testing): refactor kv index tests into testing package and integrate with bolt

* chore(kv): fix go lint error in index

* fix(kv): add test for index walk

* fix(kv): change index cursing strategy to reduce calls to source bkt.Get

* feat(kv): add delete and populate remove dangling foreign keys to index
2020-03-11 14:42:29 +00:00
Lorenzo Affetti fd729f8a38
fix(http): refactor document service and fix auth 2020-03-09 20:32:31 +01:00
Jonathan A. Sternberg 834a8740e0
refactor(task/backend): move the task/backend constants to the global package (#17133)
This moves a few types and constants to the global package so it can be
used without importing the `task/backend` package. These constants are
referenced in non tasks-specific code.

This is needed to break a dependency chain where the task backend will
call into the flux runtime to perform parsing or evaluation of a script
and to prevent the http package from inheriting that dependency.
2020-03-06 16:19:32 -06:00
Lorenzo Affetti 9defb8a77f
feat(http): add document service 2020-02-21 15:58:40 +01:00
George f239a2ed9c
fix(kv): remove URM and Authorizations by UserID Index (#16852)
* Revert "fix(kv): Don't stop when key not found from index."

This reverts commit bd9167d383.

* Revert "fix(kv): push down org ID to skip in delete URM (#16841)"

This reverts commit a5f508de77.

* Revert "fix(kv): delete authorization from correct index bucket (#16835)"

This reverts commit 7349216e94.

* Revert "feat(kv): Index Authorizations by User ID (#16818)"

This reverts commit df36fe957b.

* Revert "feat: add indexes to urm for user lookups (#16789)"

This reverts commit 9561d0a4f4.
2020-02-13 15:27:57 +00:00
Brett Buddin bd9167d383
fix(kv): Don't stop when key not found from index. 2020-02-12 21:38:17 -05:00
Chris Goller a5f508de77
fix(kv): push down org ID to skip in delete URM (#16841)
* fix(kv): push down org ID to skip in delete URM

* fix(kv): use database key rather than resource id

We are trying to skip deletes that would remove keys
that have already been deleted.  This is a rather
extreme approach and I believe we should think about how
to fix user-resource-mappings.

Co-authored-by:  Lyon Hill <lyondhill@gmail.com>
Signed-off-by: Chris Goller <goller@gmail.com>

Co-authored-by: George <me@georgemac.com>
Co-authored-by: Lyon Hill <lyondhill@gmail.com>
2020-02-12 14:33:57 -06:00
George 7349216e94
fix(kv): delete authorization from correct index bucket (#16835)
* fix(kv): delete authorization from correct index bucket

* fix(kv): return not found code when user resource mapping indexed by not in source

* chore(kv): define failing test for URM on delete
2020-02-12 18:36:45 +00:00
George df36fe957b
feat(kv): Index Authorizations by User ID (#16818)
* feat(kv): add user id index on authorizations

* chore(auths): test FindAuthorizations both with and without a populated index

* chore(kv): cleanup index skipping flag in auths service

* fix(kv): bad flag around auth by user index population

* fix(kv): auth by user index lookup use correct buckets

* chore(kv): ensure indexer is called as expected when auth user index missing

* chore(kv): add benchmarks around authorization lookup
2020-02-11 17:34:20 +00:00
Lyon Hill 9561d0a4f4
feat: add indexes to urm for user lookups (#16789) 2020-02-11 09:17:37 -07:00
Lyon Hill ec13042c57
feat(kv): Create a indexer to allow the addition of indexes (#16745)
* feat(kv): Create a indexer to allow the addition of indexes

This will allow the population of indexes to be incremental and allow
for a rolling update to the index's to be handled cleanly.
2020-02-07 09:01:37 -07:00
George bc64968b0b
chore(task): add test around paging using after (#16774)
* chore(task): add test around paging using after

* fix(kv): support filter.After in FindTasksByUser
2020-02-07 15:48:55 +00:00
George fe990aa4db
feat(kv): add cursor option skip first item (#16758) 2020-02-06 12:18:47 -07:00
Lyon Hill ef848d0aa4
fix(kv): task lookup for all tasks will now skip the after key (#16757) 2020-02-06 10:44:36 -07:00
Kelvin Wang 53d9caf1c6 fix(kv): fix patching panics 2020-02-05 11:12:03 -05:00
Greg e593119c3d
feat: report telegraf plugin usage metrics (#16378)
* Begin implementing retreival of telegraf plugin stats

* Implement storing/deletion of telegraf plugin stats

* Test plugin stats

* Initialize plugins bucket for tests

* Add comment

* Shorten time and frequency in bolt when providing telegraf plugins metrics

* Simplify ticker loop

* Leak underlying ticker while still satisfying linter
2020-02-04 08:24:58 -07:00
Lyon Hill 8bd6044398
fix(kv): update user and variables to use forward cursor (#16671) 2020-02-03 08:50:06 -07:00
Lyon Hill f385d19713
fix(kv): update tasks kv to use forward cursor (#16670) 2020-02-03 08:49:51 -07:00
Lyon Hill cb7f053d45
fix(kv): Update scrapers to use new forward cursor (#16647)
* fix(kv): Update scrapers to use new forward cursor

I also made a minor update to move a db lookup outside of a for loop to save
time and optimize

* fix(inmem): fix a potential race condition
2020-01-24 13:35:47 -07:00
Lyon Hill d1c0ddc362
fix(kv): update kv source to use the new forward cursor (#16659) 2020-01-24 09:46:01 -07:00
Lyon Hill 0fa52eec9d
fix(kv): update kv secrets to use new forward cursor (#16648) 2020-01-24 09:24:20 -07:00
Gavin Cabbage b91d778579 feat: resource log 2020-01-23 13:04:06 -05:00
Lyon Hill 379fd085fc
fix(kv): update org collection to use forward cursor (#16639) 2020-01-23 08:47:02 -07:00
Lyon Hill d509946bef
fix(kv): update notification rules to use the new forward cursor (#16637) 2020-01-23 08:46:53 -07:00
Lyon Hill 25cc4ea83d
fix(kv): update label collection to use new forward cursor (#16636) 2020-01-23 08:46:40 -07:00
Lyon Hill a7dddba728
fix(kv): Update documents in kv to use new forward cursor (#16635) 2020-01-23 08:46:23 -07:00
Lyon Hill 8894b65d2d
fix(kv): update kvlog to use the new forward cursor (#16634) 2020-01-23 08:46:09 -07:00
Lyon Hill 635be50323
fix(kv): Update dashboard collection to use new forward cursor (#16626)
By updating to the new forward pointer the query should be more efficient.
2020-01-22 11:01:09 -07:00
George 8b2109362d fix(kv): use forward cursor in find all buckets (#16605)
* fix(kv): use forward cursor in find all buckets

Co-authored-by: Lyon Hill <lyondhill@gmail.com>
2020-01-21 15:27:47 -07:00
Jacob Marble b836ab9c17
feat(storage): implement backup and restore (#16504)
* feat(backup): `influx backup` creates data backup

* feat(backup): initial restore work

* feat(restore): initial restore impl

Adds a restore tool which does offline restore of data and metadata.

* fix(restore): pr cleanup

* fix(restore): fix data dir creation

* fix(restore): pr cleanup

* chore: amend CHANGELOG

* fix: restore to empty dir fails differently

* feat(backup): backup and restore credentials

Saves the credentials file to backups and restores it from backups.

Additionally adds some logging for errors when fetching backup files.

* fix(restore): add missed commit

* fix(restore): pr cleanup

* fix(restore): fix default credentials restore path

* fix(backup): actually copy the credentials file for the backup

* fix: dirs get 0777, files get 0666

* fix: small review feedback

Co-authored-by: tmgordeeva <tanya@influxdata.com>
2020-01-21 14:22:45 -08:00
George 4f14ceabab
feat(kv): add support for prefixed cursor search (#16545)
* feat(kv): add support for prefixed cursor search

* chore(kv): ensure kv store implementation return seek missing prefix error in tests

* chore(kv): update changelog
2020-01-21 12:52:30 +00:00
Zoe Steinkamp 9d1dd1c14f
Merge pull request #16457 from influxdata/zs-improveErrorMessage16447
fix(ui): improve error message for create check (#16447)
2020-01-10 08:17:00 -07:00