bd4d97b9e4
move server's defaultBackupLocation into config struct
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-08-28 13:19:21 -07:00
0e94fa37f9
update sync controller for backup locations
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-08-28 13:19:21 -07:00
2750aa71b9
Use backup storage location during restore
...
Closes #740
Signed-off-by: Carlisia <carlisia@grokkingtech.io>
2018-08-28 13:19:21 -07:00
20f89fbcef
use the default backup storage location for restic
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-08-28 13:19:21 -07:00
cf7c8587f0
download request: fix setting of log level for plugin manager
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-08-28 13:19:21 -07:00
3234124afe
backup deletion: fix setting of log level in plugin manager
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-08-28 13:19:20 -07:00
8f5346150c
download request controller: use backup location for object store
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-08-28 13:19:20 -07:00
bab08ed1a6
backup deletion controller: use backup location for object store
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-08-28 13:19:20 -07:00
c6f488f75f
Use backup location in the backup controller
...
Fixes #739
Signed-off-by: Nolan Brubaker <nolan@heptio.com>
2018-08-28 13:19:20 -07:00
adbcd3703b
add --default-backup-storage-location flag to server cmd
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-08-28 13:14:51 -07:00
9508e4a20e
Switch Config CRD elements to server flags
...
Signed-off-by: Carlisia <carlisia@grokkingtech.io>
2018-08-16 08:51:46 -07:00
eb6f742b5d
Document CRD not found errors
...
Signed-off-by: Nolan Brubaker <nolan@heptio.com>
2018-08-02 16:41:28 -04:00
1c26fbde32
remove SnapshotService, replace with direct BlockStore usage
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-07-31 08:58:11 -07:00
130512187a
Refactor plugin management
...
Refactor plugin management:
- support multiple plugins per executable
- support restarting a plugin process in the event it terminates
- simplify plugin lifecycle management by using separate managers for
each scope (server vs backup vs restore)
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2018-07-31 08:34:57 -07:00
131afb571e
Merge pull request #607 from nrb/restore-metrics
...
Restore metrics
2018-07-27 12:25:56 -07:00
82f1cd87dc
Merge pull request #680 from carlisia/c-delete-from-etcd
...
Delete backups from etcd if they're not in storage
2018-07-25 18:25:52 -04:00
d2e629f5a8
Delete backups from etcd if they're not in storage
...
This fixes #309 .
Signed-off-by: Carlisia <carlisia@grokkingtech.io>
Co-authored-by: Steve Kriss <steve@heptio.com>
2018-07-25 14:53:50 -07:00
7cebfe2df0
Add restore attempt and success/failure counters
...
Signed-off-by: Nolan Brubaker <nolan@heptio.com>
2018-07-25 11:22:43 -04:00
b0af81e780
Merge pull request #689 from lemaral/init-schedule-metrics
...
Init schedule Prometheus metrics
2018-07-23 16:16:26 -04:00
e063b7983a
Merge pull request #682 from nrb/support-rbac-1.7
...
Support for rbac v1beta1 on kubernetes v1.7.5+
2018-07-23 12:00:50 -07:00
dd1e150511
Add RBAC support for 1.7 clusters
...
Signed-off-by: Nolan Brubaker <nolan@heptio.com>
2018-07-23 14:19:03 -04:00
c47a364ab3
Merge pull request #687 from skriss/fix-delete-issues
...
fix delete issues
2018-07-23 10:40:14 -07:00
450fa72fbb
Initialize schedule Prometheus metrics to have them created beforehand (see https://prometheus.io/docs/practices/instrumentation/#avoid-missing-metrics )
...
Signed-off-by: Alex Lemaresquier <alex+github@lemaresquier.org>
2018-07-20 23:30:24 +02:00
78cbdf95f3
delete old deletion requests for backup when processing a new one
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-07-20 11:42:42 -07:00
1df9a8a38d
exit server if not all Ark CRDs exist at startup
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-07-18 14:30:42 -07:00
e11634bfbc
Merge pull request #676 from skriss/dont-create-namespace
...
require namespace for backups/etc. to exist at server startup
2018-07-17 14:26:04 -04:00
3efe677007
Update Ark library code to work with Kubernetes 1.11
...
Signed-off-by: Mike Arpaia <mike@arpaia.co>
2018-07-16 20:46:20 -06:00
d195512062
require namespace for backups/etc. to exist at server startup
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-07-13 13:50:12 -07:00
22e8f23e2c
replace ark restic repo init cmd with automatic initialization of repos
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-06-27 13:50:37 -07:00
f2072e5868
use repo identifier from ResticRepo CR when backing up/restoring
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-06-20 10:42:53 -07:00
65ed8da4b7
add ResticRepository CRD and move repo-level actions to a controller
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-06-20 09:21:43 -07:00
cc9140b3cc
Log to stdout instead of stderr
...
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2018-06-14 14:02:12 -04:00
83658e891e
Add a metrics package to add and expose metrics
...
* add a metrics package to handle metric registration and publishing
* add a metricsAddress field to the server struct
* make metrics a part of the server
* start a metrics endpoint as part of starting the controllers
* instrument backup_controller to report metrics
* update cli-reference docs
* update example deployments with prometheus annotations
* update 'pkg/install' tooling with prometheus annotations
Signed-off-by: Ashish Amarnath <ashish.amarnath@gmail.com>
2018-06-13 13:17:08 -07:00
50d4084fac
add restic integration for doing pod volume backups/restores
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-06-06 09:48:10 -07:00
18e2401e79
Merge pull request #455 from skriss/remove-aws-region
...
AWS: remove region requirement from config for backupStorageProvider
2018-05-23 11:02:57 -04:00
b029860b46
AWS: remove region requirement from config for backupStorageProvider
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-05-22 11:59:53 -07:00
20f56e9868
cleanup plugin subprocesses on server exit
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-05-11 09:48:46 -07:00
09c20b51e6
shutdown gracefully on SIGINT/SIGTERM
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-05-10 16:22:14 -07:00
ea83ed32f5
Merge pull request #470 from skriss/backup-cluster-roles-and-bindings
...
backups: include clusterroles/bindings that reference serviceaccounts
2018-05-09 13:21:13 -04:00
041cfc2173
backups: include clusterroles/bindings that reference serviceaccounts
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-05-09 10:03:55 -07:00
6754955bcd
when syncing backups, set their namespace to current cluster's Ark ns
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-05-07 09:12:51 -07:00
258f3e011e
Merge pull request #409 from skriss/fix-398
...
get backup name from restore spec when handling restore DownloadRequests
2018-04-10 13:29:07 -04:00
0d2b49acce
get backup name from restore spec when handling restore DownloadRequests
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-04-09 10:50:10 -07:00
3f2e222ae4
Don't allow deletion of in-progress backups
...
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2018-04-06 13:08:39 -04:00
74f60b1ee1
Switch backup finalizer to DeleteBackupRequest
...
We ran into a lot of problems using a finalizer on the backup to allow
the Ark server to clean up all associated backup data when deleting a
backup.
Users also found it less than desirable that deleting the heptio-ark
namespace resulted in all the backup data being deleted.
This removes the finalizer and replaces it with an explicit
DeleteBackupRequest that is created as a means of requesting the
deletion of a backup and all its associated data. This is what `ark
backup delete` does.
If you use kubectl to delete a backup or to delete the heptio-ark
namespace, this no longer deletes associated backups. Additionally, as
long as the heptio-ark namespace still exists, the Ark server's
BackupSyncController will continually sync backups into the heptio-ark
namespace from object storage.
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2018-04-05 11:16:15 -04:00
c60e47dedd
use cobra's arg-count validation & call Complete() before Validate()
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-04-03 09:45:29 -07:00
73499c2cff
Add limitranges to defaultResourcePriorities
...
This commit adds limitranges to defaultResourcePriorities as
suggested in #385 .
This is done so that pods are not restored before the LimitRange
objects, because that would lead to pods not honoring the requests
and limits set in LimitRange objects.
Fixes #385
Signed-off-by: Shubham <shubham@linux.com>
2018-03-17 21:13:44 +05:30
cab904570f
GCP: copy tags from volume to snapshot, and snapshot to volume
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-03-13 11:52:09 -07:00
e618e0e456
server: allow configurable plugin dir
...
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2018-02-27 14:50:38 -05:00
465c60b408
Add global kubecontext CLI flag
...
Signed-off-by: Blake <blake.barnett@postmates.com>
2018-02-26 10:47:39 -08:00
e130011d1e
Merge pull request #272 from ncdc/arbitrary-namespaces
...
Support running in any namespace
2018-01-05 14:15:18 -08:00
816f14c0b4
Support running in any namespace
...
Add the ability for the Ark server to run in any namespace.
Add `ark client config get/set` for manipulating the new client
configuration file in $HOME/.config/ark/config.json. This holds client
defaults, such as the Ark server's namespace (to avoid having to specify
the --namespace flag all the time).
Add a --namespace flag to all client commands.
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2018-01-05 16:51:59 -05:00
de785af89d
Support pre and post hooks.
...
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2018-01-05 10:47:34 -05:00
6b0b6373da
Merge pull request #252 from skriss/delete-backup
...
add delete backup cmd using finalizer and simplify GC process
2018-01-02 14:26:03 -05:00
1503796419
make language and casing consistent across copyrights
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-01-02 10:51:49 -08:00
1c974782fa
disable GC and backup deletion if Kubernetes is less than v1.7.5
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-12-22 10:25:32 -08:00
fbda5fe539
Add serviceaccounts to prioritized resources
...
Add serviceaccounts to the default list of prioritized resources used
when restoring.
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2017-12-20 14:59:47 -05:00
1210cb36e1
Merge pull request #253 from skriss/update-license
...
Update all license headers
2017-12-19 19:26:54 -05:00
3641c2c043
update all license headers
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-12-15 13:38:12 -08:00
56c9d68137
switch to logrus.FieldLogger and arktest.NewLogger() everywhere
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-12-13 09:38:00 -08:00
b66efd9416
rename NewPluginLogger to NewLogger
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-11-29 14:01:31 -08:00
995050390c
add plugin add/remove commands
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-11-29 12:22:30 -08:00
179b95c81d
convert restorers to plugins
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-11-28 10:58:12 -08:00
0f2d1ab82b
migrate backup actions to plugins
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-11-21 10:03:03 -08:00
c57a9b94b9
Log server version at startup
...
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2017-11-14 16:58:14 -05:00
8ba5a29679
add a logger that can be used within plugins to emit logs into Ark server
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-11-14 10:38:26 -08:00
24ce316788
switch built-in cloud providers to run as plugins
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-11-14 09:47:36 -08:00
21e2019540
rename Block/ObjectStoreAdapter -> Block/ObjectStore
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-11-08 16:58:47 -08:00
aa253bf016
Use generated deep copy functions
...
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2017-10-30 13:31:03 -04:00
15fe87aea3
Fix clientset imports
...
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2017-10-30 13:31:03 -04:00
43449885a1
React to move to k8s.io/api
...
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2017-10-30 13:30:27 -04:00
901f8e1302
Add pod exec backup hooks
...
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2017-10-20 10:20:59 -04:00
9438a8670a
when backing up PVCs, also back up claimed PVs
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-10-11 09:54:57 -07:00
e7c62b5f38
add file/line numbers to logs
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-10-10 13:14:03 -07:00
9f3ce8ab1a
add log-level flag to server command
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-10-10 13:14:03 -07:00
6dc81b42f5
Merge branch 'master' into support-multi-az-pvs
...
Signed-off-by: Ashish Amarnath <ashish.amarnath@gmail.com>
2017-10-06 23:24:41 -07:00
9fc9dbb413
Preserve PV's AZ info when snapshotting and restoring PVs.
...
- Read PV's AZ info from fault-domain label of the PV object for snapshotting.
- Store PV's AZ info in the VolumeInfo.
- Add tests for reading the label from the PV object.
- Remove availability zone validation in AWS and GCP BlockStorageAdaptor.
- Add volumeAZ as a parameter to methods in the BlockStorageAdapter interface.
- Get AZ from VolumeInfo when restoring PV snapshot.
- Remove references to PV availability zone in docs.
Signed-off-by: Ashish Amarnath <ashish.amarnath@gmail.com>
2017-10-06 23:24:41 -07:00
9ac71b03cc
add logrus hook to log error file/line/function from stack traces
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-10-05 08:49:33 -07:00
914165095a
switch logging to logrus and errors to pkg/errors
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-10-05 08:49:33 -07:00
a5d4d138b2
Merge pull request #96 from ncdc/fix-config-watch
...
Make config change detection more robust
2017-09-19 09:57:52 -07:00
97f8f2426f
Addressing PR feedback and adding tests
...
Signed-off-by: Justin Nauman <justin.r.nauman@gmail.com>
2017-09-18 17:42:16 -05:00
84fa48d57a
Make config change detection more robust
...
Fix 2 issues with config change detection:
- Objects received via Get() don't have kind and apiVersion set, while
those from Watch() do, leading to false positives.
- Compare the unmodified config (prior to applying defaults) to the
updated one from Watch().
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2017-09-14 17:12:32 -04:00
d76ed7b49e
Adding in customized user-agent
...
Signed-off-by: Justin Nauman <justin.r.nauman@gmail.com>
2017-09-13 16:35:45 -05:00
273b563c86
Add per-restore logs
...
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2017-09-13 09:51:40 -04:00
03dde45c09
Add 'ark backup logs' command for retrieval
...
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2017-09-11 15:40:48 -04:00
7648155c9a
delete restores when GC'ing backups & don't orphan cloud resources
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-09-06 10:34:08 -07:00
768aed4ddd
make --snapshot-volumes, --restore-volumes optional with sensible default behavior based on PVProvider existence
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-08-23 09:47:52 -07:00
8d5c8ffcbb
get rid of StorageAdapter structs and move validation into block/object
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-08-23 09:46:15 -07:00
ebc06fd632
make PVProvider optional in server config; disallow snap/restore PVs when not provided
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-08-23 09:46:15 -07:00
df320d7bf3
adds support for S3 SSE with KMS
...
Signed-off-by: Mathias Merscher <Mathias.Merscher@dg-i.net>
2017-08-14 18:18:19 +02:00
e0d2a2d4d0
Remove k8s.io/apiextensions-apiserver from vendor
...
Remove k8s.io/apiextensions-apiserver since we're no longer registering
CRDs in code.
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2017-08-07 12:38:07 -04:00
2fe501f527
Initial commit
...
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2017-08-02 13:27:17 -04:00
Steve Kriss
Carlisia
Nolan Brubaker
Andy Goldstein
Steve Kriss
Nolan Brubaker
Alex Lemaresquier
Mike Arpaia
Ashish Amarnath
Shubham
Blake
Justin Nauman
Mathias Merscher