Commit Graph

212 Commits (33cc85cd0c52a4efd533f39e20216d0e242a795a)

Author SHA1 Message Date
Nolan Brubaker 33cc85cd0c Compare backup and cluster objects before logging
When restoring resources that raise an already exists error, check their
equality before logging a message on the restore. If they're the same
except for some metadata, don't generate a message.

The restore process was modified so that if an object had an empty
namespace string, no namespace key is created on the object. This was to
avoid manipulating the copy of the current cluster's object by adding
the target namespace.

There are some cases right now that are known to not be equal via this
method:

- The `default` ServiceAccount in a namespace will not match, primarily
because of differing default tokens. These will be handled in their own
patch
- IP addresses for Services are recorded in the backup object, but are
either not present on the cluster object, or different. An issue for
this already exists at https://github.com/heptio/ark/issues/354
- Endpoints have differing values for `renewTime`. This may be
insubstantial, but isn't currently handled by the resetMetadataAndStatus
function.
- PersistentVolume objects do not match on spec fields, such as
claimRef and cloud provider persistent disk info

Signed-off-by: Nolan Brubaker <nolan@heptio.com>
2018-04-10 14:55:48 -04:00
Andy Goldstein 3f2e222ae4 Don't allow deletion of in-progress backups
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2018-04-06 13:08:39 -04:00
Andy Goldstein 644a75e3c1 Set DeleteBackupRequest labels if missing
When the BackupDeletionController processes a request, set the request's
backup-name and backup-uid labels if they aren't currently set.

Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2018-04-05 15:38:44 -04:00
Andy Goldstein ef57a44827 Require DeleteBackupRequest.Spec.BackupName
Make sure a DeleteBackupRequest has its Spec.BackupName filled in. If
not, record an error in the status and mark the request as processed.

Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2018-04-05 14:53:10 -04:00
Andy Goldstein 4328b67f93 backup delete/describe enhancements
Always request DeleteBackupRequests for a given backup so we can show
failed deletion attempts if you try to delete a backup that has PV
snapshots when Ark doesn't have a persistentVolumeProvider configured.

When creating a DeleteBackupRequest, include a label for the UID so we
can match based on name and UID when associated DeleteBackupRequests
with a given backup.

Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2018-04-05 14:21:45 -04:00
Andy Goldstein 1ae492da6c Expire processed deleted backup requests > 24hr
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2018-04-05 11:19:11 -04:00
Andy Goldstein 74f60b1ee1 Switch backup finalizer to DeleteBackupRequest
We ran into a lot of problems using a finalizer on the backup to allow
the Ark server to clean up all associated backup data when deleting a
backup.

Users also found it less than desirable that deleting the heptio-ark
namespace resulted in all the backup data being deleted.

This removes the finalizer and replaces it with an explicit
DeleteBackupRequest that is created as a means of requesting the
deletion of a backup and all its associated data. This is what `ark
backup delete` does.

If you use kubectl to delete a backup or to delete the heptio-ark
namespace, this no longer deletes associated backups. Additionally, as
long as the heptio-ark namespace still exists, the Ark server's
BackupSyncController will continually sync backups into the heptio-ark
namespace from object storage.

Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2018-04-05 11:16:15 -04:00
Steve Kriss 80b66434c0 move getting client into Complete()
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-04-03 20:43:42 -07:00
Steve Kriss c60e47dedd use cobra's arg-count validation & call Complete() before Validate()
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-04-03 09:45:29 -07:00
Shubham 73499c2cff Add limitranges to defaultResourcePriorities
This commit adds limitranges to defaultResourcePriorities as
suggested in #385.

This is done so that pods are not restored before the LimitRange
objects, because that would lead to pods not honoring the requests
and limits set in LimitRange objects.

Fixes #385

Signed-off-by: Shubham <shubham@linux.com>
2018-03-17 21:13:44 +05:30
Shubham 560f9504e1 Fix typo: replace ; with :
This commit replaces a ; with a : in an error message

Signed-off-by: Shubham <shubham@linux.com>
2018-03-16 18:24:34 +05:30
Andy Goldstein 3172e907a1
Merge pull request #341 from skriss/snapshot-tags
Add tags to snapshots, restored volumes
2018-03-14 12:50:40 -04:00
Andy Goldstein 07fcc927b7
Merge pull request #359 from skriss/reduce-azure-polling-interval
Azure: reduce client polling interval from default 60s to 5s
2018-03-14 12:17:54 -04:00
Steve Kriss 45cee7d222 Azure: reduce client polling interval from default 60s to 5s
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-03-14 09:07:20 -07:00
Steve Kriss b2b16b3c85 Azure: fix bug preventing snapshot deletion
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-03-13 17:03:14 -07:00
Steve Kriss 0388845bca AWS: tag snapshots during create
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-03-13 12:17:50 -07:00
Steve Kriss a05ae1a7cf add useful Ark tags to snapshots
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-03-13 12:17:50 -07:00
Steve Kriss df985bca74 Azure: copy tags from volume to snapshot, and snapshot to volume
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-03-13 12:17:50 -07:00
Steve Kriss cab904570f GCP: copy tags from volume to snapshot, and snapshot to volume
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-03-13 11:52:09 -07:00
Steve Kriss 9673e9d158 AWS: copy tags from volume to snapshot, and snapshot to volume
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-03-13 11:52:09 -07:00
Steve Kriss 51c546786e
Merge pull request #374 from sanketjpatel/feat/exclude-events-restore
Add events to nonRestorableResources and cohabitatingResources
2018-03-13 11:23:17 -07:00
Andy Goldstein f83b1de1dc
Merge pull request #356 from skriss/store-azure-snapshot-uri
Azure: store snapshot URI to support cross-resource group restores
2018-03-13 13:45:48 -04:00
Steve Kriss 409f17361d Azure: store snapshot URI to support cross-resource group restores
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-03-13 09:18:20 -07:00
Sanket Patel 7257a75f82 Add events to nonRestorableResources and cohabitatingResources
Fixes #367
Fixes #368

Signed-off-by: Sanket Patel <sanketpatel.301090@gmail.com>
2018-03-12 19:55:32 -05:00
Andy Goldstein 178a007b70
Merge pull request #362 from skriss/tweak-hook-logging
during backup, only log if hooks are actually executing
2018-03-07 13:37:03 -05:00
Steve Kriss 26944cf9a2 only log if hooks are actually executing
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-03-07 09:29:27 -08:00
Andy Goldstein 7328fd0e10
Merge pull request #343 from skriss/gc-bug
add an AddFunc to handle finalizers in GC for initial lists/resyncs
2018-03-06 16:41:57 -05:00
Andy Goldstein 331e0c28cc Remove Azure location requirement
Instead of requiring the Ark admin to specify a "location" in the azure
persistentVolumeProvider config (meaning only a single location is
supported), get info about the disk (for its location) when creating a
snapshot, and get info about the snapshot (for its location) when
creating a disk from a snapshot.

Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2018-03-05 15:20:58 -05:00
Steve Kriss c281124cba add --from-backup flag to ark restore create & allow restore name
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-03-05 10:06:43 -08:00
Andy Goldstein 6930c846e5 Fix incorrect return when err != nil
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2018-03-05 10:57:23 -05:00
Andy Goldstein cc9be4489a
Merge pull request #345 from skriss/gcp-project
extract GCP project ID from creds file, remove from config
2018-03-05 10:35:39 -05:00
Steve Kriss cbff5ef53b extract GCP project ID from creds file, remove from config
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-03-02 09:15:12 -08:00
Steve Kriss 509431909a add an AddFunc to handle finalizers in GC for initial lists/resyncs
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-02-28 16:56:23 -08:00
Steve Kriss 975dce929c
Merge pull request #331 from lypht/schedule
Include cron examples for schedule creation
2018-02-28 15:32:28 -08:00
Josh Hull f733869142
Include cron examples for schedule creation
Signed-off-by: Josh Hull <josh.hull@lypht.com>
2018-02-28 16:50:52 -06:00
Andy Goldstein e618e0e456 server: allow configurable plugin dir
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2018-02-27 14:50:38 -05:00
Andy Goldstein f13b0c00a3 server: don't assume /ark for internal plugins
If you want to test changes to the ark server without having to rebuild
and redeploy the ark container, this change allows you to do something
like this (assuming you've created your cloud credentials file):

AWS_SHARED_CREDENTIALS_FILE=credentials-minio ark server -n heptio-ark

Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2018-02-27 14:32:45 -05:00
Nolan Brubaker e73ba839f3
Merge pull request #296 from blakebarnett/bdb/add_kubecontext_flag
Adds a kubecontext global flag
2018-02-26 17:05:59 -05:00
Nolan Brubaker abeeee6643
Merge pull request #333 from ncdc/add-force-delete-backup
Allow forced backup deletion
2018-02-26 16:44:37 -05:00
Andy Goldstein d24fb232cc Allow forced backup deletion
Add --force and --confirm to `ark backup delete` to support forced
backup deletion. This forcibly removes the Ark GC finalizer (if it's
present) from a backup and will orphan any resources associated with the
backup, such as backup tarballs in object storage, persistent volume
snapshots, and restores for the backup.

If a backup has a deletion timestamp, display `Deleting` in `ark backup
describe` and `ark backup get`.

Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2018-02-26 16:25:22 -05:00
Blake 465c60b408
Add global kubecontext CLI flag
Signed-off-by: Blake <blake.barnett@postmates.com>
2018-02-26 10:47:39 -08:00
Andy Goldstein cc4d2916fa Error if backup missing when creating a restore
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2018-02-05 13:29:48 -05:00
Andy Goldstein b2cd8e1fe8 Create the config directory in case it's missing
Always try to create the config directory when saving the client config
in case it doesn't exist.

Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2018-02-02 16:01:52 -05:00
Andy Goldstein 6712e2a9ec
Merge pull request #270 from nrb/fix-232
Clarify backup and restore creation messages
2018-01-11 12:22:11 -05:00
Andy Goldstein f5f1dbac5d
Merge pull request #271 from nrb/fix-246
Invalidate restores based on non-existent backups
2018-01-10 13:24:53 -05:00
Steve Kriss af81f589f6 update GCP block store to use v1 library, and tag snapshots on create
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-01-08 09:23:32 -08:00
Nolan Brubaker 37f498cd37 Clarify backup and restore creation messages
When running `ark <resource> create`, a request is sent to the server,
but the status is not immediately known. Inform the user that a request
was sent and provide a way to get more information on it.

Signed-off-by: Nolan Brubaker <nolan@heptio.com>
2018-01-08 10:26:51 -05:00
Nolan Brubaker bb7b0a3c33 Invalidate restores based on missing backups
When creating a restore based on a backup that doesn't exist, the
restore should be marked as invalid and the error clearly communicated
so the user understands why the restore wasn't made.

Previously, the restore was left as in progress with an error attached.

Since restores are CRDs and must be updated via a controller, there's
currently not a way to give the client immediate errors.

Signed-off-by: Nolan Brubaker <nolan@heptio.com>
2018-01-08 10:10:01 -05:00
Steve Kriss b53ee44176 update GCP object store to use new/recommended lib
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-01-05 14:21:13 -08:00
Steve Kriss e130011d1e
Merge pull request #272 from ncdc/arbitrary-namespaces
Support running in any namespace
2018-01-05 14:15:18 -08:00