Commit Graph

52 Commits (33cc85cd0c52a4efd533f39e20216d0e242a795a)

Author SHA1 Message Date
Andy Goldstein 74f60b1ee1 Switch backup finalizer to DeleteBackupRequest
We ran into a lot of problems using a finalizer on the backup to allow
the Ark server to clean up all associated backup data when deleting a
backup.

Users also found it less than desirable that deleting the heptio-ark
namespace resulted in all the backup data being deleted.

This removes the finalizer and replaces it with an explicit
DeleteBackupRequest that is created as a means of requesting the
deletion of a backup and all its associated data. This is what `ark
backup delete` does.

If you use kubectl to delete a backup or to delete the heptio-ark
namespace, this no longer deletes associated backups. Additionally, as
long as the heptio-ark namespace still exists, the Ark server's
BackupSyncController will continually sync backups into the heptio-ark
namespace from object storage.

Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2018-04-05 11:16:15 -04:00
Steve Kriss a05ae1a7cf add useful Ark tags to snapshots
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-03-13 12:17:50 -07:00
Andy Goldstein d24fb232cc Allow forced backup deletion
Add --force and --confirm to `ark backup delete` to support forced
backup deletion. This forcibly removes the Ark GC finalizer (if it's
present) from a backup and will orphan any resources associated with the
backup, such as backup tarballs in object storage, persistent volume
snapshots, and restores for the backup.

If a backup has a deletion timestamp, display `Deleting` in `ark backup
describe` and `ark backup get`.

Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2018-02-26 16:25:22 -05:00
Andy Goldstein 6b0b6373da
Merge pull request #252 from skriss/delete-backup
add delete backup cmd using finalizer and simplify GC process
2018-01-02 14:26:03 -05:00
Steve Kriss 1503796419 make language and casing consistent across copyrights
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-01-02 10:51:49 -08:00
Steve Kriss 1c974782fa disable GC and backup deletion if Kubernetes is less than v1.7.5
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-12-22 10:25:32 -08:00
Steve Kriss f5123794e0 add delete backup cmd using finalizer and simplify GC process
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-12-21 10:47:37 -08:00
Andy Goldstein 1210cb36e1
Merge pull request #253 from skriss/update-license
Update all license headers
2017-12-19 19:26:54 -05:00
Steve Kriss 3641c2c043 update all license headers
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-12-15 13:38:12 -08:00
Steve Kriss 56c9d68137 switch to logrus.FieldLogger and arktest.NewLogger() everywhere
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-12-13 09:38:00 -08:00
Steve Kriss 4aea9b9a2c switch from Update() to Patch()
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-12-12 15:23:45 -08:00
Steve Kriss 038fa39451 remove ListSnapshots and GetAllSnapshots
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-11-29 13:59:29 -08:00
Steve Kriss 5b8562e73c
Merge pull request #222 from ncdc/fix-log-location-hook
Fix log location hook prefix stripping
2017-11-29 11:07:43 -08:00
Andy Goldstein c700455272 Support custom volume snapshots & restores
The main Ark code was hard-coding specific support for AWS, GCE, and
Azure volume snapshots and restores, and anything else was considered
unsupported.

Add GetVolumeID and SetVolumeID to the BlockStore interface, to allow
block store plugins to handle volume snapshots and restores.

Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2017-11-29 13:19:40 -05:00
Andy Goldstein 99f67db39d Fix log location hook prefix stripping
The log location hook was matching github.com/heptio/ark and stripping
off that + 1 more char. This meant that
github.com/heptio/ark-plugin-example/foo.go was being listed as
plugin-example/foo.go instead of
github.com/heptio/ark-plugin-example/foo.go.

Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2017-11-29 12:56:01 -05:00
Andy Goldstein 526b604237 Add boolptr helper package
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2017-11-29 12:25:10 -05:00
Steve Kriss 179b95c81d convert restorers to plugins
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-11-28 10:58:12 -08:00
Andy Goldstein 4c481f4d23 Make logSource more concise
Strip off leading .../github.com/heptio/ark/

Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2017-11-14 14:21:19 -05:00
Steve Kriss 8ba5a29679 add a logger that can be used within plugins to emit logs into Ark server
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-11-14 10:38:26 -08:00
Steve Kriss 24ce316788 switch built-in cloud providers to run as plugins
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-11-14 09:47:36 -08:00
Steve Kriss 21e2019540 rename Block/ObjectStoreAdapter -> Block/ObjectStore
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-11-08 16:58:47 -08:00
Andy Goldstein b2d80471ac Move restore warnings/errors to object storage
If you have a large number of warnings and/or errors, the restore
object's size can exceed the maximum allowed by etcd. Move them to
object storage, and add a new describe command to fetch and display them
on the fly.

Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2017-11-02 15:53:59 -04:00
Steve Kriss e7703d88ec switch UploadBackup, UploadRestoreLog, PutObject to take io.Reader
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-11-01 15:48:13 -07:00
Andy Goldstein 15fe87aea3 Fix clientset imports
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2017-10-30 13:31:03 -04:00
Andy Goldstein 43449885a1 React to move to k8s.io/api
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2017-10-30 13:30:27 -04:00
Andy Goldstein 901f8e1302 Add pod exec backup hooks
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2017-10-20 10:20:59 -04:00
Steve Kriss 07b33ace57 add --include-cluster-resources flag to backup create cmd
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-10-11 15:05:12 -07:00
Steve Kriss 9438a8670a when backing up PVCs, also back up claimed PVs
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-10-11 09:54:57 -07:00
Steve Kriss 7c0d9dcfce add missing licenses
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-10-10 13:14:03 -07:00
Steve Kriss e7c62b5f38 add file/line numbers to logs
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-10-10 13:14:03 -07:00
ashish-amarnath 6dc81b42f5 Merge branch 'master' into support-multi-az-pvs
Signed-off-by: Ashish Amarnath <ashish.amarnath@gmail.com>
2017-10-06 23:24:41 -07:00
Ashish Amarnath 9fc9dbb413 Preserve PV's AZ info when snapshotting and restoring PVs.
- Read PV's AZ info from fault-domain label of the PV object for snapshotting.
- Store PV's AZ info in the VolumeInfo.
- Add tests for reading the label from the PV object.
- Remove availability zone validation in AWS and GCP BlockStorageAdaptor.
- Add volumeAZ as a parameter to methods in the BlockStorageAdapter interface.
- Get AZ from VolumeInfo when restoring PV snapshot.
- Remove references to PV availability zone in docs.

Signed-off-by: Ashish Amarnath <ashish.amarnath@gmail.com>
2017-10-06 23:24:41 -07:00
Steve Kriss 9ac71b03cc add logrus hook to log error file/line/function from stack traces
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-10-05 08:49:33 -07:00
Steve Kriss 914165095a switch logging to logrus and errors to pkg/errors
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-10-05 08:49:33 -07:00
Steve Kriss acb3babb87 when restoring azureDisk from snapshot, update the diskURI with the new diskName
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-09-13 16:10:30 -07:00
Andy Goldstein 273b563c86 Add per-restore logs
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2017-09-13 09:51:40 -04:00
Andy Goldstein d0b7880881 Merge pull request #78 from skriss/restore-resource-inc-exc
Add --include-resources, --exclude-resources for restore operations
2017-09-12 15:53:21 -04:00
Steve Kriss 4dfce17de5 add resource includes/excludes for Restores
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-09-12 12:43:23 -07:00
Justin Nauman 98d4660d27 Adding in support for backup download
- Adding in support for a new `download` subcommand of backup
- Adjusted signing to allow for multiple types
- Adding in git sha version during build more granular version
  debugging

Signed-off-by: Justin Nauman <justin.r.nauman@gmail.com>
2017-09-11 20:17:33 -05:00
Steve Kriss 907ae6c5b0 move groupresource resolution into discovery helper
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-09-11 14:41:50 -07:00
Andy Goldstein 50a95d052e Delete all objects in backup dir
Delete all objects in backup "dir" when deleting a backup, instead of
hard-coding individual file names/types. This way, we'll be able to
delete log files and anything else we add without having to update our
deletion code.

Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2017-09-11 15:40:48 -04:00
Andy Goldstein 193000e487 Add ListObjects (for backup dir deletions)
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2017-09-11 15:40:48 -04:00
Andy Goldstein 03dde45c09 Add 'ark backup logs' command for retrieval
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2017-09-11 15:40:48 -04:00
Steve Kriss 7648155c9a delete restores when GC'ing backups & don't orphan cloud resources
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-09-06 10:34:08 -07:00
Andy Goldstein 305e7fcea0 Merge pull request #59 from jrnt30/explicit-includes-excludes-restore
Allows explicit include/exclude of namespaces on restores
2017-09-06 11:21:24 -04:00
Justin Nauman 2457d8f116 Removing instead of deprecating Namespace attr
- Per discussion, there is no reason to deal
with the complexity of backwards compatibility
with the Namespace attribute on the Restore
domain.

- Also noticed there was an error on the
validation of the BackupController where
the message would actually just be the index
of the error and not the contents of the message
itself.

Signed-off-by: Justin Nauman <justin.r.nauman@gmail.com>
2017-09-05 17:18:10 -05:00
Justin Nauman af2a792a9a Allows explicit include/exclude of namespaces on restores
- Introduces similar Include/Exclude declaration on the Restore
resource and cli flags
- Kept support for legacy Namespaces attribute until it could be
deprecating.  Defining both IncludeNamespaces and Namespaces results
in a validation error and the Restore will not be processed (shouldn't
be able to occur)

Signed-off-by: Justin Nauman <justin.r.nauman@gmail.com>
2017-09-05 17:17:41 -05:00
Steve Kriss f07a70c604 during restore, try to get backup directly from obj storage if not in cache/API
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-09-05 09:25:39 -07:00
Steve Kriss 768aed4ddd make --snapshot-volumes, --restore-volumes optional with sensible default behavior based on PVProvider existence
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-08-23 09:47:52 -07:00
Steve Kriss ebc06fd632 make PVProvider optional in server config; disallow snap/restore PVs when not provided
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-08-23 09:46:15 -07:00