vmware-tanzu
/
velero
mirror of https://github.com/vmware-tanzu/velero.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
1,569 Commits
33 Branches
161 Tags
112 MiB
Commit Graph

126 Commits (ffa3251efc7551b4ba560e53af07d573fc9d7434)

Author SHA1 Message Date
Steve Kriss 9d7ea7483c azure: support different RGs/storage accounts per backup location 
Signed-off-by: Steve Kriss <steve@heptio.com>
 2018-08-28 13:19:21 -07:00
Steve Kriss cd4e9f5336 azure: fix for breaking change in blob.GetSASURI 
Signed-off-by: Steve Kriss <steve@heptio.com>
 2018-08-28 13:19:21 -07:00
Steve Kriss 0e94fa37f9 update sync controller for backup locations 
Signed-off-by: Steve Kriss <steve@heptio.com>
 2018-08-28 13:19:21 -07:00
Steve Kriss 1c26fbde32 remove SnapshotService, replace with direct BlockStore usage 
Signed-off-by: Steve Kriss <steve@heptio.com>
 2018-07-31 08:58:11 -07:00
Andy Goldstein 130512187a Refactor plugin management 
Refactor plugin management:
- support multiple plugins per executable
- support restarting a plugin process in the event it terminates
- simplify plugin lifecycle management by using separate managers for
  each scope (server vs backup vs restore)

Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
 2018-07-31 08:34:57 -07:00
Dave Parfitt 06d6665abb check s3URL scheme upon AWS ObjectStore Init() 
Signed-off-by: Dave Parfitt <diparfitt@gmail.com>
 2018-07-24 14:01:05 -04:00
Steve Kriss 85a61b8e8d return nil error if 404 encountered when deleting snapshots 
Signed-off-by: Steve Kriss <steve@heptio.com>
 2018-07-20 11:42:34 -07:00
Mike Arpaia 3efe677007
Update Ark library code to work with Kubernetes 1.11 
Signed-off-by: Mike Arpaia <mike@arpaia.co>
 2018-07-16 20:46:20 -06:00
Steve Kriss 5ad21854f7 restic: if S3, get bucket's region up-front 
Signed-off-by: Steve Kriss <steve@heptio.com>
 2018-06-21 17:20:53 -07:00
Steve Kriss 170034787d
Merge pull request #513 from ncdc/510-follow-ups 
Fix copyright date, add happy path test case
 2018-05-23 08:54:22 -07:00
Andy Goldstein 18e2401e79
Merge pull request #455 from skriss/remove-aws-region 
AWS: remove region requirement from config for backupStorageProvider
 2018-05-23 11:02:57 -04:00
Andy Goldstein 849297e623 Fix copyright date, add happy path test case 
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
 2018-05-23 09:59:32 -04:00
Nolan Brubaker 09bbe072cd Check errors when closing GCP objects 
Writing to GCP's object store is any async operation, so errors need to
be checked both on write and close calls, since errors like permission
violations aren't reported until a close.

Signed-off-by: Nolan Brubaker <nolan@heptio.com>
 2018-05-22 17:38:49 -04:00
Steve Kriss b029860b46 AWS: remove region requirement from config for backupStorageProvider 
Signed-off-by: Steve Kriss <steve@heptio.com>
 2018-05-22 11:59:53 -07:00
Steve Kriss e81de2491f remove duplicate github.com/satori/go.uuid dependency 
Signed-off-by: Steve Kriss <steve@heptio.com>
 2018-05-07 10:02:59 -07:00
Steve Kriss 72d7e5e01d azure: pass full snapshot name when restoring disk 
Signed-off-by: Steve Kriss <steve@heptio.com>
 2018-04-23 12:51:05 -07:00
Steve Kriss a466208975 add backwards-compatibility for azure snapshot ID 
Signed-off-by: Steve Kriss <steve@heptio.com>
 2018-04-23 11:34:37 -07:00
Steve Kriss 0d2b49acce get backup name from restore spec when handling restore DownloadRequests 
Signed-off-by: Steve Kriss <steve@heptio.com>
 2018-04-09 10:50:10 -07:00
Andy Goldstein 3172e907a1
Merge pull request #341 from skriss/snapshot-tags 
Add tags to snapshots, restored volumes
 2018-03-14 12:50:40 -04:00
Andy Goldstein 07fcc927b7
Merge pull request #359 from skriss/reduce-azure-polling-interval 
Azure: reduce client polling interval from default 60s to 5s
 2018-03-14 12:17:54 -04:00
Steve Kriss 45cee7d222 Azure: reduce client polling interval from default 60s to 5s 
Signed-off-by: Steve Kriss <steve@heptio.com>
 2018-03-14 09:07:20 -07:00
Steve Kriss b2b16b3c85 Azure: fix bug preventing snapshot deletion 
Signed-off-by: Steve Kriss <steve@heptio.com>
 2018-03-13 17:03:14 -07:00
Steve Kriss 0388845bca AWS: tag snapshots during create 
Signed-off-by: Steve Kriss <steve@heptio.com>
 2018-03-13 12:17:50 -07:00
Steve Kriss a05ae1a7cf add useful Ark tags to snapshots 
Signed-off-by: Steve Kriss <steve@heptio.com>
 2018-03-13 12:17:50 -07:00
Steve Kriss df985bca74 Azure: copy tags from volume to snapshot, and snapshot to volume 
Signed-off-by: Steve Kriss <steve@heptio.com>
 2018-03-13 12:17:50 -07:00
Steve Kriss cab904570f GCP: copy tags from volume to snapshot, and snapshot to volume 
Signed-off-by: Steve Kriss <steve@heptio.com>
 2018-03-13 11:52:09 -07:00
Steve Kriss 9673e9d158 AWS: copy tags from volume to snapshot, and snapshot to volume 
Signed-off-by: Steve Kriss <steve@heptio.com>
 2018-03-13 11:52:09 -07:00
Steve Kriss 409f17361d Azure: store snapshot URI to support cross-resource group restores 
Signed-off-by: Steve Kriss <steve@heptio.com>
 2018-03-13 09:18:20 -07:00
Andy Goldstein 331e0c28cc Remove Azure location requirement 
Instead of requiring the Ark admin to specify a "location" in the azure
persistentVolumeProvider config (meaning only a single location is
supported), get info about the disk (for its location) when creating a
snapshot, and get info about the snapshot (for its location) when
creating a disk from a snapshot.

Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
 2018-03-05 15:20:58 -05:00
Steve Kriss cbff5ef53b extract GCP project ID from creds file, remove from config 
Signed-off-by: Steve Kriss <steve@heptio.com>
 2018-03-02 09:15:12 -08:00
Steve Kriss af81f589f6 update GCP block store to use v1 library, and tag snapshots on create 
Signed-off-by: Steve Kriss <steve@heptio.com>
 2018-01-08 09:23:32 -08:00
Steve Kriss b53ee44176 update GCP object store to use new/recommended lib 
Signed-off-by: Steve Kriss <steve@heptio.com>
 2018-01-05 14:21:13 -08:00
Steve Kriss 656428d0b9
Merge pull request #250 from ncdc/backup-controller-do-as-much-as-possible 
BackupController: do as much as possible
 2018-01-03 12:10:51 -08:00
Steve Kriss 1503796419 make language and casing consistent across copyrights 
Signed-off-by: Steve Kriss <steve@heptio.com>
 2018-01-02 10:51:49 -08:00
Andy Goldstein 1e581f1ead BackupController: do as much as possible 
When running a backup, try to do as much as possible, collecting errors
along the way, and return an aggregate at the end. This way, if a backup
fails for most reasons, we'll be able to upload the backup log file to
object storage, which wasn't happening before.

Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
 2017-12-20 14:49:27 -05:00
Andy Goldstein 1210cb36e1
Merge pull request #253 from skriss/update-license 
Update all license headers
 2017-12-19 19:26:54 -05:00
Steve Kriss 3641c2c043 update all license headers 
Signed-off-by: Steve Kriss <steve@heptio.com>
 2017-12-15 13:38:12 -08:00
Steve Kriss 56c9d68137 switch to logrus.FieldLogger and arktest.NewLogger() everywhere 
Signed-off-by: Steve Kriss <steve@heptio.com>
 2017-12-13 09:38:00 -08:00
Steve Kriss cbcd15d603 add/update documentation for plugins 
Signed-off-by: Steve Kriss <steve@heptio.com>
 2017-11-29 14:24:23 -08:00
Steve Kriss 038fa39451 remove ListSnapshots and GetAllSnapshots 
Signed-off-by: Steve Kriss <steve@heptio.com>
 2017-11-29 13:59:29 -08:00
Andy Goldstein c700455272 Support custom volume snapshots & restores 
The main Ark code was hard-coding specific support for AWS, GCE, and
Azure volume snapshots and restores, and anything else was considered
unsupported.

Add GetVolumeID and SetVolumeID to the BlockStore interface, to allow
block store plugins to handle volume snapshots and restores.

Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
 2017-11-29 13:19:40 -05:00
Steve Kriss 24ce316788 switch built-in cloud providers to run as plugins 
Signed-off-by: Steve Kriss <steve@heptio.com>
 2017-11-14 09:47:36 -08:00
Steve Kriss 21e2019540 rename Block/ObjectStoreAdapter -> Block/ObjectStore 
Signed-off-by: Steve Kriss <steve@heptio.com>
 2017-11-08 16:58:47 -08:00
Andy Goldstein b2d80471ac Move restore warnings/errors to object storage 
If you have a large number of warnings and/or errors, the restore
object's size can exceed the maximum allowed by etcd. Move them to
object storage, and add a new describe command to fetch and display them
on the fly.

Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
 2017-11-02 15:53:59 -04:00
Steve Kriss e7703d88ec switch UploadBackup, UploadRestoreLog, PutObject to take io.Reader 
Signed-off-by: Steve Kriss <steve@heptio.com>
 2017-11-01 15:48:13 -07:00
Andy Goldstein c1bc52eb65 Update gofmt 
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
 2017-10-30 13:31:03 -04:00
Andy Goldstein 15fe87aea3 Fix clientset imports 
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
 2017-10-30 13:31:03 -04:00
ashish-amarnath 6dc81b42f5 Merge branch 'master' into support-multi-az-pvs 
Signed-off-by: Ashish Amarnath <ashish.amarnath@gmail.com>
 2017-10-06 23:24:41 -07:00
Ashish Amarnath 9fc9dbb413 Preserve PV's AZ info when snapshotting and restoring PVs. 
- Read PV's AZ info from fault-domain label of the PV object for snapshotting.
- Store PV's AZ info in the VolumeInfo.
- Add tests for reading the label from the PV object.
- Remove availability zone validation in AWS and GCP BlockStorageAdaptor.
- Add volumeAZ as a parameter to methods in the BlockStorageAdapter interface.
- Get AZ from VolumeInfo when restoring PV snapshot.
- Remove references to PV availability zone in docs.

Signed-off-by: Ashish Amarnath <ashish.amarnath@gmail.com>
 2017-10-06 23:24:41 -07:00
Steve Kriss 914165095a switch logging to logrus and errors to pkg/errors 
Signed-off-by: Steve Kriss <steve@heptio.com>
 2017-10-05 08:49:33 -07:00
Steve Kriss 5424eba417 change Azure API timeout default to 2m 
Signed-off-by: Steve Kriss <steve@heptio.com>
 2017-09-13 16:20:12 -07:00
Andy Goldstein 136b8f200d Support restore logs when backup name has - 
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
 2017-09-13 16:04:46 -04:00
Andy Goldstein b43b65b7bf Revert to prior name format for tarballs 
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
 2017-09-13 09:51:40 -04:00
Andy Goldstein 7b892b63ed make update 
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
 2017-09-13 09:51:40 -04:00
Andy Goldstein 273b563c86 Add per-restore logs 
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
 2017-09-13 09:51:40 -04:00
Justin Nauman 98d4660d27 Adding in support for backup download 
- Adding in support for a new `download` subcommand of backup
- Adjusted signing to allow for multiple types
- Adding in git sha version during build more granular version
  debugging

Signed-off-by: Justin Nauman <justin.r.nauman@gmail.com>
 2017-09-11 20:17:33 -05:00
Andy Goldstein ab99fcc2e2 Update backup and log file name formats 
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
 2017-09-11 15:40:49 -04:00
Andy Goldstein 50a95d052e Delete all objects in backup dir 
Delete all objects in backup "dir" when deleting a backup, instead of
hard-coding individual file names/types. This way, we'll be able to
delete log files and anything else we add without having to update our
deletion code.

Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
 2017-09-11 15:40:48 -04:00
Andy Goldstein 193000e487 Add ListObjects (for backup dir deletions) 
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
 2017-09-11 15:40:48 -04:00
Andy Goldstein 03dde45c09 Add 'ark backup logs' command for retrieval 
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
 2017-09-11 15:40:48 -04:00
Andy Goldstein 9848a7a55b Save backup log file to object storage 
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
 2017-09-11 15:40:48 -04:00
Andy Goldstein 860eb7e845 Match Azure sdk changes 
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
 2017-09-11 15:40:47 -04:00
Steve Kriss 7648155c9a delete restores when GC'ing backups & don't orphan cloud resources 
Signed-off-by: Steve Kriss <steve@heptio.com>
 2017-09-06 10:34:08 -07:00
Steve Kriss ba299f880f in GetAllBackups don't error if single backup is unreadable 
Signed-off-by: Steve Kriss <steve@heptio.com>
 2017-09-05 09:56:37 -07:00
Andy Goldstein 43c74e33c2 Merge pull request #57 from skriss/restore_missing_backup_fix 
during restore, try to get backup directly from obj storage if not in cache/API
 2017-09-05 12:47:13 -04:00
Steve Kriss f07a70c604 during restore, try to get backup directly from obj storage if not in cache/API 
Signed-off-by: Steve Kriss <steve@heptio.com>
 2017-09-05 09:25:39 -07:00
Justin Nauman f017a23d41 AWS Paging Support 
- Adding in paging support for the S3 and Snapshot
AWS integration.

As a testing note, you can add in a a MaxKeys to the S3
request as an easy way to ensure that paging is working
properly without having to creation over 1k backups.

Signed-off-by: Justin Nauman <justin.r.nauman@gmail.com>
 2017-08-28 05:49:39 -05:00
Steve Kriss 8d5c8ffcbb get rid of StorageAdapter structs and move validation into block/object 
Signed-off-by: Steve Kriss <steve@heptio.com>
 2017-08-23 09:46:15 -07:00
Andy Goldstein c088470ba5 Merge pull request #35 from skriss/server_config_validation 
validate cloud-provider config at startup & make PVProvider optional
 2017-08-14 16:03:15 -04:00
Steve Kriss 726bbbb393 add zone/location validation to cloudprovider constructors 
Signed-off-by: Steve Kriss <steve@heptio.com>
 2017-08-14 12:56:12 -07:00
Andy Goldstein a865cb87ee Merge pull request #37 from skriss/iops_fix 
only save/use iops for io1 volumes
 2017-08-14 15:24:30 -04:00
Steve Kriss c2271776cc remove unused params from GCP objectStorageAdapter 
Signed-off-by: Steve Kriss <steve@heptio.com>
 2017-08-14 10:56:05 -07:00
Steve Kriss aff57e0571 switch to int64 for iops val 
Signed-off-by: Steve Kriss <steve@heptio.com>
 2017-08-14 09:42:43 -07:00
Mathias Merscher df320d7bf3
adds support for S3 SSE with KMS 
Signed-off-by: Mathias Merscher <Mathias.Merscher@dg-i.net>
 2017-08-14 18:18:19 +02:00
Steve Kriss c41c958777 only save/use iops for io1 volumes 
Signed-off-by: Steve Kriss <steve@heptio.com>
 2017-08-11 13:33:01 -07:00
Andy Goldstein 2fe501f527 Initial commit 
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
 2017-08-02 13:27:17 -04:00