Commit Graph

834 Commits (cd860771c28a2509668540c2ad0c7e00cfe8937f)

Author SHA1 Message Date
John Naulty Jr cd860771c2 Add error return to pvRenamer func (#2138)
Migrate logic from NewUUID function into the pvRenamer function.

PR #2133 switched to a new NewUUID function that returns an error, but
the invocation of that function needs to happen within the pvRenamer
closure. Because the new function returns an error, the pvRenamer should
return the error, the signature needs to be changed and the return
checked.

Signed-off-by: John Naulty <johnnaulty@bitgo.com>
2019-12-17 10:23:58 -08:00
John Naulty ff889283b8
Switch from satori/go.uuid to gofrs/uuid
satori/go.uuid has a known issue with random uuid generation.
gofrs/uuid is still maintained and has fixed the random uuid generation
issue present in satori/go.uuid

Signed-off-by: John Naulty <johnnaulty@bitgo.com>
2019-12-14 10:27:00 -08:00
Steve Kriss 7c60829f38 bug fix: don't restore cluster-scoped resources by default when restoring specific namespaces (#2118)
* add failing test case for bug

Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-12-12 12:45:41 -08:00
Steve Kriss 96297ea437 add replicasets.apps to prioritized restore resources (#2120)
* add replicasets.apps to prioritized restore resources

Signed-off-by: Steve Kriss <krisss@vmware.com>

* changelog

Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-12-11 15:32:33 -05:00
Dinesh Yadav 83ef4eb4d0 Support AWS_PROFILE for restic backups/restore operations (#2096)
* Support AWS_PROFILE for restic backups/restore operations

It enables Velero to switch credentials if multiple s3-compatible
backupLocations are present.

Signed-off-by: dinesh <dinesh1042@gmail.com>

* better comments and fixing typos

Signed-off-by: dinesh <dinesh1042@gmail.com>

* add changelog entry

Signed-off-by: dinesh <dinesh1042@gmail.com>
2019-12-09 09:46:02 -05:00
Steve Kriss f42406723c bug fix: deep-copy backup's labels for snapshot tags (#2075)
* bug fix: deep-copy backup's labels for snapshot tags

Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-12-05 12:54:19 -08:00
Roman Klimenko 87be775139 Add support for private registry with custom port in restic-helper image (#1999)
* Add support for private registry with custom port in restic-helper image definition

Signed-off-by: Roman Klimenko <cognoz@ya.ru>
2019-12-04 08:01:39 -08:00
Steve Kriss e5a85f140a use field selector on name when watching download requests (#2065)
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-11-14 16:50:24 -08:00
Steve Kriss 5a31a78df5 add better error message when cluster config can't be found
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-11-08 15:01:12 -07:00
Antony S Bett 121dc02d2f Ensure object store plugin processes are cleaned up after restore and after BSL validation during server start up (#2041)
* Ensure child object store plugin processes are cleaned up during restore

Signed-off-by: Antony Bett <antony.bett@gmail.com>
2019-11-05 08:58:08 -08:00
Steve Kriss 4fb1bc2ef3
restic: don't try to restore PVBs with no snapshotID (#2031)
* restic: don't try to restore PVBs with no snapshotID

Signed-off-by: Steve Kriss <krisss@vmware.com>

* changelog

Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-11-04 16:18:08 -07:00
Adnan Abdulhussein 636a5b9db6
add velero install --crds-only to easily update CRDs (#2038)
* add velero install --crds-only to easily update CRDs

Signed-off-by: Adnan Abdulhussein <aadnan@vmware.com>

* make update

Signed-off-by: Adnan Abdulhussein <aadnan@vmware.com>
2019-11-04 14:36:04 -08:00
Steve Kriss 83752d28d7 Restic restore fixes (#2013)
* restic: use restore cmd's env when getting snapshot size

Signed-off-by: Steve Kriss <krisss@vmware.com>

* restic: remove code that considers 0-byte backups an error

Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-10-29 07:42:12 -07:00
Steve Kriss 5d6b129ea5 fix typo in download request validation tag (#2009)
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-10-28 16:00:03 -07:00
Steve Kriss 0c1fc8195a add --allow-partially-failed flag to velero restore create (#1994)
* add --allow-partially-failed flag to velero restore create

Signed-off-by: Steve Kriss <krisss@vmware.com>

* remove extraneous client creation

Signed-off-by: Steve Kriss <krisss@vmware.com>

* add godoc to helper func

Signed-off-by: Steve Kriss <krisss@vmware.com>

* todo

Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-10-24 14:20:57 -07:00
Antony S Bett 0450567bba Allow backup storage locations to specify backup sync period or toggle off sync (#1936)
* Allow backup storage locations to specify backup sync period or toggle off sync

Signed-off-by: Antony Bett <antony.bett@gmail.com>

* Add --backup-sync-period flag to backup location create command

Signed-off-by: Antony Bett <antony.bett@gmail.com>
2019-10-24 08:32:45 -06:00
KubeKween d26bf05b33 Extract providers (#1985)
* Remove cloud providers and reorg code

Signed-off-by: Carlisia <carlisia@vmware.com>

* Update dependencies

Signed-off-by: Carlisia <carlisia@vmware.com>

* Fix tests

Signed-off-by: Carlisia <carlisia@vmware.com>

* fix dependency issues

Signed-off-by: Carlisia <carlisia@vmware.com>

* Delete dup test

Signed-off-by: Carlisia <carlisia@vmware.com>

* Add back spaces to file

Signed-off-by: Carlisia <carlisia@vmware.com>

* Remove and update docs

Signed-off-by: Carlisia <carlisia@vmware.com>

* Make the plugins flag required

Signed-off-by: Carlisia <carlisia@vmware.com>

* Add changelog

Signed-off-by: Carlisia <carlisia@vmware.com>

* Make the plugins flag conditional

Signed-off-by: Carlisia <carlisia@vmware.com>
2019-10-22 15:31:27 -07:00
Adnan Abdulhussein 69f993aebd update gcr.io/heptio-images/ references to velero/ (#1988)
* update gcr.io/heptio-images/ references to velero/

Signed-off-by: Adnan Abdulhussein <aadnan@vmware.com>
2019-10-22 14:39:43 -07:00
Alexander Demichev 1564317cef Restore action for cluster/namespace role bindings (#1974)
Signed-off-by: Alexander Demichev <ademicev@redhat.com>
2019-10-21 15:11:26 -06:00
Adnan Abdulhussein f2eb072694 fix controller-gen version in builder image (#1984)
Signed-off-by: Adnan Abdulhussein <aadnan@vmware.com>
2019-10-21 13:49:32 -07:00
FrankChen7 ae33441500 Add --no-default-storage-location flag to velero install to enable velero deployment without default BSL (#1931)
This closes #1853.

Signed-off-by: Yunqian <specyq@gmail.com>
2019-10-18 08:41:59 -06:00
Steve Kriss 92ea44e8e6 tweak test case names for clarity (#1976)
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-10-17 12:39:42 -07:00
Scott Seago 970f17c1ef If includeClusterResources is nil/auto, pull in necessary CRDs (#1831)
Related issue: https://github.com/heptio/velero/issues/1830

This accomplishes everything
that's needed, although there might be room for improvement in avoiding
a GET call for matching CRDs for each resource backed up. An alternative
could be a single call to get all CRDs prior to iterating over resources
and passing this into the backupResource function.

Signed-off-by: Scott Seago <sseago@redhat.com>
2019-10-17 12:31:04 -06:00
Adnan Abdulhussein e3d64d9dd9 use pointer types for metav1.Time fields (#1951)
* use pointer types for metav1.Time fields

Signed-off-by: Adnan Abdulhussein <aadnan@vmware.com>

* simpler metav1.Time ptrs

Signed-off-by: Adnan Abdulhussein <aadnan@vmware.com>

* remove test debug println

Signed-off-by: Adnan Abdulhussein <aadnan@vmware.com>
2019-10-14 10:20:28 -06:00
KubeKween d868b71e29 Move validation of plugins (#1953)
* Move validation of plugins

Signed-off-by: Carlisia <carlisia@vmware.com>
2019-10-09 10:48:37 -06:00
Steve Kriss 557d170541 restic: only backup ReadWriteMany PVC's once per velero backup (#1896)
* restic: only backup ReadWriteMany PVC's once per velero backup

Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-10-08 17:16:35 -04:00
Steve Kriss d3e30dd58b Azure china/germany support (#1938)
* feat: add azure china support

Signed-off-by: andyzhangx <xiazhang@microsoft.com>

* remove AZURE_CLOUD_NAME from required env var fetching

Signed-off-by: Steve Kriss <krisss@vmware.com>

* minor simplification of parseAzureEnvironment

Signed-off-by: Steve Kriss <krisss@vmware.com>

* changelog

Signed-off-by: Steve Kriss <krisss@vmware.com>

* remove cloudNameEnvVar from getRequiredValues call

Signed-off-by: Steve Kriss <krisss@vmware.com>

* just check for err != nil

Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-10-08 17:13:51 -04:00
Nolan Brubaker f009fe9bd1 Add --plugins flag to velero install (#1930)
* Add plugins flag to install

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2019-10-03 16:44:52 -07:00
Steve Chung db59d8d4bc Revert "Revert "allow self signed certs with insecureSkipVerify"" (#1793)
* handle self-signed certs with insecureSkipTLSVerify server config

Signed-off-by: Steven Chung <schung@d2iq.com>
2019-10-03 13:46:46 -07:00
Joshua Wong 46822aea2c Add support for GKE Workload Identity (#1810)
* Allow the velero server to be created on GCP even without a provided service account key in order to support workload identity and default compute engine credentials. Add option for adding service account annotations.

Signed-off-by: Joshua Wong <joshua99wong@gmail.com>
2019-10-03 13:45:18 -07:00
Moritz Schmitz von Hülst eadac44e10 Add option for Azure cross subscription backups (#1895)
* add option for different subscription id

Signed-off-by: Moritz Schmitz von Hülst <msc@marketlogicsoftware.com>
2019-10-03 13:31:35 -07:00
KubeKween e054a2f0b3 Make "not restored" message a bit better (#1901)
* Make "not restoring" message a bit better

Signed-off-by: Carlisia <carlisia@vmware.com>

* Tweak error msg

Signed-off-by: Carlisia <carlisia@vmware.com>

* Address feedback

Signed-off-by: Carlisia <carlisia@vmware.com>
2019-10-01 16:47:21 -04:00
Steve Kriss 48792ece1f Updates for org move to vmware-tanzu (#1920)
* update import paths to github.com/vmware-tanzu/...

Signed-off-by: Steve Kriss <krisss@vmware.com>

* update other GH org refs to vmware-tanzu

Signed-off-by: Steve Kriss <krisss@vmware.com>

* site and docs: update GH org to vmware-tanzu

Signed-off-by: Steve Kriss <krisss@vmware.com>

* update travis badge links on docs readmes

Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-09-30 17:26:56 -04:00
Steve Kriss d96186473c sync controller: replace revision file with full diff each interval (#1892)
* backup sync controller: replace revision file with full diff each interval

Signed-off-by: Steve Kriss <krisss@vmware.com>

* remove getting/setting of metadata/revision file

Signed-off-by: Steve Kriss <krisss@vmware.com>

* changelog

Signed-off-by: Steve Kriss <krisss@vmware.com>

* tweak logging

Signed-off-by: Steve Kriss <krisss@vmware.com>

* don't keep podVolumeBackup log field around after syncing PVBs

Signed-off-by: Steve Kriss <krisss@vmware.com>

* update generated CRDs

Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-09-27 16:23:11 -04:00
Christopher Angelo Phillips ca6c0a5a3d Increment logging for item backupper (#1904)
Signed-off-by: Christopher Phillips <cphillips918@gmail.com>
2019-09-27 13:17:54 -07:00
lintongj 2d845683a2 Add LD_LIBRARY_PATH as an env varible for the use of vsphere plugin (#1893)
* Add LD_LIBRARY_PATH as an env varible for the use of vsphere plugin

Signed-off-by: Lintong Jiang <lintongj@vmware.com>
2019-09-27 13:10:00 -07:00
Steve Kriss 2e849dcb99 Fix for #1888: check item's original namespace, not remapped one, for inclusion/exclusion (#1909)
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-09-25 18:01:16 -07:00
Adnan Abdulhussein dd088e3475 velero API type changes for structural schema CRDs (#1898)
* velero API type changes for structural schema CRDs

- make optional fields consistent with comment and omitempty struct tags
- adds kubebuilder annotations for structural schema validations

Signed-off-by: Adnan Abdulhussein <aadnan@vmware.com>

* update generated crds

Signed-off-by: Adnan Abdulhussein <aadnan@vmware.com>

* update velero install to use structural schema generated crds

Signed-off-by: Adnan Abdulhussein <aadnan@vmware.com>

* move enum declarations closer to type declarations

Signed-off-by: Adnan Abdulhussein <aadnan@vmware.com>

* add labels to generated crds

Signed-off-by: Adnan Abdulhussein <aadnan@vmware.com>

* allow null values in some fields

Signed-off-by: Adnan Abdulhussein <aadnan@vmware.com>

* allow labelSelector to be null

Signed-off-by: Adnan Abdulhussein <aadnan@vmware.com>

* remove debug statement

Signed-off-by: Adnan Abdulhussein <aadnan@vmware.com>

* make update

Signed-off-by: Adnan Abdulhussein <aadnan@vmware.com>

* add enum validations for CRD phases

Signed-off-by: Adnan Abdulhussein <aadnan@vmware.com>

* changelog

Signed-off-by: Adnan Abdulhussein <aadnan@vmware.com>
2019-09-24 18:37:28 -04:00
Adnan Abdulhussein 5e22f9c1c8 Generate CRDs with structural schema (#1885)
* generating CRDs with controller-tools and embedding in binary

Signed-off-by: Adnan Abdulhussein <aadnan@vmware.com>
2019-09-23 10:25:48 -06:00
Nolan Brubaker 8ec1548b3c Add features package (#1849)
* Add features package

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2019-09-18 10:57:04 -06:00
Steve Kriss fdd04b4d90 GCP: support specifying Cloud KMS key name for backup storage locations (#1879)
GCP: support specifying Cloud KMS key for backup storage location

Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-09-17 16:35:28 -04:00
Steve Kriss bc60412f29 AWS: add support for arbitrary SSE algorithms, e.g. AES256 (#1869)
* AWS: add support for arbitrary SSE algorithms, e.g. AES256

Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-09-12 10:13:46 -07:00
Adnan Abdulhussein 23376e6293 record restic restore progress in PodVolumeRestore (#1854)
* record restic restore progress in PodVolumeRestore

Signed-off-by: Adnan Abdulhussein <aadnan@vmware.com>

* add progress output to velero describe restore --details

Signed-off-by: Adnan Abdulhussein <aadnan@vmware.com>

* test getVolumeSize

Signed-off-by: Adnan Abdulhussein <aadnan@vmware.com>

* block on send to finish goroutine

Signed-off-by: Adnan Abdulhussein <aadnan@vmware.com>

* remove sync import

Signed-off-by: Adnan Abdulhussein <aadnan@vmware.com>

* add changelog

Signed-off-by: Adnan Abdulhussein <aadnan@vmware.com>

* make update

Signed-off-by: Adnan Abdulhussein <aadnan@vmware.com>
2019-09-10 15:50:57 -06:00
Steve Kriss 6623e1f273 Change restic prune default interval to 7d and make user-configurable (#1864)
* change restic prune default interval to 7d, add server flag for it

Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-09-10 11:58:42 -07:00
Adnan Abdulhussein 26e06dae53 fix excluding additional items with the exclude-from-backup label (#1843)
* fix excluding additional items with the exclude-from-backup label

Signed-off-by: Adnan Abdulhussein <aadnan@vmware.com>
2019-09-10 11:45:55 -07:00
Adnan Abdulhussein 1e182e5837 record restic backup progress in PodVolumeBackup (#1821)
* record restic backup progress in PodVolumeBackup

Signed-off-by: Adnan Abdulhussein <aadnan@vmware.com>
2019-09-10 11:28:19 -07:00
KubeKween fc39ac6d51 Add a check to avoid nil pointer (#1861)
Signed-off-by: Carlisia <carlisia@vmware.com>
2019-09-10 09:29:32 -06:00
Steve Kriss 11190065fd velero install: wait for restic daemonset to be ready (#1859)
* velero install: wait for restic daemonset to be ready

Signed-off-by: Steve Kriss <krisss@vmware.com>

* changelog

Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-09-09 17:54:47 -04:00
Nolan Brubaker 8da9ef2b03 Make GCP credentials parsing errors less ambiguous (#1850)
A user encountered the following error on a GCP project:

An error occurred: some backup storage locations are invalid: error getting backup store for location "default": rpc error: code = Unknown desc = invalid character '-' in numeric literal

This error was ambiguous and took some time to track down to the fact
that their credentials file wasn't a JSON file, but instead the contents
of the private key field. This change makes the problem slightly easier
to debug.

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2019-09-09 08:22:00 -06:00
Steve Kriss 29f29f9d61
Merge pull request #1828 from carlisia/c-1826-test-bug
Sort mounts in test case to avoid intermittent fail
2019-09-03 12:39:59 -06:00