Commit Graph

867 Commits (89d9413ef44b08083f3649a31f3c1fe505fef9c4)

Author SHA1 Message Date
Ashish Amarnath 31dca0ea1b
add GroupResources for volumesnapshot types (#2288)
Signed-off-by: Ashish Amarnath <ashisham@vmware.com>
2020-02-24 11:47:35 -08:00
Nolan Brubaker 766ab5d3ee
Account for possible missing schemas on v1 CRDs (#2264)
* Account for possible missing schemas on v1 CRDs

If a v1beta1 CRD without a Schema was submitted to a Kubernets v1.16
cluster, then Kubernetes will server it back as a v1 CRD without a
schema.

However, when Velero tries to restore this document, the request will be
rejected as a v1 CRD must have a schema.

This commit has some defensive coding on the restore side, as well as
potential fixes on the backup side for getting around this.

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>

* Back up nonstructural CRDs as v1beta1

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>

* Add tests for remapping plugin

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>

* Add builders for v1 CRDs

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>

* Address review feedback

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>

* Remove extraneous log message

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>

* Add changelog

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2020-02-24 12:07:50 -07:00
Ashish Amarnath e681759178
export restic pod annotation (#2283)
Signed-off-by: Ashish Amarnath <ashisham@vmware.com>
2020-02-19 12:37:40 -07:00
JenTing Hsiao b4446bd358
Fix restic backup volume snapshot to the second location failed (#2244)
* Fix restic backup volume snapshot to the second location failed

Signed-off-by: JenTing Hsiao <jenting.hsiao@suse.com>
2020-02-19 11:01:21 -08:00
Mayank 1a1372550d
Use PV name returned from volumesnapshotter while creating a PV (#2216)
* Using PV name returned from volumesnapshotter while creating a PV

Signed-off-by: mayank <mayank.patel@mayadata.io>
2020-02-10 13:17:15 -07:00
Steve Kriss 44d3f91e97 backup describer: reword resource list not found msg
Signed-off-by: Steve Kriss <krisss@vmware.com>
2020-02-06 11:40:15 -07:00
Brad Koehn b25fea3bea
properly handle empty restic stdout during progress updates (#2231)
* properly handle empty restic stdout during progress updates

Signed-off-by: Brad Koehn <brad@koehn.com>
2020-02-03 12:49:23 -07:00
Steve Kriss 08c549a092
Restore result refactoring (#2234)
* move Result helper funcs to be methods

Signed-off-by: Steve Kriss <krisss@vmware.com>
2020-02-03 10:56:57 -08:00
Carlos Tadeu Panato Junior c9bc6646a5
use GA topology labels for PVs (#2219)
* use GA toplogy labels for PVs

Signed-off-by: Carlos Panato <ctadeu@gmail.com>
2020-02-03 10:47:18 -08:00
Ashish Amarnath 21264a11ec
Merge pull request #2218 from cpanato/GH-1168
remove schedule validation
2020-01-31 13:00:24 -08:00
Steve Kriss 37011ee5ac
bug fix: back up server-preferred version of CRD (#2230)
* bug fix: back up server-preferred version of CRD

Signed-off-by: Steve Kriss <krisss@vmware.com>
2020-01-31 08:45:46 -07:00
Carlos Panato 42b6126458
update variable name
Signed-off-by: Carlos Panato <ctadeu@gmail.com>
2020-01-31 09:22:46 +01:00
Nolan Brubaker 6745979a7b
Wait for CRDs to be ready before restoring CRs (#1937)
* Wait for CRDs to be available and ready

When restoring CRDs, we should wait for the definition to be ready and
available before moving on to restoring specific CRs.

While the CRDs are often ready by the time we get to restoring a CR,
there is a race condition where the CRD isn't ready.

This change waits on each CRD at restore time.

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2020-01-30 09:19:13 -08:00
Carlos Panato f7adc4dfd1
check if schedule have a backup
Signed-off-by: Carlos Panato <ctadeu@gmail.com>
2020-01-29 18:24:32 +01:00
Carlos Panato 445b5b781e
remove schedule validation
Signed-off-by: Carlos Panato <ctadeu@gmail.com>
2020-01-29 18:24:32 +01:00
JenTing Hsiao b19097f825 Pruning unknown fields (#2187)
* Pruning unknown fields

In CRD apiversion v1beta1, default preserveUnknownFields=true.

In CRD apiversion v1, the preserveUnknownFields can only be false.
Otherwise, the k8s validation bumps out error message for the
invalid preserveUnknownFields value.

Deploy Velero on k8s 1.16+ with CRD apiversion v1beta1, the
k8s cluster converts apiversion from v1beta1 to v1 automatically.

Fully backup and restore the cluster, restore bumps out error message
due to the preserveUnknownFields=true is not allowed on k8s 1.16+.

Since the CRD structural schema had been defined, enable the preserveUnknownFields
to false to solves the restore bumps out error message on k8s 1.16+.

Signed-off-by: JenTing Hsiao <jenting.hsiao@suse.com>

* Add changelog

Signed-off-by: JenTing Hsiao <jenting.hsiao@suse.com>
2020-01-28 14:04:28 -05:00
Nolan Brubaker 5b1280c2cd Fix restore issues for CRD v1 on Kubernetes v1.16 clusters (#2197)
Fix restore issues for CRD v1 on Kubernetes v1.16 clusters

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2020-01-27 13:59:08 -08:00
Ashish Amarnath ec22f2c88d
Merge pull request #2196 from skriss/fix-1777
recompute backup_last_successful_timestamp metric during resync
2020-01-21 17:01:51 -08:00
Ashish Amarnath 421dcd4e8d
Merge pull request #2201 from skriss/fix-2121
fix race condition in waiting for restic restores to complete
2020-01-21 15:16:08 -08:00
Steve Kriss 71201fe929 extract helper func and add unit tests
Signed-off-by: Steve Kriss <krisss@vmware.com>
2020-01-21 13:21:28 -07:00
Steve Kriss ae316193ea move log statement
Signed-off-by: Steve Kriss <krisss@vmware.com>
2020-01-21 12:36:46 -07:00
Steve Kriss 3b80e00d62 fix race condition in waiting for restic restores to complete
Signed-off-by: Steve Kriss <krisss@vmware.com>
2020-01-15 16:09:54 -07:00
Nolan Brubaker ee0cbcf11e
Merge pull request #2199 from jnaulty/jnaulty/improve-config-file-perms
Restrict file permissions for config file/dir
2020-01-15 15:13:22 -05:00
John Naulty 254a5eebb5
Restrict file permissions for config file/dir
Velero client config file should have restricted file permissions to be
read/write-able for the user that creates it--similiar to files like
`.ssh/id_rsa`

Refer to OTG-CONFIG-009: Test File Permission
> Impoper file permission configuration may result in privilledge
escalation, information explousure, DLL injection, or unauthorized file
access.
Therefore, files permission must be properly configured with minium
access permission by default.

[source](https://www.owasp.org/index.php/Test_File_Permission_(OTG-CONFIG-009))

Ticket: #1758
Signed-off-by: John Naulty <johnnaulty@bitgo.com>
2020-01-15 09:41:25 -08:00
Scott Seago aa44cf1c32 Check for nil LastMaintenanceTime in dueForMaintenance (#2200)
* Check for nil LastMaintenanceTime in dueForMaintenance

ResticRepository.dueForMaintenance causes a panic in the velero pod
("invalid memory address or nil pointer dereference") if
repository.Status.LastMaintenanceTime is nil. This fix returns 'true'
if it's nil, so the repository is due for maintenance if LastMaintenanceTime
is nil *or* the time elapsed since the last maintenance is greater than
repository.Spec.MaintenanceFrequency.Duration

Signed-off-by: Scott Seago <sseago@redhat.com>

* changelog for PR#2200

Signed-off-by: Scott Seago <sseago@redhat.com>
2020-01-15 12:06:16 -05:00
Steve Kriss 9fa302aa8b recompute backup_last_successful_timestamp metric during resync
Signed-off-by: Steve Kriss <krisss@vmware.com>
2020-01-14 14:11:21 -07:00
Steve Kriss b2acd3b683 bump go-hclog and prometheus dependency versions in prep for go modules (#2188)
* update revision of go-hclog to match go.mod requirement

Signed-off-by: Steve Kriss <krisss@vmware.com>

* update prometheus dep to prepare for go module migration

Signed-off-by: Steve Kriss <krisss@vmware.com>
2020-01-14 11:29:37 -05:00
Steve Kriss 04d8b47d3e update to k8s 1.17 deps and fix breaks (#2180)
* update to k8s 1.17 deps

Signed-off-by: Steve Kriss <krisss@vmware.com>
2020-01-10 08:55:58 -08:00
JenTing Hsiao 97b33402a4 Assign default CPU/Memory resource value if not present (#2176)
Install restic with CPU/Memory limits is optional.
If velero cannot parse resource requirements, use default value instead.

After that, the administrator won't get confused that something recovered failed.

Signed-off-by: JenTing Hsiao <jenting.hsiao@suse.com>
2020-01-09 08:48:35 -07:00
Shuwei Hao c832e52905 add ALIBABA_CLOUD_CREDENTIALS_FILE env in deployment and daemonset installation (#2166)
Signed-off-by: haoshuwei <haoshuwei24@gmail.com>
2020-01-06 14:56:23 -08:00
Ashish Amarnath 408c0dfb5a Add documentation for velero install cli (#2155)
* Add documentation for velero install cli

Signed-off-by: Ashish Amarnath <ashisham@vmware.com>

* mirror doc changes into v1.2.0

Signed-off-by: Ashish Amarnath <ashisham@vmware.com>
2020-01-06 13:13:02 -07:00
JenTing Hsiao 93e4737258 Fix incorrect output log format (#2165)
Signed-off-by: JenTing Hsiao <jenting.hsiao@suse.com>
2020-01-06 09:05:38 -07:00
Steve Kriss be140985c5 bug fix: only prioritize restoring replicasets.apps (#2157)
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-12-19 09:44:08 -08:00
John Naulty Jr cd860771c2 Add error return to pvRenamer func (#2138)
Migrate logic from NewUUID function into the pvRenamer function.

PR #2133 switched to a new NewUUID function that returns an error, but
the invocation of that function needs to happen within the pvRenamer
closure. Because the new function returns an error, the pvRenamer should
return the error, the signature needs to be changed and the return
checked.

Signed-off-by: John Naulty <johnnaulty@bitgo.com>
2019-12-17 10:23:58 -08:00
John Naulty ff889283b8
Switch from satori/go.uuid to gofrs/uuid
satori/go.uuid has a known issue with random uuid generation.
gofrs/uuid is still maintained and has fixed the random uuid generation
issue present in satori/go.uuid

Signed-off-by: John Naulty <johnnaulty@bitgo.com>
2019-12-14 10:27:00 -08:00
Steve Kriss 7c60829f38 bug fix: don't restore cluster-scoped resources by default when restoring specific namespaces (#2118)
* add failing test case for bug

Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-12-12 12:45:41 -08:00
Steve Kriss 96297ea437 add replicasets.apps to prioritized restore resources (#2120)
* add replicasets.apps to prioritized restore resources

Signed-off-by: Steve Kriss <krisss@vmware.com>

* changelog

Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-12-11 15:32:33 -05:00
Dinesh Yadav 83ef4eb4d0 Support AWS_PROFILE for restic backups/restore operations (#2096)
* Support AWS_PROFILE for restic backups/restore operations

It enables Velero to switch credentials if multiple s3-compatible
backupLocations are present.

Signed-off-by: dinesh <dinesh1042@gmail.com>

* better comments and fixing typos

Signed-off-by: dinesh <dinesh1042@gmail.com>

* add changelog entry

Signed-off-by: dinesh <dinesh1042@gmail.com>
2019-12-09 09:46:02 -05:00
Steve Kriss f42406723c bug fix: deep-copy backup's labels for snapshot tags (#2075)
* bug fix: deep-copy backup's labels for snapshot tags

Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-12-05 12:54:19 -08:00
Roman Klimenko 87be775139 Add support for private registry with custom port in restic-helper image (#1999)
* Add support for private registry with custom port in restic-helper image definition

Signed-off-by: Roman Klimenko <cognoz@ya.ru>
2019-12-04 08:01:39 -08:00
Steve Kriss e5a85f140a use field selector on name when watching download requests (#2065)
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-11-14 16:50:24 -08:00
Steve Kriss 5a31a78df5 add better error message when cluster config can't be found
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-11-08 15:01:12 -07:00
Antony S Bett 121dc02d2f Ensure object store plugin processes are cleaned up after restore and after BSL validation during server start up (#2041)
* Ensure child object store plugin processes are cleaned up during restore

Signed-off-by: Antony Bett <antony.bett@gmail.com>
2019-11-05 08:58:08 -08:00
Steve Kriss 4fb1bc2ef3
restic: don't try to restore PVBs with no snapshotID (#2031)
* restic: don't try to restore PVBs with no snapshotID

Signed-off-by: Steve Kriss <krisss@vmware.com>

* changelog

Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-11-04 16:18:08 -07:00
Adnan Abdulhussein 636a5b9db6
add velero install --crds-only to easily update CRDs (#2038)
* add velero install --crds-only to easily update CRDs

Signed-off-by: Adnan Abdulhussein <aadnan@vmware.com>

* make update

Signed-off-by: Adnan Abdulhussein <aadnan@vmware.com>
2019-11-04 14:36:04 -08:00
Steve Kriss 83752d28d7 Restic restore fixes (#2013)
* restic: use restore cmd's env when getting snapshot size

Signed-off-by: Steve Kriss <krisss@vmware.com>

* restic: remove code that considers 0-byte backups an error

Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-10-29 07:42:12 -07:00
Steve Kriss 5d6b129ea5 fix typo in download request validation tag (#2009)
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-10-28 16:00:03 -07:00
Steve Kriss 0c1fc8195a add --allow-partially-failed flag to velero restore create (#1994)
* add --allow-partially-failed flag to velero restore create

Signed-off-by: Steve Kriss <krisss@vmware.com>

* remove extraneous client creation

Signed-off-by: Steve Kriss <krisss@vmware.com>

* add godoc to helper func

Signed-off-by: Steve Kriss <krisss@vmware.com>

* todo

Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-10-24 14:20:57 -07:00
Antony S Bett 0450567bba Allow backup storage locations to specify backup sync period or toggle off sync (#1936)
* Allow backup storage locations to specify backup sync period or toggle off sync

Signed-off-by: Antony Bett <antony.bett@gmail.com>

* Add --backup-sync-period flag to backup location create command

Signed-off-by: Antony Bett <antony.bett@gmail.com>
2019-10-24 08:32:45 -06:00
KubeKween d26bf05b33 Extract providers (#1985)
* Remove cloud providers and reorg code

Signed-off-by: Carlisia <carlisia@vmware.com>

* Update dependencies

Signed-off-by: Carlisia <carlisia@vmware.com>

* Fix tests

Signed-off-by: Carlisia <carlisia@vmware.com>

* fix dependency issues

Signed-off-by: Carlisia <carlisia@vmware.com>

* Delete dup test

Signed-off-by: Carlisia <carlisia@vmware.com>

* Add back spaces to file

Signed-off-by: Carlisia <carlisia@vmware.com>

* Remove and update docs

Signed-off-by: Carlisia <carlisia@vmware.com>

* Make the plugins flag required

Signed-off-by: Carlisia <carlisia@vmware.com>

* Add changelog

Signed-off-by: Carlisia <carlisia@vmware.com>

* Make the plugins flag conditional

Signed-off-by: Carlisia <carlisia@vmware.com>
2019-10-22 15:31:27 -07:00