Commit Graph

169 Commits (89d9413ef44b08083f3649a31f3c1fe505fef9c4)

Author SHA1 Message Date
Nolan Brubaker 766ab5d3ee
Account for possible missing schemas on v1 CRDs (#2264)
* Account for possible missing schemas on v1 CRDs

If a v1beta1 CRD without a Schema was submitted to a Kubernets v1.16
cluster, then Kubernetes will server it back as a v1 CRD without a
schema.

However, when Velero tries to restore this document, the request will be
rejected as a v1 CRD must have a schema.

This commit has some defensive coding on the restore side, as well as
potential fixes on the backup side for getting around this.

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>

* Back up nonstructural CRDs as v1beta1

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>

* Add tests for remapping plugin

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>

* Add builders for v1 CRDs

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>

* Address review feedback

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>

* Remove extraneous log message

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>

* Add changelog

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2020-02-24 12:07:50 -07:00
Nolan Brubaker 6745979a7b
Wait for CRDs to be ready before restoring CRs (#1937)
* Wait for CRDs to be available and ready

When restoring CRDs, we should wait for the definition to be ready and
available before moving on to restoring specific CRs.

While the CRDs are often ready by the time we get to restoring a CR,
there is a race condition where the CRD isn't ready.

This change waits on each CRD at restore time.

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2020-01-30 09:19:13 -08:00
Nolan Brubaker 5b1280c2cd Fix restore issues for CRD v1 on Kubernetes v1.16 clusters (#2197)
Fix restore issues for CRD v1 on Kubernetes v1.16 clusters

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2020-01-27 13:59:08 -08:00
Steve Kriss be140985c5 bug fix: only prioritize restoring replicasets.apps (#2157)
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-12-19 09:44:08 -08:00
Steve Kriss 96297ea437 add replicasets.apps to prioritized restore resources (#2120)
* add replicasets.apps to prioritized restore resources

Signed-off-by: Steve Kriss <krisss@vmware.com>

* changelog

Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-12-11 15:32:33 -05:00
Antony S Bett 121dc02d2f Ensure object store plugin processes are cleaned up after restore and after BSL validation during server start up (#2041)
* Ensure child object store plugin processes are cleaned up during restore

Signed-off-by: Antony Bett <antony.bett@gmail.com>
2019-11-05 08:58:08 -08:00
Antony S Bett 0450567bba Allow backup storage locations to specify backup sync period or toggle off sync (#1936)
* Allow backup storage locations to specify backup sync period or toggle off sync

Signed-off-by: Antony Bett <antony.bett@gmail.com>

* Add --backup-sync-period flag to backup location create command

Signed-off-by: Antony Bett <antony.bett@gmail.com>
2019-10-24 08:32:45 -06:00
KubeKween d26bf05b33 Extract providers (#1985)
* Remove cloud providers and reorg code

Signed-off-by: Carlisia <carlisia@vmware.com>

* Update dependencies

Signed-off-by: Carlisia <carlisia@vmware.com>

* Fix tests

Signed-off-by: Carlisia <carlisia@vmware.com>

* fix dependency issues

Signed-off-by: Carlisia <carlisia@vmware.com>

* Delete dup test

Signed-off-by: Carlisia <carlisia@vmware.com>

* Add back spaces to file

Signed-off-by: Carlisia <carlisia@vmware.com>

* Remove and update docs

Signed-off-by: Carlisia <carlisia@vmware.com>

* Make the plugins flag required

Signed-off-by: Carlisia <carlisia@vmware.com>

* Add changelog

Signed-off-by: Carlisia <carlisia@vmware.com>

* Make the plugins flag conditional

Signed-off-by: Carlisia <carlisia@vmware.com>
2019-10-22 15:31:27 -07:00
Alexander Demichev 1564317cef Restore action for cluster/namespace role bindings (#1974)
Signed-off-by: Alexander Demichev <ademicev@redhat.com>
2019-10-21 15:11:26 -06:00
Steve Kriss 48792ece1f Updates for org move to vmware-tanzu (#1920)
* update import paths to github.com/vmware-tanzu/...

Signed-off-by: Steve Kriss <krisss@vmware.com>

* update other GH org refs to vmware-tanzu

Signed-off-by: Steve Kriss <krisss@vmware.com>

* site and docs: update GH org to vmware-tanzu

Signed-off-by: Steve Kriss <krisss@vmware.com>

* update travis badge links on docs readmes

Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-09-30 17:26:56 -04:00
Nolan Brubaker 8ec1548b3c Add features package (#1849)
* Add features package

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2019-09-18 10:57:04 -06:00
Steve Kriss 6623e1f273 Change restic prune default interval to 7d and make user-configurable (#1864)
* change restic prune default interval to 7d, add server flag for it

Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-09-10 11:58:42 -07:00
Nolan Brubaker 363c2692a1 Use custom namespace when creating Velero clients
The Velero deployment did not have a way of exposing the namespace it
was installed in to the API client. This is a problem for plugins that
need to query for resources in that namespaces, such as the restic
restore process that needs to find PodVolume(Backup|Restore)s.

While the Velero client is consulted for a configured namespace, this
cannot be set in the server pod since there is no valid home directory
in which to place it.

This change provides the namespace to the deployment via the downward
API, and updates the API client factory to use the VELERO_NAMESPACE
before looking at the config file, so that any plugins using the client
will look at the appropriate namespace.

Fixes #1743

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2019-08-13 13:06:12 -04:00
Steve Kriss ffa3251efc move contents of pkg/util/test into pkg/test
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-08-07 08:26:00 -06:00
KubeKween 4accb8512a Restore from PodVolumeBackups (#1723)
* Restore from PodVolumeBackups

Signed-off-by: Carlisia <carlisiac@vmware.com>

* Partially address code reviews

Signed-off-by: Carlisia <carlisiac@vmware.com>

* Partially address code reviews #2

Signed-off-by: Carlisia <carlisiac@vmware.com>

* Clean up struct

Signed-off-by: Carlisia <carlisiac@vmware.com>

* Fix log messages

Signed-off-by: Carlisia <carlisiac@vmware.com>

* Fix tests

Signed-off-by: Carlisia <carlisiac@vmware.com>

* Clean up

Signed-off-by: Carlisia <carlisiac@vmware.com>

* Add changelog

Signed-off-by: Carlisia <carlisiac@vmware.com>
2019-08-06 13:17:36 -07:00
carthewd 22eca22ac8 Add JSON as an option for log output (#1654)
* Add JSON as an option for log output

Signed-off-by: Donovan Carthew <donovan.carthew@gmail.com>
2019-07-30 16:29:34 -07:00
KubeKween 3b9af8c654 Store PodVolumeBackups in obj storage & use as source of truth (#1577)
* Store PodVolumeBackups in object storage

Signed-off-by: Carlisia <carlisiac@vmware.com>
2019-07-24 15:51:20 -04:00
Steve Kriss 1bb167ef90 add restore item action to change PVC/PV storage class name (#1621)
* add restore item action to change PVC/PV storage class name

Signed-off-by: Steve Kriss <krisss@vmware.com>

* code review

Signed-off-by: Steve Kriss <krisss@vmware.com>

* change existing plugin names to lowercase/hyphenated

Signed-off-by: Steve Kriss <krisss@vmware.com>

* add validation for new storage class name

Signed-off-by: Steve Kriss <krisss@vmware.com>

* add test cases

Signed-off-by: Steve Kriss <krisss@vmware.com>

* changelog

Signed-off-by: Steve Kriss <krisss@vmware.com>

* fix imports

Signed-off-by: Steve Kriss <krisss@vmware.com>

* update plugin names to be more consistent

Signed-off-by: Steve Kriss <krisss@vmware.com>

* update unit tests to use pkg/test object constructors

Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-07-15 11:19:38 -07:00
Adnan Abdulhussein 74b575200c hide server commands in velero CLI (#1561)
* hide server commands in velero CLI

Signed-off-by: Adnan Abdulhussein <aadnan@vmware.com>

* changelog

Signed-off-by: Adnan Abdulhussein <aadnan@vmware.com>

* remove unnecessary comment

Signed-off-by: Adnan Abdulhussein <aadnan@vmware.com>
2019-06-10 19:20:17 -06:00
KubeKween 0a771e6a53 Add cmd to list plugins (#1535)
* Add cmd to list plugins

Signed-off-by: Carlisia <carlisiac@vmware.com>
2019-06-05 13:41:02 -04:00
Steve Kriss 411d44a673 allow individual backup storage locations to be read-only (#1517)
* allow individual backup storage locations to be read-only

Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-05-29 14:21:25 -04:00
Steve Kriss 2406994740 remove ref to v0.10 from BSL validation error msg (#1429)
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-05-02 11:20:50 -07:00
Steve Kriss 7f36f78aee remove code that removes legacy GC finalizer from backups
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-04-15 10:17:03 -06:00
KubeKween abee09aa2d Add validation for plugin name format and dups (#1339)
* Add validation for plugin name format and dups

Signed-off-by: Carlisia <carlisiac@vmware.com>

* Bug fix

Signed-off-by: Carlisia <carlisiac@vmware.com>

* Add changelog

Signed-off-by: Carlisia <carlisiac@vmware.com>

* Address code reviews

Signed-off-by: Carlisia <carlisiac@vmware.com>

* Fix code

Signed-off-by: Carlisia <carlisiac@vmware.com>

* Address code reviews

Signed-off-by: Carlisia <carlisiac@vmware.com>

* Add documentation

Signed-off-by: Carlisia <carlisiac@vmware.com>

* Update godoc

Signed-off-by: Carlisia <carlisiac@vmware.com>

* More doc

Signed-off-by: Carlisia <carlisiac@vmware.com>
2019-04-12 08:25:04 -06:00
Aman Wangde 0e0f357cef Added ability to disable controllers (#1326)
Signed-off-by: James King <james.king@emc.com>
2019-04-10 15:41:28 -04:00
Rohan Vora 4beb8aab3c Set default backup TTL
Set default backup TTL to 30 days when TTL
is not provided in the backup yaml configuration.

Updates #138

Signed-off-by: Rohan Vora <vorar@vmware.com>
2019-04-09 14:13:29 -07:00
Fábio Franco Uechi 0167539a14 add new counter metrics for backup deletion (#1280)
* compute backup deletion metrics (attempt, success, fail)

Signed-off-by: fabito <fuechi@ciandt.com>
2019-04-04 14:25:59 -04:00
Carlisia 23b1098950
Fix copyright
Signed-off-by: Carlisia <carlisiac@vmware.com>
2019-04-01 12:06:14 -07:00
Steve Kriss 7298a4eda0 allow restic restore helper image to be specified via ConfigMap (#1311)
* allow restic restore helper image to be specified via ConfigMap

Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-03-29 17:11:34 -04:00
Steve Kriss 9b635c0e14 add additionalItems to restore item actions (#1304)
* add additionalItems to restore item actions

Signed-off-by: Steve Kriss <krisss@vmware.com>
Co-authored-by: Andy Goldstein <goldsteina@vmware.com>
2019-03-28 12:21:56 -07:00
Steve Kriss bb9c3f6a1a rename BlockStore to VolumeSnapshotter
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-03-27 14:55:28 -06:00
Steve Kriss a111eed2af update license headers to Velero contributors (#1302)
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-03-20 12:32:48 -07:00
Carlisia a71e43b2b7
Split velero plugin client into its own package
Signed-off-by: Carlisia <carlisiac@vmware.com>
2019-03-19 16:05:37 -07:00
Steve Kriss 1eac10ca9f
Merge pull request #1288 from carlisia/c-plugins-II
Split plugin framework into its own package
2019-03-19 16:44:58 -06:00
Carlisia 7dfe58d37f
Split plugin framework into its own package
Signed-off-by: Carlisia <carlisiac@vmware.com>
2019-03-19 15:36:31 -07:00
Steve Kriss 7d66fc31bd pkg/restic: check for & skip hostPath PVC/PVs
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-03-19 08:57:47 -06:00
KubeKween 73514a003b Move plugin interfaces to same package (#1264)
* Move plugin interfaces to same package

Signed-off-by: Carlisia <carlisiac@vmware.com>
2019-03-14 16:35:06 -04:00
Steve Kriss 7674332313 pass --log-level to plugins (#1278)
Plumb the log level through to plugin processes


Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-03-14 10:53:36 -07:00
Steve Kriss 4583aa7078
Merge pull request #826 from nrb/fix-691
Wait for namespace to terminate before restoring
2019-02-07 11:07:25 -07:00
Nolan Brubaker 890202f2e4 Wait for PV/namespace to delete before restore
If a PV already exists, wait for it, it's associated PVC, and associated
namespace to be deleted before attempting to restore it.

If a namespace already exists, wait for it to be deleted before
attempting to restore it.

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2019-02-06 15:43:50 -05:00
Steve Kriss 328bc361be velero server: log version and git SHA at startup
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-02-05 10:12:06 -07:00
Nolan Brubaker 43714caaec Rename Ark to Velero!!!
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2019-02-04 17:35:22 -05:00
Steve Kriss 8a58b217be show server version in ark version output using ServerStatusRequest CRD
Signed-off-by: Steve Kriss <steve@heptio.com>
2019-01-23 12:51:13 -07:00
Carlisia 1906c33eb2
Improve directions when location CRDs not deployed
Signed-off-by: Carlisia <carlisia@grokkingtech.io>
2018-12-07 10:41:40 -08:00
Andy Goldstein aa9d96f3b7
Add pprof support to ark server
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2018-12-05 10:13:39 -05:00
Andy Goldstein 62d8c642d2
Remove default token from all service accounts
Instead of only removing the default token from a service account when
it already exists in the cluster, always remove it. If the service
account already exists, continue to do the merging logic.

Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2018-12-04 11:41:18 -05:00
Andy Goldstein d85872dfb2
Merge pull request #1069 from gliptak/patch-1
Update to go 1.11
2018-11-21 08:39:06 -05:00
Gábor Lipták daf1a75515 Bring Go current in Travis
Signed-off-by: Gábor Lipták <gliptak@gmail.com>
2018-11-21 08:31:14 -05:00
Christian Beneke 90c5ed0850 Initialize empty schedule metrics on server init
When backups are run manually (outside of a schedule) the metrics
will be counted for ark_*{schedule=""}. To prevent partial NaN
metrics they will be initialised on server init.

Signed-off-by: Christian Beneke <c.beneke@syseleven.de>
2018-11-14 15:31:26 +01:00
Steve Kriss 2781e4e8de
Merge pull request #956 from ffd2subroutine/qps-burst-configurable
Added qps and burst to server's client
2018-11-09 12:41:41 -07:00