Commit Graph

1045 Commits (c4484d1c7e4211c8a301435c833815d94027dbbd)

Author SHA1 Message Date
runzexia 3a4e441af8
add kindfor func to get apiresource from gvk (#2764)
* add kindfor func to get apiresource from gvk

Signed-off-by: RyderXia <ryder.xia@sap.com>

* impl interface & changelog

Signed-off-by: RyderXia <ryder.xia@sap.com>
2020-08-17 15:52:33 -07:00
Phuong N. Hoang 14170b52a8
Enhance Backup to backup resources in specific order. (#2724)
Signed-off-by: Phuong Hoang <phuong.n.hoang@dell.com>

Co-authored-by: Phuong Hoang <phuong.n.hoang@dell.com>
2020-08-12 17:17:31 -07:00
Jason Scarano 827d5d34f5
Improve and clarify cmd help documentation, flags, and examples (#2736)
* capitalize `backup create` cmd comments & examples

Signed-off-by: Jason Scarano <scaranoj@vmware.com>

* update copyright and capitilize flags and comments

Signed-off-by: Jason Scarano <scaranoj@vmware.com>

* update copyright and capitilize flags and comments

Signed-off-by: Jason Scarano <scaranoj@vmware.com>

* update copyright and capitilize flags and comments

Signed-off-by: Jason Scarano <scaranoj@vmware.com>

* update backuplocation, restic, & restore cmd doc

Signed-off-by: Jason Scarano <scaranoj@vmware.com>

* fix local typo

Signed-off-by: Jason Scarano <scaranoj@vmware.com>

* update copyrights & capitalize pflag/help strings

Signed-off-by: Jason Scarano <scaranoj@vmware.com>

* update copyright in utils dir

Signed-off-by: Jason Scarano <scaranoj@vmware.com>

* Revert "update copyright in utils dir"

This reverts commit d116efe3a3.

Signed-off-by: Jason Scarano <scaranoj@vmware.com>

* revert copyright changes

Signed-off-by: Jason Scarano <scaranoj@vmware.com>

* restore missing file

Signed-off-by: Jason Scarano <scaranoj@vmware.com>

* revert copyright changes

Signed-off-by: Jason Scarano <scaranoj@vmware.com>

* add cacert flag back

Signed-off-by: Jason Scarano <scaranoj@vmware.com>

Co-authored-by: Carlisia Campos <carlisia@vmware.com>
2020-08-12 18:13:44 -04:00
Ashish Amarnath 9eca0fcbff
Pass default-volumes-to-restic flag from create schedule to backup (#2776)
* Pass default-volumes-to-restic flag from create schedule to backup

Signed-off-by: Ashish Amarnath <ashisham@vmware.com>
2020-08-12 12:09:07 -07:00
Ashish Amarnath e391e43192 🐛 Supply command to run restic-wait init container
Signed-off-by: Ashish Amarnath <ashisham@vmware.com>
2020-08-11 11:05:30 -07:00
Ashish Amarnath 5d6da6517b
Implement restore hooks injecting init containers into pod spec (#2787)
*  Implement restore hooks injecting init containers into pod spec

Signed-off-by: Ashish Amarnath <ashisham@vmware.com>
2020-08-11 10:38:44 -07:00
Ashish Amarnath 9b9bb62968
🐛 Make init and exec restore hooks as omitempty in restore hookSpec (#2793)
Signed-off-by: Ashish Amarnath <ashisham@vmware.com>
2020-08-11 10:13:41 -07:00
Benoit Gagnon 0e8a7a23cb
always use groupResource.String() when logging (fixes #2795) (#2796)
Signed-off-by: Benoit Gagnon <benoit.gagnon@ubisoft.com>
2020-08-06 10:10:59 -07:00
Piper Dougherty 19e65689ef
Add the ability to set the allowPrivilegeEscalation property on the Restic restore helper via plugin ConfigMap (#2792)
* Add the ability to set the `allowPrivilegeEscalation` security context attribute on the Restic restore helper init container.

Signed-off-by: Piper Dougherty <doughertypiper@gmail.com>

* Add changelog.

Signed-off-by: Piper Dougherty <doughertypiper@gmail.com>

* Fix old tests and add tests for new allowPrivilegeEscalation config option.

Signed-off-by: Piper Dougherty <doughertypiper@gmail.com>

* Correct spelling in changelog.

Signed-off-by: Piper Dougherty <doughertypiper@gmail.com>

* Switch to boolptr type.

Signed-off-by: Piper Dougherty <doughertypiper@gmail.com>

* Reorder imports for sanity.

Signed-off-by: Piper Dougherty <doughertypiper@gmail.com>
2020-08-06 13:08:36 -04:00
JenTing Hsiao 1fdb647c7f
Add cacert flag for velero backup-location create (#2778)
Signed-off-by: JenTing Hsiao <jenting.hsiao@suse.com>
2020-07-30 11:33:45 -07:00
Ashish Amarnath 9de644f1a4
Add types to implement restore hooks (#2761)
* Add types to implement restore hooks

Signed-off-by: Ashish Amarnath <ashisham@vmware.com>
2020-07-29 13:29:40 -07:00
Ashish Amarnath 028818a053
exclude vols mounting secrets and configmaps from defaultVolumesToRestic (#2762)
Signed-off-by: Ashish Amarnath <ashisham@vmware.com>
2020-07-27 20:27:49 -07:00
Andrew Reed 9011b192e9
Add hooks fields to restore context (#2755)
* Add hooks fields to restore context

Signed-off-by: Andrew Reed <andrew@replicated.com>

* Changelog

Signed-off-by: Andrew Reed <andrew@replicated.com>
2020-07-24 11:43:44 -07:00
Nolan Brubaker bbcbde084d
Create hook package (#2734)
* Move pkg/backup/item_hook_handler to internal/hoo

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>

* Add internal packages to test target

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2020-07-22 14:26:14 -07:00
Ashish Amarnath 2636730ef2 fix copyright year
Signed-off-by: Ashish Amarnath <ashisham@vmware.com>
2020-07-22 12:11:59 -07:00
Martin Odstrčilík 86efd1577e
add support for setting SecurityContext (user, group) for restic restore (#2621)
* add support for setting SecurityContext (user, group) for restic restore

Signed-off-by: Martin Odstrcilik <martin.odstrcilik@gmail.com>
2020-07-22 12:10:25 -07:00
Ashish Amarnath 91ccc4adb2
Add metrics for restic back up operation (#2719)
* add metrics for restic back up operation

Signed-off-by: Ashish Amarnath <ashisham@vmware.com>

* changelog

Signed-off-by: Ashish Amarnath <ashisham@vmware.com>
2020-07-22 15:07:52 -04:00
Thejas Babu d0d143e119
Add StartTimestamp and CompletionTimestamp in Restore Status (#2748)
Signed-off-by: thejas <thejasb99@gmail.com>
2020-07-22 11:40:39 -07:00
Carlisia Campos e60d9f2821
Remove unneeded auto-generated files (#2732)
Signed-off-by: Carlisia <carlisia@vmware.com>
2020-07-20 15:37:27 -07:00
fvsqr 01e2dcb364
StorageGrid compatibility (#2712)
* remove explicit Accept-Encoding header

For StorageGrid compatibility the Accept-Encoding header should not be set, otherwise StorageGrid compresses the already compressed log files which are only decompressed by the client once

Signed-off-by: fvsqr <48791253+fvsqr@users.noreply.github.com>

* Removed explicit gzip Accept-Encoding header

For StorageGrid compatibility the Accept-Encoding header should not be set, otherwise StorageGrid compresses the already compressed log files which are only decompressed by the client once.
Unclear, how this affects Backup endpoints from Azure or GCP

Signed-off-by: fvsqr <48791253+fvsqr@users.noreply.github.com>

* Create 2712-fvsqr

Signed-off-by: fvsqr <48791253+fvsqr@users.noreply.github.com>
2020-07-20 13:11:26 -04:00
Nolan Brubaker f42c63af1b
Address insensitive language (#2677)
* Change master to main in most uses

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2020-07-17 14:59:51 -07:00
kathpeony 243ac62e3f
Add backupValidationFailureTotal to metrics (#2714)
* Add backValidationFailureTotal to metrics

Signed-off-by: Kathrin Mao <kathrin.mao@sap.com>
2020-07-16 10:13:17 -07:00
Andrew Lavery a368370bef
k8s 1.18 import (#2651)
* k8s 1.18 import wip

backup, cmd, controller, generated, restic, restore, serverstatusrequest, test and util

Signed-off-by: Andrew Lavery <laverya@umich.edu>

* go mod tidy

Signed-off-by: Andrew Lavery <laverya@umich.edu>

* add changelog file

Signed-off-by: Andrew Lavery <laverya@umich.edu>

* go fmt

Signed-off-by: Andrew Lavery <laverya@umich.edu>

* update code-generator and controller-gen in CI

Signed-off-by: Andrew Lavery <laverya@umich.edu>

* checkout proper code-generator version, regen

Signed-off-by: Andrew Lavery <laverya@umich.edu>

* fix remaining calls

Signed-off-by: Andrew Lavery <laverya@umich.edu>

* regenerate CRDs with ./hack/update-generated-crd-code.sh

Signed-off-by: Andrew Lavery <laverya@umich.edu>

* use existing context in restic and server

Signed-off-by: Andrew Lavery <laverya@umich.edu>

* fix test cases by resetting resource version

also use main library go context, not golang.org/x/net/context, in pkg/restore/restore.go

Signed-off-by: Andrew Lavery <laverya@umich.edu>

* clarify changelog message

Signed-off-by: Andrew Lavery <laverya@umich.edu>

* use github.com/kubernetes-csi/external-snapshotter/v2@v2.2.0-rc1

Signed-off-by: Andrew Lavery <laverya@umich.edu>

* run 'go mod tidy' to remove old external-snapshotter version

Signed-off-by: Andrew Lavery <laverya@umich.edu>
2020-07-16 12:21:37 -04:00
Carlisia Campos 13e1eeabfe
Return early from a BSL controller (#2709)
Signed-off-by: Carlisia <carlisia@vmware.com>
2020-07-14 15:09:11 -07:00
Carlisia Campos dbd0aa4915
Add a BSL controller to handle validation + update BSL status phase (#2674)
* Add BSL controller

Signed-off-by: Carlisia <carlisia@vmware.com>

* Add changelog

Signed-off-by: Carlisia <carlisia@vmware.com>

* Make update

Signed-off-by: Carlisia <carlisia@vmware.com>

* Update docs

Signed-off-by: Carlisia <carlisia@vmware.com>

* Add kubebuilder dependency

Signed-off-by: Carlisia <carlisia@vmware.com>

* Add export

Signed-off-by: Carlisia <carlisia@vmware.com>

* add kubebuilder binaries into velero builder image

Signed-off-by: Ashish Amarnath <ashisham@vmware.com>

* Reset velero dockerfile

Signed-off-by: Carlisia <carlisia@vmware.com>

* Consolidate all logic

Signed-off-by: Carlisia <carlisia@vmware.com>

* Add copyright header

Signed-off-by: Carlisia <carlisia@vmware.com>

* Clean up + add "last validated" column

Signed-off-by: Carlisia <carlisia@vmware.com>

* Better tests

Signed-off-by: Carlisia <carlisia@vmware.com>

* Add more tests

Signed-off-by: Carlisia <carlisia@vmware.com>

* Better logging

Signed-off-by: Carlisia <carlisia@vmware.com>

* Format

Signed-off-by: Carlisia <carlisia@vmware.com>

* Code reviews

Signed-off-by: Carlisia <carlisia@vmware.com>

* Address code review

Signed-off-by: Carlisia <carlisia@vmware.com>

* Remove redundant logic

Signed-off-by: Carlisia <carlisia@vmware.com>

Co-authored-by: Ashish Amarnath <ashisham@vmware.com>
2020-07-14 17:47:00 -04:00
Ashish Amarnath 3d3b9e312a
Revert "🐛 fix file perissions on the manifest JSON in backup archive (#2685)" (#2700)
dae5230aae

Signed-off-by: Ashish Amarnath <ashisham@vmware.com>
2020-07-10 14:07:05 -07:00
Nolan Brubaker 54aa75a4cd
Adjust restic timeout and pod values up (#2696)
* Adjust restic timeout and pod values up

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2020-07-08 14:59:19 -07:00
Ashish Amarnath dae5230aae
🐛 fix file perissions on the manifest JSON in backup archive (#2685)
Signed-off-by: Ashish Amarnath <ashisham@vmware.com>
2020-07-08 06:37:59 -07:00
Ashish Amarnath b4465e92b8
🐛 Use CRD version prior to remap_crd_version backup item action (#2683)
* 🐛 preserve crd version before remapping

Signed-off-by: Ashish Amarnath <ashisham@vmware.com>
2020-07-07 10:59:41 -07:00
Daniel Thrasher 94a9522f75
updated acceptable values on cron schedule for day of the week from 0-7 to 0-6 (#2676)
* updated acceptable values on cron schedule for day of the week from 0-7 to 0-6

Signed-off-by: Daniel Thrasher <dannythrasher@gmail.com>

* added a changelog file to changelog directory

Signed-off-by: Daniel Thrasher <dannythrasher@gmail.com>

Co-authored-by: Daniel Thrasher <dannythrasher@gmail.com>
2020-07-06 17:14:44 -04:00
Carlisia Campos 4048c020a8
Convert manifests + BSL api client to kubebuilder (#2561)
* kubebuilder init - minimalist version

Signed-off-by: Carlisia <carlisia@vmware.com>

* Add back main.go, apparently kb needs it

Signed-off-by: Carlisia <carlisia@vmware.com>

* Tweak makefile to accomodate kubebuilder expectations

Signed-off-by: Carlisia <carlisia@vmware.com>

* Port BSL to kubebuilder api client

Signed-off-by: Carlisia <carlisia@vmware.com>

* s/cache/client bc client fetches from cache
And other naming improvements

Signed-off-by: Carlisia <carlisia@vmware.com>

* So, .GetAPIReader is how we bypass the cache
In this case, the cache hasn't started yet

Signed-off-by: Carlisia <carlisia@vmware.com>

* Oh that's what this code was for... adding back

We still need to embed the CRDs as binary data in the Velero binary to
access the generated CRDs at runtime.

Signed-off-by: Carlisia <carlisia@vmware.com>

* Tie in CRD/code generation w/ existing scripts

Signed-off-by: Carlisia <carlisia@vmware.com>

* Mostly result of running update-fmt, updated file formatting

Signed-off-by: Carlisia <carlisia@vmware.com>

* Just a copyright fix

Signed-off-by: Carlisia <carlisia@vmware.com>

* All the test fixes

Signed-off-by: Carlisia <carlisia@vmware.com>

* Add changelog + some cleanup

Signed-off-by: Carlisia <carlisia@vmware.com>

* Update backup manifest

Signed-off-by: Carlisia <carlisia@vmware.com>

* Remove unneeded auto-generated files

Signed-off-by: Carlisia <carlisia@vmware.com>

* Keep everything in the same (existing) package

Signed-off-by: Carlisia <carlisia@vmware.com>

* Fix/clean scripts, generated code, and calls

Deleting the entire `generated` directory and running `make update`
works. Modifying an api and running `make verify` works as expected.

Signed-off-by: Carlisia <carlisia@vmware.com>

* Clean up schema and client calls + code reviews

Signed-off-by: Carlisia <carlisia@vmware.com>

* Move all code gen to inside builder container

Signed-off-by: Carlisia <carlisia@vmware.com>

* Address code review

Signed-off-by: Carlisia <carlisia@vmware.com>

* Fix imports/aliases

Signed-off-by: Carlisia <carlisia@vmware.com>

* More code reviews

Signed-off-by: Carlisia <carlisia@vmware.com>

* Add waitforcachesync

Signed-off-by: Carlisia <carlisia@vmware.com>

* Have manager register ALL controllers

This will allow for proper cache management.

Signed-off-by: Carlisia <carlisia@vmware.com>

* Status subresource is now enabled; cleanup

Signed-off-by: Carlisia <carlisia@vmware.com>

* More code reviews

Signed-off-by: Carlisia <carlisia@vmware.com>

* Clean up

Signed-off-by: Carlisia <carlisia@vmware.com>

* Manager registers ALL controllers for restic too

Signed-off-by: Carlisia <carlisia@vmware.com>

* More code reviews

Signed-off-by: Carlisia <carlisia@vmware.com>

* Add deprecation warning/todo

Signed-off-by: Carlisia <carlisia@vmware.com>

* Add documentation

Signed-off-by: Carlisia <carlisia@vmware.com>

* Add helpful comments

Signed-off-by: Carlisia <carlisia@vmware.com>

* Address code review

Signed-off-by: Carlisia <carlisia@vmware.com>

* More idiomatic Runnable

Signed-off-by: Carlisia <carlisia@vmware.com>

* Clean up imports

Signed-off-by: Carlisia <carlisia@vmware.com>
2020-06-24 12:55:18 -04:00
Carlisia Campos 0b8e2cbbe5
Improve velero download doc (#2660)
* Improve velero download doc

Signed-off-by: Carlisia <carlisia@vmware.com>

* Add changelog

Signed-off-by: Carlisia <carlisia@vmware.com>

* Better message

Signed-off-by: Carlisia <carlisia@vmware.com>
2020-06-23 11:12:01 -07:00
Ashish Amarnath e5e7c025dd fix copyright boilerplate
Signed-off-by: Ashish Amarnath <ashisham@vmware.com>
2020-06-18 14:14:15 -07:00
Ashish Amarnath 7abd2c6db9 doc updates
Signed-off-by: Ashish Amarnath <ashisham@vmware.com>
2020-06-18 13:57:59 -07:00
Ashish Amarnath 6a8dca6b84 add default-volumes-to-restic flag to velero installation
Signed-off-by: Ashish Amarnath <ashisham@vmware.com>
2020-06-16 14:46:36 -07:00
Ashish Amarnath 63f7690f44 more tests
Signed-off-by: Ashish Amarnath <ashisham@vmware.com>
2020-06-16 14:46:36 -07:00
Ashish Amarnath b0fd3d35c1 rename field
Signed-off-by: Ashish Amarnath <ashisham@vmware.com>
2020-06-15 15:26:44 -07:00
Ashish Amarnath b71173228a add velero server flag to allow default restic use on all pod volumes
Signed-off-by: Ashish Amarnath <ashisham@vmware.com>
2020-06-15 14:46:06 -07:00
Ashish Amarnath 8a2a852b87 use backup's defaultRestic flag to identify pod volumes using restic
Signed-off-by: Ashish Amarnath <ashisham@vmware.com>
2020-06-15 14:46:06 -07:00
Ashish Amarnath f34aab251e add default restic flag to backup create cli
Signed-off-by: Ashish Amarnath <ashisham@vmware.com>
2020-06-15 14:46:06 -07:00
Ashish Amarnath 69cceb0d7e add backup level flag to opt-in for default restic use
Signed-off-by: Ashish Amarnath <ashisham@vmware.com>
2020-06-15 14:46:06 -07:00
Steve Kriss a5346c1a87
azure: support aad-pod-identity auth when using restic (#2602)
Signed-off-by: Steve Kriss <krisss@vmware.com>
2020-06-08 11:17:43 -07:00
Nolan Brubaker d9d9dc60da
Log when a hook timeout duration can't be parsed (#2610)
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2020-06-05 12:45:50 -07:00
Steve Kriss 1c80ba903e
don't error during backup when additional items returned by plugin don't exist (#2595)
* log a warning instead of erroring if additional item can't be found

Signed-off-by: Steve Kriss <krisss@vmware.com>

* always show backup warning/error count in get/describe

Signed-off-by: Steve Kriss <krisss@vmware.com>

* changelog

Signed-off-by: Steve Kriss <krisss@vmware.com>
2020-06-04 16:29:55 -07:00
Christoph Blecker ceae7a2fdc
When creating backup from schedule, allow default name (#2569)
* When creating backup from schedule, allow default name

Signed-off-by: Christoph Blecker <admin@toph.ca>
2020-05-28 12:31:25 -07:00
Carlisia Campos 40d8511f64
Add status column to get BSL output (#2493)
* Add status column to get BSL output

Signed-off-by: Carlisia <carlisia@vmware.com>

* Address code review

Signed-off-by: Carlisia <carlisia@vmware.com>
2020-05-28 10:52:46 -07:00
Steve Kriss 5b52fd3efd
re-instantiate backup store just before persisting artifacts (#2550)
Signed-off-by: Steve Kriss <krisss@vmware.com>
2020-05-27 16:03:52 -07:00
Jonas Rosland d10eea3ba3
Add up and down votes to the issue templates and contributing guide (#2583)
Signed-off-by: jonasrosland <jrosland@vmware.com>
2020-05-27 14:09:07 -07:00
Nolan Brubaker 0effb5d74b
Merge pull request #2576 from skriss/fix-2562
bug fix: pass ca cert file to restic stats command on restore
2020-05-26 15:16:28 -04:00
Steve Kriss 98c1a34558 bug fix: pass ca cert file to restic stats command on restore
Signed-off-by: Steve Kriss <krisss@vmware.com>
2020-05-26 10:16:03 -06:00
Steve Kriss 92693ad422 update backup struct with progress updates
Signed-off-by: Steve Kriss <krisss@vmware.com>
2020-05-21 15:40:59 -06:00
Steve Kriss 3f6183b522 add missing kubebuilder tags
Signed-off-by: Steve Kriss <krisss@vmware.com>
2020-05-21 15:30:12 -06:00
Ashish Amarnath 532c6c5dc3
increment restic volumesnapshot count after successful pvb create (#2542)
Signed-off-by: Ashish Amarnath <ashisham@vmware.com>
2020-05-20 09:05:23 -07:00
Nolan Brubaker c1494d6804
Tweak VolumeSnapshotContent describe output (#2534)
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2020-05-12 13:58:47 -07:00
Nolan Brubaker e400be9c8f
Include CSI volume snapshot information in `velero backup describe` (#2448)
* Add download methods for CSI objects

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>

* Add support for downloading CSI volume objects

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>

* Add new methods to mock.

Remove generated information from file since mockery no longer appears
to work. It isn't maintained anymore and doesn't support go module-based
projects.

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>

* Add describe command for CSI

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>

* Add csi package with helpers

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>

* Remove duplicate import from server

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>

* Remove CSI API that will not be used with describe

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>

* Add VolumeSnapshotContents output to describe command

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>

* Document NewCSIListOptions function

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>

* Document csi package

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>

* Remove stutter in function name

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>

* Fix CI

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>

* Fix nil pointer error when not using CSI snapshots

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>

* Remove unused CSI download request kinds

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>

* Add back mocks

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>

* Change persistent volumes to velero-native snapshots

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>

* Remove unused function

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>

* Address review feedback

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>

* Add changelog

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>

* Remove unnecessary doc.go

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2020-05-08 12:42:30 -07:00
Nolan Brubaker e91dfe3d91
Remove comment that has been turned into an issue (#2500)
Logged comment as issue #2499

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2020-05-08 09:27:01 -07:00
Nolan Brubaker de8962ea18
Bug fix: Calculate label using backup name for CSI objects (#2510)
* Use a helper function when querying w/ backup label

Setting or querying for a backup label name should always pass the value
through the GetValidName function. This change passes query uses of the
backup label value through the GetValidName function by introducing 2
new helpers, one for making a Selector, one for making a ListOptions.

It also removes functions returning the same data, but under
unecessarily specific names.

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>

* Document using the label.GetValidName function

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>

* Update copyright year

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>

* Clarify labels.GetValidName and annotations

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>

* Move functions to pkg/label

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>

* Fix function comments

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2020-05-07 11:56:13 -07:00
Nolan Brubaker abae81ddc8
Merge pull request #2440 from skriss/backup-progress
report backup progress
2020-05-07 14:51:20 -04:00
Ashish Amarnath c49afd44bd
Use label selector to list VSCs to persist (#2502)
Signed-off-by: Ashish Amarnath <ashisham@vmware.com>
2020-05-06 11:04:34 -07:00
Ashish Amarnath 911a607cf8
Allow feature flags to be passed from install CLI (#2503)
* allow feature flags to be passed from install CLI

Signed-off-by: Ashish Amarnath <ashisham@vmware.com>
2020-05-06 10:52:44 -07:00
Ashish Amarnath 577e87d1b8
Sync CSI API objects during backup sync (#2496)
* add changelog

Signed-off-by: Ashish Amarnath <ashisham@vmware.com>

* Sync CSI API volumesnapshotcontents during backup sync

Signed-off-by: Ashish Amarnath <ashisham@vmware.com>
2020-05-06 08:00:05 -06:00
Nolan Brubaker 8671a639c9
Backup entire v1beta1 CRD instead of just changing version string (#2478)
* Switch to backing up v1beta1 CRDs from API server

Instead of simply switching out the APIVersion string on a v1
CustomResourceDefinition object, re-download the object from the API
server entirely to get the correct fields.

This should fix validation errors upon restore.

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>

* Fix existing tests

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>

* Add full example CRDs to automated tests

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>

* Move beta CRD lookup into helper function

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>

* Add case for preserveUnknownFields CRDs

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>

* Add PreserveUnknownFields case and refactor execute

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>

* Add older prometheus CRD test cases

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>

* Add changelog

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2020-05-05 13:21:28 -07:00
Ashish Amarnath 6b5a084f32
Delete dangling volumesnapshotcontents from velero backups (#2480)
* Delete dangling volumesnapshotcontents from velero backups

Signed-off-by: Ashish Amarnath <ashisham@vmware.com>

* add changelog

Signed-off-by: Ashish Amarnath <ashisham@vmware.com>

* handle not found errors from VSC delete

Signed-off-by: Ashish Amarnath <ashisham@vmware.com>

* clean up unit tests

Signed-off-by: Ashish Amarnath <ashisham@vmware.com>
2020-05-04 15:41:27 -04:00
Steve Kriss 4b0f654a1e add progress info to backup status
Signed-off-by: Steve Kriss <krisss@vmware.com>
2020-05-01 15:30:50 -06:00
Steve Kriss 30ca0e4322 split out collecting items from backing up items
Signed-off-by: Steve Kriss <krisss@vmware.com>
2020-05-01 15:30:42 -06:00
Steve Kriss 19c52434b4 simplify group/resource/item backupper structure
Signed-off-by: Steve Kriss <krisss@vmware.com>
2020-05-01 15:29:01 -06:00
Nolan Brubaker 577af5a5b1
Merge pull request #2487 from skriss/fix-error-logging
don't return an error from log hook if value is not an error type
2020-05-01 16:11:16 -04:00
Nolan Brubaker 1ed5a13012
Merge pull request #2484 from skriss/fix-2319
bug fix: fix int/float conversion issue with CRD type in restore plugin
2020-05-01 16:09:57 -04:00
Ashish Amarnath bf52caf658
Delete CSI volumesnapshots created as part of a backup in its deletion (#2411)
* delete CSI volumesnapshots in backup on backup delete

Signed-off-by: Ashish Amarnath <ashisham@vmware.com>

* set vsc deletionpolicy to delete prior to vs deletion

Signed-off-by: Ashish Amarnath <ashisham@vmware.com>

* add more unit tests

Signed-off-by: Ashish Amarnath <ashisham@vmware.com>

* add change log

Signed-off-by: Ashish Amarnath <ashisham@vmware.com>

* passes build need to pass unit tests

Signed-off-by: Ashish Amarnath <ashisham@vmware.com>

* remove GetNamespacesInBackup

Signed-off-by: Ashish Amarnath <ashisham@vmware.com>

* fix unit tests

Signed-off-by: Ashish Amarnath <ashisham@vmware.com>

* check feature flag instead of nil check

Signed-off-by: Ashish Amarnath <ashisham@vmware.com>

* cleanup

Signed-off-by: Ashish Amarnath <ashisham@vmware.com>
2020-05-01 16:05:20 -04:00
Rafael Brito 0d97f9400e
Backup all groups and versions with backward compatibility (#2373)
* Backup all API Groups versions while keeping backward compatibility

Signed-off-by: Rafael Brito <rbrito@vmware.com>

* Backup all API Groups versions while keeping backward compatibility

Signed-off-by: Rafael Brito <rbrito@vmware.com>

* Adding feature flag to enable backup of multiple API group versions

Signed-off-by: Rafael Brito <rbrito@vmware.com>
2020-05-01 15:54:57 -04:00
Martin Odstrčilík 26f67f1b10
add support for getting Azure Storage Account key from ENV variable (#2455)
Signed-off-by: Martin Odstrcilik <martin.odstrcilik@gmail.com>
2020-05-01 07:55:50 -07:00
Steve Kriss 5cc6c12eb4 don't return an error from log hook if value is not an error type
Signed-off-by: Steve Kriss <krisss@vmware.com>
2020-04-30 15:30:22 -06:00
Steve Kriss e148ddad8f
Add backwards-compatibility for flags passed to plugins (#2479)
* update plugin server to ignore unknown flags during parse

Signed-off-by: Steve Kriss <krisss@vmware.com>
2020-04-30 13:19:55 -07:00
Steve Kriss 2d55cd2ed8 bug fix: fix int/float conversion issue with CRD type in restore plugin
Signed-off-by: Steve Kriss <krisss@vmware.com>
2020-04-30 09:06:03 -06:00
Steve Kriss 15b2a1c9c6
bug fix: don't remove unresolvable includes from includes-excludes lists (#2462)
* bug fix: don't remove unresolvable includes from includes-excludes lists

Signed-off-by: Steve Kriss <krisss@vmware.com>

* changelog

Signed-off-by: Steve Kriss <krisss@vmware.com>
2020-04-27 17:17:39 -04:00
FNU Raghavendra Manjunath 98d2fc732e
clarify the wording for restore describe for namespaces included (#2449)
* clarify the wording for restore describe for namespaces included

Instead of showing it as "*" explicitly mention that all the namespaces
from the backup object are included.

refer to https://github.com/vmware-tanzu/velero/issues/1918

Signed-off-by: Raghavendra M <raghavendra@redhat.com>

* Update pkg/cmd/util/output/restore_describer.go

Co-Authored-By: Ashish Amarnath <ashisham@vmware.com>
Signed-off-by: Raghavendra M <raghavendra@redhat.com>

* Update pkg/cmd/util/output/restore_describer.go

Co-Authored-By: Ashish Amarnath <ashisham@vmware.com>
Signed-off-by: Raghavendra M <raghavendra@redhat.com>

Co-authored-by: Ashish Amarnath <ashisham@vmware.com>
2020-04-24 14:44:20 -07:00
Mayank 962f5d6859
Skipping validation for volumesnapshotlocation for backup if snapshotvolume set to false (#2450)
* Disabling validation for volumesnapshotlocation if the backup has snapshotvolume set to false

Signed-off-by: mayank <mayank.patel@mayadata.io>

* adding a changelog

Signed-off-by: mayank <mayank.patel@mayadata.io>

* addressing review comment

Signed-off-by: mayank <mayank.patel@mayadata.io>
2020-04-24 09:46:20 -07:00
Nolan Brubaker aa3abefdbf Address review feedback
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2020-04-20 13:49:18 -04:00
Nolan Brubaker 16cf2780d8 Fetch contents based on volumesnapshots found
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2020-04-20 13:49:18 -04:00
Nolan Brubaker bc04c568cd Return list of errors for encoding/zipping
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2020-04-20 13:49:18 -04:00
Nolan Brubaker eefd12b3e4 Rename CSI variables to be more descriptive
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2020-04-20 13:49:18 -04:00
Nolan Brubaker b567859655 Add data description for encoding JSON in case of error
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2020-04-20 13:49:18 -04:00
Nolan Brubaker d31951c81c Add CSI feature flag constant
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2020-04-20 13:49:18 -04:00
Nolan Brubaker 15d4c11305 Wrap CSI lister access in a nil check
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2020-04-20 13:49:18 -04:00
Nolan Brubaker b4e18b489c Clean up object contents when other files fail
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2020-04-20 13:49:18 -04:00
Nolan Brubaker ad18318bb1 Rename snapshotter factory variable, return nil in wrapper
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2020-04-20 13:49:18 -04:00
Nolan Brubaker 6c3fc8c780 Addresss simple review feedback
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2020-04-20 13:49:18 -04:00
Nolan Brubaker 9372eaa31b Reduce duplication in object serialization
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2020-04-20 13:49:18 -04:00
Nolan Brubaker 850554911f Reduce duplication in object storage uploads
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2020-04-20 13:49:18 -04:00
Nolan Brubaker 9c1a2e884b Update comments
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2020-04-20 13:49:18 -04:00
Nolan Brubaker 5202885349 Remove unused CSI client variable
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2020-04-20 13:49:18 -04:00
Nolan Brubaker 9719e4de9d Don't defer cancelFunc, since it causes issues
Infomers won't start if cancelFunc is invoked as soon as the newServer
function exits via the defer

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2020-04-20 13:49:18 -04:00
Nolan Brubaker b7142465fc Wrap usage of the CSI informer in flag checks
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2020-04-20 13:49:18 -04:00
Nolan Brubaker c71318bf19 Address more review feedback
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2020-04-20 13:49:17 -04:00
Nolan Brubaker 5bde12939e Address review feedback on object store
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2020-04-20 13:49:17 -04:00
Nolan Brubaker 5de15c450b Address some review feedback on server
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2020-04-20 13:49:17 -04:00
Nolan Brubaker aff529e5d5 Upload CSI volumesnapshots associated with backup
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2020-04-20 13:49:17 -04:00
Nolan Brubaker 6a7beaf5ce Plumb CSI listers through to backup controller
Account for having CSI enabled or not, as well as having the snapshot
CRDs installed in the kubernetes cluster.

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2020-04-20 13:49:17 -04:00
Nolan Brubaker 232e1d8927 Add clients for CSI snapshots
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2020-04-20 13:49:17 -04:00
Nolan Brubaker b4be7eccb9 Document locations for CSI support
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2020-04-20 13:49:17 -04:00