Commit Graph

173 Commits (368a1ddf3c3c8f0e27fd9424024b55937dc8313e)

Author SHA1 Message Date
Xun Jiang 368a1ddf3c Add CSI VolumeSnapshot related metrics.
Signed-off-by: Xun Jiang <jxun@vmware.com>
2022-04-09 00:57:28 +08:00
Hoang, Phuong 0171a91366 Correct copyright comment and string compare
Signed-off-by: Hoang, Phuong <phuong.n.hoang@dell.com>
2022-03-21 12:01:26 -04:00
Hoang, Phuong e9d5f7121c Add unit tests, change copyright years and revert unrelated changes.
Signed-off-by: Hoang, Phuong <phuong.n.hoang@dell.com>
2022-03-15 19:51:53 -04:00
Hoang, Phuong 0741360050 Adding PriorityClass to backup podAction and restore podAction.
Signed-off-by: Hoang, Phuong <phuong.n.hoang@dell.com>
2022-03-15 19:05:13 -04:00
Wenkai Yin(尹文开) 8aaeed4411
Merge pull request #4686 from reasonerjt/fix-4080
Bypass the remap CRD version plugin when v1beta1 CRD is not supported
2022-02-28 10:57:52 +08:00
Daniel Jiang 0070138c62 Bypass the remap CRD version plugin when v1beta1 CRD is not supported
When velero is running on clusters that don't support v1beta1 CRD, the
plugin will not try to backup v1beta1 CRD.
The plugin should be kept for backward compatibility.  It will be
removed when velero drop the support for k8s v1.21

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2022-02-22 18:59:47 +08:00
Xun Jiang 4c8d1c2693 Modify according to comments
1. rename zoneSeparator to gkeZoneSeparator
2. add example of regional PV's node affinity. modify test case description.

Signed-off-by: Xun Jiang <jxun@vmware.com>
2022-02-22 16:23:10 +08:00
Xun Jiang 7be12a9220 [fix] Add regional PV support for GKE
fix #4663.
For GKE pv, when create backup, return all zones retrived from node affinity.

Signed-off-by: Xun Jiang <jxun@vmware.com>
2022-02-21 22:04:21 +08:00
F. Gold 491942cf2d
Simplify by assuming nil is false and use boolptr util function
Signed-off-by: F. Gold <fgold@vmware.com>
2022-02-09 16:58:07 -08:00
F. Gold 95c43d5b65
Minor change to trigger GitHub actions
Signed-off-by: F. Gold <fgold@vmware.com>
2022-02-02 17:02:22 -08:00
F. Gold 131c6de407
Check for nil before logging DefaultVolumestToRestic value
Signed-off-by: F. Gold <fgold@vmware.com>
2022-02-02 10:53:03 -08:00
David L. Smith-Uchida 5677e04bb1
Consolidated code for resolving actions and plugins into ActionResolver (#4410)
* Consolidated code for resolving actions and plugins into ActionResolver.  Added BackupWithResolvers and
RestoreWithResolvers.  Introduces ItemSnapshooterResolver to bring ItemSnapshotter plugins into backup and
restore.  ItemSnapshotters are not used yet.

Added action_resolver_test

Signed-off-by: Dave Smith-Uchida <dsmithuchida@vmware.com>

* Addressed review comments

Signed-off-by: Dave Smith-Uchida <dsmithuchida@vmware.com>
2021-12-10 12:53:47 -05:00
Daniel Jiang 748cf86aa7 Add the key for GKE zone for AZ collection
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2021-11-20 15:58:31 +08:00
Daniel Jiang 5878a52843 Read Availability zone from nodeAffinity requirements
Velero to read the AZ info from `NodeAffinity` of a PV when it's taking
the snapshot.

Fixes #4332

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2021-11-16 09:44:12 +08:00
Wenkai Yin(尹文开) 6801ddc9ac
Merge pull request #4306 from alaypatel07/fix-paging
fix buggy pager func
2021-11-10 07:53:58 +08:00
Alay Patel b2fe7fe304 - fix buggy pager func
fix paging items in to use list options passed by the paging function

The client-go pager sets the Limit options for the list call
to paginate the request[1]. This PR fixes the paging function
to use the options passed by the pager instead of shadowed options
This is required for the pagination to work correctly.

- simplify the pager list implementation by using pager.List()
The List() function already implements a lot of the logic that was
needed for paging here, using it simplifies the code.

1. 3f40906dd8/staging/src/k8s.io/client-go/tools/pager/pager.go (L219)

Signed-off-by: Alay Patel <alay1431@gmail.com>
2021-11-09 10:34:26 -05:00
Scott Seago 550efddd88
Verify group before treating resource as cohabitating (#4126)
Signed-off-by: Scott Seago <sseago@redhat.com>
2021-11-03 18:11:32 -04:00
Dharma Bellamkonda dc1f17944e
Page list requests by default (#3823)
Signed-off-by: Dharma Bellamkonda <bellamko@adobe.com>
2021-08-17 18:49:41 -07:00
Bridget McErlean d98c65f69e
Use appropriate CRD API during readiness check (#4015)
* Use appropriate CRD API during readiness check

The readiness check for the Velero CRDs was still using the v1beta1 API.
This would cause the readiness check to fail on 1.22 clusters as the
v1beta1 API is no longer available. Previously, this error would be
ignored and the installation would proceed, however with #4002, we are
no longer ignoring errors from this check.

This change modifies the CRD readiness check to check the CRDs using the
same API version that was used when submitting the CRDs to the cluster.
It also introduces a new CRD builder using the V1 API for testing.

This change also fixes a bug that was identified in the polling code
where if the CRDs were not ready on the first polling iteration, they
would be added again to the list of CRDs to check resulting in
duplicates. This would cause the length check to fail on all subsequent
polls and the timeout would always be reached.

Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>

* Remove duplicate V1 CRD builder and update comment

Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>
2021-08-05 15:17:14 -04:00
Ashish Amarnath ecab583680
🐛 Do not run ItemAction plugins for unresolvable types for all types (#3059)
Signed-off-by: Ashish Amarnath <ashisham@vmware.com>
2020-11-11 09:50:57 -05:00
Mateusz Gozdek dbc83af77b
Fix various typos found by codespell (#3057)
By running the following command:

codespell -S .git,*.png,*.jpg,*.woff,*.ttf,*.gif,*.ico -L \
iam,aks,ist,bridget,ue

Signed-off-by: Mateusz Gozdek <mgozdekof@gmail.com>
2020-11-10 11:48:35 -05:00
Scott Seago d820bc5e72
restore proper lowercase/plural CRD resource (#2949)
* restore proper lowercase/plural CRD resource

This commit restores the proper resource string
"customresourcedefinitions" for CRD. The prior change to
"CustomResourceDefinition" was made because this was being used
in another place to populate the CRD "Kind" field in
remap_crd_version_action.go -- there, just use the correct Kind
string instead of pulling from Resource.

Signed-off-by: Scott Seago <sseago@redhat.com>

* add changelog

Signed-off-by: Scott Seago <sseago@redhat.com>
2020-10-02 09:48:12 -04:00
Phuong N. Hoang 14170b52a8
Enhance Backup to backup resources in specific order. (#2724)
Signed-off-by: Phuong Hoang <phuong.n.hoang@dell.com>

Co-authored-by: Phuong Hoang <phuong.n.hoang@dell.com>
2020-08-12 17:17:31 -07:00
Ashish Amarnath 5d6da6517b
Implement restore hooks injecting init containers into pod spec (#2787)
*  Implement restore hooks injecting init containers into pod spec

Signed-off-by: Ashish Amarnath <ashisham@vmware.com>
2020-08-11 10:38:44 -07:00
Benoit Gagnon 0e8a7a23cb
always use groupResource.String() when logging (fixes #2795) (#2796)
Signed-off-by: Benoit Gagnon <benoit.gagnon@ubisoft.com>
2020-08-06 10:10:59 -07:00
Nolan Brubaker bbcbde084d
Create hook package (#2734)
* Move pkg/backup/item_hook_handler to internal/hoo

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>

* Add internal packages to test target

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2020-07-22 14:26:14 -07:00
Andrew Lavery a368370bef
k8s 1.18 import (#2651)
* k8s 1.18 import wip

backup, cmd, controller, generated, restic, restore, serverstatusrequest, test and util

Signed-off-by: Andrew Lavery <laverya@umich.edu>

* go mod tidy

Signed-off-by: Andrew Lavery <laverya@umich.edu>

* add changelog file

Signed-off-by: Andrew Lavery <laverya@umich.edu>

* go fmt

Signed-off-by: Andrew Lavery <laverya@umich.edu>

* update code-generator and controller-gen in CI

Signed-off-by: Andrew Lavery <laverya@umich.edu>

* checkout proper code-generator version, regen

Signed-off-by: Andrew Lavery <laverya@umich.edu>

* fix remaining calls

Signed-off-by: Andrew Lavery <laverya@umich.edu>

* regenerate CRDs with ./hack/update-generated-crd-code.sh

Signed-off-by: Andrew Lavery <laverya@umich.edu>

* use existing context in restic and server

Signed-off-by: Andrew Lavery <laverya@umich.edu>

* fix test cases by resetting resource version

also use main library go context, not golang.org/x/net/context, in pkg/restore/restore.go

Signed-off-by: Andrew Lavery <laverya@umich.edu>

* clarify changelog message

Signed-off-by: Andrew Lavery <laverya@umich.edu>

* use github.com/kubernetes-csi/external-snapshotter/v2@v2.2.0-rc1

Signed-off-by: Andrew Lavery <laverya@umich.edu>

* run 'go mod tidy' to remove old external-snapshotter version

Signed-off-by: Andrew Lavery <laverya@umich.edu>
2020-07-16 12:21:37 -04:00
Ashish Amarnath 3d3b9e312a
Revert "🐛 fix file perissions on the manifest JSON in backup archive (#2685)" (#2700)
dae5230aae

Signed-off-by: Ashish Amarnath <ashisham@vmware.com>
2020-07-10 14:07:05 -07:00
Ashish Amarnath dae5230aae
🐛 fix file perissions on the manifest JSON in backup archive (#2685)
Signed-off-by: Ashish Amarnath <ashisham@vmware.com>
2020-07-08 06:37:59 -07:00
Ashish Amarnath b4465e92b8
🐛 Use CRD version prior to remap_crd_version backup item action (#2683)
* 🐛 preserve crd version before remapping

Signed-off-by: Ashish Amarnath <ashisham@vmware.com>
2020-07-07 10:59:41 -07:00
Ashish Amarnath e5e7c025dd fix copyright boilerplate
Signed-off-by: Ashish Amarnath <ashisham@vmware.com>
2020-06-18 14:14:15 -07:00
Ashish Amarnath 63f7690f44 more tests
Signed-off-by: Ashish Amarnath <ashisham@vmware.com>
2020-06-16 14:46:36 -07:00
Ashish Amarnath b0fd3d35c1 rename field
Signed-off-by: Ashish Amarnath <ashisham@vmware.com>
2020-06-15 15:26:44 -07:00
Ashish Amarnath b71173228a add velero server flag to allow default restic use on all pod volumes
Signed-off-by: Ashish Amarnath <ashisham@vmware.com>
2020-06-15 14:46:06 -07:00
Ashish Amarnath 8a2a852b87 use backup's defaultRestic flag to identify pod volumes using restic
Signed-off-by: Ashish Amarnath <ashisham@vmware.com>
2020-06-15 14:46:06 -07:00
Nolan Brubaker d9d9dc60da
Log when a hook timeout duration can't be parsed (#2610)
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2020-06-05 12:45:50 -07:00
Steve Kriss 1c80ba903e
don't error during backup when additional items returned by plugin don't exist (#2595)
* log a warning instead of erroring if additional item can't be found

Signed-off-by: Steve Kriss <krisss@vmware.com>

* always show backup warning/error count in get/describe

Signed-off-by: Steve Kriss <krisss@vmware.com>

* changelog

Signed-off-by: Steve Kriss <krisss@vmware.com>
2020-06-04 16:29:55 -07:00
Steve Kriss 92693ad422 update backup struct with progress updates
Signed-off-by: Steve Kriss <krisss@vmware.com>
2020-05-21 15:40:59 -06:00
Nolan Brubaker abae81ddc8
Merge pull request #2440 from skriss/backup-progress
report backup progress
2020-05-07 14:51:20 -04:00
Nolan Brubaker 8671a639c9
Backup entire v1beta1 CRD instead of just changing version string (#2478)
* Switch to backing up v1beta1 CRDs from API server

Instead of simply switching out the APIVersion string on a v1
CustomResourceDefinition object, re-download the object from the API
server entirely to get the correct fields.

This should fix validation errors upon restore.

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>

* Fix existing tests

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>

* Add full example CRDs to automated tests

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>

* Move beta CRD lookup into helper function

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>

* Add case for preserveUnknownFields CRDs

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>

* Add PreserveUnknownFields case and refactor execute

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>

* Add older prometheus CRD test cases

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>

* Add changelog

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2020-05-05 13:21:28 -07:00
Steve Kriss 4b0f654a1e add progress info to backup status
Signed-off-by: Steve Kriss <krisss@vmware.com>
2020-05-01 15:30:50 -06:00
Steve Kriss 30ca0e4322 split out collecting items from backing up items
Signed-off-by: Steve Kriss <krisss@vmware.com>
2020-05-01 15:30:42 -06:00
Steve Kriss 19c52434b4 simplify group/resource/item backupper structure
Signed-off-by: Steve Kriss <krisss@vmware.com>
2020-05-01 15:29:01 -06:00
Rafael Brito 0d97f9400e
Backup all groups and versions with backward compatibility (#2373)
* Backup all API Groups versions while keeping backward compatibility

Signed-off-by: Rafael Brito <rbrito@vmware.com>

* Backup all API Groups versions while keeping backward compatibility

Signed-off-by: Rafael Brito <rbrito@vmware.com>

* Adding feature flag to enable backup of multiple API group versions

Signed-off-by: Rafael Brito <rbrito@vmware.com>
2020-05-01 15:54:57 -04:00
Steve Kriss 15b2a1c9c6
bug fix: don't remove unresolvable includes from includes-excludes lists (#2462)
* bug fix: don't remove unresolvable includes from includes-excludes lists

Signed-off-by: Steve Kriss <krisss@vmware.com>

* changelog

Signed-off-by: Steve Kriss <krisss@vmware.com>
2020-04-27 17:17:39 -04:00
Mayank 962f5d6859
Skipping validation for volumesnapshotlocation for backup if snapshotvolume set to false (#2450)
* Disabling validation for volumesnapshotlocation if the backup has snapshotvolume set to false

Signed-off-by: mayank <mayank.patel@mayadata.io>

* adding a changelog

Signed-off-by: mayank <mayank.patel@mayadata.io>

* addressing review comment

Signed-off-by: mayank <mayank.patel@mayadata.io>
2020-04-24 09:46:20 -07:00
Steve Kriss ca58d7ab90 change dynamic client's List return type to UnstructuredList
Signed-off-by: Steve Kriss <krisss@vmware.com>
2020-04-15 12:41:47 -06:00
Steve Kriss 1826b6b67b
bug fix: populate namespace in logs for backup errors (#2438)
Signed-off-by: Steve Kriss <krisss@vmware.com>
2020-04-15 10:31:40 -07:00
Nolan Brubaker 040f6806f1
Fix int/float conversion issues in CRD version remapping plugin (#2322)
* Add builders for CRD schemas

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>

* Add test case for #2319

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>

* Add failing test case

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>

* Remove unnecessary print and temporary variable

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>

* Add some options for fixing the test case

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>

* Switch to a JSON middle step to "fix" conversions

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>

* Add comment and changelog

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2020-03-10 09:24:22 -06:00
Steve Kriss 9e19c1d3bd fix goimport noise in prep for bump to latest
Signed-off-by: Steve Kriss <krisss@vmware.com>
2020-03-02 10:55:08 -07:00