Commit Graph

138 Commits (705a3bc355781896687e8d655144d72fc9d435fb)

Author SHA1 Message Date
Scott Seago 7750e12151 Perf improvements for existing resource restore
Use informer cache with dynamic client for Get calls on restore
When enabled, also make the Get call before create.

Add server and install parameter to allow disabling this feature,
but enable by default

Signed-off-by: Scott Seago <sseago@redhat.com>
2023-10-11 10:51:39 -04:00
Xun Jiang ec4bb42117 Retrieve DataUpload into backup result ConfigMap during volume snapshot restore.
Fix issue #6117.
Add CSI plugin needs builder functions.

Signed-off-by: Xun Jiang <jxun@vmware.com>
2023-06-30 11:13:04 +08:00
Xun Jiang/Bruce Jiang e106bbf06b
Enable linter revive and resolve found errors: part 2. (#6177)
Signed-off-by: Xun Jiang <blackpiglet@gmail.com>
Co-authored-by: Xun Jiang <blackpiglet@gmail.com>
2023-04-26 14:05:22 +08:00
Xun Jiang bbc1e2e151 Enable stylecheck linter and resolve found issues.
Signed-off-by: Xun Jiang <blackpiglet@gmail.com>
2023-04-25 14:37:02 +08:00
Scott Seago 433d2d5e57 BIA/RIAv2 -- distinguish between New and InProgress operations
Signed-off-by: Scott Seago <sseago@redhat.com>
2023-03-21 18:55:59 -04:00
Scott Seago 2155b2b215 RIAv2 async operations controller work
Signed-off-by: Scott Seago <sseago@redhat.com>
2023-03-17 14:30:39 -04:00
Scott Seago a2621caa74 Wait for additional items to be ready before restoring current item
Signed-off-by: Scott Seago <sseago@redhat.com>
2023-03-13 15:26:45 -04:00
Wenkai Yin(尹文开) eb4ecd3767 Add restored resource list in the output of restore describe command
Add restored resource list in the output of restore describe command

Fixes #5199

Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
2023-02-28 10:43:03 +08:00
Wenkai Yin(尹文开) 4d0c3ac83f
Restore finalizer and managedFields (#5853)
Restore finalizer and managedFields of metadata during the restoration

Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
2023-02-20 10:26:45 +08:00
Scott Seago fa58a775e8
Merge pull request #5576 from anshulahuja98/backupresults
Publish backup results to enhance error info
2023-02-18 17:00:49 -05:00
Anshul Ahuja 8c3ddf0f73 Publish backup results extracted from backup logs
Signed-off-by: Anshul Ahuja <anshulahuja@microsoft.com>
2023-02-16 09:57:53 +05:30
Xun Jiang/Bruce Jiang 0b6b841f2a
Remove redundant client from restore controller. (#5759)
Signed-off-by: Xun Jiang <blackpiglet@gmail.com>
2023-02-03 15:57:43 +08:00
Scott Seago 9f6f13f0c5 RestoreItemAction v2 API implementation
Signed-off-by: Scott Seago <sseago@redhat.com>
2023-01-10 16:26:28 -05:00
Wenkai Yin(尹文开) c186a7d193 Enhance the restore priorities list to support specifying the low prioritized resources that need to be restored in the last
Enhance the restore priorities list to support specifying the low prioritized resources that need to be r
estored in the last

Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
2022-11-03 10:53:32 +08:00
Kira Boyle b146a880c6 update k8s.io dependencies to 0.24.0
* This also required an update to use github.com/bombsimon/logrusr/v3
* 'WithClusterName' removed as per the k8s doc reasoning:
* https://github.com/kubernetes/apimachinery/blob/release-1.24/pkg/apis/meta/v1/types.go\#L257-L259
* ('ClusterName was a legacy field that was always cleared by the system and never used')
* Test was updated accordingly

Signed-off-by: Kira Boyle <kira@replicated.com>
2022-10-21 11:16:21 +08:00
Xun Jiang/Bruce Jiang d658f6564d
Merge pull request #5444 from Lyndon-Li/remove-restic
Pod Volume Backup/Restore Refactor: Remove Restic in code
2022-10-17 15:32:17 +08:00
Lyndon-Li 5f7f69366c remove restic from code
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2022-10-13 13:45:32 +08:00
Scott Seago 45de8a782f moved RIA execute input/output structs back to velero package
The RIA refactoring moved velero.RestoreItemAction into a separate
(restoreitemaction) v1 package. Unfortunately, this change would require
plugins to make code changes to locate the RestoreItemActionExecuteInput
and RestoreItemActionExecuteOutput structs.

This commit restores those structs to the original velero package, leaving
just the RestoreItemAction interface in the new v1 package.

Signed-off-by: Scott Seago <sseago@redhat.com>
2022-10-12 13:17:47 -04:00
Scott Seago b6088356e6 plugin versioning v1 refactor for VolumeSnapshotter
Signed-off-by: Scott Seago <sseago@redhat.com>
2022-09-21 13:47:23 -04:00
Scott Seago 32ef20d317 RestoreItemAction v1 refactoring for plugin api versioning
Signed-off-by: Scott Seago <sseago@redhat.com>
2022-09-20 17:43:56 -04:00
allenxu404 dc70471909 Change the status of restore to completed from partially failed when restore empty backup
Signed-off-by: allenxu404 <qix2@vmware.com>
2022-09-08 18:40:49 +08:00
Wenkai Yin(尹文开) 092fc01e8d
Splic pkg/restic package (#5143)
This commit splits the pkg/restic package into several packages to support Kopia integration works

Fixes #5055

Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
2022-08-05 17:15:38 +08:00
Rafael Leal e0e3016efa
Cleanup resetStatus
Signed-off-by: Rafael Leal <rafaelealdias@gmail.com>
2022-05-23 17:53:32 -03:00
Rafael Leal 04aa7a849f
Fixes tests hopefully
Signed-off-by: Rafael Leal <rafaelealdias@gmail.com>
2022-05-12 11:59:05 -03:00
Shubham Pampattiwar bfdb68a35a add multiple label selector support to backup API
Signed-off-by: Shubham Pampattiwar <shubhampampattiwar7@gmail.com>

remove backup CLI bits

Signed-off-by: Shubham Pampattiwar <shubhampampattiwar7@gmail.com>

labelselectors spec option for velero restore

Signed-off-by: Shubham Pampattiwar <shubhampampattiwar7@gmail.com>

add changelog file

Signed-off-by: Shubham Pampattiwar <shubhampampattiwar7@gmail.com>

update spec name to OrLabelSelectors

Signed-off-by: Shubham Pampattiwar <shubhampampattiwar7@gmail.com>

minor fixes

Signed-off-by: Shubham Pampattiwar <shubhampampattiwar7@gmail.com>

add validations for labelSelector and orLabelSelectors

Signed-off-by: Shubham Pampattiwar <shubhampampattiwar7@gmail.com>

update crds.gp after fixing conflicts

Signed-off-by: Shubham Pampattiwar <shubhampampattiwar7@gmail.com>

fix CI and add unit tests

Signed-off-by: Shubham Pampattiwar <shubhampampattiwar7@gmail.com>

updated OrLabelSelector spec description and added validation failure unit tests

Signed-off-by: Shubham Pampattiwar <shubhampampattiwar7@gmail.com>

add comments and change log level

Signed-off-by: Shubham Pampattiwar <shubhampampattiwar7@gmail.com>

update site docs

Signed-off-by: Shubham Pampattiwar <shubhampampattiwar7@gmail.com>

wrap pager client calls in a function

Signed-off-by: Shubham Pampattiwar <shubhampampattiwar7@gmail.com>

resolve confilcts and update crds.go

Signed-off-by: Shubham Pampattiwar <shubhampampattiwar7@gmail.com>

rebase and update crds.go

Signed-off-by: Shubham Pampattiwar <shubhampampattiwar7@gmail.com>

combine listing items for a given label

Signed-off-by: Shubham Pampattiwar <shubhampampattiwar7@gmail.com>
2022-05-11 09:57:47 -04:00
Shubham Pampattiwar e3106f3e10 add existingResourcePolicy to Restore API
add updateall policy option

fix updating labels

dump updateAll policy option

remove updateAll policy refs

Signed-off-by: Shubham Pampattiwar <shubhampampattiwar7@gmail.com>

add changelog file

Signed-off-by: Shubham Pampattiwar <shubhampampattiwar7@gmail.com>

update docs

Signed-off-by: Shubham Pampattiwar <shubhampampattiwar7@gmail.com>

patch labels for sa if policy is update

Signed-off-by: Shubham Pampattiwar <shubhampampattiwar7@gmail.com>

fix existingResourcePolicy for serviceaccounts

modularize changes and add unit tests

Signed-off-by: Shubham Pampattiwar <shubhampampattiwar7@gmail.com>

fix conflicts and update crds.go

Signed-off-by: Shubham Pampattiwar <shubhampampattiwar7@gmail.com>

change log level from info to error

Signed-off-by: Shubham Pampattiwar <shubhampampattiwar7@gmail.com>

update crds.go

Signed-off-by: Shubham Pampattiwar <shubhampampattiwar7@gmail.com>
2022-05-09 13:36:24 -04:00
Scott Seago da916846b1 continue rather than return for non-matching restore action label
When iterating over applicable restore actions, if a non-matching label
selector is found, velero should continue to the next action rather than
returning from the restoreItem func, which ends up preventing the item's
restore entirely.

Signed-off-by: Scott Seago <sseago@redhat.com>
2022-05-05 08:37:48 -04:00
I538157 74db20993b fix: delete empty backups
Signed-off-by: Yuval Manor <yuvalman958@gmail.com>
2022-04-12 13:13:56 +03:00
Wenkai Yin(尹文开) fcad46ccdf Check whether the volume is provisioned by CSI driver or not by the annotation as well
Check whether the volume is provisioned by CSI driver or not by the annotation as well

Fixes #4496

Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
2022-01-12 19:05:33 +08:00
Wenkai Yin(尹文开) 62779bbcc6 Check the existence of the expected service when ignoring the NodePort already allocated error
Check the existence of the expected service when ignoring the NodePort already allocated error

Fixes 2308

Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
2022-01-12 15:06:42 +08:00
Daniel Jiang 7c2dc143d5
Merge pull request #4391 from ywk253100/211123_anno
Keep the annotation "pv.kubernetes.io/provisioned-by" when restoring PVs
2021-12-21 11:08:25 +08:00
Wenkai Yin(尹文开) 648f3ac228 Keep the annotation "pv.kubernetes.io/provisioned-by" when restoring PVs
More details please refer to https://github.com/vmware-tanzu/velero/issues/3470#issuecomment-976279606

Fixes #3470

Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
2021-12-21 09:52:56 +08:00
Wenkai Yin(尹文开) c7cd95a374
Ignore the provided port is already allocated error when restoring the LoadBalancer service (#4462)
Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
2021-12-21 05:42:27 +08:00
David L. Smith-Uchida 5677e04bb1
Consolidated code for resolving actions and plugins into ActionResolver (#4410)
* Consolidated code for resolving actions and plugins into ActionResolver.  Added BackupWithResolvers and
RestoreWithResolvers.  Introduces ItemSnapshooterResolver to bring ItemSnapshotter plugins into backup and
restore.  ItemSnapshotters are not used yet.

Added action_resolver_test

Signed-off-by: Dave Smith-Uchida <dsmithuchida@vmware.com>

* Addressed review comments

Signed-off-by: Dave Smith-Uchida <dsmithuchida@vmware.com>
2021-12-10 12:53:47 -05:00
Wenkai Yin(尹文开) 7c4e03e9f9
Ignore the `provided port is already allocated` error when restoring the `NodePort` service (#4336)
Fixes #2308

Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
2021-11-15 20:25:04 +08:00
Scott Seago 983489073f
PV remapClaimRefNS was being skipped when there was no snapshot (#3708)
Signed-off-by: Scott Seago <sseago@redhat.com>
2021-11-09 20:30:16 -05:00
Bridget McErlean 368098b727 Use LabelSelector in Action ResourceSelector
Instead of converting the unstructured item to check for the presence of
the `kube-aggregator.kubernetes.io/automanaged` label, use this label in
the `AppliesTo` to enable the restore logic to select the item. This
means that any item that matches the selector will have restore skipped.

Also add a new test case to the restore action test to check that label
selectors are applied correctly.

Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>
2021-08-11 17:33:31 -04:00
Pranav Gaikwad c46fe71b12
Restore progress reporting (#3125)
* restore progress reporting

Signed-off-by: Pranav Gaikwad <pgaikwad@redhat.com>

* add restore statistics to describe restore

Signed-off-by: Pranav Gaikwad <pgaikwad@redhat.com>

* address feedback, include namespaces in the count

Signed-off-by: Pranav Gaikwad <pgaikwad@redhat.com>
2021-03-04 16:21:44 -05:00
Ashish Amarnath dc6762a895
🐛 Use namespace and name to match PVB to Pod restore (#3051)
Signed-off-by: Ashish Amarnath <ashisham@vmware.com>
2020-11-10 11:36:49 -05:00
Nolan Brubaker 28a46d3a8b
Ensure PVs and PVCs remain bound when doing a restore (#3007)
* Only remove the UID from a PV's claimRef

The UID is the only part of a claimRef that might prevent it from being
rebound correctly on a restore. The namespace and name within the
claimRef should be preserved in order to ensure that the PV is claimed
by the correct PVC on restore.

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>

* Remap PVs claimRef.namespace on relevant restores

When remapping namespaces, any included PVs need to have their claimRef
updated to point remapped namespaces to the new namespace name in order
to be bound to the correct PVC.

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>

* Update tests and ensure claimRef namespace remaps

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>

* Remove lowercased uid field from unstructured PV

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>

* Fix issues that prevented PVs from being restored

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>

* Add changelog

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>

* Dynamically reprovision volumes without snapshots

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>

* Update test for lower case uid field

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>

* Remove stray debugging print statement

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>

* Fix typo, remove extra code, add tests.

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2020-10-15 16:57:43 -07:00
Pawan Prakash Sharma debcbd2f8e
fix: rename the PV if VolumeSnapshotter has modified the PV name (#2835)
* fix: rename the PV if VolumeSnapshotter has modified the PV name

When VolumeSnapshotter sets the PV name via SetVolumeID and PV is
not there in the cluster, velero does not rename the PV. Which causes
the pvc to be in the lost state as pvc points to the old PV but pv object
has been renamed by VolumeSnapshotter.

Signed-off-by: Pawan <pawan@mayadata.io>

* adding a test case for pv rename

Signed-off-by: Pawan <pawan@mayadata.io>
2020-09-01 14:25:13 -07:00
Nolan Brubaker 718a94ad05
Invoke DeleteItemActions on backup deletion (#2815)
* Add serving and listing support

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2020-08-20 17:24:29 -07:00
Andrew Lavery a368370bef
k8s 1.18 import (#2651)
* k8s 1.18 import wip

backup, cmd, controller, generated, restic, restore, serverstatusrequest, test and util

Signed-off-by: Andrew Lavery <laverya@umich.edu>

* go mod tidy

Signed-off-by: Andrew Lavery <laverya@umich.edu>

* add changelog file

Signed-off-by: Andrew Lavery <laverya@umich.edu>

* go fmt

Signed-off-by: Andrew Lavery <laverya@umich.edu>

* update code-generator and controller-gen in CI

Signed-off-by: Andrew Lavery <laverya@umich.edu>

* checkout proper code-generator version, regen

Signed-off-by: Andrew Lavery <laverya@umich.edu>

* fix remaining calls

Signed-off-by: Andrew Lavery <laverya@umich.edu>

* regenerate CRDs with ./hack/update-generated-crd-code.sh

Signed-off-by: Andrew Lavery <laverya@umich.edu>

* use existing context in restic and server

Signed-off-by: Andrew Lavery <laverya@umich.edu>

* fix test cases by resetting resource version

also use main library go context, not golang.org/x/net/context, in pkg/restore/restore.go

Signed-off-by: Andrew Lavery <laverya@umich.edu>

* clarify changelog message

Signed-off-by: Andrew Lavery <laverya@umich.edu>

* use github.com/kubernetes-csi/external-snapshotter/v2@v2.2.0-rc1

Signed-off-by: Andrew Lavery <laverya@umich.edu>

* run 'go mod tidy' to remove old external-snapshotter version

Signed-off-by: Andrew Lavery <laverya@umich.edu>
2020-07-16 12:21:37 -04:00
Steve Kriss 583ef4258a refactor restore priorities code to use single loop and lazy discovery
Signed-off-by: Steve Kriss <krisss@vmware.com>
2020-03-10 08:57:50 -06:00
Mayank 1a1372550d
Use PV name returned from volumesnapshotter while creating a PV (#2216)
* Using PV name returned from volumesnapshotter while creating a PV

Signed-off-by: mayank <mayank.patel@mayadata.io>
2020-02-10 13:17:15 -07:00
John Naulty Jr cd860771c2 Add error return to pvRenamer func (#2138)
Migrate logic from NewUUID function into the pvRenamer function.

PR #2133 switched to a new NewUUID function that returns an error, but
the invocation of that function needs to happen within the pvRenamer
closure. Because the new function returns an error, the pvRenamer should
return the error, the signature needs to be changed and the return
checked.

Signed-off-by: John Naulty <johnnaulty@bitgo.com>
2019-12-17 10:23:58 -08:00
Steve Kriss 7c60829f38 bug fix: don't restore cluster-scoped resources by default when restoring specific namespaces (#2118)
* add failing test case for bug

Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-12-12 12:45:41 -08:00
Steve Kriss 4fb1bc2ef3
restic: don't try to restore PVBs with no snapshotID (#2031)
* restic: don't try to restore PVBs with no snapshotID

Signed-off-by: Steve Kriss <krisss@vmware.com>

* changelog

Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-11-04 16:18:08 -07:00
Steve Kriss 48792ece1f Updates for org move to vmware-tanzu (#1920)
* update import paths to github.com/vmware-tanzu/...

Signed-off-by: Steve Kriss <krisss@vmware.com>

* update other GH org refs to vmware-tanzu

Signed-off-by: Steve Kriss <krisss@vmware.com>

* site and docs: update GH org to vmware-tanzu

Signed-off-by: Steve Kriss <krisss@vmware.com>

* update travis badge links on docs readmes

Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-09-30 17:26:56 -04:00
Steve Kriss 2e849dcb99 Fix for #1888: check item's original namespace, not remapped one, for inclusion/exclusion (#1909)
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-09-25 18:01:16 -07:00