Commit Graph

119 Commits (a4416874cf8d2d0f88cfbb2fb3cebac62a8e3a33)

Author SHA1 Message Date
Scott Seago 9d6f4d2db5 ItemBlock model and phase 1 (single-thread) workflow changes
Signed-off-by: Scott Seago <sseago@redhat.com>
2024-09-03 19:04:18 -04:00
Matthieu MOREL c8baaa9b11
testifylint: enable more rules (#8024)
Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>
2024-07-18 10:43:16 -04:00
Matthieu MOREL 35c90f1672
testifylint: enable error-nil rule (#7670)
Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>
2024-07-16 12:23:16 -04:00
Scott Seago dc286a38fc Reuse existing plugin manager for get/put volume info
Signed-off-by: Scott Seago <sseago@redhat.com>
2024-07-12 10:15:16 -04:00
Xun Jiang cf5dfdf42d Check whether the namespaces specified in namespace filter exist.
Check whether the namespaces specified in the
backup.Spec.IncludeNamespaces exist during backup resource collcetion
If not, log error to mark the backup as PartiallyFailed.

Signed-off-by: Xun Jiang <blackpigletbruce@gmail.com>
2024-07-04 10:02:10 +08:00
Xun Jiang/Bruce Jiang d9c9f77860
Merge pull request #7671 from mmorel-35/testifylint/compare
testifylint: enable compares rule
2024-05-27 12:45:16 +08:00
Xun Jiang a91d2cb036 Modify the volume helper logic.
Signed-off-by: Xun Jiang <blackpigletbruce@gmail.com>
2024-05-23 09:57:21 +08:00
Matthieu MOREL 1010b04821 testifylint: enable compares rule
Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>
2024-05-16 20:18:43 +00:00
Xun Jiang/Bruce Jiang f25c154709
Merge pull request #7569 from ywk253100/240326_namespace
Check the existence of the namespaces provided in the "--include-namespaces" option
2024-04-11 19:00:33 +08:00
Xun Jiang 5462035469 Delete the unneeded pvRestorer action in
handleSkippedPVHasRetainPolicy

According to comment, calling executePVAction aims to reset PV's
claimRef, but the reset logic was moved into resetVolumeBindingInfo
since release-1.4.

Signed-off-by: Xun Jiang <blackpigletbruce@gmail.com>
2024-03-29 14:12:12 +08:00
Xun Jiang b06d7a467f Support update the backup VolumeInfos by the Async ops result.
1. Add PutBackupVolumeInfos method.
2. Add CompletionTimestamp in VolumeInfo.
3. Add Size in SnapshotDataMovementInfo.
4. Update CompletionTimpstmap, SnapshotHandle, RetainedSnapshot
   and Size in VolumeInfo on DataUpload Operation completes.

Signed-off-by: Xun Jiang <blackpigletbruce@gmail.com>
2024-03-28 19:52:44 +08:00
Matthieu MOREL ef04ef6361 golangci-lint: use exclude-rules instead of skip-files and skip-dirs
Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>
2024-03-27 20:17:34 +00:00
Wenkai Yin(尹文开) 35d2534e19 Check the existence of the namespaces provided in the "--include-namespaces" option
Check the existence of the namespaces provided in the "--include-namespaces" opt
ion and reports validation error if not found

Fixes #7431

Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
2024-03-27 18:37:03 +08:00
Wenkai Yin(尹文开) 8752c3a820 Bump up the versions of severel Kubernetes-related libs
Bump up the versions of severel Kubernetes-related libs

Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
2024-03-05 13:09:38 +08:00
Daniel Jiang 804b9a8d91
Merge pull request #7171 from kaovilai/tests-explicit-enableCSI
Add explicit enableCSI to TestProcessBackupCompletions
2023-12-11 14:11:37 +08:00
Xun Jiang ca97248f2a Use VolumeInfo to help restore the PV.
Add VolumeInfo for left PVs during backup.

Signed-off-by: Xun Jiang <jxun@vmware.com>
2023-12-04 08:33:37 +08:00
Tiger Kaovilai 2132506e8c
Add explicit enableCSI to TestProcessBackupCompletions
Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>
2023-12-01 14:22:40 -05:00
Xun Jiang 7f52321772 Generate VolumeInfo.
Remove CSI VolumeSnapshot listter and the informer.
Add download the VolumeInfos metadata for backup.

Signed-off-by: Xun Jiang <jxun@vmware.com>
2023-11-22 09:40:38 +08:00
Tiger Kaovilai 8c727429c4
revert test changes
Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>
2023-11-02 17:06:19 -04:00
Tiger Kaovilai 9311a4269b
refactor backup snapshot status updates into UpdateBackupSnapshotsStatus() and run in backup_finalizer_controller
Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>
2023-11-02 15:30:35 -04:00
Tiger Kaovilai 886e074b55
Add PatchResource unit test for backup status
Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>
2023-11-01 15:28:56 -04:00
Lyndon-Li 778feba3ae fix issue 6647
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2023-09-04 16:55:36 +08:00
Wenkai Yin(尹文开) 89c10ddcc0
Update metrics when backup failed with validation error (#6318)
Update metrics when backup failed with validation error

Fixes #6265

Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
2023-06-09 11:19:26 +08:00
Xun Jiang/Bruce Jiang 9743a7ce56
Fix #6118: Do not persist VolumeSnapshot and VolumeSnapshotContent for snapshot DataMover case. (#6366)
1. Because VolumeSnapshot and VolumeSnapshotContent CRs are not kept after backup completed,
   don't persist them in the backup metadata.
2. Add some builder methods needed by CSI plugin.

Signed-off-by: Xun Jiang <jxun@vmware.com>
2023-06-08 17:05:17 +08:00
Xun Jiang 7491ab1ec5 Remove Wait VolumeSnapshot to ReadyToUse logic.
Due to the logic moving to plugin, and the plugin cannot read the
Velero server's resourceTimeout setting, add the resourceTimeout
in the backup annotation to pass to plugin.
Remove VolumeSnapshotContent reset code from Velero server.

Signed-off-by: Xun Jiang <jxun@vmware.com>
2023-06-02 14:43:43 +08:00
Xun Jiang bbc1e2e151 Enable stylecheck linter and resolve found issues.
Signed-off-by: Xun Jiang <blackpiglet@gmail.com>
2023-04-25 14:37:02 +08:00
Tiger Kaovilai da0f5d5850 Field labels for Duration, Time, client.ObjectKey, KibishiiData, VeleroCLI2Version
Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>
2023-04-16 12:34:28 -04:00
Xun Jiang 56f93393d8 Modify new scope resource filters name.
Signed-off-by: Xun Jiang <blackpiglet@gmail.com>
2023-04-06 18:11:55 +08:00
lyndon 2ad43194aa
Merge pull request #5900 from blackpiglet/5416_limit_concurrent_goroutine
Limit the concurrent number for backup's VolumeSnapshot operation
2023-03-21 18:07:27 +08:00
Daniel Jiang 458560795b
Remove Reference of itemsnapshotter (#5997)
It will be removed as we choose to use BIAv2 for progress monitoring.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2023-03-21 09:05:29 +08:00
Xun Jiang c677c433e0 Limit the concurrent number of go routine for VolumeSnapshot and VolumeSnapshotContent operations.
Signed-off-by: Xun Jiang <blackpiglet@gmail.com>
2023-03-16 11:42:38 +08:00
Xun Jiang a1e4f54488 Add new resource filters can separate cluster and namespace scope resources.
Signed-off-by: Xun Jiang <blackpiglet@gmail.com>
2023-03-14 07:07:22 +00:00
Scott Seago dd63e8182c Follow-on fixes for BIAv2 controller work
Signed-off-by: Scott Seago <sseago@redhat.com>
2023-03-13 10:29:36 -04:00
Xun Jiang/Bruce Jiang 7d7e3fff0d
Refoctor backup controller with controller-runtime. (#5969)
Signed-off-by: Ming <mqiu@vmware.com>
Signed-off-by: Xun Jiang <blackpiglet@gmail.com>
Co-authored-by: Ming <mqiu@vmware.com>
2023-03-10 08:59:40 +08:00
Scott Seago c3d1d83da5 BIAv2 async operations controller work
Signed-off-by: Scott Seago <sseago@redhat.com>
2023-03-01 11:49:17 -05:00
Zhizhen He fb1dc110f2
Replace k8s.io/apimachinery/pkg/util/clock with k8s.io/utils/clock
Signed-off-by: Zhizhen He <hezhizhen.yi@gmail.com>
2023-02-21 11:56:17 +08:00
Scott Seago 9a54142257 BackupItemAction v2 API implementation
Signed-off-by: Scott Seago <sseago@redhat.com>
2023-01-09 18:36:47 -05:00
Xun Jiang/Bruce Jiang 5027aae194
Add more nil pointer check for CSI related code in backup controller. (#5388)
Add some corner cases checking for CSI snapshot in backup controller.

Signed-off-by: Xun Jiang <blackpiglet@gmail.com>
2022-10-24 10:42:08 +08:00
Xun Jiang 9693aca1f3 Fix Test_prepareBackupRequest_BackupStorageLocation UT failure.
Signed-off-by: Xun Jiang <blackpiglet@gmail.com>
2022-09-26 17:10:23 +08:00
Daniel Jiang c83447f394
Merge pull request #5362 from niulechuan/add-backup-controller-ut
Added backupController's UT to test the prepareBackupRequest() method BackupStorageLocation processing logic
2022-09-26 14:17:16 +08:00
lyndon c81f0db886
rename pvbr param (#5370)
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2022-09-23 09:13:36 +08:00
Niu Lechuan a5f1e7ac11 Added backupController's UT to test the prepareBackupRequest() method BackupStorageLocation processing logic
Signed-off-by: Niu Lechuan <lechuan.niu@daocloud.io>
2022-09-20 11:26:03 +08:00
Scott Seago 596114b427 Add credentials to volume snapshot locations.
This is analogous to the BSL creds work that was done previously, but for VSLs instead.

Signed-off-by: Scott Seago <sseago@redhat.com>
2022-09-08 08:59:17 -04:00
Hoang, Phuong b54424bdc6 Refactor BackupItemAction to backupitemaction/v1
Signed-off-by: Hoang, Phuong <phuong.n.hoang@dell.com>
2022-08-25 17:10:09 -04:00
Wenkai Yin(尹文开) 6fa4d7d606 Mark in-progress CRs as failed when starting the server
Mark in-progress CRs as failed when starting the server

Fixes #4953

Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
2022-06-07 17:02:17 +08:00
Shubham Pampattiwar bfdb68a35a add multiple label selector support to backup API
Signed-off-by: Shubham Pampattiwar <shubhampampattiwar7@gmail.com>

remove backup CLI bits

Signed-off-by: Shubham Pampattiwar <shubhampampattiwar7@gmail.com>

labelselectors spec option for velero restore

Signed-off-by: Shubham Pampattiwar <shubhampampattiwar7@gmail.com>

add changelog file

Signed-off-by: Shubham Pampattiwar <shubhampampattiwar7@gmail.com>

update spec name to OrLabelSelectors

Signed-off-by: Shubham Pampattiwar <shubhampampattiwar7@gmail.com>

minor fixes

Signed-off-by: Shubham Pampattiwar <shubhampampattiwar7@gmail.com>

add validations for labelSelector and orLabelSelectors

Signed-off-by: Shubham Pampattiwar <shubhampampattiwar7@gmail.com>

update crds.gp after fixing conflicts

Signed-off-by: Shubham Pampattiwar <shubhampampattiwar7@gmail.com>

fix CI and add unit tests

Signed-off-by: Shubham Pampattiwar <shubhampampattiwar7@gmail.com>

updated OrLabelSelector spec description and added validation failure unit tests

Signed-off-by: Shubham Pampattiwar <shubhampampattiwar7@gmail.com>

add comments and change log level

Signed-off-by: Shubham Pampattiwar <shubhampampattiwar7@gmail.com>

update site docs

Signed-off-by: Shubham Pampattiwar <shubhampampattiwar7@gmail.com>

wrap pager client calls in a function

Signed-off-by: Shubham Pampattiwar <shubhampampattiwar7@gmail.com>

resolve confilcts and update crds.go

Signed-off-by: Shubham Pampattiwar <shubhampampattiwar7@gmail.com>

rebase and update crds.go

Signed-off-by: Shubham Pampattiwar <shubhampampattiwar7@gmail.com>

combine listing items for a given label

Signed-off-by: Shubham Pampattiwar <shubhampampattiwar7@gmail.com>
2022-05-11 09:57:47 -04:00
Wenkai Yin(尹文开) dfc86566b8 Make in-progress backup/restore as failed when doing the reconcile
Make in-progress backup/restore as failed when doing the reconcile to avoid hanging in in-progress status

Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
2022-04-26 11:05:03 +08:00
David L. Smith-Uchida 5677e04bb1
Consolidated code for resolving actions and plugins into ActionResolver (#4410)
* Consolidated code for resolving actions and plugins into ActionResolver.  Added BackupWithResolvers and
RestoreWithResolvers.  Introduces ItemSnapshooterResolver to bring ItemSnapshotter plugins into backup and
restore.  ItemSnapshotters are not used yet.

Added action_resolver_test

Signed-off-by: Dave Smith-Uchida <dsmithuchida@vmware.com>

* Addressed review comments

Signed-off-by: Dave Smith-Uchida <dsmithuchida@vmware.com>
2021-12-10 12:53:47 -05:00
Bridget McErlean 9dbd238c89
Use controller-runtime client to get restic secrets (#3320)
* Use kubebuilder client for fetching restic secrets

Instead of using a SecretInformer for fetching secrets for restic, use
the cached client provided by the controller-runtime manager.

In order to use this client, the scheme for Secrets must be added to the
scheme used by the manager so this is added when creating the manager in
both the velero and restic servers.

This change also refactors some of the tests to add a shared utility for
creating a fake controller-runtime client which is now used among all
tests which use that client. This has been added to ensure that all
tests use the same client with the same scheme.

Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>

* Add builder for SecretKeySelector

Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>
2021-02-18 10:30:52 -08:00
Bridget McErlean 38c08e087b
Replace NewObjectBackupStore with interface (#3329)
In preparation for modifying the instantiation of `BackupStores` to be
able to load credentials, change the function `NewObjectBackupStore` to
be an interface that is passed in to all controllers.

Previously, the function to get a new backup store was configurable but
for many controllers was fixed to use `NewObjectBackupStore`. This
change introduces an interface for getting the backup store and wraps
the functionality from `NewObjectBackupStore` in a type which implements
this interface. This will allow more flexibility when introducing
credentials for a specific backup store as it will allow us to create a
new `ObjectBackupStoreGetter` type which can be configured to add
credentials config when creating the ObjectBackupStore without needing
to change the API used by the controllers.

Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>
2021-02-08 13:04:08 -05:00