Commit Graph

772 Commits (069c9a07512c48743ecd7bf22089f5b1c5d9b49d)

Author SHA1 Message Date
Xun Jiang 9693aca1f3 Fix Test_prepareBackupRequest_BackupStorageLocation UT failure.
Signed-off-by: Xun Jiang <blackpiglet@gmail.com>
2022-09-26 17:10:23 +08:00
Daniel Jiang c83447f394
Merge pull request #5362 from niulechuan/add-backup-controller-ut
Added backupController's UT to test the prepareBackupRequest() method BackupStorageLocation processing logic
2022-09-26 14:17:16 +08:00
danfengl 081b70d0eb Add backup opt-in/opt-out E2E test
Signed-off-by: danfengl <danfengl@vmware.com>
2022-09-23 09:29:32 +00:00
Lyndon-Li 6c8981b0ad fix issue 5386
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2022-09-23 16:52:38 +08:00
qiuming 66f6365988
Merge pull request #5355 from danfengliu/add-schedule-backup-timing-e2e-test
Add schedule backup timing E2E test
2022-09-23 15:51:47 +08:00
Xun Jiang/Bruce Jiang ce247a3d90
Remove snapshot related lister, informer and client from backup controller. (#5299)
Signed-off-by: Xun Jiang <blackpiglet@gmail.com>
2022-09-23 15:44:39 +08:00
Xun Jiang/Bruce Jiang b7f5cbd0c0
Merge pull request #5318 from sseago/volumesnapshotter-refactor
plugin versioning v1 refactor for VolumeSnapshotter
2022-09-23 09:54:01 +08:00
lyndon c81f0db886
rename pvbr param (#5370)
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2022-09-23 09:13:36 +08:00
Scott Seago b6088356e6 plugin versioning v1 refactor for VolumeSnapshotter
Signed-off-by: Scott Seago <sseago@redhat.com>
2022-09-21 13:47:23 -04:00
lyndon ee254c644f
Merge pull request #5353 from cleverhu/add-crd-field-doc-for-schedule
add useOwnerReferencesInBackup field doc
2022-09-21 12:12:38 +08:00
danfengl 80430542df Add schedule backup timing E2E test
Signed-off-by: danfengl <danfengl@vmware.com>
2022-09-21 02:32:26 +00:00
Niu Lechuan 1ab7ebd80e add change log
Signed-off-by: Niu Lechuan <lechuan.niu@daocloud.io>
2022-09-21 10:24:19 +08:00
Scott Seago 32ef20d317 RestoreItemAction v1 refactoring for plugin api versioning
Signed-off-by: Scott Seago <sseago@redhat.com>
2022-09-20 17:43:56 -04:00
qiuming 07da9b9cf8
Merge pull request #5279 from ywk253100/220829_pause
Support pause/unpause schedules
2022-09-20 16:06:17 +08:00
qiuming 648311a0f5
Merge pull request #5350 from blackpiglet/5019-snapshot-volumes-help-message-modification
Modify help message of parameter --snapshot-volumes
2022-09-20 14:18:00 +08:00
cleverhu f51c8bf44b add useOwnerReferencesInBackup field doc
Signed-off-by: cleverhu <shouping.hu@daocloud.io>
2022-09-19 18:50:51 +08:00
Lyndon-Li 18bda60791 fix issue 5358
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2022-09-19 16:46:03 +08:00
lyndon 429e204992
Merge pull request #5329 from kaovilai/veleroDownloadRequestReturnSomething-veleromain
Cancel downloadRequest when timeout without downloadURL
2022-09-19 14:20:18 +08:00
lyndon ede7b197ae
Merge pull request #5297 from blackpiglet/remove_apex_log
Remove github.com/apex/log logger.
2022-09-16 16:47:36 +08:00
Xun Jiang/Bruce Jiang e42352b2e4
Merge pull request #5271 from sseago/plugin-versioning-impl
plugin/clientmgmt and framework refactoring for BackupItemAction v1
2022-09-16 16:41:33 +08:00
Xun Jiang d134783282 Clarify the help message for the default value of parameter --snapshot-volumes, when it's not set.
Signed-off-by: Xun Jiang <blackpiglet@gmail.com>
2022-09-16 16:03:05 +08:00
Xun Jiang/Bruce Jiang 4768c2acf4
Merge pull request #5314 from allenxu404/issue3565
Change the status of restore to completed from partiallyFailed when restore empty backup
2022-09-16 15:00:36 +08:00
Scott Seago 4262b47536 plugin/clientmgmt refactoring for BackupItemAction v1
Refactors the clientmgmt package to implement the plugin versioning changes
needed for BIA v1 and overall package refactoring to support plugin versions
in different packages. This should be all that's needed to move on to
v2 for BackupItemAction. The remaining plugin types still need similar
refactoring to what's being done here for BIA before attempting a
v2 implementation.

Signed-off-by: Scott Seago <sseago@redhat.com>
2022-09-15 11:15:30 -04:00
Shubham Pampattiwar 1e138af1cf
Merge pull request #5263 from sseago/biav1-refactor
Don't move velero v1 plugins to new proto dir
2022-09-15 11:14:05 -04:00
Xun Jiang/Bruce Jiang 745ebbe081
Merge pull request #5347 from qiuming-best/fix-restic-resotre
Fix restore cmd extraflag overwrite bug
2022-09-15 15:37:15 +08:00
Xun Jiang/Bruce Jiang d0954dddd4
Merge pull request #5322 from qiuming-best/fix-get-parent-snapshot
Fix PVB finds wrong parent snapshot
2022-09-15 15:23:27 +08:00
Ming 4022020d5f Fix restore cmd extraflag overwrite bug
Signed-off-by: Ming <mqiu@vmware.com>
2022-09-15 06:09:41 +00:00
Xun Jiang/Bruce Jiang a05fc498b1
Merge pull request #5291 from cleverhu/add-shorthand-for-labels-columns
Add shorthand for labels columns
2022-09-15 11:24:06 +08:00
Wenkai Yin(尹文开) 4b9dbfa416 Support pause/unpause schedules
Support pause/unpause schedule

Fixes #2363

Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
2022-09-15 10:42:48 +08:00
Tiger Kaovilai 2c759f395a cancel downloadRequest checkFunc if timeout passed
Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>
2022-09-14 14:13:09 -04:00
Tiger Kaovilai 876238e33d Resolve gopkg.in/yaml.v3 vulnerabilities
as shown from https://security.snyk.io/package/golang/gopkg.in%2Fyaml.v3

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>
2022-09-14 10:48:52 -04:00
lyndon 100d6b4430
Merge pull request #5319 from Lyndon-Li/issue-fix-4874
Issue fix 4874 and 4752
2022-09-14 11:04:56 +08:00
Lyndon-Li fbb2606102 issue fix 4874 update
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2022-09-13 11:15:52 +08:00
Shubham Pampattiwar be40d7eb19
Merge pull request #4864 from sseago/vsl-creds-main
Add credentials to volume snapshot locations.
2022-09-12 09:56:49 -04:00
Ming c612853bd5 Fix PVB finds wrong parent snapshot
Signed-off-by: Ming <mqiu@vmware.com>
2022-09-09 08:30:57 +00:00
Xun Jiang/Bruce Jiang 325b8c0d05
Merge pull request #5308 from Lyndon-Li/repository-ensurer-refactor
Repo Ensurer Refactor
2022-09-08 21:17:31 +08:00
Scott Seago be0a1cf361
Merge pull request #5293 from cleverhu/optimize-restore
optimize code for restore exists resources
2022-09-08 09:01:44 -04:00
Scott Seago 596114b427 Add credentials to volume snapshot locations.
This is analogous to the BSL creds work that was done previously, but for VSLs instead.

Signed-off-by: Scott Seago <sseago@redhat.com>
2022-09-08 08:59:17 -04:00
allenxu404 dc70471909 Change the status of restore to completed from partially failed when restore empty backup
Signed-off-by: allenxu404 <qix2@vmware.com>
2022-09-08 18:40:49 +08:00
Lyndon-Li 8496b43e37 refactor repo ensurer
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2022-09-08 16:10:51 +08:00
lyndon 8888f8765e
Merge pull request #5282 from Lyndon-Li/kopia-pvbr-smoking-test
Kopia Pod Volume Backup/Restore Smoking Test Changes
2022-09-07 14:20:44 +08:00
Lyndon-Li a90ba3db7c kopia pvbr smoking test changes
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2022-09-07 13:22:49 +08:00
Xun Jiang b49e39c021 Remove github.com/apex/log logger.
Signed-off-by: Xun Jiang <blackpiglet@gmail.com>
2022-09-07 08:52:16 +08:00
cleverhu 7de6f2a2fc trim isAlreadyExistsError for restore
Signed-off-by: cleverhu <shouping.hu@daocloud.io>
2022-09-07 03:55:20 +08:00
cleverhu 067a3ec03a change CSISnapshotTimeout from point to normal var
Signed-off-by: cleverhu <shouping.hu@daocloud.io>
2022-09-07 01:30:47 +08:00
cleverhu 5e6111e6c0 add shorthand for labels columns
Signed-off-by: cleverhu <shouping.hu@daocloud.io>
2022-09-06 20:14:13 +08:00
Scott Seago b15c59ba69 Don't move velero v1 plugins to new proto dir
To preserve backwards compatibility, don't move velero
v1 plugins to new proto dir.

Signed-off-by: Scott Seago <sseago@redhat.com>
2022-09-05 16:01:56 -04:00
Daniel Jiang c7bd2b9c02
Merge pull request #5252 from cleverhu/add-csiSnapshotTimeout-for-describe
add CSISnapshotTimeout for describe backup
2022-09-05 14:44:36 +08:00
lyndon 0282e65221
Kopia Pod Volume Backup/Restore (#5259)
* kopia pvbr

Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2022-09-05 10:29:30 +08:00
qiuming a5a3df193d
Merge pull request #5236 from sseago/no-namespaces-fix
Check for empty ns list before checking nslist[0]
2022-09-05 09:40:19 +08:00
lyndon 5f1bf9eb35
Merge pull request #5214 from qiuming-best/uploader-restic
Uploader Implementation: Restic backup and restore
2022-09-02 11:11:38 +08:00
Daniel Jiang 218bab987d
Merge pull request #5239 from shubham-pampattiwar/fix-already-exists
Fix edge cases for already exists resources
2022-08-31 23:01:56 +08:00
Shubham Pampattiwar 94a9a7c795
Merge pull request #4943 from phuongatemc/refactor_plugin_biav1
Refactor BackupItemAction to backupitemaction/v1
2022-08-30 09:48:26 -04:00
Xun Jiang/Bruce Jiang 6fea973c57
Migrate backup sync controller from code-generator to kubebuilder (#5218)
* Migrate backup sync controller from code-generator to kubebuilder

1. use kubebuilder's reconcile logic to replace controller's old logic.
2. use ginkgo and gomega to replace testing.
3. modify BSL reconciler registration method.

Signed-off-by: Xun Jiang <blackpiglet@gmail.com>

* Add Option for PeriodicalEnqueueSource.

Signed-off-by: Xun Jiang <blackpiglet@gmail.com>

Signed-off-by: Xun Jiang <blackpiglet@gmail.com>
Co-authored-by: Xun Jiang <blackpiglet@gmail.com>
2022-08-30 16:55:39 +08:00
Xun Jiang/Bruce Jiang 4bc3a3a784
Merge pull request #5248 from allenxu404/gc-freq
equip gc controller with configurable frequency
2022-08-30 11:12:12 +08:00
allenxu404 a8ba4875f0 equip gc controller with configurable frequency
Signed-off-by: allenxu404 <qix2@vmware.com>
2022-08-30 10:13:06 +08:00
qiuming c8818ec1c9
Merge pull request #5241 from blackpiglet/update_enabled_runtime_controllers
Controller refactor code modifications.
2022-08-30 09:30:35 +08:00
Shubham Pampattiwar 93a875873b fix edge cases for already exists resources
Signed-off-by: Shubham Pampattiwar <spampatt@redhat.com>

add changelog file

Signed-off-by: Shubham Pampattiwar <spampatt@redhat.com>

update changelog filename

Signed-off-by: Shubham Pampattiwar <spampatt@redhat.com>

change log level and error type

Signed-off-by: Shubham Pampattiwar <spampatt@redhat.com>
2022-08-29 17:34:02 -04:00
Scott Seago 71648750cc
Merge pull request #5247 from divolgin/storage-class-panic
Don't panic when storageClassName is not set in stateful sets
2022-08-29 10:06:32 -04:00
Xun Jiang/Bruce Jiang eaf9fab711
Merge branch 'main' into update_enabled_runtime_controllers 2022-08-29 20:06:31 +08:00
Ming f15757a3d8 Uploader Implementation: Restic backup and restore
Signed-off-by: Ming <mqiu@vmware.com>
2022-08-29 09:33:37 +00:00
qiuming 3e30a3d388
Merge pull request #5233 from Lyndon-Li/udmrepo-dev-06
Kopia Integration:Kopia Lib - Implementation
2022-08-28 22:33:12 +08:00
cleverhu e77aaa32ca add CSISnapshotTimeout for describe backup
Signed-off-by: cleverhu <shouping.hu@daocloud.io>
2022-08-28 15:14:03 +08:00
Xun Jiang/Bruce Jiang 86762f442a
Merge pull request #5217 from lilongfeng0902/lilf-branch
check vsc null pointer
2022-08-27 10:45:13 +08:00
Lyndon-Li 3769cd218a kopia lib
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2022-08-26 15:25:20 +08:00
Hoang, Phuong 4a043bdab9 Add changelog
Signed-off-by: Hoang, Phuong <phuong.n.hoang@dell.com>
2022-08-25 17:10:14 -04:00
divolgin ad4e733ef2 Don't panic when storageClassName is not set in stateful sets
Signed-off-by: divolgin <dmitriy@replicated.com>
2022-08-25 15:02:40 +00:00
Scott Seago 55bf2de15d Check for empty ns list before checking nslist[0]
In determining whether a backup includes all namespaces, item_collector
checks for an empty string in the first element of the ns list. If processing
includes+excludes results in an empty list, treat this as another case
of a not-all-namespaces backup rather than crashing velero.

Signed-off-by: Scott Seago <sseago@redhat.com>
2022-08-24 14:32:19 -04:00
Xun Jiang/Bruce Jiang e8494418d4
Merge pull request #5215 from allenxu404/issue5023
Refactor GC controller with kubebuilder
2022-08-23 15:59:03 +08:00
Xun Jiang 71e5027bfb Controller refactor code modifications.
1. Add some refactored controllers initiation code into enabledRuntimeControllers.
2. Add reconciler struct initiation function for DownloadRequest and ServerStatusRequest controllers.

Signed-off-by: Xun Jiang <blackpiglet@gmail.com>
2022-08-23 10:32:19 +08:00
qiuming 082d680d7b
Merge pull request #5221 from qiuming-best/uploader-kopia
Uploader Implementation: Kopia backup and restore
2022-08-22 20:00:27 +08:00
李龙峰 5cddaeae6c check vsc null pointer
Signed-off-by: 李龙峰 <lilongfeng@inspur.com>
2022-08-22 14:44:17 +08:00
allenxu404 0378020c8d refactor gc controller with kubebuilder
Signed-off-by: allenxu404 <qix2@vmware.com>
2022-08-22 12:16:56 +08:00
Daniel Jiang 3e435eeb44 Remove reference to non-existent doc
fixes #4923

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2022-08-19 15:44:00 +08:00
Ming 839c2ed98f Merge branch 'main' of https://github.com/qiuming-best/velero into uploader-kopia 2022-08-19 06:40:58 +00:00
Ming 262de19f52 Uploader Implementation: Kopia backup and restore
Signed-off-by: Ming <mqiu@vmware.com>
2022-08-19 06:39:28 +00:00
Lyndon-Li d58abb2477 repo init
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2022-08-18 19:45:49 +08:00
Daniel Jiang 4e25f59dc1
Add parameter "uploader-type" to velero server (#5212)
This commit adds the parameter "uploader-type" to velero server, add exposes the
setting via "velero install" in CLI.

fixes #5062

Signed-off-by: Daniel Jiang <jiangd@vmware.com>

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2022-08-16 13:50:28 +08:00
qiuming 71e4430840
Merge pull request #5179 from Lyndon-Li/udmrepo-dev-02
Kopia Integration: Unified Repository Provider - Implementation
2022-08-16 09:39:40 +08:00
qiuming 36d8d176dd
Merge pull request #5163 from qiuming-best/uploader
kopia uploader integration on shim progress uploader module
2022-08-11 17:38:21 +08:00
Ming 49e151739f Kopia Uploader integration on shim progress uploader
Signed-off-by: Ming <mqiu@vmware.com>
2022-08-10 11:35:24 +00:00
Lyndon-Li 649c3a77df unified repo provider impl
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2022-08-09 20:48:31 +08:00
Xun Jiang/Bruce Jiang a5f4f8f9fc
Merge branch 'main' into 5048-CSI-snapshot-timeout-configurable 2022-08-08 17:18:44 +08:00
Xun Jiang f8d9cfdb84 Make CSI snapshot creation timeout configurable for backup and schedule.
Signed-off-by: Xun Jiang <jxun@vmware.com>
2022-08-08 17:07:26 +08:00
Xun Jiang/Bruce Jiang 201c43d683
Merge pull request #5178 from allenxu404/i2413
fix issue#2413: treat namespaces with exclude label as excludedNamespaces
2022-08-08 13:35:38 +08:00
Daniel Jiang ba50458ae2
Merge pull request #5181 from blackpiglet/restic-volume-path-change
Add annotation "pv.kubernetes.io/migrated-to" for CSI checking.
2022-08-08 12:03:47 +08:00
Xun Jiang 7af1e23614 Add annotation "pv.kubernetes.io/migrated-to" for CSI checking.
1. Also checking annotation "pv.kubernetes.io/migrated-to" to find out whether volume is provisioned by CSI.
2. Add UT cases.

Signed-off-by: Xun Jiang <jxun@vmware.com>
2022-08-08 10:59:41 +08:00
Xun Jiang/Bruce Jiang cea5e7f218
Merge pull request #5174 from blackpiglet/reduce-crd-size-new
Reduce CRD size.
2022-08-05 18:46:29 +08:00
Wenkai Yin(尹文开) 092fc01e8d
Splic pkg/restic package (#5143)
This commit splits the pkg/restic package into several packages to support Kopia integration works

Fixes #5055

Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
2022-08-05 17:15:38 +08:00
allenxu404 eb08bdeb62 fix issue#2413 by treating namespaces with exclude label as excludedNamespaces
Signed-off-by: allenxu404 <qix2@vmware.com>
2022-08-04 20:00:11 +08:00
lyndon 088eb9b83c
repo credentials (#5167)
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2022-08-04 15:20:02 +08:00
qiuming 701256d296
Merge pull request #5172 from qiuming-best/multi-bsl-fix
Fix restic backups to multiple backup storage locations bug
2022-08-03 14:35:42 +08:00
Xun Jiang e8da5df57a Reduce CRD size.
1. Make the Restore hook.InitConatianer server side field pruing disable.
2. Remove restore patch in update-generate-crd-code.sh.
3. Modify related testcases.
4. Add Container fields validation in Restore Init hook.

Signed-off-by: Xun Jiang <jxun@vmware.com>
2022-08-02 19:45:01 +08:00
Ming 828e28aa43 Fix restic backups to multiple backup storage locations bug
Signed-off-by: Ming <mqiu@vmware.com>
2022-08-02 10:26:23 +00:00
Daniel Jiang 108c81d84c
Skip registering "crd-remap-version" plugin when feature flag (#5165)
"EnableAPIGroupVersions" is set

The crd-remap-version plugin will always backup v1b1 resource for some
CRD. It impacts the feature flag `EnableAPIGroupVersions` which means to
backup all versions, and make migration fail.

In this commit the featureSet was removed from plugin server struct b/c
it blocks the parm `--features` to be populated correctly.  This change
should not have negative impact b/c the attribute in server struct is never used.

Fixes #5146

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2022-08-02 15:47:57 +08:00
Daniel Jiang 2b0d0959da
Merge pull request #5148 from blackpiglet/vsl-refactor
VolumeSnapshotLocation refactor with kubebuilder.
2022-08-01 21:14:18 +08:00
lyndon 52fd18e9db
Kopia Integration Change - Storage Configuration (#5142)
* unified repo storge config

Signed-off-by: Lyndon-Li <lyonghui@vmware.com>

* add UT

Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2022-07-29 16:06:18 +08:00
qiuming f2ef40c983
Merge pull request #5157 from blackpiglet/modify-changelog-check-action-condition
Add labeled and unlabeled events for PR changelog check action.
2022-07-29 16:00:33 +08:00
Xun Jiang cd643bbac9 Add labeled and unlabeled events for PR changelog check action.
Signed-off-by: Xun Jiang <jxun@vmware.com>
2022-07-28 17:45:26 +08:00
Xun Jiang 396e68b810 VolumeSnapshotLocation refactor with kubebuilder.
1. modify VSL CRD API file name. Add kubebuilder related marker.
2. Add Labels init code for VSL create command.

Signed-off-by: Xun Jiang <jxun@vmware.com>
2022-07-25 15:46:43 +08:00