Commit Graph

1780 Commits (831be07dd338e138462c619b1f6b7ff27333057e)

Author SHA1 Message Date
Lyndon-Li 40aae5ebdd fix issue 5935
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2023-03-09 14:24:51 +08:00
Shubham Pampattiwar c6059a93d2
Merge pull request #5873 from blackpiglet/add_psa_support 2023-03-08 17:41:53 -08:00
杨刚 (成都) 38a7707ce3
Unify pkg/install deployment args array. (#5961)
Signed-off-by: yanggang <gang.yang@daocloud.io>
2023-03-08 10:29:30 +08:00
Lyndon-Li 28d636bd71 fix client wait problem
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2023-03-08 09:29:16 +08:00
Scott Seago c3d1d83da5 BIAv2 async operations controller work
Signed-off-by: Scott Seago <sseago@redhat.com>
2023-03-01 11:49:17 -05:00
Wenkai Yin(尹文开) eb4ecd3767 Add restored resource list in the output of restore describe command
Add restored resource list in the output of restore describe command

Fixes #5199

Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
2023-02-28 10:43:03 +08:00
Xun Jiang c2ac76165e Make restore controller adopting the controller-runtime framework.
Signed-off-by: Xun Jiang <blackpiglet@gmail.com>
2023-02-25 11:55:02 +08:00
Xun Jiang/Bruce Jiang a467488f1a
Merge pull request #5894 from Lyndon-Li/issue-fix-5881
Fix issue 5881
2023-02-24 17:23:10 +08:00
Shubham Pampattiwar b23c541010
Merge pull request #5859 from hezhizhen/clock
Replace k8s.io/apimachinery/pkg/util/clock with k8s.io/utils/clock
2023-02-23 13:37:33 -08:00
danfengl 979fb9ccab Roll back pkg client code for Velero server
Signed-off-by: danfengl <danfengl@vmware.com>
2023-02-22 14:11:02 +00:00
qiuming d7defa7fb5
Merge pull request #5893 from reasonerjt/sa-rb
Do not create the rolebinding if user customize the service account
2023-02-22 14:35:06 +08:00
Lyndon-Li 08b8498afb fix issue 5881
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2023-02-21 19:17:38 +08:00
Daniel Jiang 42a92e9b3d Do not create the rolebinding if user customize the service account
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2023-02-21 18:20:38 +08:00
Xun Jiang/Bruce Jiang 5555f7d4e7
Merge pull request #5865 from allenxu404/issue4816
Add a json output to velero backup describe
2023-02-21 18:19:06 +08:00
allenxu404 16bf3e2d90 Add a json output to velero backup describe
Signed-off-by: allenxu404 <qix2@vmware.com>
2023-02-21 17:16:34 +08:00
Zhizhen He fb1dc110f2
Replace k8s.io/apimachinery/pkg/util/clock with k8s.io/utils/clock
Signed-off-by: Zhizhen He <hezhizhen.yi@gmail.com>
2023-02-21 11:56:17 +08:00
Scott Seago 5f039b7f7c
Merge pull request #5224 from kaovilai/patch-3
gh-issue-template: Comment out the subheading description
2023-02-20 10:14:39 -05:00
Xun Jiang 6be07a1df3 Add labels for created namespace during velero installation to adopt k8s v1.25's PSS and PSA.
Signed-off-by: Xun Jiang <blackpiglet@gmail.com>
2023-02-20 21:27:04 +08:00
Wenkai Yin(尹文开) 4d0c3ac83f
Restore finalizer and managedFields (#5853)
Restore finalizer and managedFields of metadata during the restoration

Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
2023-02-20 10:26:45 +08:00
Scott Seago fa58a775e8
Merge pull request #5576 from anshulahuja98/backupresults
Publish backup results to enhance error info
2023-02-18 17:00:49 -05:00
Anshul Ahuja 8c3ddf0f73 Publish backup results extracted from backup logs
Signed-off-by: Anshul Ahuja <anshulahuja@microsoft.com>
2023-02-16 09:57:53 +05:30
lyndon c5efb542d0
Merge pull request #5786 from qiuming-best/unsupport-type-error
Set Kopia IgnoreUnknownTypes in ErrorHandlingPolicy to True
2023-02-13 15:08:48 +08:00
lyndon 44bcc0959e
Merge pull request #5840 from kaovilai/pvb-pvr-should-fail-during-startup
move UpdatePVXStatusToFailed to controller pkg
2023-02-13 13:58:43 +08:00
Xun Jiang/Bruce Jiang 8f76907aff
Merge pull request #5843 from ywk253100/230208_secret
Add secret restore item action to handle service account token secret
2023-02-13 11:57:32 +08:00
Fish-pro ef05af13bf
Clean up unused variables and functions (#5818)
Signed-off-by: Fish-pro <zechun.chen@daocloud.io>
2023-02-09 10:53:24 +08:00
Scott Seago 0be05c9bc8
Merge pull request #5788 from blackpiglet/enable_staticcheck
Enable staticcheck linter. Disable check for folder /pkg and /test by…
2023-02-08 19:26:22 -05:00
Wenkai Yin(尹文开) 731a484275 Add secret restore item action to handle service account token secret
Add secret restore item action to handle service account token secret:
1. Skip the restoration for the auto-created service account token secret
2. Remove several fields for non-auto-created service account token secret to make sure the secret can be restored

Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
2023-02-08 15:52:11 +08:00
lyndon 7139daf07a
Merge pull request #5769 from Lyndon-Li/issue-fix-5458
Issue fix 5458
2023-02-08 09:59:15 +08:00
Tiger Kaovilai 3be7c33d3b move UpdatePVXStatusToFailed to controller pkg
Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>
2023-02-07 10:20:30 -05:00
lyndon 53c3f4b436
Issue fix 5226 (#5768)
* fix issue 5226

Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2023-02-07 17:47:25 +08:00
Ming 0fd5af3300 Set Kopia IgnoreUnknownTypes in ErrorHandlingPolicy to True for ignoring backup unknown file type
Signed-off-by: Ming <mqiu@vmware.com>
2023-02-07 08:30:58 +00:00
Scott Seago 5db9437f5e
Merge pull request #5540 from wenterjoy/main
change image repository by configmap
2023-02-06 20:31:47 -05:00
Scott Seago 19b855660a
Merge pull request #5828 from kaovilai/pvb-pvr-should-fail-during-startup
Use updated PVB/PVR for patching Failed Phase during startup
2023-02-06 15:08:59 -05:00
Scott Seago 53f3d13d7c
Merge pull request #5807 from shubham-pampattiwar/update-crd-desc
Update CRD descriptions for SnapshotVolumes and restorePVs
2023-02-06 14:56:28 -05:00
Xun Jiang/Bruce Jiang 218fd76411
Add mapped selected-node existence check. (#5806)
Signed-off-by: Xun Jiang <blackpiglet@gmail.com>
2023-02-06 17:03:35 +08:00
Fish-pro a761111ba1
Optimize string contrast judgment (#5821)
Signed-off-by: Fish-pro <zechun.chen@daocloud.io>
2023-02-06 14:52:01 +08:00
Fish-pro 843c70959f
Clean up unnecessary convert (#5819)
Signed-off-by: Fish-pro <zechun.chen@daocloud.io>
2023-02-06 14:50:31 +08:00
Tiger Kaovilai d7738532c8 Use updated PVB/PVR for patching Failed Phase during startup
Use the same pvb/pvr update functions across pkg/controller and pkg/cli/nodeagent for consistency of behavior

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>
2023-02-03 14:22:56 -05:00
Xun Jiang/Bruce Jiang 0b6b841f2a
Remove redundant client from restore controller. (#5759)
Signed-off-by: Xun Jiang <blackpiglet@gmail.com>
2023-02-03 15:57:43 +08:00
wenterjoy 9d1ccedd44 modified according to review comments
Signed-off-by: wenterjoy <zhaoxin26@lenovo.com>
2023-02-02 02:27:31 +00:00
Daniel Jiang 8b0afa3c44
Add option "--service-account-name" to install cmd (#5802)
The option "--service-account-name" is to be added to that user can use
an existing service account for velero and node-agent pods.  This is
helpful for users who wanna use IRSA.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2023-02-01 17:12:24 +08:00
qiuming 51568525cb
Merge pull request #5779 from allenxu404/issue-matrics
add prometheus metrics to record warning total and last status of backups
2023-01-31 16:02:30 +08:00
Shubham Pampattiwar 428415c004 update CRD descriptions for SnapshotVolumes and restorePVs
Signed-off-by: Shubham Pampattiwar <spampatt@redhat.com>

add changelog file

Signed-off-by: Shubham Pampattiwar <spampatt@redhat.com>
2023-01-31 00:01:40 -08:00
allenxu404 a5a165b0c3 add prometheus metrics to record warning total and last status of backups
Signed-off-by: allenxu404 <qix2@vmware.com>
2023-01-30 15:40:59 +08:00
Xun Jiang 955eec7033 Enable staticcheck linter. Disable check for folder /pkg and /test by now.
Signed-off-by: Xun Jiang <blackpiglet@gmail.com>
2023-01-20 13:52:13 +08:00
Shubham Pampattiwar d14879ff74
Merge pull request #5752 from sseago/itemoperations-json
Define itemoperations.json format and update DownloadRequest API.
2023-01-19 13:24:24 -05:00
lyndon b0a16ceac1
Merge pull request #5760 from Lyndon-Li/issue-fix-5043
Issue fix 5043
2023-01-19 16:44:14 +08:00
Lyndon-Li d23418b5b5 add change log
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2023-01-17 18:23:43 +08:00
Lyndon-Li 2574229fb0 add change log for 5769
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2023-01-16 19:15:49 +08:00
Lyndon-Li 2c4cfe5611 Merge branch 'main' into issue-fix-5458 2023-01-16 17:46:58 +08:00
Lyndon-Li fc98268181 Merge branch 'main' into issue-fix-5043 2023-01-12 16:54:50 +08:00
Scott Seago 9f6f13f0c5 RestoreItemAction v2 API implementation
Signed-off-by: Scott Seago <sseago@redhat.com>
2023-01-10 16:26:28 -05:00
Shubham Pampattiwar ab642ffff2
Merge pull request #5442 from sseago/biav2-impl2
BackupItemAction v2 API implementation
2023-01-10 09:34:03 -05:00
Scott Seago 9a54142257 BackupItemAction v2 API implementation
Signed-off-by: Scott Seago <sseago@redhat.com>
2023-01-09 18:36:47 -05:00
Scott Seago 70b4238013 Define itemoperations.json format and update DownloadRequest API.
This is to support uploading async operation metadata to
object storage to support progress monitoring.

Signed-off-by: Scott Seago <sseago@redhat.com>
2023-01-09 18:19:21 -05:00
lyndon c27c395d50
Merge pull request #5730 from yanggangtony/log
fix log info : pkg/util/kube/isProvisionedByCSI
2023-01-09 14:27:42 +08:00
Daniel Jiang 722aead2fd
Merge pull request #5716 from jelipo/fix-json-name
fix json serialize name
2023-01-03 13:25:53 +08:00
Daniel Jiang 78682d7cc3
Merge pull request #5710 from sseago/async-cr-phases
backup/restore phases for async plugin operations
2023-01-03 13:24:58 +08:00
yanggang 62c00ba841
fix log info : pkg/util/kube/isProvisionedByCSI
Signed-off-by: yanggang <gang.yang@daocloud.io>
2023-01-03 10:37:19 +08:00
qiuming 5b03da2637
Merge pull request #5715 from Lyndon-Li/issue-fix-5696
[Main] Fix issue 5696
2022-12-22 10:49:25 +08:00
Scott Seago 2abb176bd8 backup/restore phases for async plugin operations
Signed-off-by: Scott Seago <sseago@redhat.com>
2022-12-21 10:15:32 -05:00
haining.cao 544df59f58 fix json serialize name
Signed-off-by: haining.cao <haining.cao@daocloud.io>
2022-12-21 18:54:45 +08:00
Lyndon-Li 32eb8655cc fix issue 5696
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2022-12-21 12:49:23 +08:00
Ming 3b903e678f [Cherry-Pick]Fix error with Restic backup empty volumes
Signed-off-by: Ming <mqiu@vmware.com>
2022-12-20 06:33:22 +00:00
Lyndon-Li 88a1317f48 fix issue 5043
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2022-12-15 20:00:54 +08:00
Dylan Murray 55873c1c37
Prevent nil panic on exec restore hooks (#5675)
* Prevent nil panic on exec restore hooks

Signed-off-by: Dylan Murray <dymurray@redhat.com>
2022-12-13 13:45:52 +08:00
Lyndon-Li b51a17138e fix issue 5458
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2022-12-01 15:12:07 +08:00
Daniel Jiang 2dce4a7cb5 Update the api-types doc
This commit makes update to the update api-types docs to add missing
fields.
It also includes misc changes to the inline comment, and a change to
Dockerfile to make sure the build-image works on mac

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2022-11-25 14:39:41 +08:00
Wenkai Yin(尹文开) 8194e8d723 Restore ClusterBootstrap before Cluster
Restore ClusterBootstrap before Cluster otherwise a new default ClusterBootstrap object is create for the cluster

Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
2022-11-22 17:25:13 +08:00
Ming bf1122b633 Fix controller problematic log output
Signed-off-by: Ming <mqiu@vmware.com>
2022-11-09 06:46:45 +00:00
Lyndon-Li b06cb9ec60 remove gofrs uuid
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2022-11-07 16:28:33 +08:00
lyndon cc9d492479
Merge pull request #5485 from Obmondo/fix/error-message
Fix pointer deference to string in error message
2022-11-04 14:15:56 +08:00
Xun Jiang/Bruce Jiang 1865aab28d
Merge pull request #5535 from ywk253100/221102_order
[cherry-pick]Enhance the restore priorities list to support specifying the low prioritized resources that need to be restored in the last
2022-11-03 14:53:10 +08:00
Wenkai Yin(尹文开) c186a7d193 Enhance the restore priorities list to support specifying the low prioritized resources that need to be restored in the last
Enhance the restore priorities list to support specifying the low prioritized resources that need to be r
estored in the last

Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
2022-11-03 10:53:32 +08:00
Xun Jiang/Bruce Jiang 345abb3142
Merge pull request #5534 from qiuming-best/backup-progress
fix restic backup progress error
2022-11-02 17:15:30 +08:00
Ming 32637da16b fix restic backup progress error
Signed-off-by: Ming <mqiu@vmware.com>
2022-11-02 07:49:00 +00:00
Xun Jiang/Bruce Jiang 5db3da5aea
Merge pull request #5512 from Lyndon-Li/issue-fix-5505
Issue fix 5505
2022-11-01 19:38:39 +08:00
Xun Jiang/Bruce Jiang ae1e42cfd7
Merge pull request #5478 from Lyndon-Li/s3-compatible-provider
Issue fix 5477
2022-11-01 19:38:26 +08:00
Ming 52a49d1945 fix backup failure with self-signed certification
Signed-off-by: Ming <mqiu@vmware.com>
2022-11-01 07:05:20 +00:00
Xun Jiang 41fc641298 Add credential file store in the backup deletion controller.
Signed-off-by: Xun Jiang <blackpiglet@gmail.com>
2022-10-31 16:59:26 +08:00
Lyndon-Li 3571339fd6 issue fix 5505
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2022-10-31 11:09:46 +08:00
Xun Jiang/Bruce Jiang 150570feec
Remove redundancy code block left by #5388. (#5483)
Signed-off-by: Xun Jiang <blackpiglet@gmail.com>
2022-10-27 14:20:26 +08:00
Faizan Ahmad 9d01432007
Fix pointer deference to string in error message
Signed-off-by: Faizan Ahmad <faizan@obmondo.com>
2022-10-25 07:52:00 +05:30
Lyndon-Li 9695340c12 repo config for s3 compatible store
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2022-10-24 19:48:09 +08:00
Xun Jiang/Bruce Jiang 660841dfbd
Merge pull request #5471 from ywk253100/221021_lib
update k8s.io dependencies to 0.24.0
2022-10-24 14:46:43 +08:00
Xun Jiang/Bruce Jiang 5027aae194
Add more nil pointer check for CSI related code in backup controller. (#5388)
Add some corner cases checking for CSI snapshot in backup controller.

Signed-off-by: Xun Jiang <blackpiglet@gmail.com>
2022-10-24 10:42:08 +08:00
Kira Boyle b146a880c6 update k8s.io dependencies to 0.24.0
* This also required an update to use github.com/bombsimon/logrusr/v3
* 'WithClusterName' removed as per the k8s doc reasoning:
* https://github.com/kubernetes/apimachinery/blob/release-1.24/pkg/apis/meta/v1/types.go\#L257-L259
* ('ClusterName was a legacy field that was always cleared by the system and never used')
* Test was updated accordingly

Signed-off-by: Kira Boyle <kira@replicated.com>
2022-10-21 11:16:21 +08:00
Shubham Pampattiwar c5339227fe
increase ensure restic repository timeout (#5335)
Signed-off-by: Shubham Pampattiwar <spampatt@redhat.com>
2022-10-19 16:26:45 +08:00
Daniel Jiang fc038041fb
Merge pull request #5294 from cleverhu/change-CSISnapshotTimeout-to-normal-var
change CSISnapshotTimeout from pointer to normal variable
2022-10-19 15:31:47 +08:00
lyndon 162680b39c
Merge branch 'main' into velero-docker-image-go-1.18
Signed-off-by: lyndon <98304688+Lyndon-Li@users.noreply.github.com>
2022-10-19 09:29:36 +08:00
Shubham Pampattiwar 34cca77533 Add nil check before execution of csi snapshot delete
Signed-off-by: Shubham Pampattiwar <spampatt@redhat.com>

add changelog

Signed-off-by: Shubham Pampattiwar <spampatt@redhat.com>
2022-10-18 11:51:55 -04:00
Lyndon-Li c92f06ef17 fix lint loop iterator problem
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2022-10-18 22:40:48 +08:00
Lyndon-Li d7b4583b2b fix lint errors
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2022-10-18 21:53:47 +08:00
Xun Jiang/Bruce Jiang 9cb46deb73
Add CSI VolumeSnapshot client back. (#5449)
Signed-off-by: Xun Jiang <blackpiglet@gmail.com>
2022-10-18 14:58:03 +08:00
lyndon ad7e3ab8d5
Merge pull request #5446 from allenxu404/i5098
Change subcommand `velero restic` to `velero repo`
2022-10-17 18:28:51 +08:00
Xun Jiang/Bruce Jiang d658f6564d
Merge pull request #5444 from Lyndon-Li/remove-restic
Pod Volume Backup/Restore Refactor: Remove Restic in code
2022-10-17 15:32:17 +08:00
lyndon d52ec8c079
Pod Volume Backup/Restore Refactor: Rename Init Helper (#5432)
* restore helper refactor

Signed-off-by: Lyndon-Li <lyonghui@vmware.com>

* resolve codespell

Signed-off-by: Lyndon-Li <lyonghui@vmware.com>

Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2022-10-17 13:42:09 +08:00
danfengl cf32cabddd fix-ctx-issue
1. Fix issue of kubectl client and server mismatch version in GitAction E2E job, refer to https://github.com/elastic/cloud-on-k8s/issues/4737;
2. Adapt to the changing of keyword for involing Kpoia as fs backupper, new installtion breaked upgrade and migration tests;
3. Accept multi-labels of Ginkgo focus as input of E2E make command;
4. Distinguish workload namespace from each tests;
5. Fix issues of not using Velero util to perform Velero commands;
6. Add snapshot test case for NamespaceMapping E2E test;
7. Collect debug bundle after catching error of Velero backup or restore command;
Signed-off-by: danfengl <danfengl@vmware.com>
2022-10-16 13:18:39 +00:00
allenxu404 154f5551c6 Change subcommand restic to repo
Signed-off-by: allenxu404 <qix2@vmware.com>
2022-10-13 19:45:52 +08:00
Lyndon-Li 5f7f69366c remove restic from code
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2022-10-13 13:45:32 +08:00
Scott Seago 45de8a782f moved RIA execute input/output structs back to velero package
The RIA refactoring moved velero.RestoreItemAction into a separate
(restoreitemaction) v1 package. Unfortunately, this change would require
plugins to make code changes to locate the RestoreItemActionExecuteInput
and RestoreItemActionExecuteOutput structs.

This commit restores those structs to the original velero package, leaving
just the RestoreItemAction interface in the new v1 package.

Signed-off-by: Scott Seago <sseago@redhat.com>
2022-10-12 13:17:47 -04:00
Daniel Jiang b5b4db29cd
Merge pull request #5429 from reasonerjt/skip-additional-resources-main
Skip the exclusion check for additional resources returned by BIA
2022-10-12 09:46:09 +08:00
lyndon 83ea1cc58b
rename daemonset (#5390)
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>

Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2022-10-11 14:18:21 +08:00
Daniel Jiang 28c543a9ec Skip the exclusion check for additional resources returned by BIA
This commit provides a simple contract that if the BackupItemAction
plugin sets an annotation in a resource it has handled, the additional
items will considered "must include" i.e. each of them will skip the
"include-exclude" filter, such that the plugin developer can make sure
they are included in the backup disregarding the filter setting in the
bakcup CR.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2022-10-11 12:03:44 +08:00
Xun Jiang/Bruce Jiang f3e3cfcdaf
Merge pull request #5412 from allenxu404/i5085
Change B/R describe CLI to support kopia
2022-10-03 14:32:04 +08:00
allenxu404 47f8eb5f9b Change B/R describe cli to support kopia
Signed-off-by: allenxu404 <qix2@vmware.com>
2022-09-30 17:30:18 +08:00
Xun Jiang/Bruce Jiang a80c96c8f8
update velero using klog to version v2.9.0 (#5396)
Signed-off-by: Xun Jiang <blackpiglet@gmail.com>

Signed-off-by: Xun Jiang <blackpiglet@gmail.com>
Co-authored-by: Xun Jiang <blackpiglet@gmail.com>
2022-09-30 14:08:43 +08:00
Xun Jiang/Bruce Jiang 3f3a5050d6
Exclude "csinodes.storage.k8s.io" and "volumeattachments.storage.k8s.io" from backup and restore by default. (#5064)
Signed-off-by: Xun Jiang <jxun@vmware.com>

Signed-off-by: Xun Jiang <jxun@vmware.com>
Signed-off-by: Xun Jiang/Bruce Jiang <59276555+blackpiglet@users.noreply.github.com>
2022-09-29 15:24:06 +08:00
Xun Jiang/Bruce Jiang eec27e942e
Add backup status checking schedule controller. (#5283)
Signed-off-by: Xun Jiang <blackpiglet@gmail.com>

Signed-off-by: Xun Jiang <blackpiglet@gmail.com>
Signed-off-by: Xun Jiang/Bruce Jiang <59276555+blackpiglet@users.noreply.github.com>
Co-authored-by: Xun Jiang <blackpiglet@gmail.com>
2022-09-29 15:01:30 +08:00
qiuming eacc10347b
Fix restore error with flag namespace-mappings (#5377)
Signed-off-by: Ming <mqiu@vmware.com>
2022-09-29 11:54:51 +08:00
Xun Jiang 9693aca1f3 Fix Test_prepareBackupRequest_BackupStorageLocation UT failure.
Signed-off-by: Xun Jiang <blackpiglet@gmail.com>
2022-09-26 17:10:23 +08:00
Daniel Jiang c83447f394
Merge pull request #5362 from niulechuan/add-backup-controller-ut
Added backupController's UT to test the prepareBackupRequest() method BackupStorageLocation processing logic
2022-09-26 14:17:16 +08:00
Lyndon-Li 6c8981b0ad fix issue 5386
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2022-09-23 16:52:38 +08:00
Xun Jiang/Bruce Jiang ce247a3d90
Remove snapshot related lister, informer and client from backup controller. (#5299)
Signed-off-by: Xun Jiang <blackpiglet@gmail.com>
2022-09-23 15:44:39 +08:00
Xun Jiang/Bruce Jiang b7f5cbd0c0
Merge pull request #5318 from sseago/volumesnapshotter-refactor
plugin versioning v1 refactor for VolumeSnapshotter
2022-09-23 09:54:01 +08:00
lyndon c81f0db886
rename pvbr param (#5370)
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2022-09-23 09:13:36 +08:00
Scott Seago b6088356e6 plugin versioning v1 refactor for VolumeSnapshotter
Signed-off-by: Scott Seago <sseago@redhat.com>
2022-09-21 13:47:23 -04:00
Scott Seago 32ef20d317 RestoreItemAction v1 refactoring for plugin api versioning
Signed-off-by: Scott Seago <sseago@redhat.com>
2022-09-20 17:43:56 -04:00
qiuming 07da9b9cf8
Merge pull request #5279 from ywk253100/220829_pause
Support pause/unpause schedules
2022-09-20 16:06:17 +08:00
qiuming 648311a0f5
Merge pull request #5350 from blackpiglet/5019-snapshot-volumes-help-message-modification
Modify help message of parameter --snapshot-volumes
2022-09-20 14:18:00 +08:00
Niu Lechuan a5f1e7ac11 Added backupController's UT to test the prepareBackupRequest() method BackupStorageLocation processing logic
Signed-off-by: Niu Lechuan <lechuan.niu@daocloud.io>
2022-09-20 11:26:03 +08:00
Lyndon-Li 18bda60791 fix issue 5358
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2022-09-19 16:46:03 +08:00
Xun Jiang/Bruce Jiang 32b48d0dad
Merge pull request #5356 from Lyndon-Li/issue-fix-5352
Fix issue 5352
2022-09-19 14:47:27 +08:00
lyndon 429e204992
Merge pull request #5329 from kaovilai/veleroDownloadRequestReturnSomething-veleromain
Cancel downloadRequest when timeout without downloadURL
2022-09-19 14:20:18 +08:00
Lyndon-Li dedb3e0098 fix issue 5352
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2022-09-19 11:39:11 +08:00
lyndon ede7b197ae
Merge pull request #5297 from blackpiglet/remove_apex_log
Remove github.com/apex/log logger.
2022-09-16 16:47:36 +08:00
Xun Jiang/Bruce Jiang e42352b2e4
Merge pull request #5271 from sseago/plugin-versioning-impl
plugin/clientmgmt and framework refactoring for BackupItemAction v1
2022-09-16 16:41:33 +08:00
Xun Jiang d134783282 Clarify the help message for the default value of parameter --snapshot-volumes, when it's not set.
Signed-off-by: Xun Jiang <blackpiglet@gmail.com>
2022-09-16 16:03:05 +08:00
Xun Jiang/Bruce Jiang 4768c2acf4
Merge pull request #5314 from allenxu404/issue3565
Change the status of restore to completed from partiallyFailed when restore empty backup
2022-09-16 15:00:36 +08:00
Scott Seago e3e2a8dfa0 plugin/framework refactoring for BackupItemAction v1
Refactors the framework  package to implement the plugin versioning changes
needed for BIA v1 and overall package refactoring to support plugin versions
in different packages. This should be all that's needed to move on to
v2 for BackupItemAction. The remaining plugin types still need similar
refactoring to what's being done here for BIA before attempting a
v2 implementation.

Signed-off-by: Scott Seago <sseago@redhat.com>
2022-09-15 11:15:30 -04:00
Scott Seago 4262b47536 plugin/clientmgmt refactoring for BackupItemAction v1
Refactors the clientmgmt package to implement the plugin versioning changes
needed for BIA v1 and overall package refactoring to support plugin versions
in different packages. This should be all that's needed to move on to
v2 for BackupItemAction. The remaining plugin types still need similar
refactoring to what's being done here for BIA before attempting a
v2 implementation.

Signed-off-by: Scott Seago <sseago@redhat.com>
2022-09-15 11:15:30 -04:00
Shubham Pampattiwar 1e138af1cf
Merge pull request #5263 from sseago/biav1-refactor
Don't move velero v1 plugins to new proto dir
2022-09-15 11:14:05 -04:00
Xun Jiang/Bruce Jiang 745ebbe081
Merge pull request #5347 from qiuming-best/fix-restic-resotre
Fix restore cmd extraflag overwrite bug
2022-09-15 15:37:15 +08:00
Xun Jiang/Bruce Jiang d0954dddd4
Merge pull request #5322 from qiuming-best/fix-get-parent-snapshot
Fix PVB finds wrong parent snapshot
2022-09-15 15:23:27 +08:00
Ming 4022020d5f Fix restore cmd extraflag overwrite bug
Signed-off-by: Ming <mqiu@vmware.com>
2022-09-15 06:09:41 +00:00
Xun Jiang/Bruce Jiang a05fc498b1
Merge pull request #5291 from cleverhu/add-shorthand-for-labels-columns
Add shorthand for labels columns
2022-09-15 11:24:06 +08:00
Wenkai Yin(尹文开) 4b9dbfa416 Support pause/unpause schedules
Support pause/unpause schedule

Fixes #2363

Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
2022-09-15 10:42:48 +08:00
Tiger Kaovilai 2c759f395a cancel downloadRequest checkFunc if timeout passed
Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>
2022-09-14 14:13:09 -04:00
lyndon 100d6b4430
Merge pull request #5319 from Lyndon-Li/issue-fix-4874
Issue fix 4874 and 4752
2022-09-14 11:04:56 +08:00
Lyndon-Li fbb2606102 issue fix 4874 update
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2022-09-13 11:15:52 +08:00
Shubham Pampattiwar be40d7eb19
Merge pull request #4864 from sseago/vsl-creds-main
Add credentials to volume snapshot locations.
2022-09-12 09:56:49 -04:00
Ming c612853bd5 Fix PVB finds wrong parent snapshot
Signed-off-by: Ming <mqiu@vmware.com>
2022-09-09 08:30:57 +00:00
lyndon 30a70cbd0d
Merge branch 'vmware-tanzu:main' into issue-fix-4874 2022-09-09 09:20:38 +08:00
Xun Jiang/Bruce Jiang 325b8c0d05
Merge pull request #5308 from Lyndon-Li/repository-ensurer-refactor
Repo Ensurer Refactor
2022-09-08 21:17:31 +08:00
Scott Seago be0a1cf361
Merge pull request #5293 from cleverhu/optimize-restore
optimize code for restore exists resources
2022-09-08 09:01:44 -04:00
Scott Seago 596114b427 Add credentials to volume snapshot locations.
This is analogous to the BSL creds work that was done previously, but for VSLs instead.

Signed-off-by: Scott Seago <sseago@redhat.com>
2022-09-08 08:59:17 -04:00
allenxu404 dc70471909 Change the status of restore to completed from partially failed when restore empty backup
Signed-off-by: allenxu404 <qix2@vmware.com>
2022-09-08 18:40:49 +08:00
Lyndon-Li 8496b43e37 refactor repo ensurer
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2022-09-08 16:10:51 +08:00
Lyndon-Li a12024887f fix issue 4874
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2022-09-07 21:44:45 +08:00
lyndon 8888f8765e
Merge pull request #5282 from Lyndon-Li/kopia-pvbr-smoking-test
Kopia Pod Volume Backup/Restore Smoking Test Changes
2022-09-07 14:20:44 +08:00
Lyndon-Li a90ba3db7c kopia pvbr smoking test changes
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2022-09-07 13:22:49 +08:00
Xun Jiang b49e39c021 Remove github.com/apex/log logger.
Signed-off-by: Xun Jiang <blackpiglet@gmail.com>
2022-09-07 08:52:16 +08:00
cleverhu 7de6f2a2fc trim isAlreadyExistsError for restore
Signed-off-by: cleverhu <shouping.hu@daocloud.io>
2022-09-07 03:55:20 +08:00
cleverhu 067a3ec03a change CSISnapshotTimeout from point to normal var
Signed-off-by: cleverhu <shouping.hu@daocloud.io>
2022-09-07 01:30:47 +08:00
cleverhu 5e6111e6c0 add shorthand for labels columns
Signed-off-by: cleverhu <shouping.hu@daocloud.io>
2022-09-06 20:14:13 +08:00
Scott Seago b15c59ba69 Don't move velero v1 plugins to new proto dir
To preserve backwards compatibility, don't move velero
v1 plugins to new proto dir.

Signed-off-by: Scott Seago <sseago@redhat.com>
2022-09-05 16:01:56 -04:00
Daniel Jiang c7bd2b9c02
Merge pull request #5252 from cleverhu/add-csiSnapshotTimeout-for-describe
add CSISnapshotTimeout for describe backup
2022-09-05 14:44:36 +08:00
Daniel Jiang 78b4914661
Merge pull request #5261 from Abirdcfly/main
chore: remove duplicate word in comments
2022-09-05 14:32:47 +08:00
lyndon 0282e65221
Kopia Pod Volume Backup/Restore (#5259)
* kopia pvbr

Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2022-09-05 10:29:30 +08:00
qiuming a5a3df193d
Merge pull request #5236 from sseago/no-namespaces-fix
Check for empty ns list before checking nslist[0]
2022-09-05 09:40:19 +08:00
lyndon 5f1bf9eb35
Merge pull request #5214 from qiuming-best/uploader-restic
Uploader Implementation: Restic backup and restore
2022-09-02 11:11:38 +08:00
Ming eb974687a7 merge upstream code
Signed-off-by: Ming <mqiu@vmware.com>
2022-09-02 01:45:12 +00:00
Daniel Jiang 218bab987d
Merge pull request #5239 from shubham-pampattiwar/fix-already-exists
Fix edge cases for already exists resources
2022-08-31 23:01:56 +08:00
Shubham Pampattiwar 94a9a7c795
Merge pull request #4943 from phuongatemc/refactor_plugin_biav1
Refactor BackupItemAction to backupitemaction/v1
2022-08-30 09:48:26 -04:00
Xun Jiang/Bruce Jiang 6fea973c57
Migrate backup sync controller from code-generator to kubebuilder (#5218)
* Migrate backup sync controller from code-generator to kubebuilder

1. use kubebuilder's reconcile logic to replace controller's old logic.
2. use ginkgo and gomega to replace testing.
3. modify BSL reconciler registration method.

Signed-off-by: Xun Jiang <blackpiglet@gmail.com>

* Add Option for PeriodicalEnqueueSource.

Signed-off-by: Xun Jiang <blackpiglet@gmail.com>

Signed-off-by: Xun Jiang <blackpiglet@gmail.com>
Co-authored-by: Xun Jiang <blackpiglet@gmail.com>
2022-08-30 16:55:39 +08:00
Abirdcfly fab86caa2f
chore: remove duplicate word in comments
Signed-off-by: Abirdcfly <fp544037857@gmail.com>
2022-08-30 16:34:23 +08:00
Xun Jiang/Bruce Jiang 4bc3a3a784
Merge pull request #5248 from allenxu404/gc-freq
equip gc controller with configurable frequency
2022-08-30 11:12:12 +08:00
allenxu404 a8ba4875f0 equip gc controller with configurable frequency
Signed-off-by: allenxu404 <qix2@vmware.com>
2022-08-30 10:13:06 +08:00
qiuming c8818ec1c9
Merge pull request #5241 from blackpiglet/update_enabled_runtime_controllers
Controller refactor code modifications.
2022-08-30 09:30:35 +08:00
Shubham Pampattiwar 93a875873b fix edge cases for already exists resources
Signed-off-by: Shubham Pampattiwar <spampatt@redhat.com>

add changelog file

Signed-off-by: Shubham Pampattiwar <spampatt@redhat.com>

update changelog filename

Signed-off-by: Shubham Pampattiwar <spampatt@redhat.com>

change log level and error type

Signed-off-by: Shubham Pampattiwar <spampatt@redhat.com>
2022-08-29 17:34:02 -04:00
Scott Seago 91ac570d81 some additional debug logs
Signed-off-by: Scott Seago <sseago@redhat.com>
2022-08-29 16:55:14 -04:00
Scott Seago 71648750cc
Merge pull request #5247 from divolgin/storage-class-panic
Don't panic when storageClassName is not set in stateful sets
2022-08-29 10:06:32 -04:00
Xun Jiang/Bruce Jiang eaf9fab711
Merge branch 'main' into update_enabled_runtime_controllers 2022-08-29 20:06:31 +08:00
Ming ed71e65486 Merge branch 'main' of https://github.com/qiuming-best/velero into uploader-restic 2022-08-29 11:33:59 +00:00
Ming f15757a3d8 Uploader Implementation: Restic backup and restore
Signed-off-by: Ming <mqiu@vmware.com>
2022-08-29 09:33:37 +00:00
Wenkai Yin(尹文开) 4a5647a891 Move the code in pkg/restic/common.go to the proper package
Move the code in pkg/restic/common.go to the proper package

Fixes #5243

Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
2022-08-29 10:04:58 +08:00
qiuming 3e30a3d388
Merge pull request #5233 from Lyndon-Li/udmrepo-dev-06
Kopia Integration:Kopia Lib - Implementation
2022-08-28 22:33:12 +08:00
cleverhu e77aaa32ca add CSISnapshotTimeout for describe backup
Signed-off-by: cleverhu <shouping.hu@daocloud.io>
2022-08-28 15:14:03 +08:00
Xun Jiang/Bruce Jiang 86762f442a
Merge pull request #5217 from lilongfeng0902/lilf-branch
check vsc null pointer
2022-08-27 10:45:13 +08:00
Lyndon-Li 3769cd218a kopia lib
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2022-08-26 15:25:20 +08:00
Scott Seago 5a5a4c184e Updated plugin/framework server files to cope with protoc changes
Signed-off-by: Scott Seago <sseago@redhat.com>
2022-08-25 21:28:07 -04:00
Scott Seago 4f2c2d2679 updated to newer protoc/protoc-gen-go
Signed-off-by: Scott Seago <sseago@redhat.com>
2022-08-25 21:28:03 -04:00
Hoang, Phuong 2c037b7491 Add missing file
Signed-off-by: Hoang, Phuong <phuong.n.hoang@dell.com>
2022-08-25 17:10:14 -04:00
Hoang, Phuong b54424bdc6 Refactor BackupItemAction to backupitemaction/v1
Signed-off-by: Hoang, Phuong <phuong.n.hoang@dell.com>
2022-08-25 17:10:09 -04:00
divolgin ad4e733ef2 Don't panic when storageClassName is not set in stateful sets
Signed-off-by: divolgin <dmitriy@replicated.com>
2022-08-25 15:02:40 +00:00
Scott Seago 55bf2de15d Check for empty ns list before checking nslist[0]
In determining whether a backup includes all namespaces, item_collector
checks for an empty string in the first element of the ns list. If processing
includes+excludes results in an empty list, treat this as another case
of a not-all-namespaces backup rather than crashing velero.

Signed-off-by: Scott Seago <sseago@redhat.com>
2022-08-24 14:32:19 -04:00
Xun Jiang/Bruce Jiang e8494418d4
Merge pull request #5215 from allenxu404/issue5023
Refactor GC controller with kubebuilder
2022-08-23 15:59:03 +08:00
Xun Jiang 71e5027bfb Controller refactor code modifications.
1. Add some refactored controllers initiation code into enabledRuntimeControllers.
2. Add reconciler struct initiation function for DownloadRequest and ServerStatusRequest controllers.

Signed-off-by: Xun Jiang <blackpiglet@gmail.com>
2022-08-23 10:32:19 +08:00
Ming 2bf054ad0b update upstream codes
Signed-off-by: Ming <mqiu@vmware.com>
2022-08-22 09:20:47 +00:00
Tiger Kaovilai 0bee6dd9fd gh-issue-template: Comment out the subheading description
This stops subheading description from showing in posted issues by default.

Signed-off-by: Tiger Kaovilai <passawit.kaovilai@gmail.com>
Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>
2022-08-22 02:59:35 -04:00
李龙峰 5cddaeae6c check vsc null pointer
Signed-off-by: 李龙峰 <lilongfeng@inspur.com>
2022-08-22 14:44:17 +08:00
allenxu404 0378020c8d refactor gc controller with kubebuilder
Signed-off-by: allenxu404 <qix2@vmware.com>
2022-08-22 12:16:56 +08:00
Ming 839c2ed98f Merge branch 'main' of https://github.com/qiuming-best/velero into uploader-kopia 2022-08-19 06:40:58 +00:00
Ming 262de19f52 Uploader Implementation: Kopia backup and restore
Signed-off-by: Ming <mqiu@vmware.com>
2022-08-19 06:39:28 +00:00
Lyndon-Li d58abb2477 repo init
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2022-08-18 19:45:49 +08:00
lyndon 775943c858
code spell check (#5230)
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>

Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2022-08-18 18:17:39 +08:00
Wenkai Yin(尹文开) 047c7531fa Refactor the restic repo related code for Kopia integration
Refactor the restic repo related code for Kopia integration

Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
2022-08-17 11:49:53 +08:00
Daniel Jiang 4e25f59dc1
Add parameter "uploader-type" to velero server (#5212)
This commit adds the parameter "uploader-type" to velero server, add exposes the
setting via "velero install" in CLI.

fixes #5062

Signed-off-by: Daniel Jiang <jiangd@vmware.com>

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2022-08-16 13:50:28 +08:00
qiuming 71e4430840
Merge pull request #5179 from Lyndon-Li/udmrepo-dev-02
Kopia Integration: Unified Repository Provider - Implementation
2022-08-16 09:39:40 +08:00
qiuming 36d8d176dd
Merge pull request #5163 from qiuming-best/uploader
kopia uploader integration on shim progress uploader module
2022-08-11 17:38:21 +08:00
Ming 49e151739f Kopia Uploader integration on shim progress uploader
Signed-off-by: Ming <mqiu@vmware.com>
2022-08-10 11:35:24 +00:00
danfengliu a71237cc64
Merge pull request #5124 from danfengliu/add-migration-e2e-test
Add migration E2E test
2022-08-10 13:38:32 +08:00
Lyndon-Li 649c3a77df unified repo provider impl
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2022-08-09 20:48:31 +08:00
danfengl fb445b3c0d Add migration E2E test
Signed-off-by: danfengl <danfengl@vmware.com>
2022-08-09 08:31:39 +00:00
Xun Jiang/Bruce Jiang a5f4f8f9fc
Merge branch 'main' into 5048-CSI-snapshot-timeout-configurable 2022-08-08 17:18:44 +08:00
Xun Jiang f8d9cfdb84 Make CSI snapshot creation timeout configurable for backup and schedule.
Signed-off-by: Xun Jiang <jxun@vmware.com>
2022-08-08 17:07:26 +08:00
Xun Jiang/Bruce Jiang 201c43d683
Merge pull request #5178 from allenxu404/i2413
fix issue#2413: treat namespaces with exclude label as excludedNamespaces
2022-08-08 13:35:38 +08:00
allenxu404 e5d828a2a4 modify variables defination and expose err
Signed-off-by: allenxu404 <qix2@vmware.com>
2022-08-08 12:05:08 +08:00
Daniel Jiang ba50458ae2
Merge pull request #5181 from blackpiglet/restic-volume-path-change
Add annotation "pv.kubernetes.io/migrated-to" for CSI checking.
2022-08-08 12:03:47 +08:00
Xun Jiang 7af1e23614 Add annotation "pv.kubernetes.io/migrated-to" for CSI checking.
1. Also checking annotation "pv.kubernetes.io/migrated-to" to find out whether volume is provisioned by CSI.
2. Add UT cases.

Signed-off-by: Xun Jiang <jxun@vmware.com>
2022-08-08 10:59:41 +08:00
Xun Jiang/Bruce Jiang cea5e7f218
Merge pull request #5174 from blackpiglet/reduce-crd-size-new
Reduce CRD size.
2022-08-05 18:46:29 +08:00
Wenkai Yin(尹文开) 092fc01e8d
Splic pkg/restic package (#5143)
This commit splits the pkg/restic package into several packages to support Kopia integration works

Fixes #5055

Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
2022-08-05 17:15:38 +08:00
allenxu404 eb08bdeb62 fix issue#2413 by treating namespaces with exclude label as excludedNamespaces
Signed-off-by: allenxu404 <qix2@vmware.com>
2022-08-04 20:00:11 +08:00
lyndon 088eb9b83c
repo credentials (#5167)
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2022-08-04 15:20:02 +08:00
qiuming 701256d296
Merge pull request #5172 from qiuming-best/multi-bsl-fix
Fix restic backups to multiple backup storage locations bug
2022-08-03 14:35:42 +08:00
Xun Jiang e8da5df57a Reduce CRD size.
1. Make the Restore hook.InitConatianer server side field pruing disable.
2. Remove restore patch in update-generate-crd-code.sh.
3. Modify related testcases.
4. Add Container fields validation in Restore Init hook.

Signed-off-by: Xun Jiang <jxun@vmware.com>
2022-08-02 19:45:01 +08:00
Ming 828e28aa43 Fix restic backups to multiple backup storage locations bug
Signed-off-by: Ming <mqiu@vmware.com>
2022-08-02 10:26:23 +00:00
Daniel Jiang 108c81d84c
Skip registering "crd-remap-version" plugin when feature flag (#5165)
"EnableAPIGroupVersions" is set

The crd-remap-version plugin will always backup v1b1 resource for some
CRD. It impacts the feature flag `EnableAPIGroupVersions` which means to
backup all versions, and make migration fail.

In this commit the featureSet was removed from plugin server struct b/c
it blocks the parm `--features` to be populated correctly.  This change
should not have negative impact b/c the attribute in server struct is never used.

Fixes #5146

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2022-08-02 15:47:57 +08:00
Daniel Jiang 2b0d0959da
Merge pull request #5148 from blackpiglet/vsl-refactor
VolumeSnapshotLocation refactor with kubebuilder.
2022-08-01 21:14:18 +08:00
lyndon 52fd18e9db
Kopia Integration Change - Storage Configuration (#5142)
* unified repo storge config

Signed-off-by: Lyndon-Li <lyonghui@vmware.com>

* add UT

Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2022-07-29 16:06:18 +08:00
qiuming f68ae92fd0
Merge pull request #5145 from blackpiglet/5140-fix-restic-backup-with-customize-CA
Delay CA file deletion in PVB controller
2022-07-26 10:32:59 +08:00
Xun Jiang 396e68b810 VolumeSnapshotLocation refactor with kubebuilder.
1. modify VSL CRD API file name. Add kubebuilder related marker.
2. Add Labels init code for VSL create command.

Signed-off-by: Xun Jiang <jxun@vmware.com>
2022-07-25 15:46:43 +08:00
Daniel Jiang b5583bc2d9
Merge pull request #5129 from kaovilai/initialize-typo
comment typo initialise -> initialize
2022-07-25 13:45:00 +08:00
Xun Jiang 82ac228a01 Delay CA file deletion in PVB controller
Fix #5140.

Signed-off-by: Xun Jiang <jxun@vmware.com>
2022-07-22 19:36:17 +08:00
Daniel Jiang f562a7ce2b Update the CRD for kopia integration
This commit adds additional fields to podvolumebackup
and podvolumerestore.  The resticrepository will be renamed to
backuprepository

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2022-07-22 00:10:24 +08:00
Tiger Kaovilai 9a5c3aceff typo initialise -> initialize
Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>
2022-07-18 12:05:10 -04:00
Scott Seago 64a8c44104 Modify BackupStoreGetter to avoid BSL spec changes
Pass in a new copy of the map of config values rather than
modifying the BSL Spec.Config and then pass in that field.

Signed-off-by: Scott Seago <sseago@redhat.com>
2022-07-18 10:31:35 -04:00
Xun Jiang/Bruce Jiang eaf97e7510
Merge pull request #5110 from reasonerjt/plugin-panic-trace
Dump stack trace when the plugin server handles panic
2022-07-11 22:47:17 +08:00
Daniel Jiang 9102f53131 Dump stack trace when the plugin server handles panic
Mitigate the issue mentioned in #4782
When there's a bug or misconfiguration that causes nil pointer there
will be more stack trace information to help us debug.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2022-07-11 02:01:13 +08:00
Scott Seago 77c1549d4d
Merge pull request #5101 from ywk253100/220628_bsl
Fix bsl validation bug
2022-07-08 15:32:51 -04:00
Wenkai Yin(尹文开) f550f8e3cd Fix bsl validation bug
Fix bsl validation bug: the BSL is validated continually and doesn't respect the validation period configured

Fixes #5056

Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
2022-07-06 14:20:03 +08:00
niulechuan fb897471c0
Move 'velero.io/exclude-from-backup' label string to const (#5053)
* move 'velero.io/exclude-from-backup' label name to const

Signed-off-by: Niu Lechuan <lechuan.niu@daocloud.io>

* add changelog file (in changelogs/unreleased) of this PR

Signed-off-by: Niu Lechuan <lechuan.niu@daocloud.io>
2022-07-04 14:31:20 +08:00
Scott Seago 6378c266c3
Merge pull request #4980 from anshulahuja98/main
Add support to download CSI json from object store
2022-06-16 10:37:55 -04:00
Scott Seago 79be75e183 When spec.RestoreStatus is empty, don't restore status
Signed-off-by: Scott Seago <sseago@redhat.com>
2022-06-15 13:02:31 -04:00
Anshul Ahuja 7b51bfe03d
Merge branch 'main' into main 2022-06-15 14:16:38 +05:30
Anshul Ahuja a0ff46a3de Add kubebuilder annotation for autogen
Signed-off-by: GitHub <noreply@github.com>
2022-06-13 13:28:26 +00:00
Anshul Ahuja 735d506a7d Add support to download CSI json from object store
Signed-off-by: GitHub <noreply@github.com>
2022-06-13 13:28:26 +00:00
Xun Jiang 806234e043 Add more information for failing to get path or snapshot in restic backup and restore.
Signed-off-by: Xun Jiang <jxun@vmware.com>
2022-06-13 11:32:55 +08:00
Wenkai Yin(尹文开) d16bdf1f80 Bugs fixing: use patch to update status and set default GC period
1. Use patch rather status patch in backup sync controller as we have disable status as sub resource
2. Set the GC period with default value if it isn't set

Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
2022-06-10 17:15:19 +08:00
Wenkai Yin(尹文开) c3a32a116f
Merge pull request #4976 from reasonerjt/vsc-policy-reset-undo
Undo setting the VSC's deletion policy during backup
2022-06-10 09:12:46 +08:00
Daniel Jiang f5649bcc1f Undo setting the VSC's deletion policy during backup
It's not necessary to set the deletion policy as the delete item action
plugin in CSI plugin will set it to Delete when the backup is deleted.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2022-06-09 16:26:05 +08:00
Wenkai Yin(尹文开) 34087fe5f4 Disable status as sub resource in CRDs
When enabling the status as sub resource in CRD, the status will be ignored when creating the CR with status, this will cause issues when syncing backups/pvbs

Fixes #4950

Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
2022-06-09 07:52:08 +08:00
Wenkai Yin(尹文开) 6fa4d7d606 Mark in-progress CRs as failed when starting the server
Mark in-progress CRs as failed when starting the server

Fixes #4953

Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
2022-06-07 17:02:17 +08:00
Scott Seago a79c1a5877
Merge pull request #4785 from RafaeLeal/restore-status
Add ability to restore status on selected resources
2022-05-24 09:41:18 -04:00
Rafael Leal e0e3016efa
Cleanup resetStatus
Signed-off-by: Rafael Leal <rafaelealdias@gmail.com>
2022-05-23 17:53:32 -03:00
Scott Seago 471e357952
Merge pull request #4914 from blackpiglet/4912-fix-completion-zsh
Make velero completion zsh command output can be used by `source` com…
2022-05-23 10:05:12 -04:00
Rafael Leal 131c8a920f
Remove break line
Signed-off-by: Rafael Leal <rafaelealdias@gmail.com>
2022-05-19 17:36:21 -03:00
Rafael Leal bcef5e1d5e
Refactor to simplify
Signed-off-by: Rafael Leal <rafaelealdias@gmail.com>
2022-05-17 10:39:14 -03:00
Wenkai Yin(尹文开) 44199db79d Enhance the map flag to support parsing input value contains entry delimiters
Enhance the map flag to support parsing input value contains entry delimiters

Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
2022-05-17 13:56:41 +08:00
Xun Jiang 3db40a58a4 Make velero completion zsh command output can be used by `source` command.
Signed-off-by: Xun Jiang <jxun@vmware.com>
2022-05-17 10:01:46 +08:00
Rafael Leal d85ed612cb
Cleanup
Signed-off-by: Rafael Leal <rafaelealdias@gmail.com>
2022-05-12 12:01:00 -03:00
Rafael Leal 206709b978
Cleanup
Signed-off-by: Rafael Leal <rafaelealdias@gmail.com>
2022-05-12 12:01:00 -03:00
Rafael Leal 8ecc11fad2
Add restore status to cli
Signed-off-by: Rafael Leal <rafaelealdias@gmail.com>
2022-05-12 12:00:34 -03:00
Rafael Leal 7f22974719
Update CRDs
Signed-off-by: Rafael Leal <rafaelealdias@gmail.com>
2022-05-12 11:59:49 -03:00
Rafael Leal 04aa7a849f
Fixes tests hopefully
Signed-off-by: Rafael Leal <rafaelealdias@gmail.com>
2022-05-12 11:59:05 -03:00
Rafael Leal 0b9d6ae73d
Add restore status mechanism
Signed-off-by: Rafael Leal <rafaelealdias@gmail.com>
2022-05-12 11:59:05 -03:00
Rafael Leal 7f1f881c28
Add resource status
Signed-off-by: Rafael Leal <rafaelealdias@gmail.com>
2022-05-12 11:58:47 -03:00
Rafael Leal e374eb9da4
Add StatusUpdater
Signed-off-by: Rafael Leal <rafaelealdias@gmail.com>
2022-05-12 11:57:41 -03:00
Rafael Leal 6a721403fb
Fix DeepEqual when status is updated
Signed-off-by: Rafael Leal <rafaelealdias@gmail.com>
2022-05-12 10:47:24 -03:00
Daniel Jiang 9b5257663a
Merge pull request #4650 from shubham-pampattiwar/add-label-selectors
Add multiple label selector support to Velero Backup and Restore APIs
2022-05-11 23:16:28 +08:00
Shubham Pampattiwar bfdb68a35a add multiple label selector support to backup API
Signed-off-by: Shubham Pampattiwar <shubhampampattiwar7@gmail.com>

remove backup CLI bits

Signed-off-by: Shubham Pampattiwar <shubhampampattiwar7@gmail.com>

labelselectors spec option for velero restore

Signed-off-by: Shubham Pampattiwar <shubhampampattiwar7@gmail.com>

add changelog file

Signed-off-by: Shubham Pampattiwar <shubhampampattiwar7@gmail.com>

update spec name to OrLabelSelectors

Signed-off-by: Shubham Pampattiwar <shubhampampattiwar7@gmail.com>

minor fixes

Signed-off-by: Shubham Pampattiwar <shubhampampattiwar7@gmail.com>

add validations for labelSelector and orLabelSelectors

Signed-off-by: Shubham Pampattiwar <shubhampampattiwar7@gmail.com>

update crds.gp after fixing conflicts

Signed-off-by: Shubham Pampattiwar <shubhampampattiwar7@gmail.com>

fix CI and add unit tests

Signed-off-by: Shubham Pampattiwar <shubhampampattiwar7@gmail.com>

updated OrLabelSelector spec description and added validation failure unit tests

Signed-off-by: Shubham Pampattiwar <shubhampampattiwar7@gmail.com>

add comments and change log level

Signed-off-by: Shubham Pampattiwar <shubhampampattiwar7@gmail.com>

update site docs

Signed-off-by: Shubham Pampattiwar <shubhampampattiwar7@gmail.com>

wrap pager client calls in a function

Signed-off-by: Shubham Pampattiwar <shubhampampattiwar7@gmail.com>

resolve confilcts and update crds.go

Signed-off-by: Shubham Pampattiwar <shubhampampattiwar7@gmail.com>

rebase and update crds.go

Signed-off-by: Shubham Pampattiwar <shubhampampattiwar7@gmail.com>

combine listing items for a given label

Signed-off-by: Shubham Pampattiwar <shubhampampattiwar7@gmail.com>
2022-05-11 09:57:47 -04:00
Wenkai Yin(尹文开) 879d03398b
Merge pull request #4894 from blackpiglet/bsl-refactor
Refactor BSL controller with periodical enqueue source
2022-05-11 19:23:51 +08:00
Xun Jiang 989a1e3ebb Refactor BSL controller with periodical enqueue source.
Add filter functions for PeriodicalEnqueueSource.
Move BSL's valication frequency check test case to PeriodicalEnqueueSource's test.

Signed-off-by: Xun Jiang <jxun@vmware.com>
2022-05-11 15:31:47 +08:00
qiuming d48e1d9db7
Merge pull request #4897 from ywk253100/220609_gc
Make garbage collection for expired backups configurable
2022-05-11 10:46:03 +08:00
Wenkai Yin(尹文开) 6a551e546e Make garbage collection for expired backups configurable
Make garbage collection for expired backups configurable

Fixes #4875

Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
2022-05-11 10:33:59 +08:00
Shubham Pampattiwar e3106f3e10 add existingResourcePolicy to Restore API
add updateall policy option

fix updating labels

dump updateAll policy option

remove updateAll policy refs

Signed-off-by: Shubham Pampattiwar <shubhampampattiwar7@gmail.com>

add changelog file

Signed-off-by: Shubham Pampattiwar <shubhampampattiwar7@gmail.com>

update docs

Signed-off-by: Shubham Pampattiwar <shubhampampattiwar7@gmail.com>

patch labels for sa if policy is update

Signed-off-by: Shubham Pampattiwar <shubhampampattiwar7@gmail.com>

fix existingResourcePolicy for serviceaccounts

modularize changes and add unit tests

Signed-off-by: Shubham Pampattiwar <shubhampampattiwar7@gmail.com>

fix conflicts and update crds.go

Signed-off-by: Shubham Pampattiwar <shubhampampattiwar7@gmail.com>

change log level from info to error

Signed-off-by: Shubham Pampattiwar <shubhampampattiwar7@gmail.com>

update crds.go

Signed-off-by: Shubham Pampattiwar <shubhampampattiwar7@gmail.com>
2022-05-09 13:36:24 -04:00
Xun Jiang/Bruce Jiang 40a36d2941
Merge pull request #4859 from qiuming-best/restic-repo-tmp
Convert Restic Repository resource/controller to the Kubebuilder framework
2022-05-09 17:57:01 +08:00
Wenkai Yin(尹文开) 1d8d2bdb4f
Merge pull request #4890 from sseago/restoreitemaction-label-fix
continue rather than return for non-matching restore action label
2022-05-09 14:34:03 +08:00
Ming ef7e5a8388 change func name and repository description
Signed-off-by: Ming <mqiu@vmware.com>
2022-05-09 13:57:02 +08:00
Xun Jiang/Bruce Jiang 62dde34f86
Merge pull request #4893 from ywk253100/220506_restart
Make in-progress PVB/PVR as failed when restic controller restarts to avoid hanging backup/restore
2022-05-07 17:32:40 +08:00
Wenkai Yin(尹文开) 034cda2571 Make in-progress PVB/PVR as failed when restic controller restarts to avoid hanging backup/restore
Make in-progress PVB/PVR as failed when restic controller restarts to avoid hanging backup/restore

Fixes #4772

Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
2022-05-07 16:51:06 +08:00
Ming 752b28166f Merge branch 'main' of https://github.com/qiuming-best/velero into restic-repo-tmp 2022-05-06 14:43:00 +08:00
Scott Seago da916846b1 continue rather than return for non-matching restore action label
When iterating over applicable restore actions, if a non-matching label
selector is found, velero should continue to the next action rather than
returning from the restoreItem func, which ends up preventing the item's
restore entirely.

Signed-off-by: Scott Seago <sseago@redhat.com>
2022-05-05 08:37:48 -04:00
Xun Jiang a69bd8b1f3 Remove VolumeSnapshot created during backup, when the VolumeSnapshotClass's DeletionPolicy is set to Delete.
1. Delete VolumeSnapshot directly when DeletionPolicy set to Retain.
2. Change VolumeSnapshotContent's DeletionPolicy to Retain, then delete VolumeSnapshot. After that delete VolumeSnapshotContent and change VSC DeletionPolicy to Delete back, then re-create the VolumeSnapshotContent.

Signed-off-by: Xun Jiang <jxun@vmware.com>
2022-05-05 18:30:59 +08:00
Xun Jiang 6421f8ad02 Make waiting VolumeSnapshot to ready process parallel.
Signed-off-by: Xun Jiang <jxun@vmware.com>
2022-05-05 14:32:55 +08:00
Wenkai Yin(尹文开) aa71427576
Merge pull request #4887 from reasonerjt/delete-orphan-vs
Delete orphan CSI snapshots in backup sync controller
2022-05-05 11:27:44 +08:00
Daniel Jiang 89e90d9be4 Delete orphan CSI snapshots in backup sync controller
This commit makes backup sync controller delete the volumesnapshot and
volumesnapshotcontent created by the backup which is cleaned up as orphan

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2022-05-05 10:54:34 +08:00
Daniel Jiang 628164668f
Merge pull request #4655 from ywk253100/220214_pod_volume_restore
Convert Pod Volume Restore resource/controller to the Kubebuilder framework
2022-05-05 10:33:41 +08:00
Ming 06d3d731ed Refactor restic repository
Signed-off-by: Ming <mqiu@vmware.com>
2022-04-29 10:26:00 +08:00
Daniel Jiang ad6c5a5ad6 Change the name of UT to be more consistent
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2022-04-28 14:47:37 +08:00
Xun Jiang/Bruce Jiang 001229a8b3
Merge pull request #4855 from reasonerjt/bak-delete-refact
Refactor backup deletion controller based on kubebuilder
2022-04-28 10:48:11 +08:00
Wenkai Yin(尹文开) d42e353610
Merge pull request #4866 from reasonerjt/clusterclass-restore-order
Add ClusterClasses to the restore priority list
2022-04-28 09:04:41 +08:00
Scott Seago 58a8371ca2
Merge pull request #4833 from ywk253100/220413_restart
Make in-progress backup/restore as failed when doing the reconcile
2022-04-27 11:45:01 -04:00
Daniel Jiang 74fd158a1c Refactor backup deletion controller based on kubebuilder
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2022-04-27 17:00:54 +08:00
Daniel Jiang f89691c5c1 Add ClusterClasses to the restore priority list
Make sure ClusterClasses are stored before Clusters.
Fixes #4767

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2022-04-27 15:53:55 +08:00
Wenkai Yin(尹文开) c2c211fefb Convert Pod Volume Restore resource/controller to the Kubebuilder framework
Convert Pod Volume Restore resource/controller to the Kubebuilder framework

Fixes #4134

Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
2022-04-26 16:23:38 +08:00
Wenkai Yin(尹文开) dfc86566b8 Make in-progress backup/restore as failed when doing the reconcile
Make in-progress backup/restore as failed when doing the reconcile to avoid hanging in in-progress status

Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
2022-04-26 11:05:03 +08:00
Xun Jiang 22dd0b2ebb Modify CSI VolumeSnapshot metric related code.
Signed-off-by: Xun Jiang <jxun@vmware.com>
2022-04-23 12:59:56 +08:00
Daniel Jiang 3ec96e2eac
Merge pull request #4838 from ywk253100/220418_marker
Use controller-gen to generate the deep copy methods for objects
2022-04-21 14:00:48 +08:00
Xun Jiang/Bruce Jiang ff4f63b6e3
Merge pull request #4844 from ywk253100/220418_schedule
Return the error when running reconcile to trigger the handling logic again
2022-04-20 17:57:20 +08:00
Wenkai Yin(尹文开) 9e786d681b
Merge pull request #4832 from reasonerjt/fix-rm-csi-bak-tmp
Make the vsc created by backup sync controller deletable
2022-04-20 17:19:09 +08:00
Scott Seago de17605c62
Merge pull request #4769 from jibutech/4754-fix
Skip povdolumerestore creation when restore excludes pv/pvc
2022-04-19 11:54:30 -04:00
Daniel Jiang 4f9e445142 Make the vsc created by backup sync controller deletable
Fixes #4760

This commit make changes in 2 parts:
1) When a volumesnapshotcontent is persisted during backup, velero will reset its
   `Source` field to remove the VolumeHandle, so that the
   csi-snapshotter will not try to call `CreateSnapshot` when its synced
   to another cluster with a backup.
2) Make sure the referenced volumesnapshotclasses are persisted and
   synced with the backup, so that when the volumesnapshotcontent is
   deleted the storage snapshot is also removed.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2022-04-19 15:06:59 +08:00
Wenkai Yin(尹文开) fb48c5a301 Return the error when running reconcile to trigger the handling logic again
Return the error when running reconcile to trigger the handling logic again

Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
2022-04-19 09:23:40 +08:00
Xun Jiang 8064421e83 Update integrated Restic version and add insecureSkipTLSVerify for Restic CLI
1. Add --insecure-tls for ResticManager's commands.
2. Add --insecure-tls in PodVolumeBackup and PodVolumeRestore controller.
3. Upgrade integrated Restic version to v0.13.1
4. Change --last flag in Restic command to --latest=1 due to Restic version update.

Signed-off-by: Xun Jiang <jxun@vmware.com>
2022-04-18 21:29:33 +08:00
Wenkai Yin(尹文开) 28bc8b7eef Use controller-gen to generate the deep copy methods for objects
As we are refactoring controllers with kubebuilder, use the controller-gen rather than code-generator to generate the deep copy methods for objects

Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
2022-04-18 21:18:00 +08:00
qiuming 65db2585fb
Merge pull request #4822 from yuvalman/patch-1
fix: use the right kind in getRestartableProcess logger
2022-04-15 16:58:46 +08:00
qiuming 9373c8a383
Merge pull request #4818 from blackpiglet/2499-add-metric-for-csi-snapshot
Add CSI VolumeSnapshot related metrics.
2022-04-15 16:55:37 +08:00
Xun Jiang 9aa4e9e860 Update according to comments. Remove csiVolumeSnapshotsStorageTotal related code.
Signed-off-by: Xun Jiang <jxun@vmware.com>
2022-04-15 08:05:56 +00:00
Xun Jiang/Bruce Jiang 22189e8e19
Merge pull request #4436 from codegold79/3454-convert-pod-volume-backup-controller-to-kubebuilder-v3-framework
Convert Pod Volume Backup resource/controller to the Kubebuilder V3 framework
2022-04-15 15:07:09 +08:00