Commit Graph

68 Commits (a4416874cf8d2d0f88cfbb2fb3cebac62a8e3a33)

Author SHA1 Message Date
Scott Seago 9d6f4d2db5 ItemBlock model and phase 1 (single-thread) workflow changes
Signed-off-by: Scott Seago <sseago@redhat.com>
2024-09-03 19:04:18 -04:00
Lyndon-Li faa704d909 data mover ms watcher
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2024-07-25 10:47:52 +08:00
Xun Jiang/Bruce Jiang 8df4e6aded
Merge branch 'main' into merge_csi
Signed-off-by: Xun Jiang/Bruce Jiang <59276555+blackpiglet@users.noreply.github.com>
2024-04-10 18:54:16 +08:00
clonefetch 474dc824e7
chore: fix function names in comment (#7633)
Signed-off-by: clonefetch <c0217@outlook.com>
2024-04-10 15:15:29 +05:30
Xun Jiang 31e140919a Merge CSI plugin code.
Signed-off-by: Xun Jiang <blackpigletbruce@gmail.com>
2024-04-10 14:53:29 +08:00
Wenkai Yin(尹文开) 8752c3a820 Bump up the versions of severel Kubernetes-related libs
Bump up the versions of severel Kubernetes-related libs

Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
2024-03-05 13:09:38 +08:00
Tiger Kaovilai 270b1de6a1
Do not attempt restore resource with no available GVK in cluster (#7322)
Check for GVK before attempting restore.

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>
2024-01-22 09:50:13 +08:00
Wenkai Yin(尹文开) 427a254136 Specify the Kind explicitly in the API resource
Specify the Kind explicitly in the API resource to avoid wrong Kind conversion

Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
2024-01-19 12:51:56 +08:00
Wenkai Yin(尹文开) 956248e8a6 Create informer per resources to avoid huge memory consumption
Create informer per resources to avoid huge memory consumption

Fixes #7323

Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
2024-01-18 10:47:00 +08:00
Wenkai Yin(尹文开) 2a533d01bf
Merge pull request #7046 from kaovilai/backup-patch-status-unittest
Update Backup.Status.CSIVolumeSnapshotsCompleted during finalize
2023-11-15 15:32:51 +08:00
Xun Jiang/Bruce Jiang cb5ffe2753
Merge pull request #7061 from blackpiglet/6595_backward_compatability
Add DataUpload Result and CSI VolumeSnapshot check for restore PV.
2023-11-10 09:37:19 +08:00
Xun Jiang 1fb0529d98 Add DataUpload Result and CSI VolumeSnapshot check for restore PV.
Signed-off-by: Xun Jiang <jxun@vmware.com>
2023-11-06 22:40:03 +08:00
Lyndon-Li 68579448d6 configurable data path concurrency: UT
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2023-11-06 20:29:33 +08:00
Xun Jiang 56b5e982d9 Remove dependency of generated client part 3
Replace generated discovery client with client-go client.
Remove generated client from PVR action.
Remove generated client from pkg/cmd directory.
Delete velero generate client from client factory.

Signed-off-by: Xun Jiang <jxun@vmware.com>
2023-11-06 11:34:39 +08:00
Xun Jiang a221a88945 Remove dependency of generated client part 2
Remove dependecy of generate client from pkg/cmd/cli/snapshotLocation.
Remove the Velero generated informer from PVB and PVR. 
Remove dependency of generated client from pkg/podvolume directory.
Replace generated codec with runtime codec. 

Signed-off-by: Xun Jiang <jxun@vmware.com>
2023-11-03 17:11:36 +08:00
Tiger Kaovilai 9311a4269b
refactor backup snapshot status updates into UpdateBackupSnapshotsStatus() and run in backup_finalizer_controller
Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>
2023-11-02 15:30:35 -04:00
Scott Seago 7750e12151 Perf improvements for existing resource restore
Use informer cache with dynamic client for Get calls on restore
When enabled, also make the Get call before create.

Add server and install parameter to allow disabling this feature,
but enable by default

Signed-off-by: Scott Seago <sseago@redhat.com>
2023-10-11 10:51:39 -04:00
Xun Jiang ec4bb42117 Retrieve DataUpload into backup result ConfigMap during volume snapshot restore.
Fix issue #6117.
Add CSI plugin needs builder functions.

Signed-off-by: Xun Jiang <jxun@vmware.com>
2023-06-30 11:13:04 +08:00
danfengliu ef443fece0
Add UT for pkg/cmd/cli/backup (#6400)
Signed-off-by: danfengl <danfengl@vmware.com>
2023-06-21 11:10:13 +08:00
lyndon e3e0ce32ed
Merge pull request #6368 from Lyndon-Li/issue-fix-6341
Add UT for pkg/util
2023-06-15 14:05:53 +08:00
Ming e2bb5b3fe8 Add UT for pkg/discovery
Signed-off-by: Ming <mqiu@vmware.com>
2023-06-15 02:45:34 +00:00
Lyndon-Li 217b1dd066 add UT for pkg/util
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2023-06-08 14:00:58 +08:00
lyndon 5b75f35262
Add data mover CRD under v2alpha1 (#6176)
* add data mover CRD under v1alpha1

Signed-off-by: Lyndon-Li <lyonghui@vmware.com>

* data mover CRDs to v2alpha1

Signed-off-by: Lyndon-Li <lyonghui@vmware.com>

* data mover crd changes

Signed-off-by: Lyndon-Li <lyonghui@vmware.com>

---------

Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2023-05-16 15:09:54 -04:00
Lyndon-Li 307b82a2ec bump up kopia v0.13.0
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2023-05-15 07:23:38 +08:00
Xun Jiang f051ecaee9 Enable errcheck linter and resolve found issues.
Signed-off-by: Xun Jiang <blackpiglet@gmail.com>
2023-04-28 09:33:42 +08:00
Xun Jiang/Bruce Jiang a8a17d725a
Merge pull request #6194 from blackpiglet/linter_more
Enable more linters, and remove mal-functioned milestoned issue action.
2023-04-26 14:12:47 +08:00
Xun Jiang/Bruce Jiang e106bbf06b
Enable linter revive and resolve found errors: part 2. (#6177)
Signed-off-by: Xun Jiang <blackpiglet@gmail.com>
Co-authored-by: Xun Jiang <blackpiglet@gmail.com>
2023-04-26 14:05:22 +08:00
Xun Jiang 443f732e51 Enable more linters, and remove mal-functioned milestoned issue action.
Signed-off-by: Xun Jiang <blackpiglet@gmail.com>
2023-04-25 23:31:51 +08:00
Xun Jiang/Bruce Jiang 51c67089f5
Merge branch 'main' into linter_staticcheck
Signed-off-by: Xun Jiang/Bruce Jiang <59276555+blackpiglet@users.noreply.github.com>
2023-04-25 15:49:05 +08:00
Xun Jiang bbc1e2e151 Enable stylecheck linter and resolve found issues.
Signed-off-by: Xun Jiang <blackpiglet@gmail.com>
2023-04-25 14:37:02 +08:00
Xun Jiang cb0ada1e1c Enable staticcheck and resolve found issues.
Signed-off-by: Xun Jiang <blackpiglet@gmail.com>
2023-04-25 11:33:40 +08:00
Xun Jiang/Bruce Jiang 1fd28e8a36
Fix usestdlibvars and whitespace linters issues. (#6162)
Signed-off-by: Xun Jiang <blackpiglet@gmail.com>
Co-authored-by: Xun Jiang <blackpiglet@gmail.com>
2023-04-24 09:10:55 +08:00
Xun Jiang/Bruce Jiang 15d44724e7
Modify golangci.yaml file. Resolve found lint issues. (#6008)
Signed-off-by: Xun Jiang <blackpiglet@gmail.com>
Co-authored-by: Xun Jiang <blackpiglet@gmail.com>
2023-03-24 12:15:08 +08:00
杨刚 (成都) ad9c6e8dee
delete io/ioutil package. (#5955)
Signed-off-by: yanggang <gang.yang@daocloud.io>
2023-03-16 09:25:58 +08:00
Xun Jiang a1e4f54488 Add new resource filters can separate cluster and namespace scope resources.
Signed-off-by: Xun Jiang <blackpiglet@gmail.com>
2023-03-14 07:07:22 +00:00
Xun Jiang/Bruce Jiang 5027aae194
Add more nil pointer check for CSI related code in backup controller. (#5388)
Add some corner cases checking for CSI snapshot in backup controller.

Signed-off-by: Xun Jiang <blackpiglet@gmail.com>
2022-10-24 10:42:08 +08:00
Rafael Leal 04aa7a849f
Fixes tests hopefully
Signed-off-by: Rafael Leal <rafaelealdias@gmail.com>
2022-05-12 11:59:05 -03:00
Daniel Jiang 74fd158a1c Refactor backup deletion controller based on kubebuilder
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2022-04-27 17:00:54 +08:00
F. Gold 2dd85c9c42
Convert PodVolumeBackup to Kubebuilder framework
Signed-off-by: F. Gold <fgold@vmware.com>
2022-01-26 20:09:24 -08:00
Wenkai Yin(尹文开) 62779bbcc6 Check the existence of the expected service when ignoring the NodePort already allocated error
Check the existence of the expected service when ignoring the NodePort already allocated error

Fixes 2308

Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
2022-01-12 15:06:42 +08:00
David L. Smith-Uchida 5677e04bb1
Consolidated code for resolving actions and plugins into ActionResolver (#4410)
* Consolidated code for resolving actions and plugins into ActionResolver.  Added BackupWithResolvers and
RestoreWithResolvers.  Introduces ItemSnapshooterResolver to bring ItemSnapshotter plugins into backup and
restore.  ItemSnapshotters are not used yet.

Added action_resolver_test

Signed-off-by: Dave Smith-Uchida <dsmithuchida@vmware.com>

* Addressed review comments

Signed-off-by: Dave Smith-Uchida <dsmithuchida@vmware.com>
2021-12-10 12:53:47 -05:00
Scott Seago 550efddd88
Verify group before treating resource as cohabitating (#4126)
Signed-off-by: Scott Seago <sseago@redhat.com>
2021-11-03 18:11:32 -04:00
Wenkai Yin(尹文开) 9f0ea22c60
Fix CVE-2020-29652 and CVE-2020-26160 (#4274)
Bump up restic to v0.12.1 to fix CVE-2020-26160.
Bump up module "github.com/vmware-tanzu/crash-diagnostics" to v0.3.7 to fix CVE-2020-29652.
The "github.com/vmware-tanzu/crash-diagnostics" updates client-go to v0.22.2 which introduces several break changes, this commit updates the related codes as well

Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
2021-11-03 13:30:26 -07:00
Daniel Jiang 7c75cd6cf8
Implement `velero debug` (#4022)
This PR added a subcommand `velero debug`, which leverages `crashd` to
collect logs and specs of velero server components and bundle them in a
tarball.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2021-08-30 18:37:18 -07:00
Bridget McErlean b9a8c0b254
Pass configured BSL credential to plugin via config (#3442)
* Load credentials and pass to ObjectStorage plugins

Update NewObjectBackupStore to take a CredentialsGetter which can be
used to get the credentials for a BackupStorageLocation if it has been
configured with a Credential. If the BSL has a credential, use that
SecretKeySelector to fetch the secret, write the contents to a temp file
and then pass that file through to the plugin via the config map using
the key `credentialsFile`. This relies on the plugin being able to use
this new config field.

This does not yet handle VolumeSnapshotLocations or ResticRepositories.

Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>

* Address code reviews

Add godocs and comments.
Improve formatting and test names.

Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>

* Address code reviews

Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>
2021-03-04 13:43:15 -08:00
Suraj Banakar ff1a31db4a
Support cli uninstall (#3399)
* Add uninstall cmd
- init fn to uninstall velero
- abstract dynamic client creation to a separate fn
- creates a separate client per unstructured resource
- add delete client for CRDs
- export appendUnstructured
- add uninstall command to main cmd
- export `podTemplateOption`
- uninstall resources in the reverse order of installation
- fallback to `velero` if no ns is provided during uninstall
- skip deletion if the resource doesn't exist
- handle resource not found error
- match log formatting with cli install logs
- add Delete fn to fake client
- fix import order
- add changelog
- add comment doc for CreateClient fn

Signed-off-by: Suraj Banakar <suraj@infracloud.io>

* Re-use uninstall code from test suite
- move helper functions out of test suite
- this is to prevent cyclic imports
- move uninstall helpers to uninstall cmd
- call them from test suite
- revert export of variables/fns from install code
- because not required anymore

Signed-off-by: Suraj Banakar <suraj@infracloud.io>

* Revert `PodTemplateOption` -> `podTemplateOption`

Signed-off-by: Suraj Banakar <suraj@infracloud.io>

* Use uninstall helper under VeleroUninstall
- as a wrapper
- fix import related errors in test suite

Signed-off-by: Suraj Banakar <suraj@infracloud.io>
2021-03-04 14:16:40 -05:00
Bridget McErlean 9dbd238c89
Use controller-runtime client to get restic secrets (#3320)
* Use kubebuilder client for fetching restic secrets

Instead of using a SecretInformer for fetching secrets for restic, use
the cached client provided by the controller-runtime manager.

In order to use this client, the scheme for Secrets must be added to the
scheme used by the manager so this is added when creating the manager in
both the velero and restic servers.

This change also refactors some of the tests to add a shared utility for
creating a fake controller-runtime client which is now used among all
tests which use that client. This has been added to ensure that all
tests use the same client with the same scheme.

Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>

* Add builder for SecretKeySelector

Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>
2021-02-18 10:30:52 -08:00
Nolan Brubaker 718a94ad05
Invoke DeleteItemActions on backup deletion (#2815)
* Add serving and listing support

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2020-08-20 17:24:29 -07:00
runzexia 3a4e441af8
add kindfor func to get apiresource from gvk (#2764)
* add kindfor func to get apiresource from gvk

Signed-off-by: RyderXia <ryder.xia@sap.com>

* impl interface & changelog

Signed-off-by: RyderXia <ryder.xia@sap.com>
2020-08-17 15:52:33 -07:00
Andrew Lavery a368370bef
k8s 1.18 import (#2651)
* k8s 1.18 import wip

backup, cmd, controller, generated, restic, restore, serverstatusrequest, test and util

Signed-off-by: Andrew Lavery <laverya@umich.edu>

* go mod tidy

Signed-off-by: Andrew Lavery <laverya@umich.edu>

* add changelog file

Signed-off-by: Andrew Lavery <laverya@umich.edu>

* go fmt

Signed-off-by: Andrew Lavery <laverya@umich.edu>

* update code-generator and controller-gen in CI

Signed-off-by: Andrew Lavery <laverya@umich.edu>

* checkout proper code-generator version, regen

Signed-off-by: Andrew Lavery <laverya@umich.edu>

* fix remaining calls

Signed-off-by: Andrew Lavery <laverya@umich.edu>

* regenerate CRDs with ./hack/update-generated-crd-code.sh

Signed-off-by: Andrew Lavery <laverya@umich.edu>

* use existing context in restic and server

Signed-off-by: Andrew Lavery <laverya@umich.edu>

* fix test cases by resetting resource version

also use main library go context, not golang.org/x/net/context, in pkg/restore/restore.go

Signed-off-by: Andrew Lavery <laverya@umich.edu>

* clarify changelog message

Signed-off-by: Andrew Lavery <laverya@umich.edu>

* use github.com/kubernetes-csi/external-snapshotter/v2@v2.2.0-rc1

Signed-off-by: Andrew Lavery <laverya@umich.edu>

* run 'go mod tidy' to remove old external-snapshotter version

Signed-off-by: Andrew Lavery <laverya@umich.edu>
2020-07-16 12:21:37 -04:00