Commit Graph

38 Commits (4de4d3783313e61d15e676a260ec400393a848e2)

Author SHA1 Message Date
qiuming 086dbd344f
Handle backup of volume by resource policies (#5901)
* Handle backup of volume by resource policies

Signed-off-by: Ming <mqiu@vmware.com>
2023-03-21 14:39:25 +08:00
Xun Jiang a1e4f54488 Add new resource filters can separate cluster and namespace scope resources.
Signed-off-by: Xun Jiang <blackpiglet@gmail.com>
2023-03-14 07:07:22 +00:00
Scott Seago c3d1d83da5 BIAv2 async operations controller work
Signed-off-by: Scott Seago <sseago@redhat.com>
2023-03-01 11:49:17 -05:00
Scott Seago 0be05c9bc8
Merge pull request #5788 from blackpiglet/enable_staticcheck
Enable staticcheck linter. Disable check for folder /pkg and /test by…
2023-02-08 19:26:22 -05:00
Fish-pro 843c70959f
Clean up unnecessary convert (#5819)
Signed-off-by: Fish-pro <zechun.chen@daocloud.io>
2023-02-06 14:50:31 +08:00
Xun Jiang 955eec7033 Enable staticcheck linter. Disable check for folder /pkg and /test by now.
Signed-off-by: Xun Jiang <blackpiglet@gmail.com>
2023-01-20 13:52:13 +08:00
Lyndon-Li b06cb9ec60 remove gofrs uuid
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2022-11-07 16:28:33 +08:00
lyndon d52ec8c079
Pod Volume Backup/Restore Refactor: Rename Init Helper (#5432)
* restore helper refactor

Signed-off-by: Lyndon-Li <lyonghui@vmware.com>

* resolve codespell

Signed-off-by: Lyndon-Li <lyonghui@vmware.com>

Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2022-10-17 13:42:09 +08:00
Scott Seago 32ef20d317 RestoreItemAction v1 refactoring for plugin api versioning
Signed-off-by: Scott Seago <sseago@redhat.com>
2022-09-20 17:43:56 -04:00
Scott Seago e3e2a8dfa0 plugin/framework refactoring for BackupItemAction v1
Refactors the framework  package to implement the plugin versioning changes
needed for BIA v1 and overall package refactoring to support plugin versions
in different packages. This should be all that's needed to move on to
v2 for BackupItemAction. The remaining plugin types still need similar
refactoring to what's being done here for BIA before attempting a
v2 implementation.

Signed-off-by: Scott Seago <sseago@redhat.com>
2022-09-15 11:15:30 -04:00
Wenkai Yin(尹文开) 4a5647a891 Move the code in pkg/restic/common.go to the proper package
Move the code in pkg/restic/common.go to the proper package

Fixes #5243

Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
2022-08-29 10:04:58 +08:00
Xun Jiang/Bruce Jiang cea5e7f218
Merge pull request #5174 from blackpiglet/reduce-crd-size-new
Reduce CRD size.
2022-08-05 18:46:29 +08:00
lyndon 088eb9b83c
repo credentials (#5167)
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2022-08-04 15:20:02 +08:00
Xun Jiang e8da5df57a Reduce CRD size.
1. Make the Restore hook.InitConatianer server side field pruing disable.
2. Remove restore patch in update-generate-crd-code.sh.
3. Modify related testcases.
4. Add Container fields validation in Restore Init hook.

Signed-off-by: Xun Jiang <jxun@vmware.com>
2022-08-02 19:45:01 +08:00
I538157 74db20993b fix: delete empty backups
Signed-off-by: Yuval Manor <yuvalman958@gmail.com>
2022-04-12 13:13:56 +03:00
Daniel Jiang 2939914113 Ensure the restore hook applied to new namespace based on the mapping
fixes #4720

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2022-03-28 10:24:47 +08:00
Wenkai Yin(尹文开) 72fc1d2c0b
Parse resource from backup tarball directly rather than resolving it via discovery service to avoid #4009 (#4398) 2021-12-21 19:28:55 +08:00
David L. Smith-Uchida 5677e04bb1
Consolidated code for resolving actions and plugins into ActionResolver (#4410)
* Consolidated code for resolving actions and plugins into ActionResolver.  Added BackupWithResolvers and
RestoreWithResolvers.  Introduces ItemSnapshooterResolver to bring ItemSnapshotter plugins into backup and
restore.  ItemSnapshotters are not used yet.

Added action_resolver_test

Signed-off-by: Dave Smith-Uchida <dsmithuchida@vmware.com>

* Addressed review comments

Signed-off-by: Dave Smith-Uchida <dsmithuchida@vmware.com>
2021-12-10 12:53:47 -05:00
Jack Lu 6b315ffcd2
Fix log print error using a wrong var name (#4063)
Signed-off-by: jacklu1024 <jacklu1024@outlook.com>
2021-08-24 10:50:45 -04:00
Lars Lehtonen 15d7a9dfb6
internal/delete: fix dropped error (#4016)
Signed-off-by: Lars Lehtonen <lars.lehtonen@gmail.com>
2021-08-18 15:02:41 +08:00
Bridget McErlean 198ea57407 Allow registry to be configured at build time
This adds a new `buildinfo` variable `ImageRegistry` that can set at
build time like the `Version` variable. This allows us to customise the
Velero binary to use different registries.

If the variable is set, this variable wille be used when creating the
URIs for both the main `velero` and `velero-restic-restore-helper`
images. If it is not set, default to using Dockerhub (`velero/velero`,
`velero/velero-restic-restore-helper`).

There are numerous ways in which the Velero binary can be built so all
of them have been updated to add the new link time flag to set the
variable:
* `make local` (used for local developer builds to build for the local
  OS and ARCH)
* `make build` (used by developers and also VMware internal builds to
  build a specific OS and ARCH)
* Goreleaser config (used when creating OSS release binaries)
* Dockerfile (used to build the Velero binary used within the image)

All of these workflows are currently triggered from our Makefile where
the variable `REGISTRY` is already available with the default value of
`velero` and used to build the image tag. Where the new `ImageRegistry`
build variable is needed, we pass through this Makefile variable to
those tasks so it can be used accordingly.

The GitHub action and the `./hack/docker-push.sh` script used to push
container images has not been modified. This will continue to use the
default registry specified in the Makefile and will not explicitly pass
it in.

Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>
2021-07-14 15:42:53 -04:00
Bridget McErlean b9a8c0b254
Pass configured BSL credential to plugin via config (#3442)
* Load credentials and pass to ObjectStorage plugins

Update NewObjectBackupStore to take a CredentialsGetter which can be
used to get the credentials for a BackupStorageLocation if it has been
configured with a Credential. If the BSL has a credential, use that
SecretKeySelector to fetch the secret, write the contents to a temp file
and then pass that file through to the plugin via the config map using
the key `credentialsFile`. This relies on the plugin being able to use
this new config field.

This does not yet handle VolumeSnapshotLocations or ResticRepositories.

Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>

* Address code reviews

Add godocs and comments.
Improve formatting and test names.

Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>

* Address code reviews

Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>
2021-03-04 13:43:15 -08:00
Carlisia Thompson 11bfe82342
Convert DownloadRequest resource/controller to kubebuilder (#3004)
* Migrate DownloadRequest types to kubebuilder

Signed-off-by: Carlisia <carlisia@vmware.com>

* Migrate controller to kubebuilder

Signed-off-by: Carlisia <carlisia@vmware.com>

* Migrate download request cli to kubebuilder

Signed-off-by: Carlisia <carlisia@vmware.com>

* Format w make update

Signed-off-by: Carlisia <carlisia@vmware.com>

* Remove download file

Signed-off-by: Carlisia <carlisia@vmware.com>

* Remove kubebuilder from backup/restore apis

Signed-off-by: Carlisia <carlisia@vmware.com>

* Fix test description

Signed-off-by: Carlisia <carlisia@vmware.com>

* Import cleanups

Signed-off-by: Carlisia <carlisia@vmware.com>

* Refactor for controller runtime version update

Signed-off-by: Carlisia <carlisia@vmware.com>

* Remove year from the copyright

Signed-off-by: Carlisia <carlisia@vmware.com>

* Check for expiration regardless of phase

Signed-off-by: Carlisia <carlisia@vmware.com>

* Fix typos and godoc

Signed-off-by: Carlisia <carlisia@vmware.com>

* Fix test setup and fix a test case

Signed-off-by: Carlisia <carlisia@vmware.com>
2021-03-01 13:28:46 -05:00
Zadkiel 52504b548d
fix typo in item_hook_handler (#3361)
Signed-off-by: GitHub <noreply@github.com>
2021-02-18 13:32:32 -05:00
Ron Green ef07b72dbc fix: apply patch
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
Signed-off-by: Ron Green <11993626+georgettica@users.noreply.github.com>
2021-01-26 13:06:27 +02:00
Carlisia Thompson 9e29e50773
Minor kubebuilder related items to clean up (#3180)
* Remove unnecessary files

Signed-off-by: Carlisia <carlisia@vmware.com>

* Switch to CAPI patch function for updates

Signed-off-by: Carlisia <carlisia@vmware.com>

* Improve table test format

Signed-off-by: Carlisia <carlisia@vmware.com>

* Refactor and add test for disabling controller

Signed-off-by: Carlisia <carlisia@vmware.com>

* Add tests

Signed-off-by: Carlisia <carlisia@vmware.com>

* Change test to use real word

Signed-off-by: Carlisia <carlisia@vmware.com>

* Fix CI

Signed-off-by: Carlisia <carlisia@vmware.com>

* Minor test fixes

Signed-off-by: Carlisia <carlisia@vmware.com>

* Remove rback/role generation

Signed-off-by: Carlisia <carlisia@vmware.com>
2021-01-21 18:22:34 -05:00
Carlisia bd10b7660c Improvements to BSL logic
Signed-off-by: Carlisia <carlisia@vmware.com>
2020-12-09 13:25:01 -08:00
Ashish Amarnath 7727d535a4
🐛 BSLs with validation disabled should be validated at least once (#3084)
* 🐛 BSLs with validation disabled should be validated at least once

Signed-off-by: Ashish Amarnath <ashisham@vmware.com>

* review comments

Signed-off-by: Ashish Amarnath <ashisham@vmware.com>
2020-12-03 07:52:36 -08:00
Mateusz Gozdek dbc83af77b
Fix various typos found by codespell (#3057)
By running the following command:

codespell -S .git,*.png,*.jpg,*.woff,*.ttf,*.gif,*.ico -L \
iam,aks,ist,bridget,ue

Signed-off-by: Mateusz Gozdek <mgozdekof@gmail.com>
2020-11-10 11:48:35 -05:00
Andrew Reed 0547b1d945
Restore hooks exec (#2804)
* Exec hooks in restored pods

Signed-off-by: Andrew Reed <andrew@replicated.com>

* WaitExecHookHandler implements ItemHookHandler

This required adding a context.Context argument to the ItemHookHandler
interface which is unused by the DefaultItemHookHandler implementation.
It also means passing nil for the []ResourceHook argument since that
holds BackupResourceHook.

Signed-off-by: Andrew Reed <andrew@replicated.com>

* WaitExecHookHandler unit tests

Signed-off-by: Andrew Reed <andrew@replicated.com>

* Changelog and go fmt

Signed-off-by: Andrew Reed <andrew@replicated.com>

* Fix double import

Signed-off-by: Andrew Reed <andrew@replicated.com>

* Default to first contaienr in pod

Signed-off-by: Andrew Reed <andrew@replicated.com>

* Use constants for hook error modes in tests

Signed-off-by: Andrew Reed <andrew@replicated.com>

* Revert to separate WaitExecHookHandler interface

Signed-off-by: Andrew Reed <andrew@replicated.com>

* Negative tests for invalid timeout annotations

Signed-off-by: Andrew Reed <andrew@replicated.com>

* Rename NamedExecRestoreHook PodExecRestoreHook

Also make field names more descriptive.

Signed-off-by: Andrew Reed <andrew@replicated.com>

* Cleanup test names

Signed-off-by: Andrew Reed <andrew@replicated.com>

* Separate maxHookWait and add unit tests

Signed-off-by: Andrew Reed <andrew@replicated.com>

* Comment on maxWait <= 0

Also info log container is not running for hooks to execute in.
Also add context error to hooks not executed errors.

Signed-off-by: Andrew Reed <andrew@replicated.com>

* Remove log about default for invalid timeout

There is no default wait or exec timeout.

Signed-off-by: Andrew Reed <andrew@replicated.com>

* Linting

Signed-off-by: Andrew Reed <andrew@replicated.com>

* Fix log message and rename controller to podWatcher

Signed-off-by: Andrew Reed <andrew@replicated.com>

* Comment on exactly-once semantics for handler

Signed-off-by: Andrew Reed <andrew@replicated.com>

* Fix logging and comments

Use filed logger for pod in handler.
Add comment about pod changes in unit tests.
Use kube util NamespaceAndName in messages.

Signed-off-by: Andrew Reed <andrew@replicated.com>

* Fix maxHookWait

Signed-off-by: Andrew Reed <andrew@replicated.com>
2020-09-08 11:33:15 -07:00
Carlisia Thompson be1cd03023
Refactor BSL related code (#2870)
* Refactor BSL related code

Signed-off-by: Carlisia <carlisia@vmware.com>

* Increase # of max concurrent reconciles

Signed-off-by: Carlisia <carlisia@vmware.com>

* Clean up for better, more precise interfaces

Signed-off-by: Carlisia <carlisia@vmware.com>

* Minor clean up - code reviews

Signed-off-by: Carlisia <carlisia@vmware.com>

* Address code review

Signed-off-by: Carlisia <carlisia@vmware.com>

* Add import and fix CI

Signed-off-by: Carlisia <carlisia@vmware.com>
2020-09-02 13:12:37 -04:00
Carlisia Campos c952932f1b
Migrate ServerStatusRequest controller and resource to kubebuilder (#2838)
* Convert ServerStatusRequest controller to controller-runtime

Signed-off-by: Carlisia <carlisia@vmware.com>

* Add select stm

Signed-off-by: Carlisia <carlisia@vmware.com>

* Fixed status patch bug

Signed-off-by: Carlisia <carlisia@vmware.com>

* Add mgr start

Signed-off-by: Carlisia <carlisia@vmware.com>

* Trying to sync

Signed-off-by: Carlisia <carlisia@vmware.com>

* Clean async now

Signed-off-by: Carlisia <carlisia@vmware.com>

* Clean up + move context out

Signed-off-by: Carlisia <carlisia@vmware.com>

* Bug: not closing the channel

Signed-off-by: Carlisia <carlisia@vmware.com>

* Clean up some tests

Signed-off-by: Carlisia <carlisia@vmware.com>

* Much better way to fetch an update using a backoff loop

Signed-off-by: Carlisia <carlisia@vmware.com>

* Even better way to retry: use apimachinery lib

Signed-off-by: Carlisia <carlisia@vmware.com>

* Refactor controller + add test

Signed-off-by: Carlisia <carlisia@vmware.com>

* partially fix unit tests

Signed-off-by: Ashish Amarnath <ashisham@vmware.com>

* Fix and add tests

Signed-off-by: Carlisia <carlisia@vmware.com>

* Add changelog

Signed-off-by: Carlisia <carlisia@vmware.com>

* Add ability to disable the controller + cleanups

Signed-off-by: Carlisia <carlisia@vmware.com>

* Fix bug w/ disabling controllers + fix test + clean up

Signed-off-by: Carlisia <carlisia@vmware.com>

* Move role.yaml to the correct folder

Signed-off-by: Carlisia <carlisia@vmware.com>

* Add sample serverstatusrequest.yaml

Signed-off-by: Carlisia <carlisia@vmware.com>

* Add requeue + better formatting

Signed-off-by: Carlisia <carlisia@vmware.com>

* Increase # of max concurrent reconciles

Signed-off-by: Carlisia <carlisia@vmware.com>

Co-authored-by: Ashish Amarnath <ashisham@vmware.com>
2020-09-01 14:15:23 -07:00
Nolan Brubaker 718a94ad05
Invoke DeleteItemActions on backup deletion (#2815)
* Add serving and listing support

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2020-08-20 17:24:29 -07:00
Ashish Amarnath 5d6da6517b
Implement restore hooks injecting init containers into pod spec (#2787)
*  Implement restore hooks injecting init containers into pod spec

Signed-off-by: Ashish Amarnath <ashisham@vmware.com>
2020-08-11 10:38:44 -07:00
Ashish Amarnath bc25e789e0
Add constants for restore hook annotation keys (#2750)
* add annotation key constants for restore hooks

Signed-off-by: Ashish Amarnath <ashisham@vmware.com>
2020-07-24 13:05:27 -07:00
Nolan Brubaker bbcbde084d
Create hook package (#2734)
* Move pkg/backup/item_hook_handler to internal/hoo

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>

* Add internal packages to test target

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2020-07-22 14:26:14 -07:00
Carlisia Campos dbd0aa4915
Add a BSL controller to handle validation + update BSL status phase (#2674)
* Add BSL controller

Signed-off-by: Carlisia <carlisia@vmware.com>

* Add changelog

Signed-off-by: Carlisia <carlisia@vmware.com>

* Make update

Signed-off-by: Carlisia <carlisia@vmware.com>

* Update docs

Signed-off-by: Carlisia <carlisia@vmware.com>

* Add kubebuilder dependency

Signed-off-by: Carlisia <carlisia@vmware.com>

* Add export

Signed-off-by: Carlisia <carlisia@vmware.com>

* add kubebuilder binaries into velero builder image

Signed-off-by: Ashish Amarnath <ashisham@vmware.com>

* Reset velero dockerfile

Signed-off-by: Carlisia <carlisia@vmware.com>

* Consolidate all logic

Signed-off-by: Carlisia <carlisia@vmware.com>

* Add copyright header

Signed-off-by: Carlisia <carlisia@vmware.com>

* Clean up + add "last validated" column

Signed-off-by: Carlisia <carlisia@vmware.com>

* Better tests

Signed-off-by: Carlisia <carlisia@vmware.com>

* Add more tests

Signed-off-by: Carlisia <carlisia@vmware.com>

* Better logging

Signed-off-by: Carlisia <carlisia@vmware.com>

* Format

Signed-off-by: Carlisia <carlisia@vmware.com>

* Code reviews

Signed-off-by: Carlisia <carlisia@vmware.com>

* Address code review

Signed-off-by: Carlisia <carlisia@vmware.com>

* Remove redundant logic

Signed-off-by: Carlisia <carlisia@vmware.com>

Co-authored-by: Ashish Amarnath <ashisham@vmware.com>
2020-07-14 17:47:00 -04:00
Carlisia Campos 4048c020a8
Convert manifests + BSL api client to kubebuilder (#2561)
* kubebuilder init - minimalist version

Signed-off-by: Carlisia <carlisia@vmware.com>

* Add back main.go, apparently kb needs it

Signed-off-by: Carlisia <carlisia@vmware.com>

* Tweak makefile to accomodate kubebuilder expectations

Signed-off-by: Carlisia <carlisia@vmware.com>

* Port BSL to kubebuilder api client

Signed-off-by: Carlisia <carlisia@vmware.com>

* s/cache/client bc client fetches from cache
And other naming improvements

Signed-off-by: Carlisia <carlisia@vmware.com>

* So, .GetAPIReader is how we bypass the cache
In this case, the cache hasn't started yet

Signed-off-by: Carlisia <carlisia@vmware.com>

* Oh that's what this code was for... adding back

We still need to embed the CRDs as binary data in the Velero binary to
access the generated CRDs at runtime.

Signed-off-by: Carlisia <carlisia@vmware.com>

* Tie in CRD/code generation w/ existing scripts

Signed-off-by: Carlisia <carlisia@vmware.com>

* Mostly result of running update-fmt, updated file formatting

Signed-off-by: Carlisia <carlisia@vmware.com>

* Just a copyright fix

Signed-off-by: Carlisia <carlisia@vmware.com>

* All the test fixes

Signed-off-by: Carlisia <carlisia@vmware.com>

* Add changelog + some cleanup

Signed-off-by: Carlisia <carlisia@vmware.com>

* Update backup manifest

Signed-off-by: Carlisia <carlisia@vmware.com>

* Remove unneeded auto-generated files

Signed-off-by: Carlisia <carlisia@vmware.com>

* Keep everything in the same (existing) package

Signed-off-by: Carlisia <carlisia@vmware.com>

* Fix/clean scripts, generated code, and calls

Deleting the entire `generated` directory and running `make update`
works. Modifying an api and running `make verify` works as expected.

Signed-off-by: Carlisia <carlisia@vmware.com>

* Clean up schema and client calls + code reviews

Signed-off-by: Carlisia <carlisia@vmware.com>

* Move all code gen to inside builder container

Signed-off-by: Carlisia <carlisia@vmware.com>

* Address code review

Signed-off-by: Carlisia <carlisia@vmware.com>

* Fix imports/aliases

Signed-off-by: Carlisia <carlisia@vmware.com>

* More code reviews

Signed-off-by: Carlisia <carlisia@vmware.com>

* Add waitforcachesync

Signed-off-by: Carlisia <carlisia@vmware.com>

* Have manager register ALL controllers

This will allow for proper cache management.

Signed-off-by: Carlisia <carlisia@vmware.com>

* Status subresource is now enabled; cleanup

Signed-off-by: Carlisia <carlisia@vmware.com>

* More code reviews

Signed-off-by: Carlisia <carlisia@vmware.com>

* Clean up

Signed-off-by: Carlisia <carlisia@vmware.com>

* Manager registers ALL controllers for restic too

Signed-off-by: Carlisia <carlisia@vmware.com>

* More code reviews

Signed-off-by: Carlisia <carlisia@vmware.com>

* Add deprecation warning/todo

Signed-off-by: Carlisia <carlisia@vmware.com>

* Add documentation

Signed-off-by: Carlisia <carlisia@vmware.com>

* Add helpful comments

Signed-off-by: Carlisia <carlisia@vmware.com>

* Address code review

Signed-off-by: Carlisia <carlisia@vmware.com>

* More idiomatic Runnable

Signed-off-by: Carlisia <carlisia@vmware.com>

* Clean up imports

Signed-off-by: Carlisia <carlisia@vmware.com>
2020-06-24 12:55:18 -04:00