Commit Graph

22 Commits (208d250ddc2b3b12f8af7ecd827446f3f8e2ce51)

Author SHA1 Message Date
Wenkai Yin(尹文开) 72fc1d2c0b
Parse resource from backup tarball directly rather than resolving it via discovery service to avoid #4009 (#4398) 2021-12-21 19:28:55 +08:00
David L. Smith-Uchida 5677e04bb1
Consolidated code for resolving actions and plugins into ActionResolver (#4410)
* Consolidated code for resolving actions and plugins into ActionResolver.  Added BackupWithResolvers and
RestoreWithResolvers.  Introduces ItemSnapshooterResolver to bring ItemSnapshotter plugins into backup and
restore.  ItemSnapshotters are not used yet.

Added action_resolver_test

Signed-off-by: Dave Smith-Uchida <dsmithuchida@vmware.com>

* Addressed review comments

Signed-off-by: Dave Smith-Uchida <dsmithuchida@vmware.com>
2021-12-10 12:53:47 -05:00
Jack Lu 6b315ffcd2
Fix log print error using a wrong var name (#4063)
Signed-off-by: jacklu1024 <jacklu1024@outlook.com>
2021-08-24 10:50:45 -04:00
Lars Lehtonen 15d7a9dfb6
internal/delete: fix dropped error (#4016)
Signed-off-by: Lars Lehtonen <lars.lehtonen@gmail.com>
2021-08-18 15:02:41 +08:00
Bridget McErlean 198ea57407 Allow registry to be configured at build time
This adds a new `buildinfo` variable `ImageRegistry` that can set at
build time like the `Version` variable. This allows us to customise the
Velero binary to use different registries.

If the variable is set, this variable wille be used when creating the
URIs for both the main `velero` and `velero-restic-restore-helper`
images. If it is not set, default to using Dockerhub (`velero/velero`,
`velero/velero-restic-restore-helper`).

There are numerous ways in which the Velero binary can be built so all
of them have been updated to add the new link time flag to set the
variable:
* `make local` (used for local developer builds to build for the local
  OS and ARCH)
* `make build` (used by developers and also VMware internal builds to
  build a specific OS and ARCH)
* Goreleaser config (used when creating OSS release binaries)
* Dockerfile (used to build the Velero binary used within the image)

All of these workflows are currently triggered from our Makefile where
the variable `REGISTRY` is already available with the default value of
`velero` and used to build the image tag. Where the new `ImageRegistry`
build variable is needed, we pass through this Makefile variable to
those tasks so it can be used accordingly.

The GitHub action and the `./hack/docker-push.sh` script used to push
container images has not been modified. This will continue to use the
default registry specified in the Makefile and will not explicitly pass
it in.

Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>
2021-07-14 15:42:53 -04:00
Bridget McErlean b9a8c0b254
Pass configured BSL credential to plugin via config (#3442)
* Load credentials and pass to ObjectStorage plugins

Update NewObjectBackupStore to take a CredentialsGetter which can be
used to get the credentials for a BackupStorageLocation if it has been
configured with a Credential. If the BSL has a credential, use that
SecretKeySelector to fetch the secret, write the contents to a temp file
and then pass that file through to the plugin via the config map using
the key `credentialsFile`. This relies on the plugin being able to use
this new config field.

This does not yet handle VolumeSnapshotLocations or ResticRepositories.

Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>

* Address code reviews

Add godocs and comments.
Improve formatting and test names.

Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>

* Address code reviews

Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>
2021-03-04 13:43:15 -08:00
Carlisia Thompson 11bfe82342
Convert DownloadRequest resource/controller to kubebuilder (#3004)
* Migrate DownloadRequest types to kubebuilder

Signed-off-by: Carlisia <carlisia@vmware.com>

* Migrate controller to kubebuilder

Signed-off-by: Carlisia <carlisia@vmware.com>

* Migrate download request cli to kubebuilder

Signed-off-by: Carlisia <carlisia@vmware.com>

* Format w make update

Signed-off-by: Carlisia <carlisia@vmware.com>

* Remove download file

Signed-off-by: Carlisia <carlisia@vmware.com>

* Remove kubebuilder from backup/restore apis

Signed-off-by: Carlisia <carlisia@vmware.com>

* Fix test description

Signed-off-by: Carlisia <carlisia@vmware.com>

* Import cleanups

Signed-off-by: Carlisia <carlisia@vmware.com>

* Refactor for controller runtime version update

Signed-off-by: Carlisia <carlisia@vmware.com>

* Remove year from the copyright

Signed-off-by: Carlisia <carlisia@vmware.com>

* Check for expiration regardless of phase

Signed-off-by: Carlisia <carlisia@vmware.com>

* Fix typos and godoc

Signed-off-by: Carlisia <carlisia@vmware.com>

* Fix test setup and fix a test case

Signed-off-by: Carlisia <carlisia@vmware.com>
2021-03-01 13:28:46 -05:00
Zadkiel 52504b548d
fix typo in item_hook_handler (#3361)
Signed-off-by: GitHub <noreply@github.com>
2021-02-18 13:32:32 -05:00
Ron Green ef07b72dbc fix: apply patch
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
Signed-off-by: Ron Green <11993626+georgettica@users.noreply.github.com>
2021-01-26 13:06:27 +02:00
Carlisia Thompson 9e29e50773
Minor kubebuilder related items to clean up (#3180)
* Remove unnecessary files

Signed-off-by: Carlisia <carlisia@vmware.com>

* Switch to CAPI patch function for updates

Signed-off-by: Carlisia <carlisia@vmware.com>

* Improve table test format

Signed-off-by: Carlisia <carlisia@vmware.com>

* Refactor and add test for disabling controller

Signed-off-by: Carlisia <carlisia@vmware.com>

* Add tests

Signed-off-by: Carlisia <carlisia@vmware.com>

* Change test to use real word

Signed-off-by: Carlisia <carlisia@vmware.com>

* Fix CI

Signed-off-by: Carlisia <carlisia@vmware.com>

* Minor test fixes

Signed-off-by: Carlisia <carlisia@vmware.com>

* Remove rback/role generation

Signed-off-by: Carlisia <carlisia@vmware.com>
2021-01-21 18:22:34 -05:00
Carlisia bd10b7660c Improvements to BSL logic
Signed-off-by: Carlisia <carlisia@vmware.com>
2020-12-09 13:25:01 -08:00
Ashish Amarnath 7727d535a4
🐛 BSLs with validation disabled should be validated at least once (#3084)
* 🐛 BSLs with validation disabled should be validated at least once

Signed-off-by: Ashish Amarnath <ashisham@vmware.com>

* review comments

Signed-off-by: Ashish Amarnath <ashisham@vmware.com>
2020-12-03 07:52:36 -08:00
Mateusz Gozdek dbc83af77b
Fix various typos found by codespell (#3057)
By running the following command:

codespell -S .git,*.png,*.jpg,*.woff,*.ttf,*.gif,*.ico -L \
iam,aks,ist,bridget,ue

Signed-off-by: Mateusz Gozdek <mgozdekof@gmail.com>
2020-11-10 11:48:35 -05:00
Andrew Reed 0547b1d945
Restore hooks exec (#2804)
* Exec hooks in restored pods

Signed-off-by: Andrew Reed <andrew@replicated.com>

* WaitExecHookHandler implements ItemHookHandler

This required adding a context.Context argument to the ItemHookHandler
interface which is unused by the DefaultItemHookHandler implementation.
It also means passing nil for the []ResourceHook argument since that
holds BackupResourceHook.

Signed-off-by: Andrew Reed <andrew@replicated.com>

* WaitExecHookHandler unit tests

Signed-off-by: Andrew Reed <andrew@replicated.com>

* Changelog and go fmt

Signed-off-by: Andrew Reed <andrew@replicated.com>

* Fix double import

Signed-off-by: Andrew Reed <andrew@replicated.com>

* Default to first contaienr in pod

Signed-off-by: Andrew Reed <andrew@replicated.com>

* Use constants for hook error modes in tests

Signed-off-by: Andrew Reed <andrew@replicated.com>

* Revert to separate WaitExecHookHandler interface

Signed-off-by: Andrew Reed <andrew@replicated.com>

* Negative tests for invalid timeout annotations

Signed-off-by: Andrew Reed <andrew@replicated.com>

* Rename NamedExecRestoreHook PodExecRestoreHook

Also make field names more descriptive.

Signed-off-by: Andrew Reed <andrew@replicated.com>

* Cleanup test names

Signed-off-by: Andrew Reed <andrew@replicated.com>

* Separate maxHookWait and add unit tests

Signed-off-by: Andrew Reed <andrew@replicated.com>

* Comment on maxWait <= 0

Also info log container is not running for hooks to execute in.
Also add context error to hooks not executed errors.

Signed-off-by: Andrew Reed <andrew@replicated.com>

* Remove log about default for invalid timeout

There is no default wait or exec timeout.

Signed-off-by: Andrew Reed <andrew@replicated.com>

* Linting

Signed-off-by: Andrew Reed <andrew@replicated.com>

* Fix log message and rename controller to podWatcher

Signed-off-by: Andrew Reed <andrew@replicated.com>

* Comment on exactly-once semantics for handler

Signed-off-by: Andrew Reed <andrew@replicated.com>

* Fix logging and comments

Use filed logger for pod in handler.
Add comment about pod changes in unit tests.
Use kube util NamespaceAndName in messages.

Signed-off-by: Andrew Reed <andrew@replicated.com>

* Fix maxHookWait

Signed-off-by: Andrew Reed <andrew@replicated.com>
2020-09-08 11:33:15 -07:00
Carlisia Thompson be1cd03023
Refactor BSL related code (#2870)
* Refactor BSL related code

Signed-off-by: Carlisia <carlisia@vmware.com>

* Increase # of max concurrent reconciles

Signed-off-by: Carlisia <carlisia@vmware.com>

* Clean up for better, more precise interfaces

Signed-off-by: Carlisia <carlisia@vmware.com>

* Minor clean up - code reviews

Signed-off-by: Carlisia <carlisia@vmware.com>

* Address code review

Signed-off-by: Carlisia <carlisia@vmware.com>

* Add import and fix CI

Signed-off-by: Carlisia <carlisia@vmware.com>
2020-09-02 13:12:37 -04:00
Carlisia Campos c952932f1b
Migrate ServerStatusRequest controller and resource to kubebuilder (#2838)
* Convert ServerStatusRequest controller to controller-runtime

Signed-off-by: Carlisia <carlisia@vmware.com>

* Add select stm

Signed-off-by: Carlisia <carlisia@vmware.com>

* Fixed status patch bug

Signed-off-by: Carlisia <carlisia@vmware.com>

* Add mgr start

Signed-off-by: Carlisia <carlisia@vmware.com>

* Trying to sync

Signed-off-by: Carlisia <carlisia@vmware.com>

* Clean async now

Signed-off-by: Carlisia <carlisia@vmware.com>

* Clean up + move context out

Signed-off-by: Carlisia <carlisia@vmware.com>

* Bug: not closing the channel

Signed-off-by: Carlisia <carlisia@vmware.com>

* Clean up some tests

Signed-off-by: Carlisia <carlisia@vmware.com>

* Much better way to fetch an update using a backoff loop

Signed-off-by: Carlisia <carlisia@vmware.com>

* Even better way to retry: use apimachinery lib

Signed-off-by: Carlisia <carlisia@vmware.com>

* Refactor controller + add test

Signed-off-by: Carlisia <carlisia@vmware.com>

* partially fix unit tests

Signed-off-by: Ashish Amarnath <ashisham@vmware.com>

* Fix and add tests

Signed-off-by: Carlisia <carlisia@vmware.com>

* Add changelog

Signed-off-by: Carlisia <carlisia@vmware.com>

* Add ability to disable the controller + cleanups

Signed-off-by: Carlisia <carlisia@vmware.com>

* Fix bug w/ disabling controllers + fix test + clean up

Signed-off-by: Carlisia <carlisia@vmware.com>

* Move role.yaml to the correct folder

Signed-off-by: Carlisia <carlisia@vmware.com>

* Add sample serverstatusrequest.yaml

Signed-off-by: Carlisia <carlisia@vmware.com>

* Add requeue + better formatting

Signed-off-by: Carlisia <carlisia@vmware.com>

* Increase # of max concurrent reconciles

Signed-off-by: Carlisia <carlisia@vmware.com>

Co-authored-by: Ashish Amarnath <ashisham@vmware.com>
2020-09-01 14:15:23 -07:00
Nolan Brubaker 718a94ad05
Invoke DeleteItemActions on backup deletion (#2815)
* Add serving and listing support

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2020-08-20 17:24:29 -07:00
Ashish Amarnath 5d6da6517b
Implement restore hooks injecting init containers into pod spec (#2787)
*  Implement restore hooks injecting init containers into pod spec

Signed-off-by: Ashish Amarnath <ashisham@vmware.com>
2020-08-11 10:38:44 -07:00
Ashish Amarnath bc25e789e0
Add constants for restore hook annotation keys (#2750)
* add annotation key constants for restore hooks

Signed-off-by: Ashish Amarnath <ashisham@vmware.com>
2020-07-24 13:05:27 -07:00
Nolan Brubaker bbcbde084d
Create hook package (#2734)
* Move pkg/backup/item_hook_handler to internal/hoo

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>

* Add internal packages to test target

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2020-07-22 14:26:14 -07:00
Carlisia Campos dbd0aa4915
Add a BSL controller to handle validation + update BSL status phase (#2674)
* Add BSL controller

Signed-off-by: Carlisia <carlisia@vmware.com>

* Add changelog

Signed-off-by: Carlisia <carlisia@vmware.com>

* Make update

Signed-off-by: Carlisia <carlisia@vmware.com>

* Update docs

Signed-off-by: Carlisia <carlisia@vmware.com>

* Add kubebuilder dependency

Signed-off-by: Carlisia <carlisia@vmware.com>

* Add export

Signed-off-by: Carlisia <carlisia@vmware.com>

* add kubebuilder binaries into velero builder image

Signed-off-by: Ashish Amarnath <ashisham@vmware.com>

* Reset velero dockerfile

Signed-off-by: Carlisia <carlisia@vmware.com>

* Consolidate all logic

Signed-off-by: Carlisia <carlisia@vmware.com>

* Add copyright header

Signed-off-by: Carlisia <carlisia@vmware.com>

* Clean up + add "last validated" column

Signed-off-by: Carlisia <carlisia@vmware.com>

* Better tests

Signed-off-by: Carlisia <carlisia@vmware.com>

* Add more tests

Signed-off-by: Carlisia <carlisia@vmware.com>

* Better logging

Signed-off-by: Carlisia <carlisia@vmware.com>

* Format

Signed-off-by: Carlisia <carlisia@vmware.com>

* Code reviews

Signed-off-by: Carlisia <carlisia@vmware.com>

* Address code review

Signed-off-by: Carlisia <carlisia@vmware.com>

* Remove redundant logic

Signed-off-by: Carlisia <carlisia@vmware.com>

Co-authored-by: Ashish Amarnath <ashisham@vmware.com>
2020-07-14 17:47:00 -04:00
Carlisia Campos 4048c020a8
Convert manifests + BSL api client to kubebuilder (#2561)
* kubebuilder init - minimalist version

Signed-off-by: Carlisia <carlisia@vmware.com>

* Add back main.go, apparently kb needs it

Signed-off-by: Carlisia <carlisia@vmware.com>

* Tweak makefile to accomodate kubebuilder expectations

Signed-off-by: Carlisia <carlisia@vmware.com>

* Port BSL to kubebuilder api client

Signed-off-by: Carlisia <carlisia@vmware.com>

* s/cache/client bc client fetches from cache
And other naming improvements

Signed-off-by: Carlisia <carlisia@vmware.com>

* So, .GetAPIReader is how we bypass the cache
In this case, the cache hasn't started yet

Signed-off-by: Carlisia <carlisia@vmware.com>

* Oh that's what this code was for... adding back

We still need to embed the CRDs as binary data in the Velero binary to
access the generated CRDs at runtime.

Signed-off-by: Carlisia <carlisia@vmware.com>

* Tie in CRD/code generation w/ existing scripts

Signed-off-by: Carlisia <carlisia@vmware.com>

* Mostly result of running update-fmt, updated file formatting

Signed-off-by: Carlisia <carlisia@vmware.com>

* Just a copyright fix

Signed-off-by: Carlisia <carlisia@vmware.com>

* All the test fixes

Signed-off-by: Carlisia <carlisia@vmware.com>

* Add changelog + some cleanup

Signed-off-by: Carlisia <carlisia@vmware.com>

* Update backup manifest

Signed-off-by: Carlisia <carlisia@vmware.com>

* Remove unneeded auto-generated files

Signed-off-by: Carlisia <carlisia@vmware.com>

* Keep everything in the same (existing) package

Signed-off-by: Carlisia <carlisia@vmware.com>

* Fix/clean scripts, generated code, and calls

Deleting the entire `generated` directory and running `make update`
works. Modifying an api and running `make verify` works as expected.

Signed-off-by: Carlisia <carlisia@vmware.com>

* Clean up schema and client calls + code reviews

Signed-off-by: Carlisia <carlisia@vmware.com>

* Move all code gen to inside builder container

Signed-off-by: Carlisia <carlisia@vmware.com>

* Address code review

Signed-off-by: Carlisia <carlisia@vmware.com>

* Fix imports/aliases

Signed-off-by: Carlisia <carlisia@vmware.com>

* More code reviews

Signed-off-by: Carlisia <carlisia@vmware.com>

* Add waitforcachesync

Signed-off-by: Carlisia <carlisia@vmware.com>

* Have manager register ALL controllers

This will allow for proper cache management.

Signed-off-by: Carlisia <carlisia@vmware.com>

* Status subresource is now enabled; cleanup

Signed-off-by: Carlisia <carlisia@vmware.com>

* More code reviews

Signed-off-by: Carlisia <carlisia@vmware.com>

* Clean up

Signed-off-by: Carlisia <carlisia@vmware.com>

* Manager registers ALL controllers for restic too

Signed-off-by: Carlisia <carlisia@vmware.com>

* More code reviews

Signed-off-by: Carlisia <carlisia@vmware.com>

* Add deprecation warning/todo

Signed-off-by: Carlisia <carlisia@vmware.com>

* Add documentation

Signed-off-by: Carlisia <carlisia@vmware.com>

* Add helpful comments

Signed-off-by: Carlisia <carlisia@vmware.com>

* Address code review

Signed-off-by: Carlisia <carlisia@vmware.com>

* More idiomatic Runnable

Signed-off-by: Carlisia <carlisia@vmware.com>

* Clean up imports

Signed-off-by: Carlisia <carlisia@vmware.com>
2020-06-24 12:55:18 -04:00