* Use pod namespace from backup when matching PVBs
In #3051, we introduced an additional check to ensure that a PVB matched
a particular pod by checking both the name and the namespace of the pod.
This caused an issue when using a namespace mapping on restore. In the
case where a namespace mapping is being used, the check for whether a
PVB matches a particular pod will fail as the PVB was created for the
original pod namespace and is not aware of the new namespace mapping
being used. This resulted in PVRs not being created for pods that were
being restored into new namespaces. The restic init containers were
being created to wait on the volume restore, however this would cause
the restored pods to block indefinitely as they would be waiting for a
volume restore that was not scheduled.
To fix this, we use the original namespace of the pod from the backup to
match the PVB to the pod being restored, not the new namespace where
the pod is being restored into.
Fixes#3467.
Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>
* Explain why the namespace mapping can't be used
Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>
* Use kubebuilder client for fetching restic secrets
Instead of using a SecretInformer for fetching secrets for restic, use
the cached client provided by the controller-runtime manager.
In order to use this client, the scheme for Secrets must be added to the
scheme used by the manager so this is added when creating the manager in
both the velero and restic servers.
This change also refactors some of the tests to add a shared utility for
creating a fake controller-runtime client which is now used among all
tests which use that client. This has been added to ensure that all
tests use the same client with the same scheme.
Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>
* Add builder for SecretKeySelector
Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>
* Restore API group version by priority
Signed-off-by: F. Gold <fgold@vmware.com>
* Add changelog
Signed-off-by: F. Gold <fgold@vmware.com>
* Correct spelling
Signed-off-by: F. Gold <fgold@vmware.com>
* Refactor userResourceGroupVersionPriorities(...) to accept config map, adjust unit test
Signed-off-by: F. Gold <fgold@vmware.com>
* Move some unit tests into e2e
Signed-off-by: F. Gold <fgold@vmware.com>
* Add three e2e tests using Testify Suites
Summary of changes
Makefile - add testify e2e test target
go.sum - changed with go mod tidy
pkg/install/install.go - increased polling timeout
test/e2e/restore_priority_group_test.go - deleted
test/e2e/restore_test.go - deleted
test/e2e/velero_utils.go - made restic optional in velero install
test/e2e_testify/Makefile - makefile for testify e2e tests
test/e2e_testify/README.md - example command for running tests
test/e2e_testify/common_test.go - helper functions
test/e2e_testify/e2e_suite_test.go - prepare for tests and run
test/e2e_testify/restore_priority_apigv_test.go - test cases
Signed-off-by: F. Gold <fgold@vmware.com>
* Make changes per @nrb code review
Signed-off-by: F. Gold <fgold@vmware.com>
* Wait for pods in e2e tests
Signed-off-by: F. Gold <fgold@vmware.com>
* Remove testify suites e2e scaffolding moved to PR #3354
Signed-off-by: F. Gold <fgold@vmware.com>
* Make changes per @brito-rafa and Velero maintainers code reviews
- Made changes suggested by @brito-rafa in GitHub.
- We had a code review meeting with @carlisia, @dsu-igeek, @zubron, and @nrb
- and changes were made based on their suggetions:
- pull in logic from 'meetsAPIGVResotreReqs()' to restore.go.
- add TODO to remove APIGroupVersionFeatureFlag check
- have feature flag and backup version format checks in separate `if` statements.
- rename variables to be sourceGVs, targetGVs, and userGVs.
Signed-off-by: F. Gold <fgold@vmware.com>
* Convert Testify Suites e2e tests to existing Ginkgo framework
Signed-off-by: F. Gold <fgold@vmware.com>
* Made changes per @zubron PR review
Signed-off-by: F. Gold <fgold@vmware.com>
* Run go mod tidy after resolving go.sum merge conflict
Signed-off-by: F. Gold <fgold@vmware.com>
* Add feature documentation to velero.io site
Signed-off-by: F. Gold <fgold@vmware.com>
* Add config map e2e test; rename e2e test file and name
Signed-off-by: F. Gold <fgold@vmware.com>
* Update go.{mod,sum} files
Signed-off-by: F. Gold <fgold@vmware.com>
* Move CRDs and CRs to testdata folder
Signed-off-by: F. Gold <fgold@vmware.com>
* Fix typos in cert-manager to pass codespell CICD check
Signed-off-by: F. Gold <fgold@vmware.com>
* Make changes per @nrb code review round 2
- make checkAndReadDir function private
- add info level messages when priorties 1-3 API group versions can not be used
Signed-off-by: F. Gold <fgold@vmware.com>
* Make user config map rules less strict
Signed-off-by: F. Gold <fgold@vmware.com>
* Update e2e test image version in example
Signed-off-by: F. Gold <fgold@vmware.com>
* Update case A music-system controller code
Signed-off-by: F. Gold <fgold@vmware.com>
* Documentation updates
Signed-off-by: F. Gold <fgold@vmware.com>
* Update migration case documentation
Signed-off-by: F. Gold <fgold@vmware.com>
* Use label to select Velero deployment in plugin cmd
Signed-off-by: F. Gold <fgold@vmware.com>
* Move veleroLabel constant closer to usage
Signed-off-by: F. Gold <fgold@vmware.com>
* Add changelog
Signed-off-by: F. Gold <fgold@vmware.com>
* Remove year from copyright in new file
Signed-off-by: F. Gold <fgold@vmware.com>
* Export and use install.Labels() function
Signed-off-by: F. Gold <fgold@vmware.com>
Restoring CAPI workload clusters without this ordering caused the
capi-controller-manager code to panic, resulting in an unhealthy cluster
state.
This can be worked around
(https://community.pivotal.io/s/article/5000e00001pJyN41611954332537?language=en_US),
but we provide the inclusion of these resources as a default in order to
provide a better out-of-the-box experience.
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
* Add colors to describe command
* Add colors to describe backups/restore/schedules commands
* Make name in the output bold
* Disable colors via `--colorized` flag or if velero isn't in TTY
Co-authored-by: Clay Kauzlaric <ckauzlaric@vmware.com>
Signed-off-by: Clay Kauzlaric <ckauzlaric@vmware.com>
Signed-off-by: Mikael Manukyan <mmanukyan@vmware.com>
* Add changelog
* and run make update
Co-authored-by: Mikael Manukyan <mmanukyan@vmware.com>
Signed-off-by: Mikael Manukyan <mmanukyan@vmware.com>
Signed-off-by: Clay Kauzlaric <ckauzlaric@vmware.com>
* Add colorized to the client config file
Co-authored-by: Mikael Manukyan <mmanukyan@vmware.com>
Signed-off-by: Clay Kauzlaric <ckauzlaric@vmware.com>
Co-authored-by: Mikael Manukyan <mmanukyan@vmware.com>
* allow client config to use string values
* the command `velero client config set colorized=false` writes a string
value of "false" into the config. This change allows that string to be
accepted and converted into a boolean when used in program.
Signed-off-by: Clay Kauzlaric <ckauzlaric@vmware.com>
* Add docs about colored CLI output
Co-authored-by: Mikael Manukyan <mmanukyan@vmware.com>
Signed-off-by: Clay Kauzlaric <ckauzlaric@vmware.com>
* Update site/content/docs/main/customize-installation.md
Co-authored-by: JenTing Hsiao <jenting.hsiao@suse.com>
Signed-off-by: Clay Kauzlaric <ckauzlaric@vmware.com>
* docs: remove comma
* as per @carlisia 's suggestion
Signed-off-by: Clay Kauzlaric <ckauzlaric@vmware.com>
Co-authored-by: Clay Kauzlaric <ckauzlaric@vmware.com>
Co-authored-by: Clay Kauzlaric <clay.kauzlaric@gmail.com>
Co-authored-by: JenTing Hsiao <jenting.hsiao@suse.com>
In preparation for modifying the instantiation of `BackupStores` to be
able to load credentials, change the function `NewObjectBackupStore` to
be an interface that is passed in to all controllers.
Previously, the function to get a new backup store was configurable but
for many controllers was fixed to use `NewObjectBackupStore`. This
change introduces an interface for getting the backup store and wraps
the functionality from `NewObjectBackupStore` in a type which implements
this interface. This will allow more flexibility when introducing
credentials for a specific backup store as it will allow us to create a
new `ObjectBackupStoreGetter` type which can be configured to add
credentials config when creating the ObjectBackupStore without needing
to change the API used by the controllers.
Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>
now versions are working and there are code changes that need to happen
- release candidate versions are aligned and working
- replaces fields are removed and not required anymore
controller runtime has been changed during the 'make' command
Signed-off-by: Ron Green <11993626+georgettica@users.noreply.github.com>
* Remove unnecessary files
Signed-off-by: Carlisia <carlisia@vmware.com>
* Switch to CAPI patch function for updates
Signed-off-by: Carlisia <carlisia@vmware.com>
* Improve table test format
Signed-off-by: Carlisia <carlisia@vmware.com>
* Refactor and add test for disabling controller
Signed-off-by: Carlisia <carlisia@vmware.com>
* Add tests
Signed-off-by: Carlisia <carlisia@vmware.com>
* Change test to use real word
Signed-off-by: Carlisia <carlisia@vmware.com>
* Fix CI
Signed-off-by: Carlisia <carlisia@vmware.com>
* Minor test fixes
Signed-off-by: Carlisia <carlisia@vmware.com>
* Remove rback/role generation
Signed-off-by: Carlisia <carlisia@vmware.com>
* added useOwnerReferencesInBackup to crd velerio.io_schedules
Signed-off-by: matheusjuvelino <matheus.juvelino@outlook.com>
* added UseOwnerReferencesInBackup property to schedule.go
Signed-off-by: matheusjuvelino <matheus.juvelino@outlook.com>
* deepcopy schedule configured for reference the property UseOwnerReferencesInBackup
Signed-off-by: matheusjuvelino <matheus.juvelino@outlook.com>
* added UseOwnerReferencesInBackup property verification to modify OwnerReferences from backup
Signed-off-by: matheusjuvelino <matheus.juvelino@outlook.com>
* created changelog
Signed-off-by: matheusjuvelino <matheus.juvelino@outlook.com>
* removed deepcopy schedule configured for reference the property UseOwnerReferencesInBackup
Signed-off-by: matheusjuvelino <matheus.juvelino@outlook.com>
* running make update
Signed-off-by: matheusjuvelino <matheus.juvelino@outlook.com>
* running make update
Signed-off-by: matheusjuvelino <matheus.juvelino@outlook.com>
* updated the year at the top of the schedule.go file for 2020
Signed-off-by: matheusjuvelino <matheus.juvelino@outlook.com>
* -> Preserve nodePort support when restoring via "--preserve-nodeports" flag
Signed-off-by: Yusuf Güngör <yusuf.gungor@hepsiburada.com>
* -> Added changelog.
Signed-off-by: Yusuf Güngör <yusuf.gungor@hepsiburada.com>
* -> Unit test added.
-> Using boolptr.IsSetToTrue for bool ptr check.
Signed-off-by: Yusuf Güngör <yusuf.gungor@hepsiburada.com>
* -> Unit test added.
-> Using boolptr.IsSetToTrue for bool ptr check.
Signed-off-by: Yusuf Güngör <yusuf.gungor@hepsiburada.com>
* -> Other restore errors log level changed from info to error.
-> Documentation updated about Velero nodePort restore logic and preservation of them.
Signed-off-by: Yusuf Güngör <yusuf.gungor@hepsiburada.com>
Co-authored-by: Yusuf Güngör <yusuf.gungor@hepsiburada.com>
* Add default field to BSL CRD
Signed-off-by: JenTing Hsiao <jenting.hsiao@suse.com>
* Add a new flag `--default` under `velero backup-location create`
add a new flag `--default` under `velero backup-location create`
to specify this new location to be the new default BSL.
Signed-off-by: JenTing Hsiao <jenting.hsiao@suse.com>
* Add a new default field under `velero backup-location get`
add a new default field under `velero backup-location get` to indicate
which BSL is the default one.
Signed-off-by: JenTing Hsiao <jenting.hsiao@suse.com>
* Add a new sub-command and flag under `velero backup-location`
Add a new sub-command called `velero backup-location set` sub-command
and a new flag `velero backup-cation set --default` to configure which
BSL is the default one.
Signed-off-by: JenTing Hsiao <jenting.hsiao@suse.com>
* Add new flag to get the default backup-location
Add a new flag `--default` under `velero backup-location get`
to displays the current default BSL.
Signed-off-by: JenTing Hsiao <jenting.hsiao@suse.com>
* Configures default BSL in BSL controller
When upgrade the BSL CRDs, none of the BSL has been labeled as default.
Sets the BSL default field to true if the BSL name matches to the default BSL setting.
Signed-off-by: JenTing Hsiao <jenting.hsiao@suse.com>
* Configures the default BSL in BSL controller for velero upgrade
When upgrade the BSL CRDs, none of the BSL be marked as the default.
Sets the BSL `.spec.default: true` if the BSL name matches against the
`velero server --default-backup-storage-location`.
Signed-off-by: JenTing Hsiao <jenting.hsiao@suse.com>
* Add unit test to test default BSL behavior
Signed-off-by: JenTing Hsiao <jenting.hsiao@suse.com>
* Update check which one is the default BSL in backup/backup_sync/restore controller
Signed-off-by: JenTing Hsiao <jenting.hsiao@suse.com>
* Add changelog
Signed-off-by: JenTing Hsiao <jenting.hsiao@suse.com>
* Update docs locations.md and upgrade-to-1.6.md
Signed-off-by: JenTing Hsiao <jenting.hsiao@suse.com>
* 🐛 BSLs with validation disabled should be validated at least once
Signed-off-by: Ashish Amarnath <ashisham@vmware.com>
* review comments
Signed-off-by: Ashish Amarnath <ashisham@vmware.com>
* feat: add delete sub-command for backup-location
Signed-off-by: JenTing Hsiao <jenting.hsiao@suse.com>
* Change to use kubebuilder/runtimecontroller API
Signed-off-by: JenTing Hsiao <jenting.hsiao@suse.com>
* fix get BSL by label doesn't work
Signed-off-by: JenTing Hsiao <jenting.hsiao@suse.com>
* Update changelog
Signed-off-by: JenTing Hsiao <jenting.hsiao@suse.com>
* Ordering by alphabet
Signed-off-by: JenTing Hsiao <jenting.hsiao@suse.com>
* Better example format for help message
Signed-off-by: JenTing Hsiao <jenting.hsiao@suse.com>
* Capital the comments
Signed-off-by: JenTing Hsiao <jenting.hsiao@suse.com>
* Don't fail backup if downloading tarball fails
Previously, we would always attempt to download the tarball for a backup
for processing DeleteItemAction plugins, even if there weren't any.
This caused an issue for some users in the case where the backup tarball
had been deleted from object storage as the backup deletion would fail.
Now, we only attempt to download the tarball in the case where there are
DeleteItemAction plugins. If downloading that tarball fails, we log
the error, skip the processing of the DeleteItemAction plugins and
proceed with the rest of the deletion.
Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>
* Skip file removal in closeAndRemoveFile if nil
Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>
* pass annotations from scheduler to created backup
Signed-off-by: Michael <michael.ketslah@tufin.com>
* add change log
Signed-off-by: Michael <michael.ketslah@tufin.com>
* add test for annotations in controller
Signed-off-by: Michael <michael.ketslah@tufin.com>
* If no annotations are set - do not copy empty list
Signed-off-by: Michael <michael.ketslah@tufin.com>
* remove unneeded var
Signed-off-by: Michael <michael.ketslah@tufin.com>
* add empty annotations and actually check annotations in backups
Signed-off-by: Michael <michael.ketslah@tufin.com>
* add empty missing label and empty annotations
Signed-off-by: Michael <michael.ketslah@tufin.com>
* revert empty annotations as seems they are nil as expected
Signed-off-by: Michael <michael.ketslah@tufin.com>
* fix typo in changelog
Signed-off-by: Michael <michael.ketslah@tufin.com>
Co-authored-by: Michael <michael.ketslah@tufin.com>
By running the following command:
codespell -S .git,*.png,*.jpg,*.woff,*.ttf,*.gif,*.ico -L \
iam,aks,ist,bridget,ue
Signed-off-by: Mateusz Gozdek <mgozdekof@gmail.com>
* fixing label for 'velero.io/change-pvc-node-selector' plugin in site document
Signed-off-by: mayank <mayank.patel@mayadata.io>
* Fixing "velero.io/change-pvc-node-selector" to fetch config using plugin name
Signed-off-by: mayank <mayank.patel@mayadata.io>
* adding changelog
Signed-off-by: mayank <mayank.patel@mayadata.io>
This change modifies the kubebuilder annotations for the Velero CRDs to
include `additionalPrinterColumns` so that more information is exposed
when using `kubectl get`.
For each of the CRDs, annotations have been added to make the output
for `kubectl get` match the output from the equivalent `velero get`
command as closely as possible. There are some cases where this output
could not be replicated, such as the `EXPIRES` column for Backups, due
to the limitations of JSONPath expressions within the resulting CRD
defition. Some columns undergo processing and formatting before being
printed by the Velero CLI which cannot be replicated using JSONPath. In
these cases, these printer columns have been omitted.
For other CRDs where there is no `velero get` equivalent, such as
`PodVolumeBackup` and `PodVolumeRestore`, a best effort has been made to
expose information that provides value.
Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>
* Adding handling of restic-wait init container at any order with warning.
Signed-off-by: Piper Dougherty <doughertypiper@gmail.com>
* Adding newline at end of files to match convention.
Signed-off-by: Piper Dougherty <doughertypiper@gmail.com>
* Formatting.
Signed-off-by: Piper Dougherty <doughertypiper@gmail.com>
* Update copyright year on modified files.
Signed-off-by: Piper Dougherty <doughertypiper@gmail.com>
* Only remove the UID from a PV's claimRef
The UID is the only part of a claimRef that might prevent it from being
rebound correctly on a restore. The namespace and name within the
claimRef should be preserved in order to ensure that the PV is claimed
by the correct PVC on restore.
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
* Remap PVs claimRef.namespace on relevant restores
When remapping namespaces, any included PVs need to have their claimRef
updated to point remapped namespaces to the new namespace name in order
to be bound to the correct PVC.
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
* Update tests and ensure claimRef namespace remaps
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
* Remove lowercased uid field from unstructured PV
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
* Fix issues that prevented PVs from being restored
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
* Add changelog
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
* Dynamically reprovision volumes without snapshots
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
* Update test for lower case uid field
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
* Remove stray debugging print statement
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
* Fix typo, remove extra code, add tests.
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
* create CRB with velero-<namespace>
This will allow creating multiple instances of velero,
across two different namespaces
Signed-off-by: Alay Patel <alay1431@gmail.com>
* add changelog
Signed-off-by: Alay Patel <alay1431@gmail.com>
* add package var DefaultVeleroNamespace and use it wherever needed
Signed-off-by: Alay Patel <alay1431@gmail.com>
* restore proper lowercase/plural CRD resource
This commit restores the proper resource string
"customresourcedefinitions" for CRD. The prior change to
"CustomResourceDefinition" was made because this was being used
in another place to populate the CRD "Kind" field in
remap_crd_version_action.go -- there, just use the correct Kind
string instead of pulling from Resource.
Signed-off-by: Scott Seago <sseago@redhat.com>
* add changelog
Signed-off-by: Scott Seago <sseago@redhat.com>
* Show format version on velero backup describe
Signed-off-by: JenTing Hsiao <jenting.hsiao@suse.com>
* Add changelog
Signed-off-by: JenTing Hsiao <jenting.hsiao@suse.com>
* Exec hooks in restored pods
Signed-off-by: Andrew Reed <andrew@replicated.com>
* WaitExecHookHandler implements ItemHookHandler
This required adding a context.Context argument to the ItemHookHandler
interface which is unused by the DefaultItemHookHandler implementation.
It also means passing nil for the []ResourceHook argument since that
holds BackupResourceHook.
Signed-off-by: Andrew Reed <andrew@replicated.com>
* WaitExecHookHandler unit tests
Signed-off-by: Andrew Reed <andrew@replicated.com>
* Changelog and go fmt
Signed-off-by: Andrew Reed <andrew@replicated.com>
* Fix double import
Signed-off-by: Andrew Reed <andrew@replicated.com>
* Default to first contaienr in pod
Signed-off-by: Andrew Reed <andrew@replicated.com>
* Use constants for hook error modes in tests
Signed-off-by: Andrew Reed <andrew@replicated.com>
* Revert to separate WaitExecHookHandler interface
Signed-off-by: Andrew Reed <andrew@replicated.com>
* Negative tests for invalid timeout annotations
Signed-off-by: Andrew Reed <andrew@replicated.com>
* Rename NamedExecRestoreHook PodExecRestoreHook
Also make field names more descriptive.
Signed-off-by: Andrew Reed <andrew@replicated.com>
* Cleanup test names
Signed-off-by: Andrew Reed <andrew@replicated.com>
* Separate maxHookWait and add unit tests
Signed-off-by: Andrew Reed <andrew@replicated.com>
* Comment on maxWait <= 0
Also info log container is not running for hooks to execute in.
Also add context error to hooks not executed errors.
Signed-off-by: Andrew Reed <andrew@replicated.com>
* Remove log about default for invalid timeout
There is no default wait or exec timeout.
Signed-off-by: Andrew Reed <andrew@replicated.com>
* Linting
Signed-off-by: Andrew Reed <andrew@replicated.com>
* Fix log message and rename controller to podWatcher
Signed-off-by: Andrew Reed <andrew@replicated.com>
* Comment on exactly-once semantics for handler
Signed-off-by: Andrew Reed <andrew@replicated.com>
* Fix logging and comments
Use filed logger for pod in handler.
Add comment about pod changes in unit tests.
Use kube util NamespaceAndName in messages.
Signed-off-by: Andrew Reed <andrew@replicated.com>
* Fix maxHookWait
Signed-off-by: Andrew Reed <andrew@replicated.com>
* fix: rename the PV if VolumeSnapshotter has modified the PV name
When VolumeSnapshotter sets the PV name via SetVolumeID and PV is
not there in the cluster, velero does not rename the PV. Which causes
the pvc to be in the lost state as pvc points to the old PV but pv object
has been renamed by VolumeSnapshotter.
Signed-off-by: Pawan <pawan@mayadata.io>
* adding a test case for pv rename
Signed-off-by: Pawan <pawan@mayadata.io>
* remove explicit Accept-Encoding header
For StorageGrid compatibility the Accept-Encoding header should not be set, otherwise StorageGrid compresses the already compressed log files which are only decompressed by the client once
Signed-off-by: fvsqr <48791253+fvsqr@users.noreply.github.com>
* Removed explicit gzip Accept-Encoding header
For StorageGrid compatibility the Accept-Encoding header should not be set, otherwise StorageGrid compresses the already compressed log files which are only decompressed by the client once.
Unclear, how this affects Backup endpoints from Azure or GCP
Signed-off-by: fvsqr <48791253+fvsqr@users.noreply.github.com>
* Create 2712-fvsqr
Signed-off-by: fvsqr <48791253+fvsqr@users.noreply.github.com>
* k8s 1.18 import wip
backup, cmd, controller, generated, restic, restore, serverstatusrequest, test and util
Signed-off-by: Andrew Lavery <laverya@umich.edu>
* go mod tidy
Signed-off-by: Andrew Lavery <laverya@umich.edu>
* add changelog file
Signed-off-by: Andrew Lavery <laverya@umich.edu>
* go fmt
Signed-off-by: Andrew Lavery <laverya@umich.edu>
* update code-generator and controller-gen in CI
Signed-off-by: Andrew Lavery <laverya@umich.edu>
* checkout proper code-generator version, regen
Signed-off-by: Andrew Lavery <laverya@umich.edu>
* fix remaining calls
Signed-off-by: Andrew Lavery <laverya@umich.edu>
* regenerate CRDs with ./hack/update-generated-crd-code.sh
Signed-off-by: Andrew Lavery <laverya@umich.edu>
* use existing context in restic and server
Signed-off-by: Andrew Lavery <laverya@umich.edu>
* fix test cases by resetting resource version
also use main library go context, not golang.org/x/net/context, in pkg/restore/restore.go
Signed-off-by: Andrew Lavery <laverya@umich.edu>
* clarify changelog message
Signed-off-by: Andrew Lavery <laverya@umich.edu>
* use github.com/kubernetes-csi/external-snapshotter/v2@v2.2.0-rc1
Signed-off-by: Andrew Lavery <laverya@umich.edu>
* run 'go mod tidy' to remove old external-snapshotter version
Signed-off-by: Andrew Lavery <laverya@umich.edu>
* updated acceptable values on cron schedule for day of the week from 0-7 to 0-6
Signed-off-by: Daniel Thrasher <dannythrasher@gmail.com>
* added a changelog file to changelog directory
Signed-off-by: Daniel Thrasher <dannythrasher@gmail.com>
Co-authored-by: Daniel Thrasher <dannythrasher@gmail.com>
* kubebuilder init - minimalist version
Signed-off-by: Carlisia <carlisia@vmware.com>
* Add back main.go, apparently kb needs it
Signed-off-by: Carlisia <carlisia@vmware.com>
* Tweak makefile to accomodate kubebuilder expectations
Signed-off-by: Carlisia <carlisia@vmware.com>
* Port BSL to kubebuilder api client
Signed-off-by: Carlisia <carlisia@vmware.com>
* s/cache/client bc client fetches from cache
And other naming improvements
Signed-off-by: Carlisia <carlisia@vmware.com>
* So, .GetAPIReader is how we bypass the cache
In this case, the cache hasn't started yet
Signed-off-by: Carlisia <carlisia@vmware.com>
* Oh that's what this code was for... adding back
We still need to embed the CRDs as binary data in the Velero binary to
access the generated CRDs at runtime.
Signed-off-by: Carlisia <carlisia@vmware.com>
* Tie in CRD/code generation w/ existing scripts
Signed-off-by: Carlisia <carlisia@vmware.com>
* Mostly result of running update-fmt, updated file formatting
Signed-off-by: Carlisia <carlisia@vmware.com>
* Just a copyright fix
Signed-off-by: Carlisia <carlisia@vmware.com>
* All the test fixes
Signed-off-by: Carlisia <carlisia@vmware.com>
* Add changelog + some cleanup
Signed-off-by: Carlisia <carlisia@vmware.com>
* Update backup manifest
Signed-off-by: Carlisia <carlisia@vmware.com>
* Remove unneeded auto-generated files
Signed-off-by: Carlisia <carlisia@vmware.com>
* Keep everything in the same (existing) package
Signed-off-by: Carlisia <carlisia@vmware.com>
* Fix/clean scripts, generated code, and calls
Deleting the entire `generated` directory and running `make update`
works. Modifying an api and running `make verify` works as expected.
Signed-off-by: Carlisia <carlisia@vmware.com>
* Clean up schema and client calls + code reviews
Signed-off-by: Carlisia <carlisia@vmware.com>
* Move all code gen to inside builder container
Signed-off-by: Carlisia <carlisia@vmware.com>
* Address code review
Signed-off-by: Carlisia <carlisia@vmware.com>
* Fix imports/aliases
Signed-off-by: Carlisia <carlisia@vmware.com>
* More code reviews
Signed-off-by: Carlisia <carlisia@vmware.com>
* Add waitforcachesync
Signed-off-by: Carlisia <carlisia@vmware.com>
* Have manager register ALL controllers
This will allow for proper cache management.
Signed-off-by: Carlisia <carlisia@vmware.com>
* Status subresource is now enabled; cleanup
Signed-off-by: Carlisia <carlisia@vmware.com>
* More code reviews
Signed-off-by: Carlisia <carlisia@vmware.com>
* Clean up
Signed-off-by: Carlisia <carlisia@vmware.com>
* Manager registers ALL controllers for restic too
Signed-off-by: Carlisia <carlisia@vmware.com>
* More code reviews
Signed-off-by: Carlisia <carlisia@vmware.com>
* Add deprecation warning/todo
Signed-off-by: Carlisia <carlisia@vmware.com>
* Add documentation
Signed-off-by: Carlisia <carlisia@vmware.com>
* Add helpful comments
Signed-off-by: Carlisia <carlisia@vmware.com>
* Address code review
Signed-off-by: Carlisia <carlisia@vmware.com>
* More idiomatic Runnable
Signed-off-by: Carlisia <carlisia@vmware.com>
* Clean up imports
Signed-off-by: Carlisia <carlisia@vmware.com>
* log a warning instead of erroring if additional item can't be found
Signed-off-by: Steve Kriss <krisss@vmware.com>
* always show backup warning/error count in get/describe
Signed-off-by: Steve Kriss <krisss@vmware.com>
* changelog
Signed-off-by: Steve Kriss <krisss@vmware.com>
* Use a helper function when querying w/ backup label
Setting or querying for a backup label name should always pass the value
through the GetValidName function. This change passes query uses of the
backup label value through the GetValidName function by introducing 2
new helpers, one for making a Selector, one for making a ListOptions.
It also removes functions returning the same data, but under
unecessarily specific names.
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
* Document using the label.GetValidName function
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
* Update copyright year
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
* Clarify labels.GetValidName and annotations
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
* Move functions to pkg/label
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
* Fix function comments
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
* Switch to backing up v1beta1 CRDs from API server
Instead of simply switching out the APIVersion string on a v1
CustomResourceDefinition object, re-download the object from the API
server entirely to get the correct fields.
This should fix validation errors upon restore.
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
* Fix existing tests
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
* Add full example CRDs to automated tests
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
* Move beta CRD lookup into helper function
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
* Add case for preserveUnknownFields CRDs
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
* Add PreserveUnknownFields case and refactor execute
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
* Add older prometheus CRD test cases
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
* Add changelog
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
* Backup all API Groups versions while keeping backward compatibility
Signed-off-by: Rafael Brito <rbrito@vmware.com>
* Backup all API Groups versions while keeping backward compatibility
Signed-off-by: Rafael Brito <rbrito@vmware.com>
* Adding feature flag to enable backup of multiple API group versions
Signed-off-by: Rafael Brito <rbrito@vmware.com>
* bug fix: don't remove unresolvable includes from includes-excludes lists
Signed-off-by: Steve Kriss <krisss@vmware.com>
* changelog
Signed-off-by: Steve Kriss <krisss@vmware.com>
* clarify the wording for restore describe for namespaces included
Instead of showing it as "*" explicitly mention that all the namespaces
from the backup object are included.
refer to https://github.com/vmware-tanzu/velero/issues/1918
Signed-off-by: Raghavendra M <raghavendra@redhat.com>
* Update pkg/cmd/util/output/restore_describer.go
Co-Authored-By: Ashish Amarnath <ashisham@vmware.com>
Signed-off-by: Raghavendra M <raghavendra@redhat.com>
* Update pkg/cmd/util/output/restore_describer.go
Co-Authored-By: Ashish Amarnath <ashisham@vmware.com>
Signed-off-by: Raghavendra M <raghavendra@redhat.com>
Co-authored-by: Ashish Amarnath <ashisham@vmware.com>
* Disabling validation for volumesnapshotlocation if the backup has snapshotvolume set to false
Signed-off-by: mayank <mayank.patel@mayadata.io>
* adding a changelog
Signed-off-by: mayank <mayank.patel@mayadata.io>
* addressing review comment
Signed-off-by: mayank <mayank.patel@mayadata.io>
Infomers won't start if cancelFunc is invoked as soon as the newServer
function exits via the defer
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
Account for having CSI enabled or not, as well as having the snapshot
CRDs installed in the kubernetes cluster.
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
* Add --cacert flag to velero cli commands
Adds a --cacert flag to the log and describe commands
that takes a path to a PEM-encoded certificate bundle
as an alternative to --insecure-skip-tls-verify for
dealing with self-signed certificates.
Signed-off-by: Sam Lucidi <slucidi@redhat.com>
* Add --cacert flag to the installer
Allows setting the cacert field on the BSL during
the install process using the file at the path
specified by the --cacert field.
Signed-off-by: Sam Lucidi <slucidi@redhat.com>
* Add changelog for #2368
Signed-off-by: Sam Lucidi <slucidi@redhat.com>
CSI plugin for velero will use this to return secrets as additional
resource while backing up CSI objects
Signed-off-by: Ashish Amarnath <ashisham@vmware.com>
* Support setting a custom CA certificate for a BSL
Signed-off-by: Sam Lucidi <slucidi@redhat.com>
* update CRDS
Signed-off-by: Sam Lucidi <slucidi@redhat.com>
* Add changelog for #2353
Signed-off-by: Sam Lucidi <slucidi@redhat.com>
* Clean up temp file from TestTempCACertFile
Signed-off-by: Sam Lucidi <slucidi@redhat.com>
* Add builders for CRD schemas
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
* Add test case for #2319
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
* Add failing test case
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
* Remove unnecessary print and temporary variable
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
* Add some options for fixing the test case
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
* Switch to a JSON middle step to "fix" conversions
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
* Add comment and changelog
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
* Account for possible missing schemas on v1 CRDs
If a v1beta1 CRD without a Schema was submitted to a Kubernets v1.16
cluster, then Kubernetes will server it back as a v1 CRD without a
schema.
However, when Velero tries to restore this document, the request will be
rejected as a v1 CRD must have a schema.
This commit has some defensive coding on the restore side, as well as
potential fixes on the backup side for getting around this.
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
* Back up nonstructural CRDs as v1beta1
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
* Add tests for remapping plugin
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
* Add builders for v1 CRDs
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
* Address review feedback
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
* Remove extraneous log message
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
* Add changelog
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
* Wait for CRDs to be available and ready
When restoring CRDs, we should wait for the definition to be ready and
available before moving on to restoring specific CRs.
While the CRDs are often ready by the time we get to restoring a CR,
there is a race condition where the CRD isn't ready.
This change waits on each CRD at restore time.
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
* Pruning unknown fields
In CRD apiversion v1beta1, default preserveUnknownFields=true.
In CRD apiversion v1, the preserveUnknownFields can only be false.
Otherwise, the k8s validation bumps out error message for the
invalid preserveUnknownFields value.
Deploy Velero on k8s 1.16+ with CRD apiversion v1beta1, the
k8s cluster converts apiversion from v1beta1 to v1 automatically.
Fully backup and restore the cluster, restore bumps out error message
due to the preserveUnknownFields=true is not allowed on k8s 1.16+.
Since the CRD structural schema had been defined, enable the preserveUnknownFields
to false to solves the restore bumps out error message on k8s 1.16+.
Signed-off-by: JenTing Hsiao <jenting.hsiao@suse.com>
* Add changelog
Signed-off-by: JenTing Hsiao <jenting.hsiao@suse.com>
Velero client config file should have restricted file permissions to be
read/write-able for the user that creates it--similiar to files like
`.ssh/id_rsa`
Refer to OTG-CONFIG-009: Test File Permission
> Impoper file permission configuration may result in privilledge
escalation, information explousure, DLL injection, or unauthorized file
access.
Therefore, files permission must be properly configured with minium
access permission by default.
[source](https://www.owasp.org/index.php/Test_File_Permission_(OTG-CONFIG-009))
Ticket: #1758
Signed-off-by: John Naulty <johnnaulty@bitgo.com>
* Check for nil LastMaintenanceTime in dueForMaintenance
ResticRepository.dueForMaintenance causes a panic in the velero pod
("invalid memory address or nil pointer dereference") if
repository.Status.LastMaintenanceTime is nil. This fix returns 'true'
if it's nil, so the repository is due for maintenance if LastMaintenanceTime
is nil *or* the time elapsed since the last maintenance is greater than
repository.Spec.MaintenanceFrequency.Duration
Signed-off-by: Scott Seago <sseago@redhat.com>
* changelog for PR#2200
Signed-off-by: Scott Seago <sseago@redhat.com>
* update revision of go-hclog to match go.mod requirement
Signed-off-by: Steve Kriss <krisss@vmware.com>
* update prometheus dep to prepare for go module migration
Signed-off-by: Steve Kriss <krisss@vmware.com>
Install restic with CPU/Memory limits is optional.
If velero cannot parse resource requirements, use default value instead.
After that, the administrator won't get confused that something recovered failed.
Signed-off-by: JenTing Hsiao <jenting.hsiao@suse.com>
Migrate logic from NewUUID function into the pvRenamer function.
PR #2133 switched to a new NewUUID function that returns an error, but
the invocation of that function needs to happen within the pvRenamer
closure. Because the new function returns an error, the pvRenamer should
return the error, the signature needs to be changed and the return
checked.
Signed-off-by: John Naulty <johnnaulty@bitgo.com>
satori/go.uuid has a known issue with random uuid generation.
gofrs/uuid is still maintained and has fixed the random uuid generation
issue present in satori/go.uuid
Signed-off-by: John Naulty <johnnaulty@bitgo.com>
* restic: don't try to restore PVBs with no snapshotID
Signed-off-by: Steve Kriss <krisss@vmware.com>
* changelog
Signed-off-by: Steve Kriss <krisss@vmware.com>
* restic: use restore cmd's env when getting snapshot size
Signed-off-by: Steve Kriss <krisss@vmware.com>
* restic: remove code that considers 0-byte backups an error
Signed-off-by: Steve Kriss <krisss@vmware.com>
* add --allow-partially-failed flag to velero restore create
Signed-off-by: Steve Kriss <krisss@vmware.com>
* remove extraneous client creation
Signed-off-by: Steve Kriss <krisss@vmware.com>
* add godoc to helper func
Signed-off-by: Steve Kriss <krisss@vmware.com>
* todo
Signed-off-by: Steve Kriss <krisss@vmware.com>
Related issue: https://github.com/heptio/velero/issues/1830
This accomplishes everything
that's needed, although there might be room for improvement in avoiding
a GET call for matching CRDs for each resource backed up. An alternative
could be a single call to get all CRDs prior to iterating over resources
and passing this into the backupResource function.
Signed-off-by: Scott Seago <sseago@redhat.com>
* feat: add azure china support
Signed-off-by: andyzhangx <xiazhang@microsoft.com>
* remove AZURE_CLOUD_NAME from required env var fetching
Signed-off-by: Steve Kriss <krisss@vmware.com>
* minor simplification of parseAzureEnvironment
Signed-off-by: Steve Kriss <krisss@vmware.com>
* changelog
Signed-off-by: Steve Kriss <krisss@vmware.com>
* remove cloudNameEnvVar from getRequiredValues call
Signed-off-by: Steve Kriss <krisss@vmware.com>
* just check for err != nil
Signed-off-by: Steve Kriss <krisss@vmware.com>
* Allow the velero server to be created on GCP even without a provided service account key in order to support workload identity and default compute engine credentials. Add option for adding service account annotations.
Signed-off-by: Joshua Wong <joshua99wong@gmail.com>
* update import paths to github.com/vmware-tanzu/...
Signed-off-by: Steve Kriss <krisss@vmware.com>
* update other GH org refs to vmware-tanzu
Signed-off-by: Steve Kriss <krisss@vmware.com>
* site and docs: update GH org to vmware-tanzu
Signed-off-by: Steve Kriss <krisss@vmware.com>
* update travis badge links on docs readmes
Signed-off-by: Steve Kriss <krisss@vmware.com>
* backup sync controller: replace revision file with full diff each interval
Signed-off-by: Steve Kriss <krisss@vmware.com>
* remove getting/setting of metadata/revision file
Signed-off-by: Steve Kriss <krisss@vmware.com>
* changelog
Signed-off-by: Steve Kriss <krisss@vmware.com>
* tweak logging
Signed-off-by: Steve Kriss <krisss@vmware.com>
* don't keep podVolumeBackup log field around after syncing PVBs
Signed-off-by: Steve Kriss <krisss@vmware.com>
* update generated CRDs
Signed-off-by: Steve Kriss <krisss@vmware.com>
* velero install: wait for restic daemonset to be ready
Signed-off-by: Steve Kriss <krisss@vmware.com>
* changelog
Signed-off-by: Steve Kriss <krisss@vmware.com>
A user encountered the following error on a GCP project:
An error occurred: some backup storage locations are invalid: error getting backup store for location "default": rpc error: code = Unknown desc = invalid character '-' in numeric literal
This error was ambiguous and took some time to track down to the fact
that their credentials file wasn't a JSON file, but instead the contents
of the private key field. This change makes the problem slightly easier
to debug.
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
* rename PV during restore when cloning a namespace
Signed-off-by: Steve Kriss <krisss@vmware.com>
* rename func and vars, switch to if..else
Signed-off-by: Steve Kriss <krisss@vmware.com>
* make pv renamer func configurable for testing purposes
Signed-off-by: Steve Kriss <krisss@vmware.com>
* add unit test cases
Signed-off-by: Steve Kriss <krisss@vmware.com>
* changelog
Signed-off-by: Steve Kriss <krisss@vmware.com>
* address review feedback
Signed-off-by: Steve Kriss <krisss@vmware.com>
* address review feedback
Signed-off-by: Steve Kriss <krisss@vmware.com>
* when backing up PVCs with restic, explicitly specify --parent
Signed-off-by: Steve Kriss <krisss@vmware.com>
* changelog
Signed-off-by: Steve Kriss <krisss@vmware.com>
* address review feedback
Signed-off-by: Steve Kriss <krisss@vmware.com>
Carlisia Thompson
codegold79
Bridget McErlean
JenTing Hsiao
Nolan Brubaker
Mikael Manukyan
Taeuk Kim
steve_chph
Ron Green
Madhav Jivrajani
Dave Smith-Uchida
matheusjuvelino
yusufgungor
Ashish Amarnath
Misha Ketslah
Mateusz Gozdek
Mayank
Hariharan
Piper Dougherty
Alay Patel
Antony S Bett
Scott Seago
Andrew Reed
Pawan Prakash Sharma
Dylan Murray
Benoit Gagnon
runzexia
Phuong N. Hoang
Jason Scarano
Benoit Gagnon
Martin Odstrčilík
Thejas Babu
fvsqr
kathpeony
Andrew Lavery
Daniel Thrasher
Steve Kriss
Christoph Blecker
Jonas Rosland
Rafael Brito
FNU Raghavendra Manjunath
Samuel Lucidi
Samuel Lucidi
Brad Koehn
Carlos Tadeu Panato Junior
John Naulty
Shuwei Hao
Dinesh Yadav
Roman Klimenko
Antony S Bett
Adnan Abdulhussein
Adnan Abdulhussein
Alexander Demichev
FrankChen7
Steve Chung
Joshua Wong
Moritz Schmitz von Hülst
Christopher Angelo Phillips
lintongj
KubeKween
Steven Chung