121dc02d2f
Ensure object store plugin processes are cleaned up after restore and after BSL validation during server start up ( #2041 )
...
* Ensure child object store plugin processes are cleaned up during restore
Signed-off-by: Antony Bett <antony.bett@gmail.com>
2019-11-05 08:58:08 -08:00
0450567bba
Allow backup storage locations to specify backup sync period or toggle off sync ( #1936 )
...
* Allow backup storage locations to specify backup sync period or toggle off sync
Signed-off-by: Antony Bett <antony.bett@gmail.com>
* Add --backup-sync-period flag to backup location create command
Signed-off-by: Antony Bett <antony.bett@gmail.com>
2019-10-24 08:32:45 -06:00
e3d64d9dd9
use pointer types for metav1.Time fields ( #1951 )
...
* use pointer types for metav1.Time fields
Signed-off-by: Adnan Abdulhussein <aadnan@vmware.com>
* simpler metav1.Time ptrs
Signed-off-by: Adnan Abdulhussein <aadnan@vmware.com>
* remove test debug println
Signed-off-by: Adnan Abdulhussein <aadnan@vmware.com>
2019-10-14 10:20:28 -06:00
48792ece1f
Updates for org move to vmware-tanzu ( #1920 )
...
* update import paths to github.com/vmware-tanzu/...
Signed-off-by: Steve Kriss <krisss@vmware.com>
* update other GH org refs to vmware-tanzu
Signed-off-by: Steve Kriss <krisss@vmware.com>
* site and docs: update GH org to vmware-tanzu
Signed-off-by: Steve Kriss <krisss@vmware.com>
* update travis badge links on docs readmes
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-09-30 17:26:56 -04:00
d96186473c
sync controller: replace revision file with full diff each interval ( #1892 )
...
* backup sync controller: replace revision file with full diff each interval
Signed-off-by: Steve Kriss <krisss@vmware.com>
* remove getting/setting of metadata/revision file
Signed-off-by: Steve Kriss <krisss@vmware.com>
* changelog
Signed-off-by: Steve Kriss <krisss@vmware.com>
* tweak logging
Signed-off-by: Steve Kriss <krisss@vmware.com>
* don't keep podVolumeBackup log field around after syncing PVBs
Signed-off-by: Steve Kriss <krisss@vmware.com>
* update generated CRDs
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-09-27 16:23:11 -04:00
23376e6293
record restic restore progress in PodVolumeRestore ( #1854 )
...
* record restic restore progress in PodVolumeRestore
Signed-off-by: Adnan Abdulhussein <aadnan@vmware.com>
* add progress output to velero describe restore --details
Signed-off-by: Adnan Abdulhussein <aadnan@vmware.com>
* test getVolumeSize
Signed-off-by: Adnan Abdulhussein <aadnan@vmware.com>
* block on send to finish goroutine
Signed-off-by: Adnan Abdulhussein <aadnan@vmware.com>
* remove sync import
Signed-off-by: Adnan Abdulhussein <aadnan@vmware.com>
* add changelog
Signed-off-by: Adnan Abdulhussein <aadnan@vmware.com>
* make update
Signed-off-by: Adnan Abdulhussein <aadnan@vmware.com>
2019-09-10 15:50:57 -06:00
6623e1f273
Change restic prune default interval to 7d and make user-configurable ( #1864 )
...
* change restic prune default interval to 7d, add server flag for it
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-09-10 11:58:42 -07:00
1e182e5837
record restic backup progress in PodVolumeBackup ( #1821 )
...
* record restic backup progress in PodVolumeBackup
Signed-off-by: Adnan Abdulhussein <aadnan@vmware.com>
2019-09-10 11:28:19 -07:00
ef911ff21b
when backing up PVCs with restic, explicitly specify --parent ( #1807 )
...
* when backing up PVCs with restic, explicitly specify --parent
Signed-off-by: Steve Kriss <krisss@vmware.com>
* changelog
Signed-off-by: Steve Kriss <krisss@vmware.com>
* address review feedback
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-08-27 16:37:51 -07:00
6aa0215137
create backups from schedules using velero create backup ( #1734 )
...
* add --from-schedule to `velero backup create` to create backups from schedules
Signed-off-by: Adnan Abdulhussein <aadnan@vmware.com>
2019-08-23 13:03:51 -07:00
686f41ebec
remove calls to restic check before/after prune ( #1794 )
...
* remove calls to restic check before/after prune
Signed-off-by: Steve Kriss <krisss@vmware.com>
* changelog
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-08-23 10:53:00 -07:00
8b6c505817
error if restic repo identifier can't be determined
...
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-08-22 10:19:22 -06:00
5ef0a84128
create pod volume backups in backup's namespace during sync ( #1760 )
...
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-08-13 10:20:25 -07:00
ffa3251efc
move contents of pkg/util/test into pkg/test
...
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-08-07 08:26:00 -06:00
4accb8512a
Restore from PodVolumeBackups ( #1723 )
...
* Restore from PodVolumeBackups
Signed-off-by: Carlisia <carlisiac@vmware.com>
* Partially address code reviews
Signed-off-by: Carlisia <carlisiac@vmware.com>
* Partially address code reviews #2
Signed-off-by: Carlisia <carlisiac@vmware.com>
* Clean up struct
Signed-off-by: Carlisia <carlisiac@vmware.com>
* Fix log messages
Signed-off-by: Carlisia <carlisiac@vmware.com>
* Fix tests
Signed-off-by: Carlisia <carlisiac@vmware.com>
* Clean up
Signed-off-by: Carlisia <carlisiac@vmware.com>
* Add changelog
Signed-off-by: Carlisia <carlisiac@vmware.com>
2019-08-06 13:17:36 -07:00
6522ba7c42
fix bugs preventing pod volume backups from syncing properly
...
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-08-06 09:23:49 -06:00
07525bd593
store backup resource list metadata in object storage ( #1709 )
...
* move backedUpItems to pkg/backup.Request struct
Signed-off-by: Adnan Abdulhussein <aadnan@vmware.com>
* construct resource itemKey field from gvk
Signed-off-by: Adnan Abdulhussein <aadnan@vmware.com>
* store backup resource list metadata in object storage
Signed-off-by: Adnan Abdulhussein <aadnan@vmware.com>
* remove debug log
Signed-off-by: Adnan Abdulhussein <aadnan@vmware.com>
* fix formatting
Signed-off-by: Adnan Abdulhussein <aadnan@vmware.com>
* add missing license blocks and split BackupInfo struct lines
Signed-off-by: Adnan Abdulhussein <aadnan@vmware.com>
* add test for checking BackedUpItems matches tarball contents
Signed-off-by: Adnan Abdulhussein <aadnan@vmware.com>
* add comment to explain test
Signed-off-by: Adnan Abdulhussein <aadnan@vmware.com>
2019-08-05 11:15:55 -06:00
b24a603711
periodically check for stale restic repo locks ( #1708 )
...
* periodically check for stale restic repo locks
Signed-off-by: Steve Kriss <krisss@vmware.com>
* changelog
Signed-off-by: Steve Kriss <krisss@vmware.com>
* only try to init a restic repo if it doesn't already exist
Signed-off-by: Steve Kriss <krisss@vmware.com>
* reword comment
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-07-31 10:52:23 -07:00
f2d06bc5e9
strip leading/trailing slashes from BSL bucket & prefix vals ( #1694 )
...
* strip leading/trailing slashes from BSL bucket & prefix vals
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-07-31 08:27:12 -07:00
4543258970
Update tests to use object builders in pkg/builder ( #1707 )
...
* add pkg/builder with BackupBuilder and ObjectMeta functional options
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-07-31 07:46:48 -07:00
22eca22ac8
Add JSON as an option for log output ( #1654 )
...
* Add JSON as an option for log output
Signed-off-by: Donovan Carthew <donovan.carthew@gmail.com>
2019-07-30 16:29:34 -07:00
3b9af8c654
Store PodVolumeBackups in obj storage & use as source of truth ( #1577 )
...
* Store PodVolumeBackups in object storage
Signed-off-by: Carlisia <carlisiac@vmware.com>
2019-07-24 15:51:20 -04:00
63964fc6f9
Append "/mount" to directory for CSI volumes ( #1615 )
...
CSI volumes are mounted one level deeper than "native" kubernetes
volumes, and this needs to be appended for proper restic support.
Fixes #1313 .
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2019-07-10 15:16:21 -07:00
2156124dfc
record PodVolumeBackup/Restore start and completion timestamps ( #1609 )
...
* record PodVolumeBackup start and completion timestamps
adds startTimestamp and completionTimestamp fields to the
PodVolumeBackup status spec
Signed-off-by: Adnan Abdulhussein <aadnan@vmware.com>
* record PodVolumeRestore start and completion timestamps
Signed-off-by: Adnan Abdulhussein <aadnan@vmware.com>
2019-06-28 11:58:02 -04:00
19052994ed
replace TestBackup with pkg/backup.Builder ( #1593 )
...
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-06-21 17:08:08 -04:00
d6c60b2dd5
fix panic when processing DeleteBackupRequest objs without labels ( #1556 )
...
This fix initialises an empty map if the request object's Labels map
is nil, allowing the controller to later add and modify labels on the
object.
Signed-off-by: Adnan Abdulhussein <aadnan@vmware.com>
2019-06-10 10:04:51 -07:00
0a771e6a53
Add cmd to list plugins ( #1535 )
...
* Add cmd to list plugins
Signed-off-by: Carlisia <carlisiac@vmware.com>
2019-06-05 13:41:02 -04:00
0804f34644
Don't log error if restic volume is empty ( #1480 )
...
* Don't log error if restic volume is empty
Signed-off-by: Carlisia <carlisiac@vmware.com>
2019-05-30 12:48:21 -04:00
411d44a673
allow individual backup storage locations to be read-only ( #1517 )
...
* allow individual backup storage locations to be read-only
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-05-29 14:21:25 -04:00
9287505f62
fixes for breaking changes
...
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-05-20 13:25:36 -06:00
4e12b08953
Validate restore name label length
...
Velero should handle cases when the label length exceeds 63 characters.
- if the length of the backup/restore name is <= 63 characters, use it as the value of the label
- if it's > 63 characters, take the SHA256 hash of the name. the value of
the label will be the first 57 characters of the backup/restore name
plus the first six characters of the SHA256 hash.
Fixes heptio#1021
Signed-off-by: Anshul Chandra <anshulc@vmware.com>
2019-05-01 10:25:11 -07:00
660080e6f3
remove all remaining legacy Ark references ( #1415 )
...
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-04-29 12:33:54 -07:00
8392e6d83f
add PartiallyFailed phase for backups, log+continue on errors ( #1386 )
...
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-04-26 09:14:26 -07:00
bf19623e82
Merge pull request #1389 from skriss/restore-partially-failed
...
Add PartiallyFailed phase for restores
2019-04-25 17:06:26 -04:00
6f496a8921
Bug fix
...
Signed-off-by: Carlisia <carlisiac@vmware.com>
2019-04-24 14:18:24 -07:00
eb30ec0666
move restores to PartiallyFailed if >=1 error in restore results
...
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-04-24 12:57:40 -06:00
7d28f82540
Address code review
...
Signed-off-by: Carlisia <carlisiac@vmware.com>
2019-04-24 10:54:43 -07:00
38ccb40ca1
Add back code and tests
...
Signed-off-by: Carlisia <carlisiac@vmware.com>
2019-04-23 16:19:49 -07:00
05a88345e5
Code clean up
...
Signed-off-by: Carlisia <carlisiac@vmware.com>
2019-04-22 15:22:58 -07:00
01d0b026e9
Fail backup if it already exists in object storage
...
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2019-04-22 15:22:58 -07:00
475cf2ab60
add new metrics (gauges): backup_total and restore_total
...
Signed-off-by: fabito <fuechi@ciandt.com>
2019-04-18 01:28:38 -03:00
0750b2c789
Merge pull request #1367 from skriss/restic-repo-fixes
...
Restic repository management fixes
2019-04-17 11:49:17 -04:00
5bc6695109
move RestoreResult from api to pkg/restore, rename to Result
...
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-04-16 12:57:02 -06:00
c59544cb79
remove backup.status.volumeBackups and all related code
...
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-04-15 10:17:40 -06:00
7f36f78aee
remove code that removes legacy GC finalizer from backups
...
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-04-15 10:17:03 -06:00
77e648eafa
remove Ark field from RestoreResult
...
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-04-15 10:17:02 -06:00
5d06bd4ab9
restic repo controller: lower resync period to 5min
...
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-04-12 14:43:40 -06:00
3116185e5b
instantiate plugin manager with per-restore logger so plugin logs are captured
...
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-04-12 10:36:20 -06:00
4beb8aab3c
Set default backup TTL
...
Set default backup TTL to 30 days when TTL
is not provided in the backup yaml configuration.
Updates #138
Signed-off-by: Rohan Vora <vorar@vmware.com>
2019-04-09 14:13:29 -07:00
0167539a14
add new counter metrics for backup deletion ( #1280 )
...
* compute backup deletion metrics (attempt, success, fail)
Signed-off-by: fabito <fuechi@ciandt.com>
2019-04-04 14:25:59 -04:00
2a36cdcbf6
set backup start timestamp before patching to inprogress ( #1330 )
...
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-03-29 13:33:50 -07:00
bb9c3f6a1a
rename BlockStore to VolumeSnapshotter
...
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-03-27 14:55:28 -06:00
a111eed2af
update license headers to Velero contributors ( #1302 )
...
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-03-20 12:32:48 -07:00
a71e43b2b7
Split velero plugin client into its own package
...
Signed-off-by: Carlisia <carlisiac@vmware.com>
2019-03-19 16:05:37 -07:00
183bea369d
make resticrepositories non-restorable resources ( #1296 )
...
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-03-18 11:38:37 -07:00
73514a003b
Move plugin interfaces to same package ( #1264 )
...
* Move plugin interfaces to same package
Signed-off-by: Carlisia <carlisiac@vmware.com>
2019-03-14 16:35:06 -04:00
32835c63f6
code review feedback
...
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-02-11 16:21:49 -07:00
88fc6e2141
pkg/controller: remove usage of pkg/util/collections
...
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-02-11 16:21:24 -07:00
52ecc45ec8
Merge pull request #1167 from skriss/logging-and-misc-cleanup
...
Logging and misc cleanup
2019-02-06 14:45:08 -05:00
723cda2697
use restic stats instead of check to check repo existence
...
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-02-06 12:14:11 -07:00
c0a55e136b
logging tweaks for clarity
...
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-02-05 10:08:33 -07:00
43714caaec
Rename Ark to Velero!!!
...
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2019-02-04 17:35:22 -05:00
8a58b217be
show server version in ark version output using ServerStatusRequest CRD
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2019-01-23 12:51:13 -07:00
5847dcabba
Merge pull request #1117 from wwitzel3/issue-134
...
Add backup-version file in backup tarball
2019-01-22 09:01:34 -08:00
d08c2e1b9c
Add backup-version file in backup tarball
...
Signed-off-by: Wayne Witzel III <wwitzel3@vmware.com>
2019-01-09 16:36:32 -05:00
1906c33eb2
Improve directions when location CRDs not deployed
...
Signed-off-by: Carlisia <carlisia@grokkingtech.io>
2018-12-07 10:41:40 -08:00
daf1a75515
Bring Go current in Travis
...
Signed-off-by: Gábor Lipták <gliptak@gmail.com>
2018-11-21 08:31:14 -05:00
40e33020d8
Merge pull request #1022 from shubheksha/fix/945-restore-controller-remove-checking-backup-storage-directly
...
Remove code that checks backup storage directly from restore controller
2018-11-06 15:53:13 -08:00
6c9e1f187f
lower some noisy logs to debug level
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-11-02 15:55:47 -06:00
defb8aa856
remove code that checks directly for a backup from restore controller
...
Signed-off-by: Shubheksha Jalan <jshubheksha@gmail.com>
2018-11-02 23:35:04 +05:30
c3c2ea2805
Merge pull request #985 from shubheksha/fix/977-add-prom-metrics-volume-snapshots
...
Add prometheus metrics for successful and attempted volume snapshots per backup
2018-10-30 13:25:00 -06:00
4239e61f47
add prometheus metrics for successful and attempted volume snapshots per backup
...
rename variables to reflect the metric name
fix comments for exported methods
explicitly record per schedule per schedule metric values
initialize metrics and change variable name to match with that of metric
add metric for recording failed volume snapshots
use singular variable instead of plural
remove extra field for failed snapshots, calculate using existing fields
initialize failure metric and rename methods
Signed-off-by: Shubheksha Jalan <jshubheksha@gmail.com>
2018-10-31 00:47:05 +05:30
b818cc2769
don't require a default provider VSL if there's only 1
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-10-26 12:45:40 -06:00
516422c2c4
sync controller: fill in missing .spec.storageLocation
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-10-23 16:26:07 -06:00
bca585162f
Run 'make update' to update formatting
...
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2018-10-23 13:08:42 -04:00
6cf3519c3a
Merge pull request #974 from skriss/backup-describer
...
backup describer: show snapshot summary by default, details optionally
2018-10-23 12:58:54 -04:00
6591b5a09c
Merge pull request #975 from skriss/rm-pvproviderexists
...
remove pvProviderExists param from NewRestoreController
2018-10-23 12:58:17 -04:00
7c4ac05ae5
Merge pull request #972 from skriss/support-pre-v0.10-snaps
...
Support pre v0.10 snaps
2018-10-23 09:08:52 -07:00
90d9be59d3
support restoring/deleting legacy backups with .status.volumeBackups
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-10-23 09:55:40 -06:00
ef19497205
rename variables #967
...
Signed-off-by: u5surf <u5.horie@gmail.com>
2018-10-23 08:53:35 +09:00
f014cab1fe
backup describer: show snapshot summary by default, details optionally
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-10-22 12:08:37 -06:00
8acc66d02f
remove pvProviderExists param from NewRestoreController
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-10-22 11:47:51 -06:00
57ce590fae
create a struct for multiple return of same type in restore_contoroller #967
...
Signed-off-by: u5surf <u5.horie@gmail.com>
2018-10-20 20:45:26 +09:00
db856affcb
Specify return arguments
...
Update restoreController runRestore function to specify return arguments
Signed-off-by: Atul Bhosale <atul1bhosale@gmail.com>
2018-10-19 21:41:58 +05:30
406b50a71b
update restore process using snapshot locations
...
Signed-off-by: Wayne Witzel III <wayne@riotousliving.com>
2018-10-17 13:40:42 -06:00
268080ad09
avoid panics if can't get block store during deletion
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-10-17 13:40:42 -06:00
4a03370f1d
update backup deletion controller for snapshot locations
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-10-17 13:40:42 -06:00
35bb533c2d
close gzip writer before uploading volumesnapshots file
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-10-17 13:39:50 -06:00
da9ed38c63
store volume snapshot info as JSON in backup storage
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-10-17 13:39:49 -06:00
df07b7dc9f
update backup code to work with volume snapshot locations
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-10-17 13:36:50 -06:00
1aa712d236
Default and validate VolumeSnapshotLocations
...
Signed-off-by: Carlisia <carlisia@grokkingtech.io>
2018-10-17 13:33:22 -06:00
3af43b492f
add azure-specific code to support multi-location restic
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-10-05 09:20:56 -06:00
d009163b67
update restic to support multiple backup storage locations
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-10-05 09:10:35 -06:00
eb709b8f70
only sync a backup location if it's changed since last sync
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-10-03 08:05:06 -06:00
e46e89cb61
have restic share main Ark bucket
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-09-25 13:12:22 -06:00
8bc7e4f6aa
store backups & restores in backups/, restores/ subdirs in obj storage
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-09-25 09:56:41 -06:00
94617b302d
convert all controllers to use genericController, logContext -> log
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-09-07 09:38:58 -06:00
f0edf7335f
add a BackupStore to pkg/persistence that supports prefixes
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-09-06 10:53:58 -06:00
af64069d65
create pkg/persistence and move relevant code from pkg/cloudprovider into it
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-09-06 09:05:40 -06:00
211aa7b7fd
Set schedule labels to subsequent backups
...
Signed-off-by: Zsolt Varga <zsolt.varga@console.hu>
2018-09-06 10:06:18 +02:00
Antony S Bett
Adnan Abdulhussein
Steve Kriss
KubeKween
carthewd
Nolan Brubaker
KubeKween
Anshul Chandra
Andy Goldstein
fabito
Nolan Brubaker
Rohan Vora
Fábio Franco Uechi
Steve Kriss
Steve Kriss
Wayne Witzel III
Gábor Lipták
Shubheksha Jalan
u5surf
Atul Bhosale
Wayne Witzel III
Zsolt Varga