73514a003b
Move plugin interfaces to same package ( #1264 )
...
* Move plugin interfaces to same package
Signed-off-by: Carlisia <carlisiac@vmware.com>
2019-03-14 16:35:06 -04:00
7674332313
pass --log-level to plugins ( #1278 )
...
Plumb the log level through to plugin processes
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-03-14 10:53:36 -07:00
409116fce8
add basic plugin panic handlers ( #1270 )
...
* add server-side panic handlers to all plugin methods
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-03-13 14:07:52 -04:00
503b112638
Add location resources and tests ( #1277 )
...
Add locations and tests to install package
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2019-03-13 11:23:00 -06:00
89ca2571f3
AWS zone on volume IDs
...
Signed-off-by: Travis Sturzl<travis@metismachine.com>
2019-03-12 13:04:33 -06:00
394548afcd
Merge pull request #1254 from skriss/remove-wait-for-pv
...
remove restore code that waits for a PV to become Available
2019-03-11 13:20:59 -04:00
4ee41a13a0
Merge pull request #1261 from asaf-erlich/patch-1
...
Update ark restore to not open every single file open during extracti…
2019-03-08 14:59:09 -08:00
4041044a93
Update ark restore to not open every single file open during extraction of the data
...
Original error was:
```
ark -n <redacted> restore logs <redacted>
time="2019-03-06T18:31:06Z" level=info msg="Not including resource" groupResource=nodes logSource="pkg/restore/restore.go:124"
time="2019-03-06T18:31:06Z" level=info msg="Not including resource" groupResource=events logSource="pkg/restore/restore.go:124"
time="2019-03-06T18:31:06Z" level=info msg="Not including resource" groupResource=events.events.k8s.io logSource="pkg/restore/restore.go:124"
time="2019-03-06T18:31:06Z" level=info msg="Not including resource" groupResource=backups.ark.heptio.com logSource="pkg/restore/restore.go:124"
time="2019-03-06T18:31:06Z" level=info msg="Not including resource" groupResource=restores.ark.heptio.com logSource="pkg/restore/restore.go:124"
time="2019-03-06T18:31:06Z" level=info msg="Starting restore of backup backup/<redacted>" logSource="pkg/restore/restore.go:342"
time="2019-03-06T18:31:06Z" level=info msg="error unzipping and extracting: open /tmp/604421455/resources/rolebindings.rbac.authorization.k8s.io/namespaces/<redacted>/<redacted>: too many open files" logSource="pkg/restore/restore.go:346"
```
Downloading the directory from s3 and untarring it I found 1036 files. The ulimit -n output says 1024. This is our team's best guess at a root cause. But the code fixed in the PR definitely is holding all the files open until the method closes: https://blog.learngoprogramming.com/gotchas-of-defer-in-go-1-8d070894cb01
Please note my go code abilities are not great and I did not test this. I just edited the file in github. All I did was remove the defer and put fileClose after the copy is done. Theoretically this should only hold one file open at a time now. Let me know if you want me to do any further steps.
Thank you,
-Asaf
Signed-off-by: Asaf Erlich <aerlich@groupon.com>
2019-03-07 11:23:35 -05:00
1354e2b6ff
Add original item to restore plugin interface
...
Signed-off-by: Michal Wieczorek <wieczorek-michal@wp.pl>
2019-03-05 17:09:42 -08:00
e29aa74a23
remove restore code that waits for a PV to become Available
...
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-03-05 17:04:52 -07:00
ce3f43e876
Merge pull request #1251 from skriss/backup-extractor
...
move backup extraction logic to its own type
2019-03-05 16:37:48 -05:00
5912fe66e5
Merge pull request #1250 from skriss/extract-pv-restorer
...
move pvRestorer and tests to their own files
2019-03-05 16:37:32 -05:00
1b031f0cc4
Improve `describe` output
...
* Move Phase to right under Metadata(name/namespace/label/annotations)
* Move Validation errors: section right after Phase: section and only
show it if the item has a phase of FailedValidation
* For restores move Warnings and Errors under Validation errors. Do not
show Warnings or Errors if there are none.
Signed-off-by: DheerajSShetty <dheerajs@vmware.com>
Fixes #987
2019-03-04 13:21:18 -08:00
88e6a740f2
move pvRestorer and tests to their own files
...
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-03-01 15:07:25 -07:00
0fec56f488
move backup extraction logic to its own type
...
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-03-01 15:05:58 -07:00
e21940bee1
Merge pull request #1231 from skriss/k8s-1.12-deps
...
update kubernetes and azure dependencies to 1.12
2019-02-28 15:09:06 -08:00
fcf21813a5
Merge pull request #1246 from skriss/preserve-storageclass
...
when restoring a PV, don't remove its spec.storageClassName
2019-02-28 18:02:20 -05:00
65f3926caa
Merge pull request #1146 from skriss/replace-map-utils-final
...
replace ark's map_utils.go with structured types and apimachinery's unstructured helpers
2019-02-28 14:37:07 -08:00
31501b79b2
when deleting snapshot, don't error if it doesn't exist
...
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-02-28 15:33:05 -07:00
6bf837b233
address breaking changes in Azure SDK
...
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-02-28 15:33:05 -07:00
58e471bda0
fix breaking changes
...
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-02-28 15:33:05 -07:00
61eab7dca3
update generated code using 1.12 generator
...
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-02-28 15:33:05 -07:00
21c57c46b3
when restoring a PV, don't remove its spec.storageClassName
...
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-02-28 14:34:36 -07:00
7353294b7f
Merge pull request #1236 from DheerajSShetty/squashed_feature
...
defer closing the ReadCloser in ObjectStoreGRPCServer.GetObject
2019-02-28 13:12:05 -08:00
032aaac508
defer closing the ReadCloser in ObjectStoreGRPCServer.GetObject
...
Signed-off-by: DheerajSShetty <dheerajs@vmware.com>
Fixes #1093
2019-02-28 11:51:10 -08:00
03b8f5397f
Merge pull request #1203 from skriss/v0.11-changes
...
v0.11 changes
2019-02-28 07:01:23 -08:00
e3e76c2067
pkg/restic: fix concurrency bugs causing dupe repos, panics
...
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-02-25 18:30:26 -08:00
49eeeb04f0
Merge pull request #1204 from skriss/deprecation-notices
...
add deprecation notices to pkg/apis/ark/v1 types
2019-02-12 16:02:47 -05:00
e1d414338c
Merge pull request #1200 from nrb/fix-1183
...
Fix restoring GCP regional disks
2019-02-12 13:48:42 -07:00
0ffaeb949d
Fix restoring GCP regional disks
...
To create a regional disk, the URLs for the zones in which the disk is
replicated must be provided to the GCP API.
Fixes #1183
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2019-02-12 15:42:50 -05:00
ed73be44fd
Merge pull request #1206 from nzoueidi/master
...
Add UTC time to --schedule
2019-02-12 09:19:53 -07:00
988ce573c0
Add UTC time to --schedule
...
Signed-off-by: Naeil Ezzoueidi <naeilzoueidi@ubuntu.com>
2019-02-12 11:04:10 -05:00
780dc4551f
fix compile err & test
...
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-02-11 16:45:04 -07:00
32835c63f6
code review feedback
...
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-02-11 16:21:49 -07:00
86c5c25d13
code review: remove obsolete commented code
...
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-02-11 16:21:49 -07:00
250f109c41
delete pkg/util/collections/map_utils.go & tests
...
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-02-11 16:21:49 -07:00
d8e9b772ff
move MergeMaps func into pkg/restore where it's used
...
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-02-11 16:21:48 -07:00
88fc6e2141
pkg/controller: remove usage of pkg/util/collections
...
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-02-11 16:21:24 -07:00
38ad7d71f5
pkg/restore: remove usage of pkg/util/collections
...
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-02-11 16:20:41 -07:00
e91c841c59
pkg/backup: remove usage of pkg/util/collections
...
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-02-11 16:18:51 -07:00
902c0f797f
pkg/podexec: remove usage of pkg/util/collections
...
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-02-11 16:18:37 -07:00
296dd6617e
pkg/cloudprovider: remove usage of pkg/util/collections
...
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-02-11 16:17:33 -07:00
4cd8170386
add deprecation notices to pkg/apis/ark/v1 types
...
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-02-11 15:44:52 -07:00
0df30c1e89
ark->velero in issue templates & bug cmd
...
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-02-11 12:43:51 -07:00
378011baf6
Merge pull request #1153 from daved/fix/1142-restore_log
...
Clarify conditional nature of restore in log
2019-02-11 09:22:25 -08:00
b2b1ee44ea
Clarify restore log when object unchanged
...
Signed-off-by: Daved <daved@codemodus.com>
2019-02-11 09:12:07 -08:00
4583aa7078
Merge pull request #826 from nrb/fix-691
...
Wait for namespace to terminate before restoring
2019-02-07 11:07:25 -07:00
890202f2e4
Wait for PV/namespace to delete before restore
...
If a PV already exists, wait for it, it's associated PVC, and associated
namespace to be deleted before attempting to restore it.
If a namespace already exists, wait for it to be deleted before
attempting to restore it.
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2019-02-06 15:43:50 -05:00
3c7737c8b1
Merge pull request #1166 from skriss/tweak-server-log-version
...
velero server: log version and git SHA at startup
2019-02-06 15:28:51 -05:00
52ecc45ec8
Merge pull request #1167 from skriss/logging-and-misc-cleanup
...
Logging and misc cleanup
2019-02-06 14:45:08 -05:00
46e87661c0
Merge pull request #1171 from skriss/restic-stats
...
use restic stats instead of check to check repo existence
2019-02-06 14:28:07 -05:00
723cda2697
use restic stats instead of check to check repo existence
...
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-02-06 12:14:11 -07:00
5f0ff026b0
Merge pull request #1156 from skriss/restic-v0.9.4
...
upgrade to restic v0.9.4 and replace --hostname with --host
2019-02-06 14:03:44 -05:00
c1a817b4e9
Use current info in bug template
...
The binary, deployment, and namespace have yet to change for most users,
so make sure the bug template has the correct information when filing.
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2019-02-05 16:11:56 -05:00
478d12b4ff
upgrade to restic v0.9.4 and replace --hostname with --host
...
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-02-05 10:15:04 -07:00
328bc361be
velero server: log version and git SHA at startup
...
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-02-05 10:12:06 -07:00
7913ae1867
remove extraneous use of meta.Accessor
...
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-02-05 10:09:57 -07:00
c0a55e136b
logging tweaks for clarity
...
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-02-05 10:08:33 -07:00
381149cedf
Update templates to use current command
...
Currently, the executable binary in the wild is `ark`, so leave it in
our github issue templates
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2019-02-05 11:12:33 -05:00
43714caaec
Rename Ark to Velero!!!
...
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2019-02-04 17:35:22 -05:00
25299513c1
change mode on metadata/version to 0755
...
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-01-31 16:27:13 -07:00
8a58b217be
show server version in ark version output using ServerStatusRequest CRD
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2019-01-23 12:51:13 -07:00
5847dcabba
Merge pull request #1117 from wwitzel3/issue-134
...
Add backup-version file in backup tarball
2019-01-22 09:01:34 -08:00
d08c2e1b9c
Add backup-version file in backup tarball
...
Signed-off-by: Wayne Witzel III <wwitzel3@vmware.com>
2019-01-09 16:36:32 -05:00
e38e2fc5a1
Remove YEAR from generated boilerplate comment
...
Signed-off-by: Andy Goldstein <goldsteina@vmware.com>
2019-01-03 12:22:19 -05:00
2ed241b0b7
Merge pull request #1124 from skriss/ark-version-changes
...
ark version changes
2018-12-19 16:24:07 -05:00
c34e3b9127
Merge pull request #988 from carlisia/c-crd-msg
...
Improve directions when location CRDs not deployed
2018-12-07 16:45:15 -05:00
1906c33eb2
Improve directions when location CRDs not deployed
...
Signed-off-by: Carlisia <carlisia@grokkingtech.io>
2018-12-07 10:41:40 -08:00
246d75811a
version cmd: move tree state into git sha line
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-12-07 09:59:03 -07:00
f983f4fb21
Fix output of Schedule Delete
...
It should be `\n` and not `/n/` when doing the printf on a schedule delete.
Signed-off-by: Jon Whitcraft <jwhitcraft@mac.com>
2018-12-06 15:33:14 -05:00
aa9d96f3b7
Add pprof support to ark server
...
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2018-12-05 10:13:39 -05:00
e13806e0b8
Allow to use AWS Signature v1 for creating signed AWS urls
...
Some aws implementations, for example the quobyte object storage, do not
support the v4 signing algorithm, but only v1.
This makes it possible to configure the signatureVersion.
The algorithm implementation was ported from d6c1be296e/botocore/auth.py (L860-L862)https://github.com/heptio/ark/issues/811 .
Signed-off-by: Bastian Hofmann <bashofmann@gmail.com>
2018-12-04 18:32:41 +01:00
62d8c642d2
Remove default token from all service accounts
...
Instead of only removing the default token from a service account when
it already exists in the cluster, always remove it. If the service
account already exists, continue to do the merging logic.
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2018-12-04 11:41:18 -05:00
b92b35d42b
add multizone/region support to gcp
...
Signed-off-by: Wayne Witzel III <wayne@riotousliving.com>
2018-12-04 10:24:55 -05:00
09042a9015
Merge pull request #1074 from mwieczorek/del-priority
...
Delete spec.priority in pod restore action
2018-11-26 14:05:51 -05:00
43cd2ca36f
Delete spec.priority in pod restore action
...
Signed-off-by: Michal Wieczorek <wieczorek-michal@wp.pl>
2018-11-22 09:54:49 +01:00
d85872dfb2
Merge pull request #1069 from gliptak/patch-1
...
Update to go 1.11
2018-11-21 08:39:06 -05:00
daf1a75515
Bring Go current in Travis
...
Signed-off-by: Gábor Lipták <gliptak@gmail.com>
2018-11-21 08:31:14 -05:00
d27e4f7f05
Merge pull request #1054 from cbeneke/master
...
Initialize empty schedule metrics on server init
2018-11-20 09:30:45 -07:00
411d62fe91
Merge pull request #1006 from ncdc/add-s3-public-url
...
Support a separate URL base for pre-signed URLs
2018-11-14 15:12:46 -07:00
90c5ed0850
Initialize empty schedule metrics on server init
...
When backups are run manually (outside of a schedule) the metrics
will be counted for ark_*{schedule=""}. To prevent partial NaN
metrics they will be initialised on server init.
Signed-off-by: Christian Beneke <c.beneke@syseleven.de>
2018-11-14 15:31:26 +01:00
2781e4e8de
Merge pull request #956 from ffd2subroutine/qps-burst-configurable
...
Added qps and burst to server's client
2018-11-09 12:41:41 -07:00
e5556fe608
Added qps and burst to server's client
...
Signed-off-by: Darko Radisic <ffd2subroutine@users.noreply.github.com>
2018-11-09 20:16:24 +01:00
9ae861c9e2
Support a separate URL base for pre-signed URLs
...
This allows the Ark server to use one URL for the majority of
communications with S3 (or compatible) object storage, and a different
URL base for pre-signed URLs (for streaming logs, etc. to clients).
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2018-11-08 14:43:57 -05:00
449cac5806
Merge pull request #1032 from mwieczorek/plugins
...
Move clearing up of metadata before plugin's actions
2018-11-08 12:00:33 -07:00
3788014552
Fix check for non-found PV
...
We were checking for nil, but were getting back an empty
*unstructured.Unstructured{} instead, along with a NotFound error.
Change the logic to check for the NotFound error instead of a nil
object.
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2018-11-08 10:56:14 -05:00
40e33020d8
Merge pull request #1022 from shubheksha/fix/945-restore-controller-remove-checking-backup-storage-directly
...
Remove code that checks backup storage directly from restore controller
2018-11-06 15:53:13 -08:00
6c9e1f187f
lower some noisy logs to debug level
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-11-02 15:55:47 -06:00
defb8aa856
remove code that checks directly for a backup from restore controller
...
Signed-off-by: Shubheksha Jalan <jshubheksha@gmail.com>
2018-11-02 23:35:04 +05:30
7abe115674
Move clearing up of metadata before plugin's actions
...
Signed-off-by: Michal Wieczorek <wieczorek-michal@wp.pl>
2018-11-02 09:56:33 +01:00
cb0e6f4773
Merge pull request #999 from bashofmann/fix-backup-file-deletion-order
...
AWS: Ensure that the order returned by ListObjects is consistent
2018-10-30 15:46:33 -04:00
8bbfc538f1
AWS: Ensure that the order returned by ListObjects is consistent
...
When a backup is deleted, the delete method uses ListObjects to get a list of
files it needs to delete in s3. Different s3 implementations may return
the object lists in different, even non-deterministic orders, which can
result in the deletion not working because ark tries to delete a non empty folder
before it tries to delete the files in the folder.
Signed-off-by: Bastian Hofmann <bashofmann@gmail.com>
2018-10-30 20:38:47 +01:00
c3c2ea2805
Merge pull request #985 from shubheksha/fix/977-add-prom-metrics-volume-snapshots
...
Add prometheus metrics for successful and attempted volume snapshots per backup
2018-10-30 13:25:00 -06:00
4239e61f47
add prometheus metrics for successful and attempted volume snapshots per backup
...
rename variables to reflect the metric name
fix comments for exported methods
explicitly record per schedule per schedule metric values
initialize metrics and change variable name to match with that of metric
add metric for recording failed volume snapshots
use singular variable instead of plural
remove extra field for failed snapshots, calculate using existing fields
initialize failure metric and rename methods
Signed-off-by: Shubheksha Jalan <jshubheksha@gmail.com>
2018-10-31 00:47:05 +05:30
9165d514a3
Merge pull request #1002 from nrb/fix-609
...
Verify PV doesn't exist before creating new volume
2018-10-30 13:04:15 -06:00
6231aaa875
Verify PV doesn't exist before creating new volume
...
Signed-off-by: Nolan Brubaker <nolan@heptio.com>
2018-10-30 13:57:00 -04:00
b818cc2769
don't require a default provider VSL if there's only 1
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-10-26 12:45:40 -06:00
74cb6a2150
Merge pull request #986 from skriss/fix-empty-bsls
...
sync controller: fill in missing .spec.storageLocation
2018-10-24 18:38:00 -04:00
516422c2c4
sync controller: fill in missing .spec.storageLocation
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-10-23 16:26:07 -06:00
195e6aaf00
fix bug preventing PV snapshots from v0.10 backups from restoring
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-10-23 12:44:05 -06:00
bca585162f
Run 'make update' to update formatting
...
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2018-10-23 13:08:42 -04:00
6cf3519c3a
Merge pull request #974 from skriss/backup-describer
...
backup describer: show snapshot summary by default, details optionally
2018-10-23 12:58:54 -04:00
6591b5a09c
Merge pull request #975 from skriss/rm-pvproviderexists
...
remove pvProviderExists param from NewRestoreController
2018-10-23 12:58:17 -04:00
7c4ac05ae5
Merge pull request #972 from skriss/support-pre-v0.10-snaps
...
Support pre v0.10 snaps
2018-10-23 09:08:52 -07:00
90d9be59d3
support restoring/deleting legacy backups with .status.volumeBackups
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-10-23 09:55:40 -06:00
f3a57b5b8a
Merge pull request #971 from shubheksha/fix/594-restore-storageclasses-before-pv-pvc
...
Restore storageclasses before pvs and pvcs
2018-10-23 08:14:42 -06:00
ef19497205
rename variables #967
...
Signed-off-by: u5surf <u5.horie@gmail.com>
2018-10-23 08:53:35 +09:00
596eea1b8c
restore storageclasses before pvs and pvcs
...
Signed-off-by: Shubheksha Jalan <jshubheksha@gmail.com>
2018-10-22 21:15:08 +02:00
f014cab1fe
backup describer: show snapshot summary by default, details optionally
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-10-22 12:08:37 -06:00
8acc66d02f
remove pvProviderExists param from NewRestoreController
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-10-22 11:47:51 -06:00
57ce590fae
create a struct for multiple return of same type in restore_contoroller #967
...
Signed-off-by: u5surf <u5.horie@gmail.com>
2018-10-20 20:45:26 +09:00
d0937a3433
Merge pull request #960 from Atul9/return-arguments
...
#959 Update restoreController runRestore function to specify return arguments
2018-10-19 13:03:30 -04:00
db856affcb
Specify return arguments
...
Update restoreController runRestore function to specify return arguments
Signed-off-by: Atul Bhosale <atul1bhosale@gmail.com>
2018-10-19 21:41:58 +05:30
9952dfb0ad
Address #424 : Add CRDs to list of prioritized resources
...
Signed-off-by: Domenic Rosati <domenicrosati@gmail.com>
2018-10-18 22:42:37 -03:00
b3b065a9fb
Merge pull request #949 from skriss/remove-config
...
remove Config CRD
2018-10-18 10:47:23 -04:00
cf2c27141b
fix bugs in GetBackupVolumeSnapshots and add test
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-10-17 15:10:42 -06:00
c36131a024
remove Config-related code
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-10-17 14:00:38 -06:00
406b50a71b
update restore process using snapshot locations
...
Signed-off-by: Wayne Witzel III <wayne@riotousliving.com>
2018-10-17 13:40:42 -06:00
268080ad09
avoid panics if can't get block store during deletion
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-10-17 13:40:42 -06:00
4a03370f1d
update backup deletion controller for snapshot locations
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-10-17 13:40:42 -06:00
38c72b8cc2
include snapshot locations in created schedule's backup spec
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-10-17 13:40:41 -06:00
0ec2de55c0
azure: update blockstore to allow storing snaps in different resource group
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-10-17 13:39:51 -06:00
35bb533c2d
close gzip writer before uploading volumesnapshots file
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-10-17 13:39:50 -06:00
da9ed38c63
store volume snapshot info as JSON in backup storage
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-10-17 13:39:49 -06:00
e24248e07a
add --volume-snapshot-locations flag to ark backup create
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-10-17 13:36:51 -06:00
df07b7dc9f
update backup code to work with volume snapshot locations
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-10-17 13:36:50 -06:00
4af89fa863
add unit test for getDefaultVolumeSnapshotLocations
...
Signed-off-by: Wayne Witzel III <wayne@riotousliving.com>
2018-10-17 13:35:52 -06:00
02f50b9c84
add default-volume-snapshot-locations to server cmd
...
Signed-off-by: Wayne Witzel III <wayne@riotousliving.com>
2018-10-17 13:35:50 -06:00
1aa712d236
Default and validate VolumeSnapshotLocations
...
Signed-off-by: Carlisia <carlisia@grokkingtech.io>
2018-10-17 13:33:22 -06:00
bbf769850e
add create CLI command for snapshot locations
...
Signed-off-by: Wayne Witzel III <wayne@riotousliving.com>
2018-10-17 13:33:22 -06:00
aeb221eafe
Add printer for snapshot locations
...
Signed-off-by: Wayne Witzel III <wayne@riotousliving.com>
2018-10-17 13:33:22 -06:00
ffc612ac13
Add volume snapshot CLI get command
...
Signed-off-by: Wayne Witzel III <wayne@riotousliving.com>
2018-10-17 13:33:22 -06:00
f20342aab9
Add VolumeLocation and Snapshot.
...
Signed-off-by: Carlisia <carlisia@grokkingtech.io>
2018-10-17 13:33:22 -06:00
ebc379ac02
Add --include-cluster-resources options when creating schedule
...
Signed-off-by: captjt <jtaylor007.jt@gmail.com>
2018-10-16 15:52:18 -04:00
0e7f442f00
Merge pull request #930 from skriss/fix-898
...
don't require a default backup storage location to exist
2018-10-12 12:34:44 -07:00
37031d5e7c
Merge pull request #920 from skriss/fix-225
...
only try to backup PVC's linked PV if PVC phase is Bound
2018-10-12 14:10:18 -04:00
480f1942b7
Merge pull request #927 from skriss/fix-925
...
pod restore action: check initContainers exist before processing
2018-10-12 14:09:35 -04:00
4173515435
don't require a default backup storage location to exist
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-10-11 14:43:27 -06:00
fe51be6713
Merge pull request #926 from shubheksha/fix/329-remove-get-project-gcp
...
Remove logic to get a gcp project
2018-10-11 15:45:40 -04:00
38e86ceff5
pod restore action: check initContainers exist before processing
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-10-11 13:44:02 -06:00
611bc92695
remove logic to get a gcp project
...
Signed-off-by: Shubheksha Jalan <jshubheksha@gmail.com>
2018-10-11 21:23:30 +02:00
63dc6c1d9a
only try to backup PVC's linked PV if PVC phase is Bound
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-10-10 13:34:33 -06:00
d579784692
Merge pull request #910 from james-powis/fix_restic_restore_service_user_token
...
Drop volumes matching name ServiceAccountName-token-
2018-10-09 11:32:23 -07:00
30369c2ad5
Drop volumeMounts from initContainers if SAToken
...
Signed-off-by: James Powis <powisj@gmail.com>
2018-10-05 16:00:58 -06:00
0612c5de70
templatize error message in DeleteOptions
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-10-05 13:24:30 -06:00
1da3278ad6
Merge pull request #874 from shubheksha/fix/748-ark-schedule-bulk-deletion
...
Add support for bulk deletion to `ark schedule delete`
2018-10-05 13:21:40 -06:00
66bcbc058c
add support for bulk deletion to ark schedule delete
...
refactor and move DeleteOptions struct and methods
unexport fields not used outside the package in DeleteOptions struct
refactor BindFlags() to work with name of command
fix constructor
Signed-off-by: Shubheksha Jalan <jshubheksha@gmail.com>
2018-10-05 19:45:18 +02:00
3af43b492f
add azure-specific code to support multi-location restic
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-10-05 09:20:56 -06:00
d009163b67
update restic to support multiple backup storage locations
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-10-05 09:10:35 -06:00
1d90d02a99
Merge pull request #900 from skriss/restic-restore-ns-mapping
...
bug: fix restic restores when using namespace mappings
2018-10-03 10:59:13 -04:00
0c88eefc0d
bug: fix restic restores when using namespace mappings
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-10-03 08:09:55 -06:00
eb709b8f70
only sync a backup location if it's changed since last sync
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-10-03 08:05:06 -06:00
82ab2d73ac
Merge pull request #894 from marctc/fix_backup_list_date
...
Change CreationTimestamp to StartTimestamp in backup list
2018-10-02 11:04:04 -06:00
ae4bf3d5f8
Change CreationTimestamp by StartTimestamp in backup list
...
Signed-off-by: Marc Tudurí <marctc@gmail.com>
2018-10-02 15:17:58 +02:00
0f1c5c283f
Enable restoring resources with ownerReference set
...
Signed-off-by: Michal Wieczorek <wieczorek-michal@wp.pl>
2018-10-02 12:04:51 +02:00
94b8fae15a
Merge pull request #891 from skriss/fix-restore-logs
...
remove restore log helper for accurate line #'s
2018-10-01 21:02:17 -07:00
729a688b36
remove restore log helper for accurate line #'s
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-09-30 14:45:32 -06:00
9fdf85130a
Minor code cleanup
...
Signed-off-by: ffd2subroutine <ffd2subroutine@users.noreply.github.com>
2018-09-30 20:43:11 +02:00
a57acec43d
Merge pull request #876 from skriss/restic-share-bucket
...
Move restic repos into the main Ark bucket, under restic/
2018-09-28 12:12:45 -04:00
ed2bca83bd
Merge pull request #869 from shubheksha/fix/786-fix-error-ExternalName-service-restore
...
Fix error when restoring ExternalName services
2018-09-25 16:33:36 -04:00
e46e89cb61
have restic share main Ark bucket
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-09-25 13:12:22 -06:00
42b54586cd
refactor to make valid dirs part of an object store layout
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-09-25 13:12:22 -06:00
8bc7e4f6aa
store backups & restores in backups/, restores/ subdirs in obj storage
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-09-25 09:56:41 -06:00
e3232b7eb6
add support for bulk deletion to ark restore delete
...
refactor util methods into a common package
move utility methods into cli package instead of common
rename util.go to common.go
Signed-off-by: Shubheksha Jalan <jshubheksha@gmail.com>
2018-09-25 20:37:20 +05:30
e3222a9e3f
move code dealing with node ports into a separate function
...
Signed-off-by: Shubheksha Jalan <jshubheksha@gmail.com>
2018-09-25 18:08:57 +05:30
d95b18bad8
ignore spec.ports not being there for services of type ExternalName
...
Signed-off-by: Shubheksha Jalan <jshubheksha@gmail.com>
2018-09-25 01:35:13 +05:30
907a9fbdd8
fix error during restore when spec.ports are not found
...
Signed-off-by: Shubheksha Jalan <jshubheksha@gmail.com>
2018-09-25 01:02:47 +05:30
3a60853340
Merge pull request #843 from james-powis/master
...
Discard all SA Token volumes from pod restore
2018-09-19 13:31:15 -06:00
a5cb2ce4b7
Change from regex matching default-token to prefix SA-token-
...
Signed-off-by: James Powis <powisj@gmail.com>
2018-09-19 13:06:10 -06:00
b5e4f85b8c
Merge pull request #834 from skriss/fix-restic-restores
...
get a new metadata accessor after calling backup item actions
2018-09-12 17:13:16 -04:00
fcd27a13da
get a new metadata accessor after calling backup item actions
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-09-10 16:19:38 -06:00
69e2f51fbe
Merge pull request #801 from ljakimczuk/master
...
Claim ownership of new AWS volumes by Kubernetes cluster restoring the backup
2018-09-10 14:03:06 -07:00
ffef86e38d
Adding support for the AWS_CLUSTER_NAME env variable allowing to claim volumes ownership
...
Signed-off-by: Lukasz Jakimczuk <ljakimczuk@gmail.com>
Moving check for environment variable outside the loop
Signed-off-by: Lukasz Jakimczuk <ljakimczuk@gmail.com>
Insert a note about AWS_CLUSTER_NAME in the aws-config doc
Signed-off-by: Lukasz Jakimczuk <ljakimczuk@gmail.com>
Improving implementation and documentation
Signed-off-by: Lukasz Jakimczuk <ljakimczuk@gmail.com>
Changing instructions, adding unit test for getTagsForCluster and removing duplicated Lookup
Signed-off-by: Lukasz Jakimczuk <ljakimczuk@gmail.com>
Commit after update
Signed-off-by: Lukasz Jakimczuk <ljakimczuk@gmail.com>
Correcting bad formatting in aws-config.md
Signed-off-by: Lukasz Jakimczuk <ljakimczuk@gmail.com>
2018-09-10 22:34:33 +02:00
c9a9dd4508
Merge pull request #824 from skriss/generic-controllers
...
convert all controllers to use genericController, logContext -> log
2018-09-07 14:20:22 -04:00
94617b302d
convert all controllers to use genericController, logContext -> log
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-09-07 09:38:58 -06:00
44a75cee7b
Merge pull request #820 from skriss/azure-restic-fix
...
set azure restic env vars based on default backup location's config
2018-09-06 13:01:45 -07:00
7d8813a96c
move ObjectStore mock into pkg/cloudprovider/mocks
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-09-06 10:53:58 -06:00
f0edf7335f
add a BackupStore to pkg/persistence that supports prefixes
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-09-06 10:53:58 -06:00
af64069d65
create pkg/persistence and move relevant code from pkg/cloudprovider into it
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-09-06 09:05:40 -06:00
29d75d72e2
move object and block store interfaces to their own files
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-09-06 09:05:40 -06:00
211aa7b7fd
Set schedule labels to subsequent backups
...
Signed-off-by: Zsolt Varga <zsolt.varga@console.hu>
2018-09-06 10:06:18 +02:00
d34994cb5f
set azure restic env vars based on default backup location's config
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-09-05 14:31:42 -06:00
cb321db21f
azure: refactor to not use helpers/ pkg, validate all env/config inputs
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-08-28 13:19:21 -07:00
9d7ea7483c
azure: support different RGs/storage accounts per backup location
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-08-28 13:19:21 -07:00
cd4e9f5336
azure: fix for breaking change in blob.GetSASURI
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-08-28 13:19:21 -07:00
b31e25bf6e
server: remove unused code, replace deprecated func
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-08-28 13:19:21 -07:00
729d733986
controllers: take a newPluginManager func in constructors
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-08-28 13:19:21 -07:00
133dc185ca
backup sync: process the default location first
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-08-28 13:19:21 -07:00
7a1e6d16cc
generic controller: allow controllers with only a resync func
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-08-28 13:19:21 -07:00
6f7bfe545d
remove Config CRD's BackupStorageProvider & other obsolete code
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-08-28 13:19:21 -07:00
bd4d97b9e4
move server's defaultBackupLocation into config struct
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-08-28 13:19:21 -07:00
0e94fa37f9
update sync controller for backup locations
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-08-28 13:19:21 -07:00
2750aa71b9
Use backup storage location during restore
...
Closes #740
Signed-off-by: Carlisia <carlisia@grokkingtech.io>
2018-08-28 13:19:21 -07:00
20f89fbcef
use the default backup storage location for restic
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-08-28 13:19:21 -07:00
833a6307a9
Add storage location to backup get/describe
...
Fixes #775
Also conforms ark imports to https://github.com/heptio/ark/issues/494
Signed-off-by: Nolan Brubaker <nolan@heptio.com>
2018-08-28 13:19:21 -07:00
cf7c8587f0
download request: fix setting of log level for plugin manager
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-08-28 13:19:21 -07:00
3234124afe
backup deletion: fix setting of log level in plugin manager
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-08-28 13:19:20 -07:00
74043ab428
download request controller: fix bug in determining expiration
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-08-28 13:19:20 -07:00
7007f198e1
refactor download request controller test and add test cases
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-08-28 13:19:20 -07:00
8f5346150c
download request controller: use backup location for object store
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-08-28 13:19:20 -07:00
bab08ed1a6
backup deletion controller: use backup location for object store
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-08-28 13:19:20 -07:00
c6f488f75f
Use backup location in the backup controller
...
Fixes #739
Signed-off-by: Nolan Brubaker <nolan@heptio.com>
2018-08-28 13:19:20 -07:00
06b5af449f
add create and get CLI commands for backup locations
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-08-28 13:19:19 -07:00
adbcd3703b
add --default-backup-storage-location flag to server cmd
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-08-28 13:14:51 -07:00
2a34772ed5
Add --storage-location argument to create commands
...
Closes #738
Signed-off-by: Nolan Brubaker <nolan@heptio.com>
2018-08-28 13:14:51 -07:00
56f1617049
Correct metadata for BackupStorageLocationList
...
Signed-off-by: Nolan Brubaker <nolan@heptio.com>
2018-08-28 13:14:51 -07:00
345c3c39b1
Generate clients for BackupStorageLocation
...
Signed-off-by: Nolan Brubaker <nolan@heptio.com>
2018-08-28 13:14:51 -07:00
a25eb03290
Add BackupStorageLocation API type
...
This commit only provides the data model for further work. It does not
implement any logic around locations, nor does it remove anything from
the Config API type.
Closes #736
Closes #732
Signed-off-by: Nolan Brubaker <nolan@heptio.com>
2018-08-28 13:14:51 -07:00
adc29a2db0
Merge pull request #774 from metadave/dp_report_bug
...
add an ark bug command
2018-08-24 16:17:21 -04:00
342a1c6437
add an ark bug command
...
Signed-off-by: Dave Parfitt <diparfitt@gmail.com>
2018-08-24 13:10:56 -04:00
8a5d6f9111
Merge pull request #781 from nrb/fix-777
...
Fix map merging logic
2018-08-22 11:29:02 -07:00
ea50ebf2b5
Fix map merging logic
...
Fixes #777
Signed-off-by: Nolan Brubaker <nolan@heptio.com>
2018-08-21 15:30:30 -04:00
d29c96387e
Merge pull request #757 from carlisia/c-538-config-m
...
Switch Config CRD elements to server flags
2018-08-16 09:02:48 -07:00
9508e4a20e
Switch Config CRD elements to server flags
...
Signed-off-by: Carlisia <carlisia@grokkingtech.io>
2018-08-16 08:51:46 -07:00
65cd5c602f
Merge pull request #754 from skriss/namespace-restore-label
...
update how we label restored objects
2018-08-09 15:17:50 -04:00
95b2b90006
add backup name label to restored objects
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-08-08 16:51:33 -07:00
0c3ac67b6d
start using a namespaced label on restored objects, deprecate old label
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-08-08 16:34:39 -07:00
5acccaa739
add bulk deletion support to ark backup delete
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-08-08 10:28:40 -07:00
1f7a4a1665
Merge pull request #712 from timoreimann/preserve-node-ports-when-specified-in-annotation
...
Preserve node ports during restore when annotations hold specification.
2018-08-07 09:17:27 -07:00
3aa241a74c
Preserve node ports during restore when annotations hold specification.
...
This is to better reflect the intent of the user when node ports are
specified explicitly (as opposed to being assigned by Kubernetes). The
`last-applied-configuration` annotation added by `kubectl apply` is one
such indicator we are now leveraging.
We still default to omitting the node ports when the annotation is
missing.
Signed-off-by: Timo Reimann <ttr314@googlemail.com>
2018-08-07 07:47:17 +02:00
27003af62a
Merge pull request #304 from ncdc/wait
...
Add --wait support to ark backup create
2018-08-03 10:57:20 -07:00
c5f5862c9c
Add --wait support to ark backup create
...
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2018-08-03 10:42:42 -07:00
c6ef76d2b3
Merge pull request #719 from nrb/prereq-faq
...
Document CRD not found errors
2018-08-02 16:03:09 -07:00
eb6f742b5d
Document CRD not found errors
...
Signed-off-by: Nolan Brubaker <nolan@heptio.com>
2018-08-02 16:41:28 -04:00
e7bb592602
Add --wait support to `ark restore create`
...
Signed-off-by: Carlisia <carlisia@grokkingtech.io>
2018-07-31 17:20:03 -07:00
6f061db9a2
Merge pull request #715 from skriss/remove-snapshot-service
...
remove SnapshotService, replace with direct BlockStore usage
2018-07-31 13:50:22 -07:00
8ce513acbd
Only delete unused backup if they are complete
...
Fixes #705
Signed-off-by: Carlisia <carlisia@grokkingtech.io>
2018-07-31 09:12:12 -07:00
1c26fbde32
remove SnapshotService, replace with direct BlockStore usage
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-07-31 08:58:11 -07:00
130512187a
Refactor plugin management
...
Refactor plugin management:
- support multiple plugins per executable
- support restarting a plugin process in the event it terminates
- simplify plugin lifecycle management by using separate managers for
each scope (server vs backup vs restore)
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2018-07-31 08:34:57 -07:00
131afb571e
Merge pull request #607 from nrb/restore-metrics
...
Restore metrics
2018-07-27 12:25:56 -07:00
74dbf38793
Add restore failed phase and metrics
...
Signed-off-by: Nolan Brubaker <nolan@heptio.com>
2018-07-27 14:32:11 -04:00
ca5656c279
fix bug preventing backup item action item updates from saving
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-07-26 10:38:17 -07:00
82f1cd87dc
Merge pull request #680 from carlisia/c-delete-from-etcd
...
Delete backups from etcd if they're not in storage
2018-07-25 18:25:52 -04:00
d2e629f5a8
Delete backups from etcd if they're not in storage
...
This fixes #309 .
Signed-off-by: Carlisia <carlisia@grokkingtech.io>
Co-authored-by: Steve Kriss <steve@heptio.com>
2018-07-25 14:53:50 -07:00
7cebfe2df0
Add restore attempt and success/failure counters
...
Signed-off-by: Nolan Brubaker <nolan@heptio.com>
2018-07-25 11:22:43 -04:00
06d6665abb
check s3URL scheme upon AWS ObjectStore Init()
...
Signed-off-by: Dave Parfitt <diparfitt@gmail.com>
2018-07-24 14:01:05 -04:00
b0af81e780
Merge pull request #689 from lemaral/init-schedule-metrics
...
Init schedule Prometheus metrics
2018-07-23 16:16:26 -04:00
f62045623c
cleanup service account action log statement
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-07-23 12:10:30 -07:00
e063b7983a
Merge pull request #682 from nrb/support-rbac-1.7
...
Support for rbac v1beta1 on kubernetes v1.7.5+
2018-07-23 12:00:50 -07:00
dd1e150511
Add RBAC support for 1.7 clusters
...
Signed-off-by: Nolan Brubaker <nolan@heptio.com>
2018-07-23 14:19:03 -04:00
c47a364ab3
Merge pull request #687 from skriss/fix-delete-issues
...
fix delete issues
2018-07-23 10:40:14 -07:00
450fa72fbb
Initialize schedule Prometheus metrics to have them created beforehand (see https://prometheus.io/docs/practices/instrumentation/#avoid-missing-metrics )
...
Signed-off-by: Alex Lemaresquier <alex+github@lemaresquier.org>
2018-07-20 23:30:24 +02:00
78cbdf95f3
delete old deletion requests for backup when processing a new one
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-07-20 11:42:42 -07:00
85a61b8e8d
return nil error if 404 encountered when deleting snapshots
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-07-20 11:42:34 -07:00
1df9a8a38d
exit server if not all Ark CRDs exist at startup
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-07-18 14:30:42 -07:00
e11634bfbc
Merge pull request #676 from skriss/dont-create-namespace
...
require namespace for backups/etc. to exist at server startup
2018-07-17 14:26:04 -04:00
13f893f1f9
Merge pull request #634 from marpaia/k8s-1.11
...
Update Kubernetes and Client-Go for 1.11.0 / 8.0.0
2018-07-17 08:55:47 -07:00
3efe677007
Update Ark library code to work with Kubernetes 1.11
...
Signed-off-by: Mike Arpaia <mike@arpaia.co>
2018-07-16 20:46:20 -06:00
d195512062
require namespace for backups/etc. to exist at server startup
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-07-13 13:50:12 -07:00
1e2b141e5d
Merge pull request #443 from skriss/restore-from-schedule
...
Add --from-schedule flag to `ark restore create`
2018-07-11 13:50:08 -04:00
683f7afc0d
switch to using .status.startTimestamp for sorting backups
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-07-11 10:35:55 -07:00
b71a37dbfc
Record backup completion time before uploading
...
Signed-off-by: Nolan Brubaker <nolan@heptio.com>
2018-07-11 13:21:29 -04:00
09afeb6d3e
Merge pull request #646 from skriss/remove-redundant-filesystem
...
replace pkg/restore's osFileSystem with pkg/util/filesystem's
2018-07-10 17:57:39 -04:00
fae00a7622
Merge pull request #659 from carlisia/cc-backupsync
...
Skip backup sync if it already exists on Kubernetes
2018-07-10 12:50:20 -04:00
5b89f7b6db
Skip backup sync if it already exists in k8s
...
Signed-off-by: Carlisia Pinto <carlisia@grokkingtech.io>
2018-07-09 21:55:24 -07:00
c6050845a0
restore controller: switch to 'c' for receiver name
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-07-09 15:07:38 -07:00
706ae07d0d
enable a schedule to be provided as the source for a restore
...
- ScheduleName is added as an API field to the Restore object
- Restore controller validates that exactly one of BackupName
or ScheduleName has been provided
- If ScheduleName is provided, Restore controller populates
BackupName with the name of the most recent successful backup
created from the schedule
- --from-schedule flag is added to `ark restore create` CLI cmd
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-07-09 15:07:38 -07:00
795dc26214
replace pkg/restore's osFileSystem with pkg/util/filesystem's
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-07-05 15:46:17 -07:00
2999f158db
Merge pull request #550 from skriss/restic-describers
...
Add pod volume backups/restores to ark backup/restore describe
2018-07-05 16:27:36 -04:00
8dd9cded1a
Fix testing arbitrary map key order
...
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2018-07-05 16:22:46 -04:00
42f2891485
add pod volume backups/restores to ark backup/restore describe output
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-07-05 13:13:40 -07:00
9db5e36b54
Fix test sorting function
...
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2018-07-05 16:08:05 -04:00
c18decc89b
pkg/restic unit tests
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-07-05 11:44:17 -07:00
547625c333
set --hostname flag for restic backups
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-07-02 10:52:32 -07:00
eabef08561
Update generated Ark code based on the 1.11 k8s.io/code-generator script
...
Signed-off-by: Mike Arpaia <mike@arpaia.co>
2018-07-02 09:24:19 -06:00
ae4aad0890
use pkg/util/exec for running get snapshot id cmd
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-06-29 12:06:31 -07:00
fe286ff564
Don't restore backups or restores
...
Add backups and restores the list of non restorable resources. Backups,
if applicable, are synced from object storage by the backup sync
controller. Restores are specific to a cluster and don't have value
moving across clusters.
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2018-06-28 16:58:09 -04:00
eaeb9d677e
Merge pull request #608 from skriss/no-pv-snapshot-if-restic-backup
...
don't snapshot volumes that have been backed up with restic
2018-06-28 13:27:30 -04:00
11c176c490
don't snapshot volumes that have been backed up with restic
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-06-28 10:19:34 -07:00
539de6d361
Merge pull request #564 from nrb/backup-timing
...
Record backup start and completion times, add timing metrics
2018-06-28 11:37:39 -04:00
96b72acb2d
Record backup start and completion times
...
Signed-off-by: Nolan Brubaker <nolan@heptio.com>
2018-06-28 11:18:14 -04:00
fa470170cf
Merge pull request #610 from skriss/restic-skip-invalid-volumes
...
log and skip backups for non-existent & hostPath volumes
2018-06-28 10:49:33 -04:00
255a991c6e
Stop restoring mirror pods
...
Mirror pods are pods created from static manifest files on a node.
They're mirrored to the apiserver so they're visible when querying the
apiserver for a list of pods, but it's not possible to send a pod
containing the mirror pod annotation to the apiserver and have it be
created successfully. Instead of trying to do this, log a message that
we're skipping restoring the pod because it's a mirror pod.
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2018-06-28 10:06:55 -04:00
781b7cd1aa
log and skip backups for non-existent & hostPath volumes
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-06-27 14:37:36 -07:00
22e8f23e2c
replace ark restic repo init cmd with automatic initialization of repos
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-06-27 13:50:37 -07:00
a697ad164e
refine what gets enqueued in PVB/PVR controllers, and log better
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-06-27 13:31:11 -07:00
29ac0b4a6c
Merge pull request #613 from ncdc/ignore-reclaim-policy-delete-pvs-without-snapshots
...
Skip reclaim policy Delete PVs without snapshots
2018-06-26 16:52:01 -04:00
7c283e5de8
Skip reclaim policy Delete PVs without snapshots
...
If a PV has a reclaim policy of Delete and we didn't create a snapshot
of it, don't restore the PV, as doing so would create a PV whose
underlying volume is incorrect.
Also "reset" any PVCs bound to the PV so they'll be dynamically
provisioned when restored.
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2018-06-26 10:40:49 -04:00
0da5f1ccca
Add omitempty tag to optional API fields
...
Signed-off-by: Nikhita Raghunath <nikitaraghunath@gmail.com>
2018-06-25 16:40:43 +05:30
5ad21854f7
restic: if S3, get bucket's region up-front
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-06-21 17:20:53 -07:00
25d3597c9a
add a /scratch emptyDir to ark pods to use for restic cache
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-06-20 11:48:09 -07:00
f2072e5868
use repo identifier from ResticRepo CR when backing up/restoring
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-06-20 10:42:53 -07:00
a927906e52
add --maintenance-frequency flag to ark restic repo init
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-06-20 09:21:43 -07:00
6e9e653f76
add restic repo getter and reorg restic cmds
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-06-20 09:21:43 -07:00
3481618324
move runCommand to pkg/util/exec and use in restic repo mgr
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-06-20 09:21:43 -07:00
65ed8da4b7
add ResticRepository CRD and move repo-level actions to a controller
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-06-20 09:21:43 -07:00
de12ca4882
move restic binary into /usr/bin so it's in PATH
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-06-19 11:31:27 -07:00
01b5828ee7
Merge pull request #403 from nrb/merge-serviceaccount-secrets
...
Merge serviceaccounts on restore
2018-06-15 09:46:26 -07:00
e7d00cf5fd
Add merge support for serviceaccounts
...
All properties from a backup will be merged into the ServiceAccount
except for the default token secret.
Signed-off-by: Nolan Brubaker <nolan@heptio.com>
2018-06-15 12:40:59 -04:00
2dfa7a1a72
Merge pull request #553 from ncdc/log-to-stdout
...
Log to stdout instead of stderr
2018-06-15 08:58:08 -07:00
6fb11b8087
get rid of restore staging dir by backing up/restoring within volume dir
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-06-14 13:49:27 -07:00
cc9140b3cc
Log to stdout instead of stderr
...
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2018-06-14 14:02:12 -04:00
7be81fe60e
Merge pull request #546 from skriss/rm-old-donefiles
...
remove existing .ark dir and contents during restic restores
2018-06-14 11:12:31 -04:00
05e86ee734
expose container ports in example deployment and pkg/install tooling
...
Signed-off-by: Ashish Amarnath <ashish.amarnath@gmail.com>
2018-06-13 19:22:09 -07:00
dc273e3bed
remove existing .ark dir and contents during restic restores
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-06-13 16:52:06 -07:00
9245e9d5dc
Merge pull request #547 from skriss/restic-rsync
...
use rsync to copy restored data from staging dir to volume dir
2018-06-13 16:56:42 -04:00
f7a42f378f
use rsync to copy restored data from staging dir to volume dir
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-06-13 13:41:55 -07:00
d1e3688468
Merge pull request #531 from ashish-amarnath/add-backup-metrics
...
Add metrics package to publish metrics
2018-06-13 16:33:30 -04:00
83658e891e
Add a metrics package to add and expose metrics
...
* add a metrics package to handle metric registration and publishing
* add a metricsAddress field to the server struct
* make metrics a part of the server
* start a metrics endpoint as part of starting the controllers
* instrument backup_controller to report metrics
* update cli-reference docs
* update example deployments with prometheus annotations
* update 'pkg/install' tooling with prometheus annotations
Signed-off-by: Ashish Amarnath <ashish.amarnath@gmail.com>
2018-06-13 13:17:08 -07:00
d7134b1df2
use Ark version as the tag for the restore-helper image in init container
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-06-11 10:46:10 -07:00
KubeKween
Steve Kriss
Nolan Brubaker
tsturzl
asaf-erlich
Michal Wieczorek
Steve Kriss
DheerajSShetty
Nolan Brubaker
Naeil ZOUEIDI
Daved
Nolan Brubaker
Steve Kriss
Andy Goldstein
Wayne Witzel III
Andy Goldstein
Jon Whitcraft
Bastian Hofmann
Wayne Witzel III
Gábor Lipták
Christian Beneke
Darko Radisic
Shubheksha Jalan
u5surf
Atul Bhosale
Domenic Rosati
captjt
James Powis
Marc Tudurí
Lukasz Jakimczuk
Zsolt Varga
Dave Parfitt
Timo Reimann
Alex Lemaresquier
Mike Arpaia
Nikhita Raghunath
Ashish Amarnath