5acccaa739
add bulk deletion support to ark backup delete
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-08-08 10:28:40 -07:00
1f7a4a1665
Merge pull request #712 from timoreimann/preserve-node-ports-when-specified-in-annotation
...
Preserve node ports during restore when annotations hold specification.
2018-08-07 09:17:27 -07:00
3aa241a74c
Preserve node ports during restore when annotations hold specification.
...
This is to better reflect the intent of the user when node ports are
specified explicitly (as opposed to being assigned by Kubernetes). The
`last-applied-configuration` annotation added by `kubectl apply` is one
such indicator we are now leveraging.
We still default to omitting the node ports when the annotation is
missing.
Signed-off-by: Timo Reimann <ttr314@googlemail.com>
2018-08-07 07:47:17 +02:00
27003af62a
Merge pull request #304 from ncdc/wait
...
Add --wait support to ark backup create
2018-08-03 10:57:20 -07:00
c5f5862c9c
Add --wait support to ark backup create
...
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2018-08-03 10:42:42 -07:00
c6ef76d2b3
Merge pull request #719 from nrb/prereq-faq
...
Document CRD not found errors
2018-08-02 16:03:09 -07:00
eb6f742b5d
Document CRD not found errors
...
Signed-off-by: Nolan Brubaker <nolan@heptio.com>
2018-08-02 16:41:28 -04:00
e7bb592602
Add --wait support to `ark restore create`
...
Signed-off-by: Carlisia <carlisia@grokkingtech.io>
2018-07-31 17:20:03 -07:00
6f061db9a2
Merge pull request #715 from skriss/remove-snapshot-service
...
remove SnapshotService, replace with direct BlockStore usage
2018-07-31 13:50:22 -07:00
8ce513acbd
Only delete unused backup if they are complete
...
Fixes #705
Signed-off-by: Carlisia <carlisia@grokkingtech.io>
2018-07-31 09:12:12 -07:00
1c26fbde32
remove SnapshotService, replace with direct BlockStore usage
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-07-31 08:58:11 -07:00
130512187a
Refactor plugin management
...
Refactor plugin management:
- support multiple plugins per executable
- support restarting a plugin process in the event it terminates
- simplify plugin lifecycle management by using separate managers for
each scope (server vs backup vs restore)
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2018-07-31 08:34:57 -07:00
131afb571e
Merge pull request #607 from nrb/restore-metrics
...
Restore metrics
2018-07-27 12:25:56 -07:00
74dbf38793
Add restore failed phase and metrics
...
Signed-off-by: Nolan Brubaker <nolan@heptio.com>
2018-07-27 14:32:11 -04:00
ca5656c279
fix bug preventing backup item action item updates from saving
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-07-26 10:38:17 -07:00
82f1cd87dc
Merge pull request #680 from carlisia/c-delete-from-etcd
...
Delete backups from etcd if they're not in storage
2018-07-25 18:25:52 -04:00
d2e629f5a8
Delete backups from etcd if they're not in storage
...
This fixes #309 .
Signed-off-by: Carlisia <carlisia@grokkingtech.io>
Co-authored-by: Steve Kriss <steve@heptio.com>
2018-07-25 14:53:50 -07:00
7cebfe2df0
Add restore attempt and success/failure counters
...
Signed-off-by: Nolan Brubaker <nolan@heptio.com>
2018-07-25 11:22:43 -04:00
06d6665abb
check s3URL scheme upon AWS ObjectStore Init()
...
Signed-off-by: Dave Parfitt <diparfitt@gmail.com>
2018-07-24 14:01:05 -04:00
b0af81e780
Merge pull request #689 from lemaral/init-schedule-metrics
...
Init schedule Prometheus metrics
2018-07-23 16:16:26 -04:00
f62045623c
cleanup service account action log statement
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-07-23 12:10:30 -07:00
e063b7983a
Merge pull request #682 from nrb/support-rbac-1.7
...
Support for rbac v1beta1 on kubernetes v1.7.5+
2018-07-23 12:00:50 -07:00
dd1e150511
Add RBAC support for 1.7 clusters
...
Signed-off-by: Nolan Brubaker <nolan@heptio.com>
2018-07-23 14:19:03 -04:00
c47a364ab3
Merge pull request #687 from skriss/fix-delete-issues
...
fix delete issues
2018-07-23 10:40:14 -07:00
450fa72fbb
Initialize schedule Prometheus metrics to have them created beforehand (see https://prometheus.io/docs/practices/instrumentation/#avoid-missing-metrics )
...
Signed-off-by: Alex Lemaresquier <alex+github@lemaresquier.org>
2018-07-20 23:30:24 +02:00
78cbdf95f3
delete old deletion requests for backup when processing a new one
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-07-20 11:42:42 -07:00
85a61b8e8d
return nil error if 404 encountered when deleting snapshots
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-07-20 11:42:34 -07:00
1df9a8a38d
exit server if not all Ark CRDs exist at startup
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-07-18 14:30:42 -07:00
e11634bfbc
Merge pull request #676 from skriss/dont-create-namespace
...
require namespace for backups/etc. to exist at server startup
2018-07-17 14:26:04 -04:00
13f893f1f9
Merge pull request #634 from marpaia/k8s-1.11
...
Update Kubernetes and Client-Go for 1.11.0 / 8.0.0
2018-07-17 08:55:47 -07:00
3efe677007
Update Ark library code to work with Kubernetes 1.11
...
Signed-off-by: Mike Arpaia <mike@arpaia.co>
2018-07-16 20:46:20 -06:00
d195512062
require namespace for backups/etc. to exist at server startup
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-07-13 13:50:12 -07:00
1e2b141e5d
Merge pull request #443 from skriss/restore-from-schedule
...
Add --from-schedule flag to `ark restore create`
2018-07-11 13:50:08 -04:00
683f7afc0d
switch to using .status.startTimestamp for sorting backups
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-07-11 10:35:55 -07:00
b71a37dbfc
Record backup completion time before uploading
...
Signed-off-by: Nolan Brubaker <nolan@heptio.com>
2018-07-11 13:21:29 -04:00
09afeb6d3e
Merge pull request #646 from skriss/remove-redundant-filesystem
...
replace pkg/restore's osFileSystem with pkg/util/filesystem's
2018-07-10 17:57:39 -04:00
fae00a7622
Merge pull request #659 from carlisia/cc-backupsync
...
Skip backup sync if it already exists on Kubernetes
2018-07-10 12:50:20 -04:00
5b89f7b6db
Skip backup sync if it already exists in k8s
...
Signed-off-by: Carlisia Pinto <carlisia@grokkingtech.io>
2018-07-09 21:55:24 -07:00
c6050845a0
restore controller: switch to 'c' for receiver name
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-07-09 15:07:38 -07:00
706ae07d0d
enable a schedule to be provided as the source for a restore
...
- ScheduleName is added as an API field to the Restore object
- Restore controller validates that exactly one of BackupName
or ScheduleName has been provided
- If ScheduleName is provided, Restore controller populates
BackupName with the name of the most recent successful backup
created from the schedule
- --from-schedule flag is added to `ark restore create` CLI cmd
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-07-09 15:07:38 -07:00
795dc26214
replace pkg/restore's osFileSystem with pkg/util/filesystem's
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-07-05 15:46:17 -07:00
2999f158db
Merge pull request #550 from skriss/restic-describers
...
Add pod volume backups/restores to ark backup/restore describe
2018-07-05 16:27:36 -04:00
8dd9cded1a
Fix testing arbitrary map key order
...
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2018-07-05 16:22:46 -04:00
42f2891485
add pod volume backups/restores to ark backup/restore describe output
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-07-05 13:13:40 -07:00
9db5e36b54
Fix test sorting function
...
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2018-07-05 16:08:05 -04:00
c18decc89b
pkg/restic unit tests
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-07-05 11:44:17 -07:00
547625c333
set --hostname flag for restic backups
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-07-02 10:52:32 -07:00
eabef08561
Update generated Ark code based on the 1.11 k8s.io/code-generator script
...
Signed-off-by: Mike Arpaia <mike@arpaia.co>
2018-07-02 09:24:19 -06:00
ae4aad0890
use pkg/util/exec for running get snapshot id cmd
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-06-29 12:06:31 -07:00
fe286ff564
Don't restore backups or restores
...
Add backups and restores the list of non restorable resources. Backups,
if applicable, are synced from object storage by the backup sync
controller. Restores are specific to a cluster and don't have value
moving across clusters.
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2018-06-28 16:58:09 -04:00
eaeb9d677e
Merge pull request #608 from skriss/no-pv-snapshot-if-restic-backup
...
don't snapshot volumes that have been backed up with restic
2018-06-28 13:27:30 -04:00
11c176c490
don't snapshot volumes that have been backed up with restic
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-06-28 10:19:34 -07:00
539de6d361
Merge pull request #564 from nrb/backup-timing
...
Record backup start and completion times, add timing metrics
2018-06-28 11:37:39 -04:00
96b72acb2d
Record backup start and completion times
...
Signed-off-by: Nolan Brubaker <nolan@heptio.com>
2018-06-28 11:18:14 -04:00
fa470170cf
Merge pull request #610 from skriss/restic-skip-invalid-volumes
...
log and skip backups for non-existent & hostPath volumes
2018-06-28 10:49:33 -04:00
255a991c6e
Stop restoring mirror pods
...
Mirror pods are pods created from static manifest files on a node.
They're mirrored to the apiserver so they're visible when querying the
apiserver for a list of pods, but it's not possible to send a pod
containing the mirror pod annotation to the apiserver and have it be
created successfully. Instead of trying to do this, log a message that
we're skipping restoring the pod because it's a mirror pod.
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2018-06-28 10:06:55 -04:00
781b7cd1aa
log and skip backups for non-existent & hostPath volumes
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-06-27 14:37:36 -07:00
22e8f23e2c
replace ark restic repo init cmd with automatic initialization of repos
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-06-27 13:50:37 -07:00
a697ad164e
refine what gets enqueued in PVB/PVR controllers, and log better
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-06-27 13:31:11 -07:00
29ac0b4a6c
Merge pull request #613 from ncdc/ignore-reclaim-policy-delete-pvs-without-snapshots
...
Skip reclaim policy Delete PVs without snapshots
2018-06-26 16:52:01 -04:00
7c283e5de8
Skip reclaim policy Delete PVs without snapshots
...
If a PV has a reclaim policy of Delete and we didn't create a snapshot
of it, don't restore the PV, as doing so would create a PV whose
underlying volume is incorrect.
Also "reset" any PVCs bound to the PV so they'll be dynamically
provisioned when restored.
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2018-06-26 10:40:49 -04:00
0da5f1ccca
Add omitempty tag to optional API fields
...
Signed-off-by: Nikhita Raghunath <nikitaraghunath@gmail.com>
2018-06-25 16:40:43 +05:30
5ad21854f7
restic: if S3, get bucket's region up-front
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-06-21 17:20:53 -07:00
25d3597c9a
add a /scratch emptyDir to ark pods to use for restic cache
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-06-20 11:48:09 -07:00
f2072e5868
use repo identifier from ResticRepo CR when backing up/restoring
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-06-20 10:42:53 -07:00
a927906e52
add --maintenance-frequency flag to ark restic repo init
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-06-20 09:21:43 -07:00
6e9e653f76
add restic repo getter and reorg restic cmds
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-06-20 09:21:43 -07:00
3481618324
move runCommand to pkg/util/exec and use in restic repo mgr
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-06-20 09:21:43 -07:00
65ed8da4b7
add ResticRepository CRD and move repo-level actions to a controller
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-06-20 09:21:43 -07:00
de12ca4882
move restic binary into /usr/bin so it's in PATH
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-06-19 11:31:27 -07:00
01b5828ee7
Merge pull request #403 from nrb/merge-serviceaccount-secrets
...
Merge serviceaccounts on restore
2018-06-15 09:46:26 -07:00
e7d00cf5fd
Add merge support for serviceaccounts
...
All properties from a backup will be merged into the ServiceAccount
except for the default token secret.
Signed-off-by: Nolan Brubaker <nolan@heptio.com>
2018-06-15 12:40:59 -04:00
2dfa7a1a72
Merge pull request #553 from ncdc/log-to-stdout
...
Log to stdout instead of stderr
2018-06-15 08:58:08 -07:00
6fb11b8087
get rid of restore staging dir by backing up/restoring within volume dir
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-06-14 13:49:27 -07:00
cc9140b3cc
Log to stdout instead of stderr
...
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2018-06-14 14:02:12 -04:00
7be81fe60e
Merge pull request #546 from skriss/rm-old-donefiles
...
remove existing .ark dir and contents during restic restores
2018-06-14 11:12:31 -04:00
05e86ee734
expose container ports in example deployment and pkg/install tooling
...
Signed-off-by: Ashish Amarnath <ashish.amarnath@gmail.com>
2018-06-13 19:22:09 -07:00
dc273e3bed
remove existing .ark dir and contents during restic restores
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-06-13 16:52:06 -07:00
9245e9d5dc
Merge pull request #547 from skriss/restic-rsync
...
use rsync to copy restored data from staging dir to volume dir
2018-06-13 16:56:42 -04:00
f7a42f378f
use rsync to copy restored data from staging dir to volume dir
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-06-13 13:41:55 -07:00
d1e3688468
Merge pull request #531 from ashish-amarnath/add-backup-metrics
...
Add metrics package to publish metrics
2018-06-13 16:33:30 -04:00
83658e891e
Add a metrics package to add and expose metrics
...
* add a metrics package to handle metric registration and publishing
* add a metricsAddress field to the server struct
* make metrics a part of the server
* start a metrics endpoint as part of starting the controllers
* instrument backup_controller to report metrics
* update cli-reference docs
* update example deployments with prometheus annotations
* update 'pkg/install' tooling with prometheus annotations
Signed-off-by: Ashish Amarnath <ashish.amarnath@gmail.com>
2018-06-13 13:17:08 -07:00
d7134b1df2
use Ark version as the tag for the restore-helper image in init container
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-06-11 10:46:10 -07:00
81520a9b86
rename restic-init-container to ark-restic-restore-helper
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-06-11 10:24:23 -07:00
2c6fc5bd90
don't fail the restore if cleanup of the staging directory post-restore fails
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-06-11 09:48:26 -07:00
6897c2f901
move dotfiles from staging to volume dir on restore
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-06-11 09:31:03 -07:00
e354b1c130
replace complete-restore.sh with go code
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-06-11 09:31:03 -07:00
c1f4e6d92d
rename ark daemonset to ark restic server
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-06-11 09:12:49 -07:00
0396ca1dee
Merge pull request #526 from yastij/ignore-resources-terminating
...
ignore terminating resources while doing a backup
2018-06-08 11:10:59 -04:00
17f6a14d37
ignore terminating resources while doing a backup
...
Signed-off-by: Yassine TIJANI <yasstij11@gmail.com>
2018-06-08 16:49:35 +02:00
67b40c7fc8
use fake filesystem to test key file flag for ark restic init-repo
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-06-07 10:29:59 -07:00
e2561f9073
ark restic init-repository: use key data if provided
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-06-07 10:06:15 -07:00
50d4084fac
add restic integration for doing pod volume backups/restores
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-06-06 09:48:10 -07:00
86b9cc6d15
BackupItemActionPlugin: handle nil updatedItem
...
Handle the case where a BackupItemAction may return nil for updatedItem,
meaning "no modifications to the item". The backupPVAction does this,
and we were panicking instead of accepting it.
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2018-06-01 14:03:32 -04:00
b2ec87f05f
Run post-hooks even if backup actions fail
...
Signed-off-by: Calle Pettersson <cpettsson@gmail.com>
2018-05-23 18:44:55 +02:00
fb33d93186
Merge pull request #512 from carlpett/additional-backup-error
...
Handle errors in additionalItemBackupper
2018-05-23 09:02:20 -07:00
170034787d
Merge pull request #513 from ncdc/510-follow-ups
...
Fix copyright date, add happy path test case
2018-05-23 08:54:22 -07:00
b92d086712
Add additional test
...
Signed-off-by: Calle Pettersson <cpettsson@gmail.com>
2018-05-23 17:28:51 +02:00
18e2401e79
Merge pull request #455 from skriss/remove-aws-region
...
AWS: remove region requirement from config for backupStorageProvider
2018-05-23 11:02:57 -04:00
849297e623
Fix copyright date, add happy path test case
...
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2018-05-23 09:59:32 -04:00
Steve Kriss
Steve Kriss
Timo Reimann
Andy Goldstein
Nolan Brubaker
Carlisia
Nolan Brubaker
Dave Parfitt
Alex Lemaresquier
Mike Arpaia
Nikhita Raghunath
Ashish Amarnath
Yassine TIJANI
Calle Pettersson