f62045623c
cleanup service account action log statement
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-07-23 12:10:30 -07:00
e063b7983a
Merge pull request #682 from nrb/support-rbac-1.7
...
Support for rbac v1beta1 on kubernetes v1.7.5+
2018-07-23 12:00:50 -07:00
dd1e150511
Add RBAC support for 1.7 clusters
...
Signed-off-by: Nolan Brubaker <nolan@heptio.com>
2018-07-23 14:19:03 -04:00
c47a364ab3
Merge pull request #687 from skriss/fix-delete-issues
...
fix delete issues
2018-07-23 10:40:14 -07:00
450fa72fbb
Initialize schedule Prometheus metrics to have them created beforehand (see https://prometheus.io/docs/practices/instrumentation/#avoid-missing-metrics )
...
Signed-off-by: Alex Lemaresquier <alex+github@lemaresquier.org>
2018-07-20 23:30:24 +02:00
78cbdf95f3
delete old deletion requests for backup when processing a new one
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-07-20 11:42:42 -07:00
85a61b8e8d
return nil error if 404 encountered when deleting snapshots
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-07-20 11:42:34 -07:00
1df9a8a38d
exit server if not all Ark CRDs exist at startup
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-07-18 14:30:42 -07:00
e11634bfbc
Merge pull request #676 from skriss/dont-create-namespace
...
require namespace for backups/etc. to exist at server startup
2018-07-17 14:26:04 -04:00
13f893f1f9
Merge pull request #634 from marpaia/k8s-1.11
...
Update Kubernetes and Client-Go for 1.11.0 / 8.0.0
2018-07-17 08:55:47 -07:00
3efe677007
Update Ark library code to work with Kubernetes 1.11
...
Signed-off-by: Mike Arpaia <mike@arpaia.co>
2018-07-16 20:46:20 -06:00
d195512062
require namespace for backups/etc. to exist at server startup
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-07-13 13:50:12 -07:00
1e2b141e5d
Merge pull request #443 from skriss/restore-from-schedule
...
Add --from-schedule flag to `ark restore create`
2018-07-11 13:50:08 -04:00
683f7afc0d
switch to using .status.startTimestamp for sorting backups
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-07-11 10:35:55 -07:00
b71a37dbfc
Record backup completion time before uploading
...
Signed-off-by: Nolan Brubaker <nolan@heptio.com>
2018-07-11 13:21:29 -04:00
09afeb6d3e
Merge pull request #646 from skriss/remove-redundant-filesystem
...
replace pkg/restore's osFileSystem with pkg/util/filesystem's
2018-07-10 17:57:39 -04:00
fae00a7622
Merge pull request #659 from carlisia/cc-backupsync
...
Skip backup sync if it already exists on Kubernetes
2018-07-10 12:50:20 -04:00
5b89f7b6db
Skip backup sync if it already exists in k8s
...
Signed-off-by: Carlisia Pinto <carlisia@grokkingtech.io>
2018-07-09 21:55:24 -07:00
c6050845a0
restore controller: switch to 'c' for receiver name
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-07-09 15:07:38 -07:00
706ae07d0d
enable a schedule to be provided as the source for a restore
...
- ScheduleName is added as an API field to the Restore object
- Restore controller validates that exactly one of BackupName
or ScheduleName has been provided
- If ScheduleName is provided, Restore controller populates
BackupName with the name of the most recent successful backup
created from the schedule
- --from-schedule flag is added to `ark restore create` CLI cmd
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-07-09 15:07:38 -07:00
795dc26214
replace pkg/restore's osFileSystem with pkg/util/filesystem's
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-07-05 15:46:17 -07:00
2999f158db
Merge pull request #550 from skriss/restic-describers
...
Add pod volume backups/restores to ark backup/restore describe
2018-07-05 16:27:36 -04:00
8dd9cded1a
Fix testing arbitrary map key order
...
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2018-07-05 16:22:46 -04:00
42f2891485
add pod volume backups/restores to ark backup/restore describe output
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-07-05 13:13:40 -07:00
9db5e36b54
Fix test sorting function
...
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2018-07-05 16:08:05 -04:00
c18decc89b
pkg/restic unit tests
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-07-05 11:44:17 -07:00
547625c333
set --hostname flag for restic backups
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-07-02 10:52:32 -07:00
eabef08561
Update generated Ark code based on the 1.11 k8s.io/code-generator script
...
Signed-off-by: Mike Arpaia <mike@arpaia.co>
2018-07-02 09:24:19 -06:00
ae4aad0890
use pkg/util/exec for running get snapshot id cmd
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-06-29 12:06:31 -07:00
fe286ff564
Don't restore backups or restores
...
Add backups and restores the list of non restorable resources. Backups,
if applicable, are synced from object storage by the backup sync
controller. Restores are specific to a cluster and don't have value
moving across clusters.
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2018-06-28 16:58:09 -04:00
eaeb9d677e
Merge pull request #608 from skriss/no-pv-snapshot-if-restic-backup
...
don't snapshot volumes that have been backed up with restic
2018-06-28 13:27:30 -04:00
11c176c490
don't snapshot volumes that have been backed up with restic
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-06-28 10:19:34 -07:00
539de6d361
Merge pull request #564 from nrb/backup-timing
...
Record backup start and completion times, add timing metrics
2018-06-28 11:37:39 -04:00
96b72acb2d
Record backup start and completion times
...
Signed-off-by: Nolan Brubaker <nolan@heptio.com>
2018-06-28 11:18:14 -04:00
fa470170cf
Merge pull request #610 from skriss/restic-skip-invalid-volumes
...
log and skip backups for non-existent & hostPath volumes
2018-06-28 10:49:33 -04:00
255a991c6e
Stop restoring mirror pods
...
Mirror pods are pods created from static manifest files on a node.
They're mirrored to the apiserver so they're visible when querying the
apiserver for a list of pods, but it's not possible to send a pod
containing the mirror pod annotation to the apiserver and have it be
created successfully. Instead of trying to do this, log a message that
we're skipping restoring the pod because it's a mirror pod.
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2018-06-28 10:06:55 -04:00
781b7cd1aa
log and skip backups for non-existent & hostPath volumes
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-06-27 14:37:36 -07:00
22e8f23e2c
replace ark restic repo init cmd with automatic initialization of repos
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-06-27 13:50:37 -07:00
a697ad164e
refine what gets enqueued in PVB/PVR controllers, and log better
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-06-27 13:31:11 -07:00
29ac0b4a6c
Merge pull request #613 from ncdc/ignore-reclaim-policy-delete-pvs-without-snapshots
...
Skip reclaim policy Delete PVs without snapshots
2018-06-26 16:52:01 -04:00
7c283e5de8
Skip reclaim policy Delete PVs without snapshots
...
If a PV has a reclaim policy of Delete and we didn't create a snapshot
of it, don't restore the PV, as doing so would create a PV whose
underlying volume is incorrect.
Also "reset" any PVCs bound to the PV so they'll be dynamically
provisioned when restored.
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2018-06-26 10:40:49 -04:00
0da5f1ccca
Add omitempty tag to optional API fields
...
Signed-off-by: Nikhita Raghunath <nikitaraghunath@gmail.com>
2018-06-25 16:40:43 +05:30
5ad21854f7
restic: if S3, get bucket's region up-front
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-06-21 17:20:53 -07:00
25d3597c9a
add a /scratch emptyDir to ark pods to use for restic cache
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-06-20 11:48:09 -07:00
f2072e5868
use repo identifier from ResticRepo CR when backing up/restoring
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-06-20 10:42:53 -07:00
a927906e52
add --maintenance-frequency flag to ark restic repo init
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-06-20 09:21:43 -07:00
6e9e653f76
add restic repo getter and reorg restic cmds
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-06-20 09:21:43 -07:00
3481618324
move runCommand to pkg/util/exec and use in restic repo mgr
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-06-20 09:21:43 -07:00
65ed8da4b7
add ResticRepository CRD and move repo-level actions to a controller
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-06-20 09:21:43 -07:00
de12ca4882
move restic binary into /usr/bin so it's in PATH
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-06-19 11:31:27 -07:00
01b5828ee7
Merge pull request #403 from nrb/merge-serviceaccount-secrets
...
Merge serviceaccounts on restore
2018-06-15 09:46:26 -07:00
e7d00cf5fd
Add merge support for serviceaccounts
...
All properties from a backup will be merged into the ServiceAccount
except for the default token secret.
Signed-off-by: Nolan Brubaker <nolan@heptio.com>
2018-06-15 12:40:59 -04:00
2dfa7a1a72
Merge pull request #553 from ncdc/log-to-stdout
...
Log to stdout instead of stderr
2018-06-15 08:58:08 -07:00
6fb11b8087
get rid of restore staging dir by backing up/restoring within volume dir
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-06-14 13:49:27 -07:00
cc9140b3cc
Log to stdout instead of stderr
...
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2018-06-14 14:02:12 -04:00
7be81fe60e
Merge pull request #546 from skriss/rm-old-donefiles
...
remove existing .ark dir and contents during restic restores
2018-06-14 11:12:31 -04:00
05e86ee734
expose container ports in example deployment and pkg/install tooling
...
Signed-off-by: Ashish Amarnath <ashish.amarnath@gmail.com>
2018-06-13 19:22:09 -07:00
dc273e3bed
remove existing .ark dir and contents during restic restores
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-06-13 16:52:06 -07:00
9245e9d5dc
Merge pull request #547 from skriss/restic-rsync
...
use rsync to copy restored data from staging dir to volume dir
2018-06-13 16:56:42 -04:00
f7a42f378f
use rsync to copy restored data from staging dir to volume dir
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-06-13 13:41:55 -07:00
d1e3688468
Merge pull request #531 from ashish-amarnath/add-backup-metrics
...
Add metrics package to publish metrics
2018-06-13 16:33:30 -04:00
83658e891e
Add a metrics package to add and expose metrics
...
* add a metrics package to handle metric registration and publishing
* add a metricsAddress field to the server struct
* make metrics a part of the server
* start a metrics endpoint as part of starting the controllers
* instrument backup_controller to report metrics
* update cli-reference docs
* update example deployments with prometheus annotations
* update 'pkg/install' tooling with prometheus annotations
Signed-off-by: Ashish Amarnath <ashish.amarnath@gmail.com>
2018-06-13 13:17:08 -07:00
d7134b1df2
use Ark version as the tag for the restore-helper image in init container
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-06-11 10:46:10 -07:00
81520a9b86
rename restic-init-container to ark-restic-restore-helper
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-06-11 10:24:23 -07:00
2c6fc5bd90
don't fail the restore if cleanup of the staging directory post-restore fails
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-06-11 09:48:26 -07:00
6897c2f901
move dotfiles from staging to volume dir on restore
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-06-11 09:31:03 -07:00
e354b1c130
replace complete-restore.sh with go code
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-06-11 09:31:03 -07:00
c1f4e6d92d
rename ark daemonset to ark restic server
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-06-11 09:12:49 -07:00
0396ca1dee
Merge pull request #526 from yastij/ignore-resources-terminating
...
ignore terminating resources while doing a backup
2018-06-08 11:10:59 -04:00
17f6a14d37
ignore terminating resources while doing a backup
...
Signed-off-by: Yassine TIJANI <yasstij11@gmail.com>
2018-06-08 16:49:35 +02:00
67b40c7fc8
use fake filesystem to test key file flag for ark restic init-repo
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-06-07 10:29:59 -07:00
e2561f9073
ark restic init-repository: use key data if provided
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-06-07 10:06:15 -07:00
50d4084fac
add restic integration for doing pod volume backups/restores
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-06-06 09:48:10 -07:00
86b9cc6d15
BackupItemActionPlugin: handle nil updatedItem
...
Handle the case where a BackupItemAction may return nil for updatedItem,
meaning "no modifications to the item". The backupPVAction does this,
and we were panicking instead of accepting it.
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2018-06-01 14:03:32 -04:00
b2ec87f05f
Run post-hooks even if backup actions fail
...
Signed-off-by: Calle Pettersson <cpettsson@gmail.com>
2018-05-23 18:44:55 +02:00
fb33d93186
Merge pull request #512 from carlpett/additional-backup-error
...
Handle errors in additionalItemBackupper
2018-05-23 09:02:20 -07:00
170034787d
Merge pull request #513 from ncdc/510-follow-ups
...
Fix copyright date, add happy path test case
2018-05-23 08:54:22 -07:00
b92d086712
Add additional test
...
Signed-off-by: Calle Pettersson <cpettsson@gmail.com>
2018-05-23 17:28:51 +02:00
18e2401e79
Merge pull request #455 from skriss/remove-aws-region
...
AWS: remove region requirement from config for backupStorageProvider
2018-05-23 11:02:57 -04:00
849297e623
Fix copyright date, add happy path test case
...
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2018-05-23 09:59:32 -04:00
92e9d307a5
Merge pull request #510 from nrb/failed-backup-upload-491
...
Fail backup if object store upload fails
2018-05-23 09:55:11 -04:00
09bbe072cd
Check errors when closing GCP objects
...
Writing to GCP's object store is any async operation, so errors need to
be checked both on write and close calls, since errors like permission
violations aren't reported until a close.
Signed-off-by: Nolan Brubaker <nolan@heptio.com>
2018-05-22 17:38:49 -04:00
b029860b46
AWS: remove region requirement from config for backupStorageProvider
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-05-22 11:59:53 -07:00
24dfef6f15
Handle errors in additionalItemBackupper
...
Signed-off-by: Calle Pettersson <cpettsson@gmail.com>
2018-05-22 18:32:48 +02:00
cb7bcea5c3
Add functional options for the Ark config install library
...
Signed-off-by: Mike Arpaia <mike@arpaia.co>
2018-05-22 09:21:12 -06:00
aeb5f6d832
Merge pull request #437 from marpaia/install-api
...
Add library code to install required server components
2018-05-18 16:48:42 -04:00
16f707aa11
Add library code to install required server components
...
This PR includes Go library code to assist with the installation of
required server-side components.
Signed-off-by: Mike Arpaia <mike@arpaia.co>
2018-05-18 14:43:06 -06:00
2fde1f5fc1
Merge pull request #500 from nrb/fix-497
...
Exit when failing to update a backup's phase
2018-05-17 14:23:41 -04:00
3a746a3f73
Merge pull request #496 from nrb/fix-477
...
Check restore Phase before downloading logs
2018-05-17 13:23:23 -04:00
f288902e3e
Merge pull request #490 from skriss/jsonpatch
...
Use JSON merge patches everywhere
2018-05-17 13:02:47 -04:00
7eac6675e8
Exit when failing to update a backup's phase
...
Signed-off-by: Nolan Brubaker <nolan@heptio.com>
2018-05-17 12:47:38 -04:00
f6761ddd00
Check restore Phase before downloading logs
...
Signed-off-by: Nolan Brubaker <nolan@heptio.com>
2018-05-17 12:20:55 -04:00
5d74a92cf1
Merge pull request #484 from nrb/move-groupresources
...
Consolidate commonly used GroupResource objects
2018-05-15 14:02:29 -07:00
f936c55a37
Consolidate commonly used GroupResource objects
...
Signed-off-by: Nolan Brubaker <nolan@heptio.com>
2018-05-15 15:55:15 -04:00
ad93135adb
Merge pull request #483 from skriss/graceful-shutdown-on-sigs
...
shutdown gracefully on SIGINT/SIGTERM
2018-05-15 14:33:09 -04:00
67263d2652
Merge pull request #486 from nrb/fix-475
...
Filter on resources that support get & delete
2018-05-15 10:58:56 -04:00
4fcd222777
Expand required resource verbs with get and delete
...
Some resources use GET for listing, which resulted in errors.
Signed-off-by: Nolan Brubaker <nolan@heptio.com>
2018-05-15 10:55:03 -04:00
6d6f734bc9
use json merge patches
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-05-14 14:34:24 -07:00
1c950aa17b
unit test to ensure new cohabitatingResources map for each backup
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-05-14 10:23:21 -07:00
ed7fbc9178
use new cohabitatingResources map for each backup
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-05-11 16:30:14 -07:00
20f56e9868
cleanup plugin subprocesses on server exit
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-05-11 09:48:46 -07:00
09c20b51e6
shutdown gracefully on SIGINT/SIGTERM
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-05-10 16:22:14 -07:00
fbb5ead4e9
add replicasets and daemonsets to cohabitating resources
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-05-10 14:15:01 -07:00
43b1f9a19e
Merge pull request #418 from skriss/refactor-patch-tests
...
use typed structs for decoding patch JSON in unit tests
2018-05-09 15:44:21 -04:00
ea83ed32f5
Merge pull request #470 from skriss/backup-cluster-roles-and-bindings
...
backups: include clusterroles/bindings that reference serviceaccounts
2018-05-09 13:21:13 -04:00
041cfc2173
backups: include clusterroles/bindings that reference serviceaccounts
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-05-09 10:03:55 -07:00
ef5ac7fd05
Merge pull request #463 from nrb/fix-454
...
Don't restore completed pods or jobs
2018-05-09 09:59:34 -04:00
e81de2491f
remove duplicate github.com/satori/go.uuid dependency
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-05-07 10:02:59 -07:00
51928e9177
use typed structs for decoding patch JSON in unit tests
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-05-07 09:22:20 -07:00
6754955bcd
when syncing backups, set their namespace to current cluster's Ark ns
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-05-07 09:12:51 -07:00
923870390b
Skip completed jobs and pods when restoring
...
Completed jobs and pods may be useful in the backup for auditing
purposes, but don't recreate them when restoring.
Signed-off-by: Nolan Brubaker <nolan@heptio.com>
2018-05-02 12:16:54 -04:00
18d6b233da
Merge pull request #453 from nrb/fix-444
...
Make empty excludes string more accurate
2018-04-24 14:53:33 -04:00
c8989231eb
Merge pull request #417 from skriss/upgrade-k8s-deps
...
Upgrade apimachinery, client-go, api, kubernetes dependencies
2018-04-24 14:53:11 -04:00
05cb059b1a
Make empty excludes string more accurate
...
Signed-off-by: Nolan Brubaker <nolan@heptio.com>
2018-04-24 14:34:56 -04:00
72d7e5e01d
azure: pass full snapshot name when restoring disk
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-04-23 12:51:05 -07:00
a466208975
add backwards-compatibility for azure snapshot ID
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-04-23 11:34:37 -07:00
3dc093c24a
regenerate code with updated generator
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-04-20 12:59:22 -07:00
989169dcfe
remove unused kubernetes/pkg/util/version dependency
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-04-20 12:59:22 -07:00
97e52f2b3c
add timeout arg to rest.NewRequest()
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-04-20 12:58:53 -07:00
961d7f2924
kubernetes/pkg/printers/ShortHumanDuration -> apimachinery/pkg/util/duration
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-04-20 12:58:53 -07:00
300a010959
Remove finalizers from backups before syncing
...
Signed-off-by: Nolan Brubaker <nolan@heptio.com>
2018-04-18 17:32:39 -04:00
730ddb2113
Merge pull request #391 from containscafeine/handle-restore-missing-error
...
Introduce validation logic to `ark restore logs`
2018-04-12 11:07:29 -04:00
7f3e88151b
Add bash and zsh completion support
...
This commit adds support for auto completion for bash and zsh
shells. A new root level command called "completion" has been
introduced, and the user can get the auto completion code by
running `ark completion bash/zsh`.
For bash completion, the built-in GenBashCompletion() from cobra
has been used, but for zsh, the built-in GenZshCompletion() is
known to cause issues. The workaround has been copied from zsh
completion code of kubectl.
Signed-off-by: Shubham <shubham@linux.com>
2018-04-12 13:26:23 +05:30
468c4faf1b
Introduce validation logic to `ark restore logs`
...
This commit introduces validation logic to `ark restore logs`
command, the way it already exists in other commands like `ark
restore create`.
Before the logs for a restore are fetched from the server, the
server is contacted to check if the specified restore exists. If
it does not, it errors out.
Fixes #389
Signed-off-by: Shubham <shubham@linux.com>
2018-04-12 13:06:32 +05:30
0998f6d8af
Merge pull request #405 from nrb/ignore-duplicates
...
Compare for duplicates before logging object exists errors
2018-04-11 09:47:37 -04:00
33cc85cd0c
Compare backup and cluster objects before logging
...
When restoring resources that raise an already exists error, check their
equality before logging a message on the restore. If they're the same
except for some metadata, don't generate a message.
The restore process was modified so that if an object had an empty
namespace string, no namespace key is created on the object. This was to
avoid manipulating the copy of the current cluster's object by adding
the target namespace.
There are some cases right now that are known to not be equal via this
method:
- The `default` ServiceAccount in a namespace will not match, primarily
because of differing default tokens. These will be handled in their own
patch
- IP addresses for Services are recorded in the backup object, but are
either not present on the cluster object, or different. An issue for
this already exists at https://github.com/heptio/ark/issues/354
- Endpoints have differing values for `renewTime`. This may be
insubstantial, but isn't currently handled by the resetMetadataAndStatus
function.
- PersistentVolume objects do not match on spec fields, such as
claimRef and cloud provider persistent disk info
Signed-off-by: Nolan Brubaker <nolan@heptio.com>
2018-04-10 14:55:48 -04:00
258f3e011e
Merge pull request #409 from skriss/fix-398
...
get backup name from restore spec when handling restore DownloadRequests
2018-04-10 13:29:07 -04:00
0d2b49acce
get backup name from restore spec when handling restore DownloadRequests
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-04-09 10:50:10 -07:00
5bb47d2b7a
extract a helper for comparing slices of Actions reliably
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-04-09 09:39:33 -07:00
3f2e222ae4
Don't allow deletion of in-progress backups
...
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2018-04-06 13:08:39 -04:00
644a75e3c1
Set DeleteBackupRequest labels if missing
...
When the BackupDeletionController processes a request, set the request's
backup-name and backup-uid labels if they aren't currently set.
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2018-04-05 15:38:44 -04:00
ef57a44827
Require DeleteBackupRequest.Spec.BackupName
...
Make sure a DeleteBackupRequest has its Spec.BackupName filled in. If
not, record an error in the status and mark the request as processed.
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2018-04-05 14:53:10 -04:00
4328b67f93
backup delete/describe enhancements
...
Always request DeleteBackupRequests for a given backup so we can show
failed deletion attempts if you try to delete a backup that has PV
snapshots when Ark doesn't have a persistentVolumeProvider configured.
When creating a DeleteBackupRequest, include a label for the UID so we
can match based on name and UID when associated DeleteBackupRequests
with a given backup.
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2018-04-05 14:21:45 -04:00
1ae492da6c
Expire processed deleted backup requests > 24hr
...
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2018-04-05 11:19:11 -04:00
74f60b1ee1
Switch backup finalizer to DeleteBackupRequest
...
We ran into a lot of problems using a finalizer on the backup to allow
the Ark server to clean up all associated backup data when deleting a
backup.
Users also found it less than desirable that deleting the heptio-ark
namespace resulted in all the backup data being deleted.
This removes the finalizer and replaces it with an explicit
DeleteBackupRequest that is created as a means of requesting the
deletion of a backup and all its associated data. This is what `ark
backup delete` does.
If you use kubectl to delete a backup or to delete the heptio-ark
namespace, this no longer deletes associated backups. Additionally, as
long as the heptio-ark namespace still exists, the Ark server's
BackupSyncController will continually sync backups into the heptio-ark
namespace from object storage.
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2018-04-05 11:16:15 -04:00
80b66434c0
move getting client into Complete()
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-04-03 20:43:42 -07:00
c60e47dedd
use cobra's arg-count validation & call Complete() before Validate()
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-04-03 09:45:29 -07:00
73499c2cff
Add limitranges to defaultResourcePriorities
...
This commit adds limitranges to defaultResourcePriorities as
suggested in #385 .
This is done so that pods are not restored before the LimitRange
objects, because that would lead to pods not honoring the requests
and limits set in LimitRange objects.
Fixes #385
Signed-off-by: Shubham <shubham@linux.com>
2018-03-17 21:13:44 +05:30
560f9504e1
Fix typo: replace ; with :
...
This commit replaces a ; with a : in an error message
Signed-off-by: Shubham <shubham@linux.com>
2018-03-16 18:24:34 +05:30
3172e907a1
Merge pull request #341 from skriss/snapshot-tags
...
Add tags to snapshots, restored volumes
2018-03-14 12:50:40 -04:00
07fcc927b7
Merge pull request #359 from skriss/reduce-azure-polling-interval
...
Azure: reduce client polling interval from default 60s to 5s
2018-03-14 12:17:54 -04:00
45cee7d222
Azure: reduce client polling interval from default 60s to 5s
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-03-14 09:07:20 -07:00
b2b16b3c85
Azure: fix bug preventing snapshot deletion
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-03-13 17:03:14 -07:00
0388845bca
AWS: tag snapshots during create
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-03-13 12:17:50 -07:00
a05ae1a7cf
add useful Ark tags to snapshots
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-03-13 12:17:50 -07:00
df985bca74
Azure: copy tags from volume to snapshot, and snapshot to volume
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-03-13 12:17:50 -07:00
cab904570f
GCP: copy tags from volume to snapshot, and snapshot to volume
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-03-13 11:52:09 -07:00
9673e9d158
AWS: copy tags from volume to snapshot, and snapshot to volume
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-03-13 11:52:09 -07:00
51c546786e
Merge pull request #374 from sanketjpatel/feat/exclude-events-restore
...
Add events to nonRestorableResources and cohabitatingResources
2018-03-13 11:23:17 -07:00
f83b1de1dc
Merge pull request #356 from skriss/store-azure-snapshot-uri
...
Azure: store snapshot URI to support cross-resource group restores
2018-03-13 13:45:48 -04:00
409f17361d
Azure: store snapshot URI to support cross-resource group restores
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-03-13 09:18:20 -07:00
7257a75f82
Add events to nonRestorableResources and cohabitatingResources
...
Fixes #367
Fixes #368
Signed-off-by: Sanket Patel <sanketpatel.301090@gmail.com>
2018-03-12 19:55:32 -05:00
178a007b70
Merge pull request #362 from skriss/tweak-hook-logging
...
during backup, only log if hooks are actually executing
2018-03-07 13:37:03 -05:00
26944cf9a2
only log if hooks are actually executing
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-03-07 09:29:27 -08:00
7328fd0e10
Merge pull request #343 from skriss/gc-bug
...
add an AddFunc to handle finalizers in GC for initial lists/resyncs
2018-03-06 16:41:57 -05:00
331e0c28cc
Remove Azure location requirement
...
Instead of requiring the Ark admin to specify a "location" in the azure
persistentVolumeProvider config (meaning only a single location is
supported), get info about the disk (for its location) when creating a
snapshot, and get info about the snapshot (for its location) when
creating a disk from a snapshot.
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2018-03-05 15:20:58 -05:00
c281124cba
add --from-backup flag to ark restore create & allow restore name
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-03-05 10:06:43 -08:00
6930c846e5
Fix incorrect return when err != nil
...
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2018-03-05 10:57:23 -05:00
cc9be4489a
Merge pull request #345 from skriss/gcp-project
...
extract GCP project ID from creds file, remove from config
2018-03-05 10:35:39 -05:00
cbff5ef53b
extract GCP project ID from creds file, remove from config
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-03-02 09:15:12 -08:00
509431909a
add an AddFunc to handle finalizers in GC for initial lists/resyncs
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-02-28 16:56:23 -08:00
975dce929c
Merge pull request #331 from lypht/schedule
...
Include cron examples for schedule creation
2018-02-28 15:32:28 -08:00
f733869142
Include cron examples for schedule creation
...
Signed-off-by: Josh Hull <josh.hull@lypht.com>
2018-02-28 16:50:52 -06:00
e618e0e456
server: allow configurable plugin dir
...
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2018-02-27 14:50:38 -05:00
f13b0c00a3
server: don't assume /ark for internal plugins
...
If you want to test changes to the ark server without having to rebuild
and redeploy the ark container, this change allows you to do something
like this (assuming you've created your cloud credentials file):
AWS_SHARED_CREDENTIALS_FILE=credentials-minio ark server -n heptio-ark
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2018-02-27 14:32:45 -05:00
e73ba839f3
Merge pull request #296 from blakebarnett/bdb/add_kubecontext_flag
...
Adds a kubecontext global flag
2018-02-26 17:05:59 -05:00
abeeee6643
Merge pull request #333 from ncdc/add-force-delete-backup
...
Allow forced backup deletion
2018-02-26 16:44:37 -05:00
d24fb232cc
Allow forced backup deletion
...
Add --force and --confirm to `ark backup delete` to support forced
backup deletion. This forcibly removes the Ark GC finalizer (if it's
present) from a backup and will orphan any resources associated with the
backup, such as backup tarballs in object storage, persistent volume
snapshots, and restores for the backup.
If a backup has a deletion timestamp, display `Deleting` in `ark backup
describe` and `ark backup get`.
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2018-02-26 16:25:22 -05:00
465c60b408
Add global kubecontext CLI flag
...
Signed-off-by: Blake <blake.barnett@postmates.com>
2018-02-26 10:47:39 -08:00
cc4d2916fa
Error if backup missing when creating a restore
...
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2018-02-05 13:29:48 -05:00
b2cd8e1fe8
Create the config directory in case it's missing
...
Always try to create the config directory when saving the client config
in case it doesn't exist.
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2018-02-02 16:01:52 -05:00
6712e2a9ec
Merge pull request #270 from nrb/fix-232
...
Clarify backup and restore creation messages
2018-01-11 12:22:11 -05:00
f5f1dbac5d
Merge pull request #271 from nrb/fix-246
...
Invalidate restores based on non-existent backups
2018-01-10 13:24:53 -05:00
af81f589f6
update GCP block store to use v1 library, and tag snapshots on create
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-01-08 09:23:32 -08:00
37f498cd37
Clarify backup and restore creation messages
...
When running `ark <resource> create`, a request is sent to the server,
but the status is not immediately known. Inform the user that a request
was sent and provide a way to get more information on it.
Signed-off-by: Nolan Brubaker <nolan@heptio.com>
2018-01-08 10:26:51 -05:00
bb7b0a3c33
Invalidate restores based on missing backups
...
When creating a restore based on a backup that doesn't exist, the
restore should be marked as invalid and the error clearly communicated
so the user understands why the restore wasn't made.
Previously, the restore was left as in progress with an error attached.
Since restores are CRDs and must be updated via a controller, there's
currently not a way to give the client immediate errors.
Signed-off-by: Nolan Brubaker <nolan@heptio.com>
2018-01-08 10:10:01 -05:00
b53ee44176
update GCP object store to use new/recommended lib
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-01-05 14:21:13 -08:00
e130011d1e
Merge pull request #272 from ncdc/arbitrary-namespaces
...
Support running in any namespace
2018-01-05 14:15:18 -08:00
816f14c0b4
Support running in any namespace
...
Add the ability for the Ark server to run in any namespace.
Add `ark client config get/set` for manipulating the new client
configuration file in $HOME/.config/ark/config.json. This holds client
defaults, such as the Ark server's namespace (to avoid having to specify
the --namespace flag all the time).
Add a --namespace flag to all client commands.
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2018-01-05 16:51:59 -05:00
de785af89d
Support pre and post hooks.
...
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2018-01-05 10:47:34 -05:00
d56b5f2b66
Updated generated CRD code
...
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2018-01-04 10:44:20 -05:00
656428d0b9
Merge pull request #250 from ncdc/backup-controller-do-as-much-as-possible
...
BackupController: do as much as possible
2018-01-03 12:10:51 -08:00
1b124a3146
Log backup item action error when it occurs
...
If a backup item action errors, log the error as soon as it occurs, so
it's clear when the error happened. Also include information about the
groupResource, namespace, and name of the item in the error.
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2018-01-03 13:20:05 -05:00
6b0b6373da
Merge pull request #252 from skriss/delete-backup
...
add delete backup cmd using finalizer and simplify GC process
2018-01-02 14:26:03 -05:00
56ca4bab2f
Merge pull request #268 from skriss/2018
...
update copyright header to 2018 and make language/casing consistent
2018-01-02 14:16:26 -05:00
1503796419
make language and casing consistent across copyrights
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-01-02 10:51:49 -08:00
3b157afbbd
update copyright year to 2018 for generated files
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-01-02 10:48:42 -08:00
1c974782fa
disable GC and backup deletion if Kubernetes is less than v1.7.5
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-12-22 10:25:32 -08:00
8878ba860e
don't remove annotations from PVs on restore
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-12-21 13:23:48 -08:00
8e5feec39c
include restore & schedule under ark delete
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-12-21 10:47:37 -08:00
f5123794e0
add delete backup cmd using finalizer and simplify GC process
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-12-21 10:47:37 -08:00
0fc087c967
Flatten aggregated errors
...
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2017-12-21 11:56:33 -05:00
fbda5fe539
Add serviceaccounts to prioritized resources
...
Add serviceaccounts to the default list of prioritized resources used
when restoring.
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2017-12-20 14:59:47 -05:00
1e581f1ead
BackupController: do as much as possible
...
When running a backup, try to do as much as possible, collecting errors
along the way, and return an aggregate at the end. This way, if a backup
fails for most reasons, we'll be able to upload the backup log file to
object storage, which wasn't happening before.
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2017-12-20 14:49:27 -05:00
fea701a03b
Merge pull request #256 from skriss/unbound-pvc-fix
...
don't error if PVC is not bound to a PV
2017-12-19 19:37:28 -05:00
1210cb36e1
Merge pull request #253 from skriss/update-license
...
Update all license headers
2017-12-19 19:26:54 -05:00
79c0e5a09a
don't error if PVC is not bound to a PV
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-12-19 14:10:58 -08:00
3641c2c043
update all license headers
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-12-15 13:38:12 -08:00
56c9d68137
switch to logrus.FieldLogger and arktest.NewLogger() everywhere
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-12-13 09:38:00 -08:00
4aea9b9a2c
switch from Update() to Patch()
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-12-12 15:23:45 -08:00
992940c55c
Merge pull request #227 from skriss/plugin-logger-fix
...
Bug fixes: obj/block store plugin logging and remapped namespaces issue
2017-11-30 12:17:56 -05:00
121b7153a7
when restoring, don't restore the Namespaces cluster-scoped resource independently
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-11-30 09:07:38 -08:00
bd8f433188
Merge pull request #229 from skriss/plugin-docs
...
add godoc for ResourceSelector
2017-11-30 11:25:53 -05:00
94ac3b3529
godoc for ResourceSelector
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-11-30 08:20:27 -08:00
8a0ac2117b
give each plugin its own logrusAdapter
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-11-29 14:36:21 -08:00
cbcd15d603
add/update documentation for plugins
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-11-29 14:24:23 -08:00
b66efd9416
rename NewPluginLogger to NewLogger
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-11-29 14:01:31 -08:00
038fa39451
remove ListSnapshots and GetAllSnapshots
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-11-29 13:59:29 -08:00
995050390c
add plugin add/remove commands
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-11-29 12:22:30 -08:00
0b6e78145c
Merge pull request #223 from ncdc/simplify-plugin-serve
...
Simplify serving plugins for external developers
2017-11-29 11:09:22 -08:00
5b8562e73c
Merge pull request #222 from ncdc/fix-log-location-hook
...
Fix log location hook prefix stripping
2017-11-29 11:07:43 -08:00
dc484d1566
Set logger for cloudprovider plugin
...
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2017-11-29 13:54:20 -05:00
1f7e9b65e8
Simply serving plugins for external developers
...
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2017-11-29 13:42:53 -05:00
c700455272
Support custom volume snapshots & restores
...
The main Ark code was hard-coding specific support for AWS, GCE, and
Azure volume snapshots and restores, and anything else was considered
unsupported.
Add GetVolumeID and SetVolumeID to the BlockStore interface, to allow
block store plugins to handle volume snapshots and restores.
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2017-11-29 13:19:40 -05:00
99f67db39d
Fix log location hook prefix stripping
...
The log location hook was matching github.com/heptio/ark and stripping
off that + 1 more char. This meant that
github.com/heptio/ark-plugin-example/foo.go was being listed as
plugin-example/foo.go instead of
github.com/heptio/ark-plugin-example/foo.go.
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2017-11-29 12:56:01 -05:00
526b604237
Add boolptr helper package
...
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2017-11-29 12:25:10 -05:00
f0b35cc45a
Merge pull request #213 from skriss/restore-action-plugins
...
Convert restorers to plugins
2017-11-28 14:50:41 -05:00
179b95c81d
convert restorers to plugins
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-11-28 10:58:12 -08:00
c2dc41efd8
Add backup & schedule describers
...
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2017-11-27 15:22:20 -05:00
0f2d1ab82b
migrate backup actions to plugins
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-11-21 10:03:03 -08:00
fc6da9b3db
Merge pull request #203 from ncdc/log-server-version
...
Log server version at startup
2017-11-15 12:13:59 -08:00
194d21c6a3
Update default TTL to 30 days
...
Signed-off-by: Nolan Brubaker <nolan@heptio.com>
2017-11-14 17:03:30 -05:00
c57a9b94b9
Log server version at startup
...
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2017-11-14 16:58:14 -05:00
b184c0b348
Seek to 0 before uploading restore results
...
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2017-11-14 14:39:42 -05:00
4c481f4d23
Make logSource more concise
...
Strip off leading .../github.com/heptio/ark/
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2017-11-14 14:21:19 -05:00
af189fd5f4
Add error & logSource hooks to backup & restore loggers
...
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2017-11-14 14:19:52 -05:00
34a6f492e5
Log restore start/end
...
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2017-11-14 14:19:52 -05:00
38aa43885b
Log when starting a backup
...
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2017-11-14 14:19:52 -05:00
8ba5a29679
add a logger that can be used within plugins to emit logs into Ark server
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-11-14 10:38:26 -08:00
24ce316788
switch built-in cloud providers to run as plugins
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-11-14 09:47:36 -08:00
3975187d57
add .proto files, generated code, and gen script for object/block stores
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-11-13 15:43:14 -08:00
cb49c62aaf
implement stream reader for gRPC byte streams
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-11-13 15:43:13 -08:00
21e2019540
rename Block/ObjectStoreAdapter -> Block/ObjectStore
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-11-08 16:58:47 -08:00
737b6d932a
Add more group-resource level logging
...
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2017-11-08 12:08:57 -05:00
5e4fc8f84a
Merge pull request #182 from ncdc/include-namespace
...
Fix missing namespace resource when backing up namespaces with auto cluster resources
2017-11-03 13:56:11 -07:00
efa4e57d1e
More namespace special casing
...
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2017-11-03 16:34:07 -04:00
9471f9da3c
Fix missing namespace metadata after restore.
...
If backing up specific namespaces with "auto" cluster resources, the
actual namespace objects themselves were not being included in the
backup. Restore would create them but any labels or metadata would be
lost.
Instead handle the special case of namespace as a cluster level resource
we may still need, even if excluding most cluster level resources.
Signed-off-by: Devan Goodwin <dgoodwin@redhat.com>
2017-11-03 12:12:03 -03:00
b2d80471ac
Move restore warnings/errors to object storage
...
If you have a large number of warnings and/or errors, the restore
object's size can exceed the maximum allowed by etcd. Move them to
object storage, and add a new describe command to fetch and display them
on the fly.
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2017-11-02 15:53:59 -04:00
55c038afa0
fix restore namespace remapping bug
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-11-02 10:39:54 -07:00
7f959f0184
Merge pull request #174 from skriss/obj-store-reader
...
Switch PutObject to take an io.Reader rather than an io.ReadSeeker
2017-11-02 12:57:09 -04:00
c49d11f17a
Merge pull request #171 from nrb/168-fix
...
Don't delete 'headless' services
2017-11-01 16:30:32 -07:00
e7703d88ec
switch UploadBackup, UploadRestoreLog, PutObject to take io.Reader
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-11-01 15:48:13 -07:00
d87e8ee16e
Don't delete 'headless' services
...
Deleting the clusterIP field when the service should be headless will
cause it to be assigned a new IP on restore; instead it should retain
the headless state after restoration.
Fixes #168
Signed-off-by: Nolan Brubaker <nolan@heptio.com>
2017-11-01 14:47:08 -07:00
4f59b19cdc
Merge pull request #169 from nrb/160-fix
...
Use label selector in backup, restore, schedule lists
2017-10-31 16:38:07 -07:00
36a40a0cd3
Use provided options when getting lists
...
Signed-off-by: Nolan Brubaker <nolan@heptio.com>
2017-10-31 16:13:28 -07:00
073795715e
Merge pull request #154 from ncdc/backup-download-remove-file-on-error
...
backup download: remove file if there's an error
2017-10-30 18:49:57 -07:00
0fab2e5e89
Stop passing encoder/decoder to NewHumanReadablePrinter
...
We only need them if we've got unstructured/unknown data and we want to
convert it to typed objects.
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2017-10-30 14:07:58 -04:00
c1bc52eb65
Update gofmt
...
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2017-10-30 13:31:03 -04:00
aa253bf016
Use generated deep copy functions
...
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2017-10-30 13:31:03 -04:00
293674c40e
React to upstream spdy executor changes
...
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2017-10-30 13:31:03 -04:00
322cbc19e6
dynamicResourceClient: fix resourceClient type
...
dynamicClient.Resource() now returns an interface
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2017-10-30 13:31:03 -04:00
15fe87aea3
Fix clientset imports
...
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2017-10-30 13:31:03 -04:00
4957dfce61
Regenerate code
...
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2017-10-30 13:31:03 -04:00
843345f728
Prep types for 1.8 code-generator
...
Fix genclient tags
Add deepcopy-gen interfaces to top level types
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2017-10-30 13:30:27 -04:00
43449885a1
React to move to k8s.io/api
...
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2017-10-30 13:30:27 -04:00
1ac0303283
Fix #137 . Remove clock from volume_snapshot_action.
...
Signed-off-by: Akhil Thampy <akhilthampy@yahoo.com>
2017-10-26 13:53:52 -05:00
8a53cef7e6
backup download: remove file if there's an error
...
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2017-10-26 11:09:35 -04:00
e38b6d0f4c
Support cross-compiling for clients
...
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2017-10-24 12:26:26 -04:00
a7cc58730e
add --include-cluster-resources flag to restores (optional, default true)
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-10-23 10:51:07 -07:00
e460199536
fix restore log statement
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-10-23 10:50:55 -07:00
28e7f0851a
Add verb-noun command aliases
...
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2017-10-20 13:26:34 -04:00
901f8e1302
Add pod exec backup hooks
...
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2017-10-20 10:20:59 -04:00
ed0194c09b
Restructure backups for resource prioritization.
...
Previously the directory structure separated resources depending on
whether or not they were cluster or namespace scoped. All cluster
resources were restored first, then all namespace resources. Priority
did not apply across both and you could not order any namespace
resources before any cluster resources.
This restructure sorts firstly on resource type.
resources/serviceaccounts/namespaces/ns1.json
resources/nodes/cluster/node1.json
This will break old backups as the format is no longer consistent as
announced on the Google group.
Signed-off-by: Devan Goodwin <dgoodwin@redhat.com>
2017-10-17 08:49:05 -03:00
4085e473e1
additional deployment/networkpolicy cases for TestBackupResource
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-10-11 15:05:12 -07:00
07b33ace57
add --include-cluster-resources flag to backup create cmd
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-10-11 15:05:12 -07:00
9438a8670a
when backing up PVCs, also back up claimed PVs
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-10-11 09:54:57 -07:00
fa427ebcfd
switch to logrus for per-backup/restore logs
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-10-10 14:11:21 -07:00
7c0d9dcfce
add missing licenses
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-10-10 13:14:03 -07:00
e7c62b5f38
add file/line numbers to logs
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-10-10 13:14:03 -07:00
9f3ce8ab1a
add log-level flag to server command
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-10-10 13:14:03 -07:00
82655b8359
tweak volume snapshot action logging messages
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-10-10 11:10:50 -07:00
d27b16370c
Merge pull request #102 from ashish-amarnath/support-multi-az-pvs
...
Preserve AWS & GCP volume AZ when snapshotting and restoring PVs.
2017-10-10 10:08:57 -07:00
6dc81b42f5
Merge branch 'master' into support-multi-az-pvs
...
Signed-off-by: Ashish Amarnath <ashish.amarnath@gmail.com>
2017-10-06 23:24:41 -07:00
9fc9dbb413
Preserve PV's AZ info when snapshotting and restoring PVs.
...
- Read PV's AZ info from fault-domain label of the PV object for snapshotting.
- Store PV's AZ info in the VolumeInfo.
- Add tests for reading the label from the PV object.
- Remove availability zone validation in AWS and GCP BlockStorageAdaptor.
- Add volumeAZ as a parameter to methods in the BlockStorageAdapter interface.
- Get AZ from VolumeInfo when restoring PV snapshot.
- Remove references to PV availability zone in docs.
Signed-off-by: Ashish Amarnath <ashish.amarnath@gmail.com>
2017-10-06 23:24:41 -07:00
1fd1e8deaa
Excluding nodes and endpoints from restoration
...
- Introduced a blacklist of resources that are non-restorable. The
goal being that the backup can still include these resources for
logging/auditing purposes but they are explicitly added to
ExcludedResources in the RestorController's "defaulting" logic
to ensure that if someone were to explicitly ask for nodes
that they would be expressly denied.
Signed-off-by: Justin Nauman <justin.r.nauman@gmail.com>
2017-10-05 22:37:29 -05:00
9ac71b03cc
add logrus hook to log error file/line/function from stack traces
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-10-05 08:49:33 -07:00
914165095a
switch logging to logrus and errors to pkg/errors
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-10-05 08:49:33 -07:00
a5d4d138b2
Merge pull request #96 from ncdc/fix-config-watch
...
Make config change detection more robust
2017-09-19 09:57:52 -07:00
024f65579c
Merge pull request #69 from jrnt30/user-agent
...
Adding in customized user-agent
2017-09-19 09:51:57 -07:00
97f8f2426f
Addressing PR feedback and adding tests
...
Signed-off-by: Justin Nauman <justin.r.nauman@gmail.com>
2017-09-18 17:42:16 -05:00
84fa48d57a
Make config change detection more robust
...
Fix 2 issues with config change detection:
- Objects received via Get() don't have kind and apiVersion set, while
those from Watch() do, leading to false positives.
- Compare the unmodified config (prior to applying defaults) to the
updated one from Watch().
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2017-09-14 17:12:32 -04:00
609bf80f9e
Merge pull request #90 from skriss/change-azure-apitimeout
...
change Azure API timeout default to 2m
2017-09-14 09:38:09 -04:00
5424eba417
change Azure API timeout default to 2m
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-09-13 16:20:12 -07:00
acb3babb87
when restoring azureDisk from snapshot, update the diskURI with the new diskName
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-09-13 16:10:30 -07:00
d76ed7b49e
Adding in customized user-agent
...
Signed-off-by: Justin Nauman <justin.r.nauman@gmail.com>
2017-09-13 16:35:45 -05:00
5405067a2e
Merge pull request #88 from ncdc/fix-restore-logs-when-backup-name-has-dash
...
Support restore logs when backup name has -
2017-09-13 14:14:55 -07:00
f31af75dc3
Merge pull request #82 from ncdc/tweak-version-output
...
Tweak version display
2017-09-13 14:14:44 -07:00
136b8f200d
Support restore logs when backup name has -
...
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2017-09-13 16:04:46 -04:00
81fda27440
Tweak version display
...
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2017-09-13 10:16:16 -04:00
b43b65b7bf
Revert to prior name format for tarballs
...
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2017-09-13 09:51:40 -04:00
7b892b63ed
make update
...
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2017-09-13 09:51:40 -04:00
e8a88747eb
Use pointers when printing schema.GroupResource
...
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2017-09-13 09:51:40 -04:00
9deb4d4cae
backup download: switch to --output as full path
...
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2017-09-13 09:51:40 -04:00
28fac1b156
Don't unzip backup data when streaming
...
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2017-09-13 09:51:40 -04:00
273b563c86
Add per-restore logs
...
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2017-09-13 09:51:40 -04:00
d0b7880881
Merge pull request #78 from skriss/restore-resource-inc-exc
...
Add --include-resources, --exclude-resources for restore operations
2017-09-12 15:53:21 -04:00
4dfce17de5
add resource includes/excludes for Restores
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-09-12 12:43:23 -07:00
98d4660d27
Adding in support for backup download
...
- Adding in support for a new `download` subcommand of backup
- Adjusted signing to allow for multiple types
- Adding in git sha version during build more granular version
debugging
Signed-off-by: Justin Nauman <justin.r.nauman@gmail.com>
2017-09-11 20:17:33 -05:00
907ae6c5b0
move groupresource resolution into discovery helper
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-09-11 14:41:50 -07:00
ab99fcc2e2
Update backup and log file name formats
...
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2017-09-11 15:40:49 -04:00
3bb02ac949
Use ioutil.Discard in unit test
...
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2017-09-11 15:40:49 -04:00
71fadf381e
Clarify Backup() param names
...
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2017-09-11 15:40:49 -04:00
a583f22ea9
Add ActionContext so backup actions can log
...
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2017-09-11 15:40:48 -04:00
50a95d052e
Delete all objects in backup dir
...
Delete all objects in backup "dir" when deleting a backup, instead of
hard-coding individual file names/types. This way, we'll be able to
delete log files and anything else we add without having to update our
deletion code.
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2017-09-11 15:40:48 -04:00
193000e487
Add ListObjects (for backup dir deletions)
...
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2017-09-11 15:40:48 -04:00
5c9ffae88f
Renerate files
...
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2017-09-11 15:40:48 -04:00
03dde45c09
Add 'ark backup logs' command for retrieval
...
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2017-09-11 15:40:48 -04:00
9848a7a55b
Save backup log file to object storage
...
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2017-09-11 15:40:48 -04:00
b50a78cf7b
Work around glog flag.Parse issue
...
Due to a change in glog, if you don't call flag.Parse, every log line
prints out 'ERROR: logging before flag.Parse'. This works around that
change. Ultimately we need to switch to a different logging library.
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2017-09-11 15:40:47 -04:00
860eb7e845
Match Azure sdk changes
...
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2017-09-11 15:40:47 -04:00
7648155c9a
delete restores when GC'ing backups & don't orphan cloud resources
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-09-06 10:34:08 -07:00
305e7fcea0
Merge pull request #59 from jrnt30/explicit-includes-excludes-restore
...
Allows explicit include/exclude of namespaces on restores
2017-09-06 11:21:24 -04:00
56c680f991
Merge pull request #62 from jrnt30/default-home-resolution
...
Allow default kubeconfig resolution
2017-09-06 11:16:10 -04:00
2457d8f116
Removing instead of deprecating Namespace attr
...
- Per discussion, there is no reason to deal
with the complexity of backwards compatibility
with the Namespace attribute on the Restore
domain.
- Also noticed there was an error on the
validation of the BackupController where
the message would actually just be the index
of the error and not the contents of the message
itself.
Signed-off-by: Justin Nauman <justin.r.nauman@gmail.com>
2017-09-05 17:18:10 -05:00
b50a046370
Additional Validation on Include/Exclude
...
- Adding in additional test to ensure *Namespaces attributes
don't directly conflict logically with one another
- Additional PR changes around naming/typos
Signed-off-by: Justin Nauman <justin.r.nauman@gmail.com>
2017-09-05 17:17:46 -05:00
af2a792a9a
Allows explicit include/exclude of namespaces on restores
...
- Introduces similar Include/Exclude declaration on the Restore
resource and cli flags
- Kept support for legacy Namespaces attribute until it could be
deprecating. Defining both IncludeNamespaces and Namespaces results
in a validation error and the Restore will not be processed (shouldn't
be able to occur)
Signed-off-by: Justin Nauman <justin.r.nauman@gmail.com>
2017-09-05 17:17:41 -05:00
ba299f880f
in GetAllBackups don't error if single backup is unreadable
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-09-05 09:56:37 -07:00
43c74e33c2
Merge pull request #57 from skriss/restore_missing_backup_fix
...
during restore, try to get backup directly from obj storage if not in cache/API
2017-09-05 12:47:13 -04:00
f07a70c604
during restore, try to get backup directly from obj storage if not in cache/API
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-09-05 09:25:39 -07:00
c0e94a4c1a
Allow default kubeconfig resolution
...
- Changed the default kubeconfig loading to utilize
the client-go's loader strategy. This allows users
to use the Ark client without having to explicitly
define a KUBECONFIG env var or argument.
This more closely resemebles how Kubectl works and users
are probably more used to while preserving the
current rules.
Signed-off-by: Justin Nauman <justin.r.nauman@gmail.com>
2017-08-28 07:56:57 -05:00
f017a23d41
AWS Paging Support
...
- Adding in paging support for the S3 and Snapshot
AWS integration.
As a testing note, you can add in a a MaxKeys to the S3
request as an easy way to ensure that paging is working
properly without having to creation over 1k backups.
Signed-off-by: Justin Nauman <justin.r.nauman@gmail.com>
2017-08-28 05:49:39 -05:00
768aed4ddd
make --snapshot-volumes, --restore-volumes optional with sensible default behavior based on PVProvider existence
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-08-23 09:47:52 -07:00
8d5c8ffcbb
get rid of StorageAdapter structs and move validation into block/object
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-08-23 09:46:15 -07:00
ebc06fd632
make PVProvider optional in server config; disallow snap/restore PVs when not provided
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-08-23 09:46:15 -07:00
3ea9899140
Adding in defaults for backup creation and restore
...
Signed-off-by: Justin Nauman <justin.r.nauman@gmail.com>
2017-08-21 09:47:11 -05:00
c088470ba5
Merge pull request #35 from skriss/server_config_validation
...
validate cloud-provider config at startup & make PVProvider optional
2017-08-14 16:03:15 -04:00
726bbbb393
add zone/location validation to cloudprovider constructors
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-08-14 12:56:12 -07:00
a865cb87ee
Merge pull request #37 from skriss/iops_fix
...
only save/use iops for io1 volumes
2017-08-14 15:24:30 -04:00
c2271776cc
remove unused params from GCP objectStorageAdapter
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-08-14 10:56:05 -07:00
aff57e0571
switch to int64 for iops val
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-08-14 09:42:43 -07:00
df320d7bf3
adds support for S3 SSE with KMS
...
Signed-off-by: Mathias Merscher <Mathias.Merscher@dg-i.net>
2017-08-14 18:18:19 +02:00
e966eb9ab0
Merge pull request #38 from skriss/cluster-only-restore-err
...
check for namespaces/ dir before reading it
2017-08-14 09:46:50 -04:00
49809b0a59
check for namespaces/ dir before reading it
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-08-11 14:05:06 -07:00
c41c958777
only save/use iops for io1 volumes
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-08-11 13:33:01 -07:00
680e17815b
Schedules: treat 1st cron field as minutes
...
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2017-08-10 12:44:00 -04:00
e0d2a2d4d0
Remove k8s.io/apiextensions-apiserver from vendor
...
Remove k8s.io/apiextensions-apiserver since we're no longer registering
CRDs in code.
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2017-08-07 12:38:07 -04:00
9dcff62f1a
Pull in azure, gcp, oidc client-go auth provider plugins
...
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2017-08-07 11:24:22 -04:00
2c9d22771c
Use NoError instead of NoErrorf
...
Prepping to roll back to the same version of testify that client-go
uses, and that does not have NoErrorf.
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2017-08-07 11:20:38 -04:00
d5953b3772
Simplify check for owner references
...
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2017-08-02 17:10:39 -04:00
2fe501f527
Initial commit
...
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2017-08-02 13:27:17 -04:00
Steve Kriss
Steve Kriss
Nolan Brubaker
Carlisia
Alex Lemaresquier
Nolan Brubaker
Mike Arpaia
Andy Goldstein
Nikhita Raghunath
Ashish Amarnath
Yassine TIJANI
Calle Pettersson
Shubham Minglani
Sanket Patel
Josh Hull
Blake
Devan Goodwin
Akhil Thampy
Justin Nauman
Justin Nauman
Mathias Merscher