Andy Goldstein
1e581f1ead
BackupController: do as much as possible
...
When running a backup, try to do as much as possible, collecting errors
along the way, and return an aggregate at the end. This way, if a backup
fails for most reasons, we'll be able to upload the backup log file to
object storage, which wasn't happening before.
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2017-12-20 14:49:27 -05:00
Steve Kriss
56c9d68137
switch to logrus.FieldLogger and arktest.NewLogger() everywhere
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-12-13 09:38:00 -08:00
Steve Kriss
4aea9b9a2c
switch from Update() to Patch()
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-12-12 15:23:45 -08:00
Andy Goldstein
992940c55c
Merge pull request #227 from skriss/plugin-logger-fix
...
Bug fixes: obj/block store plugin logging and remapped namespaces issue
2017-11-30 12:17:56 -05:00
Steve Kriss
121b7153a7
when restoring, don't restore the Namespaces cluster-scoped resource independently
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-11-30 09:07:38 -08:00
Andy Goldstein
bd8f433188
Merge pull request #229 from skriss/plugin-docs
...
add godoc for ResourceSelector
2017-11-30 11:25:53 -05:00
Steve Kriss
94ac3b3529
godoc for ResourceSelector
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-11-30 08:20:27 -08:00
Steve Kriss
8a0ac2117b
give each plugin its own logrusAdapter
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-11-29 14:36:21 -08:00
Steve Kriss
cbcd15d603
add/update documentation for plugins
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-11-29 14:24:23 -08:00
Steve Kriss
b66efd9416
rename NewPluginLogger to NewLogger
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-11-29 14:01:31 -08:00
Steve Kriss
038fa39451
remove ListSnapshots and GetAllSnapshots
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-11-29 13:59:29 -08:00
Steve Kriss
995050390c
add plugin add/remove commands
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-11-29 12:22:30 -08:00
Steve Kriss
0b6e78145c
Merge pull request #223 from ncdc/simplify-plugin-serve
...
Simplify serving plugins for external developers
2017-11-29 11:09:22 -08:00
Steve Kriss
5b8562e73c
Merge pull request #222 from ncdc/fix-log-location-hook
...
Fix log location hook prefix stripping
2017-11-29 11:07:43 -08:00
Andy Goldstein
dc484d1566
Set logger for cloudprovider plugin
...
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2017-11-29 13:54:20 -05:00
Andy Goldstein
1f7e9b65e8
Simply serving plugins for external developers
...
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2017-11-29 13:42:53 -05:00
Andy Goldstein
c700455272
Support custom volume snapshots & restores
...
The main Ark code was hard-coding specific support for AWS, GCE, and
Azure volume snapshots and restores, and anything else was considered
unsupported.
Add GetVolumeID and SetVolumeID to the BlockStore interface, to allow
block store plugins to handle volume snapshots and restores.
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2017-11-29 13:19:40 -05:00
Andy Goldstein
99f67db39d
Fix log location hook prefix stripping
...
The log location hook was matching github.com/heptio/ark and stripping
off that + 1 more char. This meant that
github.com/heptio/ark-plugin-example/foo.go was being listed as
plugin-example/foo.go instead of
github.com/heptio/ark-plugin-example/foo.go.
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2017-11-29 12:56:01 -05:00
Andy Goldstein
526b604237
Add boolptr helper package
...
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2017-11-29 12:25:10 -05:00
Andy Goldstein
f0b35cc45a
Merge pull request #213 from skriss/restore-action-plugins
...
Convert restorers to plugins
2017-11-28 14:50:41 -05:00
Steve Kriss
179b95c81d
convert restorers to plugins
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-11-28 10:58:12 -08:00
Andy Goldstein
c2dc41efd8
Add backup & schedule describers
...
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2017-11-27 15:22:20 -05:00
Steve Kriss
0f2d1ab82b
migrate backup actions to plugins
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-11-21 10:03:03 -08:00
Steve Kriss
fc6da9b3db
Merge pull request #203 from ncdc/log-server-version
...
Log server version at startup
2017-11-15 12:13:59 -08:00
Nolan Brubaker
194d21c6a3
Update default TTL to 30 days
...
Signed-off-by: Nolan Brubaker <nolan@heptio.com>
2017-11-14 17:03:30 -05:00
Andy Goldstein
c57a9b94b9
Log server version at startup
...
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2017-11-14 16:58:14 -05:00
Andy Goldstein
b184c0b348
Seek to 0 before uploading restore results
...
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2017-11-14 14:39:42 -05:00
Andy Goldstein
4c481f4d23
Make logSource more concise
...
Strip off leading .../github.com/heptio/ark/
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2017-11-14 14:21:19 -05:00
Andy Goldstein
af189fd5f4
Add error & logSource hooks to backup & restore loggers
...
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2017-11-14 14:19:52 -05:00
Andy Goldstein
34a6f492e5
Log restore start/end
...
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2017-11-14 14:19:52 -05:00
Andy Goldstein
38aa43885b
Log when starting a backup
...
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2017-11-14 14:19:52 -05:00
Steve Kriss
8ba5a29679
add a logger that can be used within plugins to emit logs into Ark server
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-11-14 10:38:26 -08:00
Steve Kriss
24ce316788
switch built-in cloud providers to run as plugins
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-11-14 09:47:36 -08:00
Steve Kriss
3975187d57
add .proto files, generated code, and gen script for object/block stores
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-11-13 15:43:14 -08:00
Steve Kriss
cb49c62aaf
implement stream reader for gRPC byte streams
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-11-13 15:43:13 -08:00
Steve Kriss
21e2019540
rename Block/ObjectStoreAdapter -> Block/ObjectStore
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-11-08 16:58:47 -08:00
Andy Goldstein
737b6d932a
Add more group-resource level logging
...
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2017-11-08 12:08:57 -05:00
Steve Kriss
5e4fc8f84a
Merge pull request #182 from ncdc/include-namespace
...
Fix missing namespace resource when backing up namespaces with auto cluster resources
2017-11-03 13:56:11 -07:00
Andy Goldstein
efa4e57d1e
More namespace special casing
...
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2017-11-03 16:34:07 -04:00
Devan Goodwin
9471f9da3c
Fix missing namespace metadata after restore.
...
If backing up specific namespaces with "auto" cluster resources, the
actual namespace objects themselves were not being included in the
backup. Restore would create them but any labels or metadata would be
lost.
Instead handle the special case of namespace as a cluster level resource
we may still need, even if excluding most cluster level resources.
Signed-off-by: Devan Goodwin <dgoodwin@redhat.com>
2017-11-03 12:12:03 -03:00
Andy Goldstein
b2d80471ac
Move restore warnings/errors to object storage
...
If you have a large number of warnings and/or errors, the restore
object's size can exceed the maximum allowed by etcd. Move them to
object storage, and add a new describe command to fetch and display them
on the fly.
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2017-11-02 15:53:59 -04:00
Steve Kriss
55c038afa0
fix restore namespace remapping bug
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-11-02 10:39:54 -07:00
Andy Goldstein
7f959f0184
Merge pull request #174 from skriss/obj-store-reader
...
Switch PutObject to take an io.Reader rather than an io.ReadSeeker
2017-11-02 12:57:09 -04:00
Steve Kriss
c49d11f17a
Merge pull request #171 from nrb/168-fix
...
Don't delete 'headless' services
2017-11-01 16:30:32 -07:00
Steve Kriss
e7703d88ec
switch UploadBackup, UploadRestoreLog, PutObject to take io.Reader
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-11-01 15:48:13 -07:00
Nolan Brubaker
d87e8ee16e
Don't delete 'headless' services
...
Deleting the clusterIP field when the service should be headless will
cause it to be assigned a new IP on restore; instead it should retain
the headless state after restoration.
Fixes #168
Signed-off-by: Nolan Brubaker <nolan@heptio.com>
2017-11-01 14:47:08 -07:00
Steve Kriss
4f59b19cdc
Merge pull request #169 from nrb/160-fix
...
Use label selector in backup, restore, schedule lists
2017-10-31 16:38:07 -07:00
Nolan Brubaker
36a40a0cd3
Use provided options when getting lists
...
Signed-off-by: Nolan Brubaker <nolan@heptio.com>
2017-10-31 16:13:28 -07:00
Steve Kriss
073795715e
Merge pull request #154 from ncdc/backup-download-remove-file-on-error
...
backup download: remove file if there's an error
2017-10-30 18:49:57 -07:00
Andy Goldstein
0fab2e5e89
Stop passing encoder/decoder to NewHumanReadablePrinter
...
We only need them if we've got unstructured/unknown data and we want to
convert it to typed objects.
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2017-10-30 14:07:58 -04:00