Some aws implementations, for example the quobyte object storage, do not
support the v4 signing algorithm, but only v1.
This makes it possible to configure the signatureVersion.
The algorithm implementation was ported from d6c1be296e/botocore/auth.py (L860-L862)
which is used by the aws CLI client.
This fixes https://github.com/heptio/ark/issues/811.
Signed-off-by: Bastian Hofmann <bashofmann@gmail.com>
This allows the Ark server to use one URL for the majority of
communications with S3 (or compatible) object storage, and a different
URL base for pre-signed URLs (for streaming logs, etc. to clients).
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
When a backup is deleted, the delete method uses ListObjects to get a list of
files it needs to delete in s3. Different s3 implementations may return
the object lists in different, even non-deterministic orders, which can
result in the deletion not working because ark tries to delete a non empty folder
before it tries to delete the files in the folder.
Signed-off-by: Bastian Hofmann <bashofmann@gmail.com>
Signed-off-by: Lukasz Jakimczuk <ljakimczuk@gmail.com>
Moving check for environment variable outside the loop
Signed-off-by: Lukasz Jakimczuk <ljakimczuk@gmail.com>
Insert a note about AWS_CLUSTER_NAME in the aws-config doc
Signed-off-by: Lukasz Jakimczuk <ljakimczuk@gmail.com>
Improving implementation and documentation
Signed-off-by: Lukasz Jakimczuk <ljakimczuk@gmail.com>
Changing instructions, adding unit test for getTagsForCluster and removing duplicated Lookup
Signed-off-by: Lukasz Jakimczuk <ljakimczuk@gmail.com>
Commit after update
Signed-off-by: Lukasz Jakimczuk <ljakimczuk@gmail.com>
Correcting bad formatting in aws-config.md
Signed-off-by: Lukasz Jakimczuk <ljakimczuk@gmail.com>
Refactor plugin management:
- support multiple plugins per executable
- support restarting a plugin process in the event it terminates
- simplify plugin lifecycle management by using separate managers for
each scope (server vs backup vs restore)
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
The main Ark code was hard-coding specific support for AWS, GCE, and
Azure volume snapshots and restores, and anything else was considered
unsupported.
Add GetVolumeID and SetVolumeID to the BlockStore interface, to allow
block store plugins to handle volume snapshots and restores.
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
- Read PV's AZ info from fault-domain label of the PV object for snapshotting.
- Store PV's AZ info in the VolumeInfo.
- Add tests for reading the label from the PV object.
- Remove availability zone validation in AWS and GCP BlockStorageAdaptor.
- Add volumeAZ as a parameter to methods in the BlockStorageAdapter interface.
- Get AZ from VolumeInfo when restoring PV snapshot.
- Remove references to PV availability zone in docs.
Signed-off-by: Ashish Amarnath <ashish.amarnath@gmail.com>
- Adding in paging support for the S3 and Snapshot
AWS integration.
As a testing note, you can add in a a MaxKeys to the S3
request as an easy way to ensure that paging is working
properly without having to creation over 1k backups.
Signed-off-by: Justin Nauman <justin.r.nauman@gmail.com>
Pranav Gaikwad
Carlisia
Andy Goldstein
Steve Kriss
Nolan Brubaker
Steve Kriss
Matt Stump
KubeKween
tsturzl
Nolan Brubaker
Bastian Hofmann
Lukasz Jakimczuk
Steve Kriss
Dave Parfitt
Mike Arpaia
ashish-amarnath
Justin Nauman