Influxdata
/
influxdb
mirror of https://github.com/influxdata/influxdb.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
influxdb/object_store/Cargo.toml

44 lines
1.7 KiB
TOML
Raw Normal View History

feat: Object storage in S3 and GCS
2020-05-29 15:06:06 +00:00
[package]
refactor: rename delorean_object_store --> object_store (#413)
2020-11-05 13:56:30 +00:00
name = "object_store"
feat: Object storage in S3 and GCS
2020-05-29 15:06:06 +00:00
version = "0.1.0"
authors = ["Paul Dix <paul@pauldix.net>"]
chore: use Rust edition 2021
2021-10-25 08:45:44 +00:00
edition = "2021"
feat: Object storage in S3 and GCS
2020-05-29 15:06:06 +00:00
chore: Clean up Cargo.tomls (#754) * fix: test_helpers crate should only be a dev-dep * fix: object_store no longer has a build script, so no longer needs a build dep * chore: Alphabetize all Cargo.tomls
2021-02-04 23:56:02 +00:00
[dependencies] # In alphabetical order
refactor: Extract a trait for ObjectStoreApi with associated path This is the promised cleanup. This structure gets rid of a lot of intermediate structures and encodes through associated types how the object stores and path types are related. The enums are still necessary to avoid having generics leak all over the place, but the object store variants and path variants should always match because they'll always come from the object store trait implementations that use the associated types.
2021-01-20 03:36:52 +00:00
async-trait = "0.1.42"
chore: Clean up Cargo.tomls (#754) * fix: test_helpers crate should only be a dev-dep * fix: object_store no longer has a build script, so no longer needs a build dep * chore: Alphabetize all Cargo.tomls
2021-02-04 23:56:02 +00:00
# Microsoft Azure Blob storage integration
feat: Update to latest Azure sdk to get delimiter support Needed these PRs: - https://github.com/Azure/azure-sdk-for-rust/pull/176 - https://github.com/Azure/azure-sdk-for-rust/pull/179 Also needed to enable the queue feature to get the azure_storage crate compiling; at the moment, the code is still being reorganized and the features aren't independent yet: https://github.com/Azure/azure-sdk-for-rust/issues/177
2021-02-18 19:59:06 +00:00
# In order to support tokio 1.0 and delimiters, needed to pull in unreleased azure sdk
chore: upgrade azure dependencies
2021-09-21 11:35:11 +00:00
azure_core = { git = "https://github.com/Azure/azure-sdk-for-rust.git", rev = "cb5aba657fe378009c38e979434b2bc93ac0a3d5", optional = true }
azure_storage = { git = "https://github.com/Azure/azure-sdk-for-rust.git", rev = "cb5aba657fe378009c38e979434b2bc93ac0a3d5", optional = true, default-features = false, features = ["table", "blob", "queue"] }
chore: update deps (tokio 1.0 and ecosystem) (#707) * chore: Update arrow + tokio deps * chore: Use bleeding edge azure * chore: Update aws + other deps * fix: fmt * fix: Switch to in-house version of routerify * fix: Upgrade to hyper 0.14 The hyper::error module is now private; hyper::Error is the public re-export * fix: Upgrade cloud storage to get tokio upgrade * fix: Upgrade open_telemetry * fix: Do not call `panic::set_hook` during another panic Doing so leads to a double panic which aborts the process. * fix: new h2 error who dis Co-authored-by: Carol (Nichols || Goulding) <carol.nichols@integer32.com> Co-authored-by: Jake Goulding <jake.goulding@integer32.com>
2021-01-29 21:11:55 +00:00
bytes = "1.0"
feat: Add list_with_delimiter to object store This adds a new function list_with_delimiter to the object store. This commit contains just the implementation for S3, leaving the others to be completed in follow on commits. This has a fixed delimiter to ensure a directory structure is created. This delimiter should be dependent on platform and which object store is used. For any of the cloud object stores or in memory, the delimiter should be /. For the future disk based implementation it should be dependendent on if you're running on Windows or Linux. I didn't use Stream for the return type because I found it difficult to work with and I don't think it actually added anything useful. The return ListResult struct has the next token and I prefer that the caller explicitly makes calls that go over the network so they're more aware of what's going on, where a Stream abstracts that away so it's hidden behind the scenes. We can easilsy add a Stream based version on top of this existing API if we want.
2021-01-03 17:26:20 +00:00
chrono = "0.4"
chore: Clean up Cargo.tomls (#754) * fix: test_helpers crate should only be a dev-dep * fix: object_store no longer has a build script, so no longer needs a build dep * chore: Alphabetize all Cargo.tomls
2021-02-04 23:56:02 +00:00
# Google Cloud Storage integration
chore(deps): bump cloud-storage from 0.10.2 to 0.10.3 Bumps [cloud-storage](https://github.com/ThouCheese/cloud-storage-rs) from 0.10.2 to 0.10.3. - [Release notes](https://github.com/ThouCheese/cloud-storage-rs/releases) - [Changelog](https://github.com/ThouCheese/cloud-storage-rs/blob/master/CHANGELOG.md) - [Commits](https://github.com/ThouCheese/cloud-storage-rs/commits) --- updated-dependencies: - dependency-name: cloud-storage dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com>
2021-10-04 01:43:09 +00:00
cloud-storage = {version = "0.10.3", optional = true}
feat: mutable buffer snapshotting (#1179) * feat: mutable buffer snapshotting * chore: review feedback
2021-04-13 12:14:54 +00:00
futures = "0.3"
chore: misc updates (#2886) * chore: misc updates * fix: don't downgrade nix * chore: remove orphaned comment Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-10-19 11:47:27 +00:00
indexmap = { version = "1.7", optional = true }
chore: Update itertools deps (#1750)
2021-06-17 17:56:44 +00:00
itertools = "0.10.1"
fix: Add logging when retrying AWS requests
2021-09-27 15:19:31 +00:00
observability_deps = { path = "../observability_deps" }
feat: Add object store path abstraction
2021-01-04 15:16:54 +00:00
percent-encoding = "2.1"
chore: Clean up Cargo.tomls (#754) * fix: test_helpers crate should only be a dev-dep * fix: object_store no longer has a build script, so no longer needs a build dep * chore: Alphabetize all Cargo.tomls
2021-02-04 23:56:02 +00:00
# rusoto crates are for Amazon S3 integration
fix: Update all the rusoto crates together
2021-08-16 23:59:26 +00:00
rusoto_core = { version = "0.47.0", optional = true}
build(deps): bump rusoto_credential from 0.46.0 to 0.47.0 Bumps [rusoto_credential](https://github.com/rusoto/rusoto) from 0.46.0 to 0.47.0. - [Release notes](https://github.com/rusoto/rusoto/releases) - [Changelog](https://github.com/rusoto/rusoto/blob/master/CHANGELOG.md) - [Commits](https://github.com/rusoto/rusoto/compare/mock-v0.46.0...rusoto-v0.47.0) --- updated-dependencies: - dependency-name: rusoto_credential dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com>
2021-08-16 16:35:09 +00:00
rusoto_credential = { version = "0.47.0", optional = true}
fix: Update all the rusoto crates together
2021-08-16 23:59:26 +00:00
rusoto_s3 = { version = "0.47.0", optional = true}
feat: disable Snafu futures feature (#2573) Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-09-19 11:36:24 +00:00
snafu = "0.6.10"
fix: Add missing features to object_store crate to enable running tests separately
2021-10-25 19:58:25 +00:00
tokio = { version = "1.11", features = ["macros", "fs", "io-util", "rt-multi-thread", "time"] }
feat: Add a local filesystem implementation of the object store
2020-06-27 00:54:35 +00:00
# Filesystem integration
chore(deps): bump tokio-util from 0.6.8 to 0.6.9 Bumps [tokio-util](https://github.com/tokio-rs/tokio) from 0.6.8 to 0.6.9. - [Release notes](https://github.com/tokio-rs/tokio/releases) - [Commits](https://github.com/tokio-rs/tokio/compare/tokio-util-0.6.8...tokio-util-0.6.9) --- updated-dependencies: - dependency-name: tokio-util dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com>
2021-11-01 01:28:41 +00:00
tokio-util = { version = "0.6.9", features = [ "codec", "io" ] }
chore: `object_store` needs `reqwest` only for azure
2021-09-22 09:16:00 +00:00
reqwest = { version = "0.11", optional = true }
chore: Clean up Cargo.tomls (#754) * fix: test_helpers crate should only be a dev-dep * fix: object_store no longer has a build script, so no longer needs a build dep * chore: Alphabetize all Cargo.tomls
2021-02-04 23:56:02 +00:00
# Filesystem integration
fix: Make file behave the same as other object stores with paths
2021-01-14 21:05:10 +00:00
walkdir = "2"
feat: Allow to put streams of unknown length to objectstore Addresses the API aspect of #818 Adds a utility module that helps computing the length of a stream while buffering it for later replay (in-memory or spilling it in a temporary file).
2021-02-17 16:48:31 +00:00
tempfile = "3.1.0"
feat: Add a local filesystem implementation of the object store
2020-06-27 00:54:35 +00:00
refactor: Make aws/azure/gcs optional features and stop compiling 100 dependencies during dev (#1933) * feat: make aws, gcp, azure dependencies optional * fix: only run object store tests if the features are enabled * fix: clean up testing * fix: rename step * fix: add to list of jobs * fix: remove test with object store * fix: review comments
2021-07-09 11:38:30 +00:00
[features]
chore: `object_store` needs `reqwest` only for azure
2021-09-22 09:16:00 +00:00
azure = ["azure_core", "azure_storage", "indexmap", "reqwest"]
refactor: Make aws/azure/gcs optional features and stop compiling 100 dependencies during dev (#1933) * feat: make aws, gcp, azure dependencies optional * fix: only run object store tests if the features are enabled * fix: clean up testing * fix: rename step * fix: add to list of jobs * fix: remove test with object store * fix: review comments
2021-07-09 11:38:30 +00:00
gcp = ["cloud-storage"]
aws = ["rusoto_core", "rusoto_credential", "rusoto_s3"]
chore: Clean up Cargo.tomls (#754) * fix: test_helpers crate should only be a dev-dep * fix: object_store no longer has a build script, so no longer needs a build dep * chore: Alphabetize all Cargo.tomls
2021-02-04 23:56:02 +00:00
[dev-dependencies] # In alphabetical order
test: Only run GCS and S3 tests if the env vars are set I don't really like this because the tests will silently not be compiled if you haven't set the environment variables, so you'd only notice you weren't running the tests if you looked for those tests' output lines and saw they weren't there. Ideally, I'd like to print a warning, but this isn't possible because: - Anything printed in tests doesn't show up by default - Cargo's build scripts can't tell whether you're building as a dependency or building for that crate's tests, so the warning would show up even if you just depended on delorean_object_store (https://github.com/rust-lang/cargo/issues/2549)
2020-06-10 21:14:48 +00:00
dotenv = "0.15.0"
chore: Clean up Cargo.tomls (#754) * fix: test_helpers crate should only be a dev-dep * fix: object_store no longer has a build script, so no longer needs a build dep * chore: Alphabetize all Cargo.tomls
2021-02-04 23:56:02 +00:00
tempfile = "3.1.0"
feat: Allow to put streams of unknown length to objectstore Addresses the API aspect of #818 Adds a utility module that helps computing the length of a stream while buffering it for later replay (in-memory or spilling it in a temporary file).
2021-02-17 16:48:31 +00:00
futures-test = "0.3.12"