10912c2b97
feat: Write owner info in the database's object store directory
...
Use this as an extra check that the database thinks the current server
is its owner when a server tries to initialize and own a database.
If the owner info doesn't match the current server, which could happen
if a server's config was updated but the database's owner info wasn't,
put the database in an error state that requires operator intervention.
2021-10-18 21:13:20 -04:00
c95d6a3a4a
docs: Clarify purpose of new return values
2021-10-18 14:20:18 -04:00
2bb5b4aba4
feat: Read server config file to discover database names and locations on startup
2021-10-18 14:20:18 -04:00
d5ab29711e
fix: Serialize relative db object store paths to the server config
...
So that they can be deserialized, without parsing, to create a new
iox object store from the location listed in the server config.
Notably, the locations serialized don't start with the object storage's
prefix like "s3:" or "file:". The location is the same object storage as
the server configuration that was just read from object storage. Having
the server config on one type of object storage and the database files
on another type is not supported.
2021-10-18 08:37:36 -04:00
c64403d8b5
docs: Clarify warning about parsing object storage paths
2021-10-15 11:14:25 -04:00
afd6e826e5
feat: Write out server config files listing database name and locations
2021-10-15 09:46:20 -04:00
188fb2282c
feat: Add logic to IoxObjectStore to get and put the server config file
2021-10-15 09:46:20 -04:00
064b7aa613
fix: Remove the database_name method from IoxObjectStore
2021-10-08 15:25:10 -04:00
63d74be490
refactor: make `ChunkId` a UUID
2021-10-07 10:23:27 +02:00
92583aee82
fix: Remove streaming API since we're not streaming anyway
2021-09-29 08:19:32 -04:00
d05528bcfd
refactor: Use s3_request for put requests
...
Which meant we also needed to change the byte stream to be a closure
that can generate a byte stream
2021-09-29 08:19:32 -04:00
cef5aeee52
refactor: introduce `ChunkId` type
2021-09-20 13:10:41 +02:00
51a40b31bf
feat: Add a --detailed option to the database list CLI
...
That will list both active and deleted databases with their generations.
Closes #2462 .
2021-09-17 15:27:23 -04:00
423a976744
refactor: Use iterator adaptors rather than for loops
2021-09-17 15:27:23 -04:00
44a89cdf75
refactor: Change DeletedDatabase to DetailedDatabase
...
So this info can be reused for active databases in detailed database
lists.
2021-09-17 15:27:22 -04:00
7b6d8f9327
feat: Add an API for restoring a database that was marked deleted
2021-09-15 16:59:37 -04:00
368f0369ee
chore: Rust 1.55
2021-09-10 12:36:49 +02:00
b67610d9b9
chore(deps): bump tokio from 1.10.1 to 1.11.0
...
Bumps [tokio](https://github.com/tokio-rs/tokio ) from 1.10.1 to 1.11.0.
- [Release notes](https://github.com/tokio-rs/tokio/releases )
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.10.1...tokio-1.11.0 )
---
updated-dependencies:
- dependency-name: tokio
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2021-09-06 09:11:38 +00:00
ce6030a3cb
feat: Wire list deleted databases through gRPC and CLI APIs
2021-09-02 15:48:07 -04:00
581e49e38f
feat: Add a list_deleted_databases API
...
Fixes #2198 .
2021-09-02 14:57:27 -04:00
6da9cf246f
feat: Introduce a public DeletedDatabase type
2021-09-02 14:57:27 -04:00
8703aaff63
feat: Introduce a public GenerationId wrapper type
2021-09-02 14:57:27 -04:00
f47d7e6d92
feat: Return the time the database was deleted
2021-09-02 14:57:27 -04:00
51b6631d8f
feat: Implement a list_all_databases API that returns generation info too
2021-09-02 14:57:27 -04:00
1d50eb9880
test: Add a unit-ish test for list_possible_databases
2021-09-02 14:57:27 -04:00
70a5ffeae7
test: allow creation of deterministic chunks and transactions
2021-09-01 11:26:05 +02:00
d688678464
feat: Add an iox_object_store API for writing the tombstone file
...
Connects to #1871 .
2021-08-30 10:42:45 -04:00
5566e1926c
refactor: Use max_by_key instead of map max
2021-08-26 09:14:23 -04:00
82b566a3bb
fix: Warn when a non-generation item is found in a database directory
2021-08-26 09:14:23 -04:00
c16b4b1bff
refactor: Have GenerationPath save a Generation field
2021-08-26 09:14:23 -04:00
63efda8213
fix: Avoid leaking generation id in an error type
2021-08-26 09:14:23 -04:00
9edffea878
docs: Add and clarify some caveats and intentions
2021-08-26 09:14:23 -04:00
7b6093092a
refactor: make the `existing` method private and share more code
2021-08-26 09:14:23 -04:00
2c42a195ea
fix: Remove now-unused generation_id method on iox object store
2021-08-26 09:14:23 -04:00
199d212b18
refactor: Move find-or-create IoxObjectStore logic into tests
...
This is the only place this logic is used; it's not appropriate for
production usage as we only ever want to either find and error or create
and error in real life.
2021-08-26 09:14:23 -04:00
c7eceac8a3
refactor: Have server determine database generation from object store
2021-08-26 09:14:23 -04:00
1f0e37c9d1
refactor: Use the parsed_path macro to make path creation shorter
2021-08-26 09:14:22 -04:00
6a79f69bfc
refactor: Make methods on paths to produce other paths
2021-08-26 09:14:22 -04:00
5e1b57de9a
refactor: Borrow arcs instead of as_ref
2021-08-26 09:14:22 -04:00
cee2f21d47
feat: Add a find_or_create object store function for tests
2021-08-26 09:14:22 -04:00
d42cbaeef8
feat: Add a method to find one active database in object storage
2021-08-26 09:14:22 -04:00
18ba3b5c59
feat: Create database directories with a generation ID
2021-08-26 09:14:22 -04:00
a8ada048dd
test: Move success case first so it's easier to see when assumptions are broken
2021-08-26 09:14:22 -04:00
375347478a
test: Add basic functional tests for the database rules file methods ( #2366 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-08-24 13:27:58 +00:00
cc214fa9b9
fix: The path to db rules.pb shouldn't end in a slash
2021-08-19 21:07:33 -04:00
61263c8774
feat: Add a debugging-suitable way to get the object storage path of a database
2021-08-18 11:32:39 -04:00
997f61c5f6
refactor: Rename Paths to their more specific exported names
2021-08-18 11:32:39 -04:00
10c9acf5c4
refactor: Mark internally-pub items with pub crate to make their privacy status clearer
2021-08-18 11:32:39 -04:00
cb05b29917
docs: Explain reasoning behind the crate design more
2021-08-18 11:32:39 -04:00
fbf3ceb1e2
refactor: Extract listing of all databases into iox_object_store
2021-08-18 11:32:39 -04:00
Carol (Nichols || Goulding)
Marco Neumann
dependabot[bot]
Jake Goulding