e4e494f1d9
fix: Remove now-unused transition function. Fixes #2762 .
2021-10-28 13:30:43 -04:00
0bccd35f8c
fix: List details about databases only prints active dbs and their UUIDs
...
We can no longer list deleted databases because the server no longer
knows about them, and we now have UUIDs that are useful to know about,
so change the detailed listing of databases to return UUID instead of
possible deleted at time.
2021-10-28 13:20:29 -04:00
f160712f5e
fix: Remove the concept of generations
...
From iox_object_store and everywhere that fails to compile as a result.
Also make path handling in iox_object_store a bit more consistent.
2021-10-28 13:20:28 -04:00
10912c2b97
feat: Write owner info in the database's object store directory
...
Use this as an extra check that the database thinks the current server
is its owner when a server tries to initialize and own a database.
If the owner info doesn't match the current server, which could happen
if a server's config was updated but the database's owner info wasn't,
put the database in an error state that requires operator intervention.
2021-10-18 21:13:20 -04:00
26484309e0
fix: Re-export prost errors instead of wrapping them
2021-10-15 13:44:53 -04:00
afd6e826e5
feat: Write out server config files listing database name and locations
2021-10-15 09:46:20 -04:00
5348c9e503
refactor: Move ProstError to root of generated_types to be useful elsewhere
2021-10-15 09:46:20 -04:00
3dfe400e6b
feat: migrate write path to TimeProvider ( #2722 ) ( #2807 )
2021-10-12 12:09:08 +00:00
c4a2641764
refactor: remove `time_closed`
...
The "time closed" is a leftover from an old lifecycle system, where
chunks moved through the system (open=>closed=>persisted) without being
merged. Now we have the compaction as well as the split query for
persistence that can merge chunks, so a single "time closed" doesn't
make sense any longer. So in fact it is `None` for many chunks and is
also not persisted. Also the current lifecycle policy doesn't use this
value. So let's just remove it.
Closes #1846 .
2021-10-11 15:49:34 +02:00
15b396c720
fix: Set a UUID if none was read from object storage on server startup
...
This enables a smoother transition to this state when this gets
deployed.
2021-10-07 10:19:17 -04:00
ae7d893199
feat: Add UUID to databases
...
Connects to #2675 .
When a database is created, assign it a UUID and serialize the UUID to
object storage by wrapping the database rules in a new
`PersistedDatabaseRules` type that also contains the UUID.
All APIs to the end user involving rules should continue using only
`DatabaseRules` so the UUID is an internal implementation detail.
2021-10-07 10:19:14 -04:00
27e7a1f925
refactor: Organize use statements
2021-10-07 10:17:19 -04:00
63d74be490
refactor: make `ChunkId` a UUID
2021-10-07 10:23:27 +02:00
2a584420b3
refactor: make data_types optional dependency ( #2739 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-10-05 15:07:45 +00:00
97881079e8
refactor: make `ChunkOrder` non-zero
...
This will make it easier to handle missing values.
Helps with #2633 .
2021-10-04 17:49:12 +02:00
b402423e9e
feat: remove move lifecycle action ( #2674 )
...
* feat: remove move_chunk lifecycle action
* chore: review feedback
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-09-30 16:58:05 +00:00
86cee568d5
feat: use upstream pbjson ( #2650 )
...
* feat: use upstream pbjson
* chore: fmt
2021-09-28 16:29:26 +00:00
cef5aeee52
refactor: introduce `ChunkId` type
2021-09-20 13:10:41 +02:00
44a89cdf75
refactor: Change DeletedDatabase to DetailedDatabase
...
So this info can be reused for active databases in detailed database
lists.
2021-09-17 15:27:22 -04:00
37b615f301
feat: migrate operations CLI to use pbjson ( #2562 )
...
* feat: migrate operations CLI to use pbjson
* fix: reserve removed field
* chore: review feedback
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-09-16 19:15:24 +00:00
1d55d9a1b5
feat: add pbjson support ( #2468 )
...
* feat: add pbjson support
* chore: fix test
2021-09-16 07:33:27 +00:00
6e7fa3e574
feat: migrate http ingest metrics ( #2542 )
...
* feat: migrate http ingest metrics
* chore: review feedback
* refactor: RAII entry ingest recorder
2021-09-15 19:01:10 +00:00
1ea4335ff3
fix: report correct DatabaseStateCode ( #2543 )
...
* fix: report correct DatabaseStateCode
* chore: fix lint
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-09-15 18:02:54 +00:00
becef1c75f
refactor: introduce `ChunkOrder` type
2021-09-14 17:10:23 +02:00
8a531be05b
feat: expose chunk order via API and in system table
2021-09-14 13:00:55 +02:00
3f2e46c397
feat: prune old transactions from preserved catalog
2021-09-14 12:08:17 +02:00
00df7b064c
feat: finally have the delete predicate parsed
2021-09-08 17:30:10 -04:00
bbb8898d36
refactor: make writer buffer auto-creation types nicer to read
2021-09-08 11:13:48 +02:00
0a823cd75c
refactor: simplify error handling
2021-09-07 18:24:59 +02:00
e7cf91bfc4
refactor: more structures write buffer auto-creation config
2021-09-07 18:18:20 +02:00
d5662328b0
refactor: `n_sequencers` should be non-zero
2021-09-07 18:18:20 +02:00
dbe4bcff22
chore: merge main to branch
2021-09-07 10:54:59 -04:00
9de3b79a90
refactor: more cleanup
2021-09-06 01:45:47 -04:00
de0bd80c3d
refactor: cleanup
2021-09-06 01:07:07 -04:00
4801b2c238
feat: Have the ParseDelete message and its corresponding ProvidedParseDelete struct ready for building delete parser
2021-09-06 00:13:59 -04:00
ce6030a3cb
feat: Wire list deleted databases through gRPC and CLI APIs
2021-09-02 15:48:07 -04:00
81e672db14
feat: add `auto_create_sequencers` write buffer flag
2021-09-02 16:41:35 +02:00
ecf1f99ddb
refactor: more flexible writer buffer config
...
This allows:
- different types (instead of guessing through the connection URL)
- sequencer counts (not used yet but will be by #2455 )
- extensible configs (e.g. to configure Kafka in a more granular way,
not wired up yet)
- future extensions (since we use a message now instead of a single
string)
**BREAKING: This requires changes for deployed systems / existing DBs!**
2021-09-02 16:41:35 +02:00
779b027271
feat: Store only the database rules sent by the client (do not store default values) ( #2430 )
...
* feat: add omit_default to protobuf definition
* feat: Persist only the client provided rules
* fix: Remove race conditions
* fix: merge confit
* refactor: do not use macro
* refactor: restore use of glob import
* fix: review comments
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-08-28 10:26:32 +00:00
499af57299
chore: merge mian to branch and resolve conflicts
2021-08-27 17:51:07 -04:00
b79eaa34d1
refactor: address review comments
2021-08-27 15:53:27 -04:00
bcd39e225c
feat: Management API for delete
2021-08-26 17:31:21 -04:00
7cf7fb02ed
refactor: Rename database ObjectStore state types to DatabaseObjectStore
2021-08-26 09:14:23 -04:00
c7eceac8a3
refactor: Have server determine database generation from object store
2021-08-26 09:14:23 -04:00
1773bf5d37
feat: add storage client to influxdb_iox_client ( #2404 )
...
* feat: add storage client to influxdb_iox_client
* chore: fix type_url
* refactor: split storage into separate crate
* chore: fix doctest
* chore: review feedback
* chore: add generated_types cleanup ticket
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-08-26 10:30:59 +00:00
60ea637c38
refactor: remove `drop_non_persisted`
...
This flag should always be `!persist`.
Closes #2212 .
2021-08-17 16:53:45 +02:00
77892a0998
feat: add API to drop entire partitions
2021-08-17 09:44:35 +02:00
3de6b44e23
build: use new rustdoc lint name ( #2261 )
...
* fix: nocache feature code rot
The MBChunk::snapshot code when using the "nocache" option no longer
compiles - this commit updates it to match the not(nocache) code.
* build: use updated broken_intra_doc_links name
The broken_intra_doc_links lint was renamed
rustdoc::broken_intra_doc_links
https://doc.rust-lang.org/rustdoc/lints.html
2021-08-11 19:48:51 +00:00
2344c28f4e
feat: drain database jobs on shutdown ( #2239 )
...
* feat: drain database jobs on shutdown
* chore: fmt
* chore: review feedback
* chore: use join() not member directly
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-08-10 16:47:37 +00:00
1f450ef371
feat: add Database abstraction ( #2186 ) ( #2203 )
...
* feat: add Database abstraction
* chore: minor tweaks
* chore: remove redundant test fixture restart
* chore: review feedback
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-08-08 17:14:23 +00:00
Carol (Nichols || Goulding)
Carol (Nichols || Goulding)
Raphael Taylor-Davies
Marco Neumann
Nga Tran
Andrew Lamb
Dom