51a40b31bf
feat: Add a --detailed option to the database list CLI
...
That will list both active and deleted databases with their generations.
Closes #2462 .
2021-09-17 15:27:23 -04:00
44a89cdf75
refactor: Change DeletedDatabase to DetailedDatabase
...
So this info can be reused for active databases in detailed database
lists.
2021-09-17 15:27:22 -04:00
23cc980d9e
Merge branch 'main' into cn/restore
2021-09-17 17:52:56 +00:00
37b615f301
feat: migrate operations CLI to use pbjson ( #2562 )
...
* feat: migrate operations CLI to use pbjson
* fix: reserve removed field
* chore: review feedback
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-09-16 19:15:24 +00:00
91fd32d506
fix: Reset restored db's state instead of restarting background worker
2021-09-15 19:04:05 -04:00
7b6d8f9327
feat: Add an API for restoring a database that was marked deleted
2021-09-15 16:59:37 -04:00
1ea4335ff3
fix: report correct DatabaseStateCode ( #2543 )
...
* fix: report correct DatabaseStateCode
* chore: fix lint
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-09-15 18:02:54 +00:00
c5ebf4a2e6
refactor: remove unused predicate serialization variants
2021-09-15 18:05:11 +02:00
44eb3b994d
feat: `Predicate` serialization
...
Closes #2493 .
2021-09-15 16:37:25 +02:00
8a531be05b
feat: expose chunk order via API and in system table
2021-09-14 13:00:55 +02:00
45cb00d8c0
refactor: track chunk order in chunks
2021-09-14 13:00:55 +02:00
3f2e46c397
feat: prune old transactions from preserved catalog
2021-09-14 12:08:17 +02:00
00df7b064c
feat: finally have the delete predicate parsed
2021-09-08 17:30:10 -04:00
bbb8898d36
refactor: make writer buffer auto-creation types nicer to read
2021-09-08 11:13:48 +02:00
e7cf91bfc4
refactor: more structures write buffer auto-creation config
2021-09-07 18:18:20 +02:00
dbe4bcff22
chore: merge main to branch
2021-09-07 10:54:59 -04:00
9de3b79a90
refactor: more cleanup
2021-09-06 01:45:47 -04:00
de0bd80c3d
refactor: cleanup
2021-09-06 01:07:07 -04:00
4801b2c238
feat: Have the ParseDelete message and its corresponding ProvidedParseDelete struct ready for building delete parser
2021-09-06 00:13:59 -04:00
d8b3208896
refactor: sstart building parser in the client side
2021-09-03 17:56:45 -04:00
ce6030a3cb
feat: Wire list deleted databases through gRPC and CLI APIs
2021-09-02 15:48:07 -04:00
81e672db14
feat: add `auto_create_sequencers` write buffer flag
2021-09-02 16:41:35 +02:00
ecf1f99ddb
refactor: more flexible writer buffer config
...
This allows:
- different types (instead of guessing through the connection URL)
- sequencer counts (not used yet but will be by #2455 )
- extensible configs (e.g. to configure Kafka in a more granular way,
not wired up yet)
- future extensions (since we use a message now instead of a single
string)
**BREAKING: This requires changes for deployed systems / existing DBs!**
2021-09-02 16:41:35 +02:00
a4183de411
feat: more progress on the delete flow from grpc API to catalog chunks
2021-08-31 17:42:07 -04:00
d688678464
feat: Add an iox_object_store API for writing the tombstone file
...
Connects to #1871 .
2021-08-30 10:42:45 -04:00
2f49e47a23
feat: return DatabaseRules for ListDatabases request ( #2431 )
2021-08-28 10:53:24 +00:00
779b027271
feat: Store only the database rules sent by the client (do not store default values) ( #2430 )
...
* feat: add omit_default to protobuf definition
* feat: Persist only the client provided rules
* fix: Remove race conditions
* fix: merge confit
* refactor: do not use macro
* refactor: restore use of glob import
* fix: review comments
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-08-28 10:26:32 +00:00
499af57299
chore: merge mian to branch and resolve conflicts
2021-08-27 17:51:07 -04:00
b79eaa34d1
refactor: address review comments
2021-08-27 15:53:27 -04:00
6625b3a768
feat: "persist partition" gRPC
2021-08-27 15:16:13 +02:00
bcd39e225c
feat: Management API for delete
2021-08-26 17:31:21 -04:00
7cf7fb02ed
refactor: Rename database ObjectStore state types to DatabaseObjectStore
2021-08-26 09:14:23 -04:00
c7eceac8a3
refactor: Have server determine database generation from object store
2021-08-26 09:14:23 -04:00
60ea637c38
refactor: remove `drop_non_persisted`
...
This flag should always be `!persist`.
Closes #2212 .
2021-08-17 16:53:45 +02:00
fcf2bee443
feat: drop partition gRPC
2021-08-17 09:44:35 +02:00
77892a0998
feat: add API to drop entire partitions
2021-08-17 09:44:35 +02:00
42d5f9f3a1
feat: skip replay via CLI
2021-08-16 13:47:07 +02:00
2344c28f4e
feat: drain database jobs on shutdown ( #2239 )
...
* feat: drain database jobs on shutdown
* chore: fmt
* chore: review feedback
* chore: use join() not member directly
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-08-10 16:47:37 +00:00
1f450ef371
feat: add Database abstraction ( #2186 ) ( #2203 )
...
* feat: add Database abstraction
* chore: minor tweaks
* chore: remove redundant test fixture restart
* chore: review feedback
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-08-08 17:14:23 +00:00
98d4c9fca1
feat: switch protobuf write service to canonical definition ( #2182 )
...
* feat: switch protobuf write service to canonical definition
The protobuf definition used for the proto write endpoint was a WIP. Now
that a canonical definition exists at
https://github.com/influxdata/influxdb-pb-data-protocol/ we can switch
to that.
* chore: lint etc
* chore: fix rustdoc nit in proto definition comment
2021-08-04 00:16:49 +00:00
fe7f65bfa7
feat(iox): Implement max_active_compactions_cpu_fraction
2021-07-28 17:31:17 +02:00
657e769c8c
chore: add go_package to management protos ( #2126 )
2021-07-27 13:11:17 +00:00
094945a72d
feat: Add '/dev/null' sink
2021-07-26 19:19:11 +02:00
2478547ad7
refactor: Remove deprecated target field in RoutingConfig
2021-07-26 17:29:39 +02:00
009c77d864
Merge branch 'main' into cn/parquet-first-last
2021-07-26 14:59:54 +00:00
5d9ad4bc31
docs: Fix description of parquet file time of first/last write
...
Co-authored-by: Andrew Lamb <alamb@influxdata.com>
2021-07-26 09:42:31 -04:00
e5ee252876
feat: Add kafka sink variant
2021-07-26 11:08:02 +02:00
d58a3ccbc7
refactor: Add sink to routing config
...
This deprecates the "target" field in the RoutingConfig and replaces it with the "sink"
field, which has a variant that accepts a node group.
This commit is backward compatible in that it will accept existing configs.
The configs will roundtrip to the new format though (i.e. `database get` will render
the sink field).
2021-07-26 11:08:01 +02:00
16a82ba350
refactor: Generailize sinks: Rename Shard to Sink
...
The ShardConfig applies matchers that resolve to a shard number.
The config then applies a mapping between shard numbers to targets.
The type that encapsulated the target that a shard points to was also called
a "Shard". This is confusing. This commit changes it to "Sink", i.e. a destination
for traffic to go to. Subsequent commits will expand the definition of a Sink to
encompass different kinds of sinks (like kafka write buffer, "devnull", ...)
This changes only the name of the protobuf message and the related rust types,
it doesn't change any name of the json-rendered protobuf configs.
2021-07-26 11:08:00 +02:00
d928bc84e6
feat: Thread time_of_{first,last}_write through Parquet metadata
2021-07-23 14:07:35 -04:00
Carol (Nichols || Goulding)
kodiakhq[bot]
Raphael Taylor-Davies
Marco Neumann
Nga Tran
Andrew Lamb
Jacob Marble
Marko Mikulicic
Carol (Nichols || Goulding)
Jake Goulding