Commit Graph

372 Commits (16157daad8564b6dfb93aed7479df76c2fec6669)

Author SHA1 Message Date
Raphael Taylor-Davies 1d55d9a1b5
feat: add pbjson support (#2468)
* feat: add pbjson support

* chore: fix test
2021-09-16 07:33:27 +00:00
Carol (Nichols || Goulding) 8f54dbcb58 test: Check for other error Database States
Connects to #2539.
2021-09-15 14:44:14 -04:00
Raphael Taylor-Davies 1ea4335ff3
fix: report correct DatabaseStateCode (#2543)
* fix: report correct DatabaseStateCode

* chore: fix lint

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-09-15 18:02:54 +00:00
Marco Neumann 8a531be05b feat: expose chunk order via API and in system table 2021-09-14 13:00:55 +02:00
Marco Neumann 3f2e46c397 feat: prune old transactions from preserved catalog 2021-09-14 12:08:17 +02:00
kodiakhq[bot] c06943e355
Merge branch 'main' into jpg/delete-e2e-test 2021-09-13 18:17:52 +00:00
Nga Tran 40499b222e chore: merge main to branch 2021-09-13 12:15:16 -04:00
Nga Tran 8292c4d2e4 refactor: address review comments 2021-09-13 11:44:18 -04:00
Jake Goulding 88555388be test: Ensure that a deleted and recreated database returns new data
Closes #2454
2021-09-13 11:42:43 -04:00
Nga Tran 3798ca09bb feat: save delete predicates in chunks 2021-09-10 17:16:18 -04:00
kodiakhq[bot] faa05f394b
Merge branch 'main' into ntran/parse_delete_2 2021-09-09 18:28:39 +00:00
Raphael Taylor-Davies 3e3437ba39
fix: test_logging port collision (#2492)
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-09-09 15:49:14 +00:00
Raphael Taylor-Davies 3cee899f77
feat: migrate catalog timestamp summary to `metric` crate (#2486)
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-09-09 14:52:25 +00:00
Nga Tran 19848eb132 chore: turn a test on 2021-09-08 17:30:55 -04:00
Nga Tran 00df7b064c feat: finally have the delete predicate parsed 2021-09-08 17:30:10 -04:00
Marco Neumann bbb8898d36 refactor: make writer buffer auto-creation types nicer to read 2021-09-08 11:13:48 +02:00
Marco Neumann 801cf08be7 feat: auto-creation of sequencers by write buffer
For Kafka, that basically means that we create a topic if it doesn't
exist yet.

Closed #2455.
Fixes #2189.
2021-09-07 18:24:57 +02:00
Marco Neumann d5662328b0 refactor: `n_sequencers` should be non-zero 2021-09-07 18:18:20 +02:00
Nga Tran dbe4bcff22 chore: merge main to branch 2021-09-07 10:54:59 -04:00
Raphael Taylor-Davies a4b0cbc0e7
feat: migrate jemalloc metrics to `metric` crate (#2435)
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-09-06 09:18:27 +00:00
Nga Tran 9de3b79a90 refactor: more cleanup 2021-09-06 01:45:47 -04:00
Nga Tran de0bd80c3d refactor: cleanup 2021-09-06 01:07:07 -04:00
Nga Tran d8b3208896 refactor: sstart building parser in the client side 2021-09-03 17:56:45 -04:00
kodiakhq[bot] 2d41fd519f
Merge branch 'main' into cn/list-soft-deleted 2021-09-03 15:16:32 +00:00
Marco Neumann 3c968ac092 feat: correctly account MUB sizes
Fixes #1565.
2021-09-03 09:15:49 +02:00
Nga Tran bbc0fcf4c0 refactor: cleanup 2021-09-02 17:21:55 -04:00
Nga Tran e2274a9f41 feat: parser for delete predicate 2021-09-02 17:02:05 -04:00
Carol (Nichols || Goulding) ce6030a3cb feat: Wire list deleted databases through gRPC and CLI APIs 2021-09-02 15:48:07 -04:00
Marco Neumann a63eb53ac5 feat: forward connection config to Kafka write buffer 2021-09-02 16:53:31 +02:00
Marco Neumann ecf1f99ddb refactor: more flexible writer buffer config
This allows:

- different types (instead of guessing through the connection URL)
- sequencer counts (not used yet but will be by #2455)
- extensible configs (e.g. to configure Kafka in a more granular way,
  not wired up yet)
- future extensions (since we use a message now instead of a single
  string)

**BREAKING: This requires changes for deployed systems / existing DBs!**
2021-09-02 16:41:35 +02:00
Marco Neumann 4a863993ec feat: "dump catalog" debug CLI 2021-09-02 08:08:20 +02:00
Nga Tran efcdc8e50f feat: implement delete parser 2021-09-01 17:31:49 -04:00
kodiakhq[bot] e183ecb3e7
Merge branch 'main' into cn/list-but-not-deleted-databases 2021-09-01 13:03:09 +00:00
Nga Tran a4183de411 feat: more progress on the delete flow from grpc API to catalog chunks 2021-08-31 17:42:07 -04:00
Marco Neumann 79ad48ac3a chore: rename "labels" to "attributes" 2021-08-31 11:31:15 +02:00
Marco Neumann 7e7855f727 test: avoid picking the same port twice under multi-threaded tests 2021-08-31 10:16:22 +02:00
Marco Neumann 69eae490ef test: avoid picking the same port twice 2021-08-31 10:16:22 +02:00
Marco Neumann e6afb844ff test: use random free port for end2end tests
Fixes #786.
Fixes #2403.
2021-08-31 10:16:22 +02:00
Carol (Nichols || Goulding) 3558e46cae test: Check deleted databases are omitted from management API list 2021-08-30 14:14:32 -04:00
Carol (Nichols || Goulding) 8dc375100a feat: Wire database deletion through the CLI
Fixes #2432.
2021-08-30 14:14:22 -04:00
Carol (Nichols || Goulding) e67624dd37 fix: Assert on which error getting a deleted database returns 2021-08-30 11:29:25 -04:00
Carol (Nichols || Goulding) e420fe13a4 test: Add delete database to the end-to-end tests 2021-08-30 10:49:19 -04:00
Marco Neumann 96b0026203 fix: make "persist partition" a bit more stable
- add longer wait times to tests
- exclude chunks that have active lifecycle actions early (instead of
  failing the whole set)
- properly catch the "no chunks" case

Fixes #2434.
2021-08-30 13:11:12 +02:00
Andrew Lamb 2f49e47a23
feat: return DatabaseRules for ListDatabases request (#2431) 2021-08-28 10:53:24 +00:00
Andrew Lamb 779b027271
feat: Store only the database rules sent by the client (do not store default values) (#2430)
* feat: add omit_default to protobuf definition

* feat: Persist only the client provided rules

* fix: Remove race conditions

* fix: merge confit

* refactor: do not use macro

* refactor: restore use of glob import

* fix: review comments

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-08-28 10:26:32 +00:00
Nga Tran 499af57299 chore: merge mian to branch and resolve conflicts 2021-08-27 17:51:07 -04:00
Marco Neumann 589bb31497 fix: make error msg clearer 2021-08-27 15:49:16 +02:00
Marco Neumann 8591aac761 test: improve error checking 2021-08-27 15:16:13 +02:00
Marco Neumann 90c60fd181 feat: "persist partition" CLI 2021-08-27 15:16:13 +02:00
Marco Neumann 6625b3a768 feat: "persist partition" gRPC 2021-08-27 15:16:13 +02:00