Commit Graph

16 Commits (0a740856c77aae0074b8a96a920b44f5b3cb4cff)

Author SHA1 Message Date
William Baker 3a81166812
feat: added metrics collection for replications (#22906)
* feat: added metrics collection for replications

* fix: fixed panic when restarting

* fix: fix panic pt2

* chore: self-review fixes

* chore: simplify test
2021-11-22 11:40:03 -06:00
Dane Strandboge 6ee472725f
refactor: use remote write func in NewDurableQueueManager (#22888) 2021-11-19 11:31:10 -06:00
William Baker ad52815e19
feat: add field for dropping data resulting in non-retryable errors to individual replications (#22885)
* feat: add field for dropping data resulting in non-retryable errors to individual replications
2021-11-16 13:41:54 -07:00
Dane Strandboge 40d9587ece
feat: add replications queue scanner (#22873)
Co-authored-by: “mcfarlm3” <“58636946+mcfarlm3@users.noreply.github.com”>
2021-11-16 10:30:52 -06:00
Daniel Moran 6b56af3c3f
feat: mirror writes to registered replications (#22833) 2021-11-10 08:25:47 -05:00
mcfarlm3 cd0243d2b4
feat: added replications queue management to launcher tasks (#22820)
* feat: added replications queue management to launcher tasks

* refactor: separated sql logic into replications service rather than durable queue manager

* refactor: extended replications feature flag to launcher code and minor change to startup function param

* chore: added unit test coverage for replications server startup queue management

* refactor: made error messages reusable and factored out unecessary string from queue management tests

* refactor: changed queue management error names to pass linter check
2021-11-09 11:32:07 -08:00
Daniel Moran 1aac92c5ee
refactor: remove replications.current_queue_size_bytes from sqlite (#22832)
Maintaining the current queue size in a SQL column would require
updating the DB on every queue operation. Avoid that contention by
instead looking up the current size on the in-memory durable queue
struct, which is already tracked & updated as data enters & leaves
the queue.
2021-11-05 14:35:12 -04:00
William Baker f7573f43a7
feat: sql migrator can do down migrations (#22806)
* feat: sql down migrations

* refactor: different name for up migrations

* chore: update migrations ref in svc tests

* build: add lint step to verify sql migration names match
2021-11-01 14:30:18 -06:00
mcfarlm3 8825cd5d50
feat: replication apis durable queue management (#22719)
* feat: added durable queue management to replications service

* refactor: improved mapping of replication streams to durable queues

* refactor: modified replication stream durable queues to use user-specified engine path

* chore: generated test mocks for replications DurableQueueManager

* chore: add test coverage for replications durable queue manager

* refactor: made changes based on code review, added mutex to durableQueueManager, improved error logging

* chore: ran make fmt

* refactor: further improvements to error logging
2021-10-26 12:14:29 -07:00
Daniel Moran 58139c47b2
feat: add auth to remotes & replications APIs (#22744) 2021-10-26 11:32:35 -04:00
Daniel Moran 7c19225bed
feat: implement replication validation (#22581) 2021-10-05 14:34:38 -04:00
Daniel Moran 153a89dba0
feat: deleting a bucket also deletes all associated replications (#22424) 2021-09-09 15:22:36 -04:00
Daniel Moran 1fa0ccf24a
refactor: move interfaces for remotes & replication services out of root package (#22417) 2021-09-07 16:21:29 -04:00
Daniel Moran 12c8fd28d2
feat: implement metadata management for replications (#22302) 2021-09-01 12:01:41 -04:00
Daniel Moran b37ad79e20
feat: add logging and metrics middlewares to replications API (#22291) 2021-08-24 14:56:56 -04:00
Daniel Moran 641c02f9a8
feat: add APIs for management of replication streams (#22287) 2021-08-24 14:19:03 -04:00