Commit Graph

20 Commits (jdstrand/update-golang-jwt)

Author SHA1 Message Date
Jeffrey Smith II 77fd64a975
fix: handle replication missing queue (#24123)
* fix: replications should startup after backup/restore

* chore: refactor

* test: improve logging and handle test better
2023-03-09 13:10:53 -05:00
suitableZebraCaller ec7fdd3a58
fix: Show Replication Queue size and Replication TCP Errors (#23960)
* feat: Show remaining replication queue size

* fix: Show non-http related error messages

* fix: Show non-http related error messages with backoff

* fix: Updates for replication tests

* chore: formatting

* chore: formatting

* chore: formatting

* chore: formatting

* chore: lowercase json field

---------

Co-authored-by: Geoffrey <suitableZebraCaller@users.noreply.github.com>
Co-authored-by: Jeffrey Smith II <jeffreyssmith2nd@gmail.com>
2023-02-02 09:47:45 -05:00
Ole Kristian (Zee) 666cabb1f4
fix: fix wrong max age transformation from seconds (#23684)
* fix: fix wrong max age transformation from seconds

* refactor: clarify max age intent

* refactor: remove unnecessary duration
2022-11-16 16:18:43 -05:00
Dane Strandboge 9e556864a3
fix: replications remote write failure can deadlock remote writer (#23458) 2022-06-16 11:57:24 -05:00
Dane Strandboge 359fcc46b5
feat: add maximum age to replication queues (#23206)
Co-authored-by: Sam Arnold <sarnold@influxdata.com>
2022-03-25 13:06:05 -05:00
mcfarlm3 60234964d0
refactor: replications local write optimization (#22993)
* refactor: eliminate sqlite query in case of no configured replications

* refactor: updated write-related tests to reflect tracking of orgID and localBucket by the queue manager

* refactor: removed redundant trackedReplications field

* refactor: corrected slice init in GetReplications and added TestGetReplications

* refactor: eliminated tracked package and moved TrackedReplication struct to influxdb package via replication.go

* chore: ran make fmt

* fix: added closeRq function back in to address flaky tests

* refactor: small changes to queue manager test based on code review
2021-12-15 12:32:46 -08:00
William Baker a7a5233432
feat: advance queue scanner periodically instead of every remote write (#22981) 2021-12-13 10:09:36 -06:00
William Baker e5cbd279ee
fix: advance replications queue after successful remote writes (#22967)
* fix: advance replications queue after successful remote writes to prevent data duplication on errors

* fix: loop on sendwrite

* chore: remove flaky test

* chore: add TODO about future optimization
2021-12-08 12:52:46 -06:00
William Baker 6096ee2ad4
feat: replications metrics include failure to enqueue (#22962)
* feat: replications metrics include failure to enqueue
2021-12-02 14:42:55 -06:00
William Baker e4e16335f5
fix: replications remote writes do not block server shutdown (#22958)
* fix: replications remote writes do not block server shutdown

* fix: don't leak goroutine
2021-12-02 12:04:52 -06:00
William Baker 3460f1cc52
feat: replication remote writes do not block local writes (#22956)
* feat: replication remote writes do not block local writes
2021-12-01 15:37:10 -06:00
William Baker f05d0136f1
feat: metrics collection for replications remote writes (#22952)
* feat: metrics collection for replications remote writes

* fix: don't update metrics with 204 error code on successful writes
2021-12-01 12:41:24 -06:00
William Baker 9873ccd657
feat: remote write function for replications (#22942)
* feat: remote write function for replications

* chore: implement UpdateResponseInfo store method

* chore: only set gzip heading for non-empty requests

* fix: address review feedback
2021-11-30 15:33:42 -06:00
William Baker 3a81166812
feat: added metrics collection for replications (#22906)
* feat: added metrics collection for replications

* fix: fixed panic when restarting

* fix: fix panic pt2

* chore: self-review fixes

* chore: simplify test
2021-11-22 11:40:03 -06:00
Dane Strandboge 6ee472725f
refactor: use remote write func in NewDurableQueueManager (#22888) 2021-11-19 11:31:10 -06:00
Dane Strandboge 40d9587ece
feat: add replications queue scanner (#22873)
Co-authored-by: “mcfarlm3” <“58636946+mcfarlm3@users.noreply.github.com”>
2021-11-16 10:30:52 -06:00
Daniel Moran 6b56af3c3f
feat: mirror writes to registered replications (#22833) 2021-11-10 08:25:47 -05:00
mcfarlm3 cd0243d2b4
feat: added replications queue management to launcher tasks (#22820)
* feat: added replications queue management to launcher tasks

* refactor: separated sql logic into replications service rather than durable queue manager

* refactor: extended replications feature flag to launcher code and minor change to startup function param

* chore: added unit test coverage for replications server startup queue management

* refactor: made error messages reusable and factored out unecessary string from queue management tests

* refactor: changed queue management error names to pass linter check
2021-11-09 11:32:07 -08:00
Daniel Moran 1aac92c5ee
refactor: remove replications.current_queue_size_bytes from sqlite (#22832)
Maintaining the current queue size in a SQL column would require
updating the DB on every queue operation. Avoid that contention by
instead looking up the current size on the in-memory durable queue
struct, which is already tracked & updated as data enters & leaves
the queue.
2021-11-05 14:35:12 -04:00
mcfarlm3 8825cd5d50
feat: replication apis durable queue management (#22719)
* feat: added durable queue management to replications service

* refactor: improved mapping of replication streams to durable queues

* refactor: modified replication stream durable queues to use user-specified engine path

* chore: generated test mocks for replications DurableQueueManager

* chore: add test coverage for replications durable queue manager

* refactor: made changes based on code review, added mutex to durableQueueManager, improved error logging

* chore: ran make fmt

* refactor: further improvements to error logging
2021-10-26 12:14:29 -07:00