Commit Graph

18 Commits (f73e80b8d15412689029e4f4303b49c037a0d947)

Author SHA1 Message Date
Joseph Crail 5fccee3d16 Fix spelling errors in comments and strings. 2015-06-28 02:54:34 -04:00
Jason Wilder 67d4ef0e28 Don't queue write failures that due to type conflicts
These will never succeed and will stay in the queue indefinitely.
2015-06-10 14:52:59 -06:00
Cory LaNou 8a5cf394d8 add ability to silence logging for testing 2015-06-10 10:27:57 -05:00
Jason Wilder 5dab8de7e0 Log hinted handoff processing duration 2015-06-09 14:49:18 -06:00
Jason Wilder c9f9b37753 Add hinted handoff queue rate limiting
Basic throughput limiting to dynamically maintain a bytes/sec
limit if configured for hinted handoff retry queues.  If a batch is
larger than the limit, the limit will slow the processing down to one
write per second.

By default, the limit is unbounded.  It can be enabled to help prevent
retstarting nodes that have queued writes for them from being overloaded
at startup.
2015-06-09 14:46:13 -06:00
Jason Wilder 07d1aac50f Fix comment in hinted handoff processor 2015-06-08 16:51:48 -06:00
Jason Wilder ede254484d Process hinted handoff queues concurrently 2015-06-08 15:05:44 -06:00
Jason Wilder d86b15953c Fix queue advance not writing updated position
The advance function was not writing the updated position in the
queue until after the next advance call was called.  Resulted in
the very last message would get replayed on restart each time.
2015-06-08 12:09:31 -06:00
Jason Wilder e75208f15c Log remote write erros 2015-06-08 09:09:10 -06:00
Jason Wilder 2ccf97e6a0 Add wait group to hinted handoff service 2015-06-05 22:16:52 -06:00
Jason Wilder 4ec77d8b84 Fix comments 2015-06-05 22:16:52 -06:00
Jason Wilder e2f3ff26a5 Fix data race when close hinted handoff service 2015-06-05 22:16:51 -06:00
Jason Wilder f4cea559d3 Purge hinted handoff queue if entries stick around past a cutoff 2015-06-05 22:16:51 -06:00
Jason Wilder 9d67f9bf62 Add hinted handoff max age and throughput rate limit config options 2015-06-05 22:16:51 -06:00
Jason Wilder 809b9b8a83 Add basic hinted handoff support
If a remote write fails, it will be queued to a per-node, local disk
queue and retried later.
2015-06-05 22:16:51 -06:00
Jason Wilder 89c01cd37d Add hinted handoff queue
This adds a disk backed queue that will be to persist hinted
handoff writes.
2015-06-05 22:16:51 -06:00
Jason Wilder 75b72c60fe Add hinted handoff service
The hinted handoff service will queue a write to a remote node if
that write fails and periodically retry the write.
2015-06-05 22:16:51 -06:00
Jason Wilder 170c584b45 Add hinted handoff config and tests 2015-06-05 22:16:51 -06:00