Commit Graph

35 Commits (fd85ae5b7fada9142aac66603200e6cc124f9aed)

Author SHA1 Message Date
Jason Wilder 5abdb42a7d Use a bounded pool for remote writes
Under highly conncurrent write load, the coordinating node would
create a connection to any other node that is part of the replica
group.  Since each connection can be expensive, OOM sitations could
occur because there was no bounds on the number of new connections
that would be created.  If writes on a remote node were slow, connections
could pile up an exacerbate the problem.

This switches the pool to be bounded and has a checkout that is blocking
with a timeout.  If a connection is available, it's returned immediately.
If the pool still has room for more connections, it will create one if needed.
Otherwise, the call will block until a connection becomes available or
the timeout expires.  In the case of a timeout, it is propogated back up
to the PointsWriter that determine what do return to the client.
2016-01-26 17:08:36 -07:00
Paul Dix fb9181d240 Fix meta-service build 2016-01-21 15:28:33 -05:00
Paul Dix f385945058 Update Server to work with new metaservice/client 2016-01-21 15:28:33 -05:00
Cory LaNou 8d878fff91 buildable meta -> services/meta 2016-01-21 15:28:32 -05:00
Nathaniel Cook 1719a6107c PointsWriter will drop writes to subscriber service for any in-flight writes 2015-11-05 16:25:00 -07:00
Philip O'Toole de7919240f Migrate internal stats to consistent names
Go style -- and existing runtime stats -- do not use underscores, but
instead use camel case. This change makes the internal stats adhere to
that convention.
2015-10-28 21:07:45 -07:00
Charles Chan 9382d5b534 Fix typos.
* non-existant --> non-existent
* propogate --> propagate
2015-10-17 07:36:56 -07:00
Nathaniel Cook 8b31007aa7 Adds subscriber service for creating/dropping subscriptions to the
InfluxDB data stream.
2015-10-14 15:23:45 -06:00
Daniel Morsing 62dff895e2 wire up INTO queries.
Since INTO queries need to have absolute information about the database
to work, we need to create a loopback interface back to the cluster
in order to perform them.
2015-10-13 15:00:36 +00:00
Mint 3cbc1936e5 Changes to make the cluster sub package golint-able
Issue: Enable golint on the code base #4098
2015-09-28 21:40:58 -05:00
Cory LaNou 72f6f7d268 Merge pull request #4134 from influxdb/issue-3447
Refactor Points and Rows to dedicated packages
2015-09-17 15:27:48 -05:00
Philip O'Toole 19384efde7 Return an error-on-write if RP does not exist 2015-09-16 18:40:29 -07:00
Cory LaNou d19a510ad2 refactor Points and Rows to dedicated packages 2015-09-16 15:33:08 -05:00
Philip O'Toole f0bbec6699 Add stats to PointsWriter 2015-09-08 19:30:07 -07:00
Jason Wilder 1d4ee6c3fa Add tests for influx consistency level parsing 2015-09-02 09:22:15 -06:00
Takayuki Usui da8efa56e1 Fix writes possibly blocked with relaxed write consistency level
Immediately return once the required number of writes are completed,
otherwise requests running with relaxed consistency levels (e.g. any
or one) would be blocked unexpectedly, for instance, waiting for dead
nodes to respond.
2015-09-02 11:08:04 +09:00
Ben Johnson 767307eed6 convert meta shard owners to objects
This commit converts meta.ShardInfo.OwnerIDs from a slice of ids
to a slice of objects. This is to support adding statuses for a
shard for a given node. For example, a node may have a shard
assigned to it but it is currently copying the shard and is not
ready to serve data for it.

The old `OwnerIDs` is marked as deprecated, however, the code
still supports loading from older protobuf-encoded data.
2015-08-31 16:33:13 -06:00
Philip O'Toole a84c48bff6 Allow the PointWriter timeout to be configurable 2015-07-02 12:50:12 -04:00
Joseph Crail 5fccee3d16 Fix spelling errors in comments and strings. 2015-06-28 02:54:34 -04:00
David Norton 7c39ede6ba fix #2920: create collectd database on startup 2015-06-11 09:40:42 -04:00
Jason Wilder 67d4ef0e28 Don't queue write failures that due to type conflicts
These will never succeed and will stay in the queue indefinitely.
2015-06-10 14:52:59 -06:00
Jason Wilder 999f4a4c41 Return field type errors as client write errors
Fixes #2849
2015-06-10 14:52:26 -06:00
Philip O'Toole 952fb49368 Move parsing consistency levels to cluster package
Errors parsing the consistency level are indicated by returning a nil
Graphite input.
2015-06-09 14:21:12 -07:00
Philip O'Toole fd0de2fb1b Add function to parser consistency levels 2015-06-09 14:21:12 -07:00
Jason Wilder eb1cd44b8d Log write errors
Since the client only receives a "write failed" or "partial write" error
message, log more context in the logs.
2015-06-09 14:49:22 -06:00
Jason Wilder 1024965db7 Create shard received from cluster writer 2015-06-05 22:16:51 -06:00
Jason Wilder 75b72c60fe Add hinted handoff service
The hinted handoff service will queue a write to a remote node if
that write fails and periodically retry the write.
2015-06-05 22:16:51 -06:00
Ben Johnson b925e1c1af Multi-node clustering.
This commit adds the ability to cluster multiple nodes together to share
the same metadata through raft consensus.
2015-06-05 14:41:19 -06:00
Jason Wilder 156e7df346 Rename PointsWrite.Store to TSDBStore
Matches MetaStore naming convention better.
2015-06-02 14:47:59 -06:00
Jason Wilder e400e8f2d6 Use default retention policy if not specified during writes 2015-06-01 17:16:44 -06:00
Jason Wilder 497cd506f9 Remove temporary INFLUXDB_ALPHA write path enable flag
Real thing exists now.
2015-06-01 16:45:08 -06:00
Cory LaNou 17bdf1c114 get both json/line protocol endpoints working 2015-06-01 12:35:57 -06:00
Cory LaNou 3597565955 reading and writing yo! 2015-06-01 11:59:58 -06:00
Ben Johnson bf823d9887 Integrating cmd/influxd/run. 2015-05-30 14:06:36 -06:00
Ben Johnson e1fc0958e7 Rename cluster.Coordinator to cluster.PointsWriter. 2015-05-30 14:05:27 -06:00