Commit Graph

22 Commits (0.11)

Author SHA1 Message Date
Jason Wilder 43118ce78e Send database and retention policy with remote writes
There was a race where a remote write could arrive before a meta
client cache update arrived.  When this happened, the receiving node
would drop the write because it could not determine what database
and retention policy the shard it was supposed to create belonged
to.

This change sends the db and rp along with the write so that the
receiving node does not need to consult the meta store.  It also
allows us to not send writes for shards that no longer exist instead
of always sending them and having the receiving node logs fill up
with dropped write requests.  This second situation can occur when
shards are deleted and some nodes still have writes queued in hinted
handoff for those shards.

Fixes #5610
2016-03-01 21:23:03 -07:00
Ben Johnson 0dda9f6608 add remote execution
This commit adds remote execution to the query engine.
2016-02-25 08:41:20 -07:00
David Norton 4d4e382ddf Add a Meta Executor.
The Meta Executor will make allow data nodes to execute queries
remotely on each other, via RPC calls.
2016-02-24 11:24:22 -05:00
Ady af17b9d8a7 Zero timeout set to support all platforms 2016-02-16 21:21:39 +05:30
Ady 58cccaa202 Update timeout for Dial, in order to work in Windows 2016-02-15 22:22:44 +05:30
Ady b850c13dc5 Modify Dial timeout in test to 0 nano sec to make it pass in Windows 2016-02-15 21:40:13 +05:30
Ady 80a2874361 Merge branch 'master' of https://github.com/influxdata/influxdb into mvadu-patch-ErrDialTimeout
Accomodate name change from influxdb->influxdata
2016-02-13 23:01:01 +05:30
Ady 0a00a59f28 test with more data points 2016-02-11 07:50:15 +05:30
Ady e68dd9a85a Modify WriteTLV with debug prints. Revert chnages to shard_writer 2016-02-11 07:29:47 +05:30
Ben Johnson 5a0d1ab7c1 rename influxdb/influxdb to influxdata/influxdb
This commit changes all the import and URL references from:

    github.com/influxdb/influxdb

to:

    github.com/influxdata/influxdb
2016-02-10 10:26:18 -07:00
Jason Wilder 5abdb42a7d Use a bounded pool for remote writes
Under highly conncurrent write load, the coordinating node would
create a connection to any other node that is part of the replica
group.  Since each connection can be expensive, OOM sitations could
occur because there was no bounds on the number of new connections
that would be created.  If writes on a remote node were slow, connections
could pile up an exacerbate the problem.

This switches the pool to be bounded and has a checkout that is blocking
with a timeout.  If a connection is available, it's returned immediately.
If the pool still has room for more connections, it will create one if needed.
Otherwise, the call will block until a connection becomes available or
the timeout expires.  In the case of a timeout, it is propogated back up
to the PointsWriter that determine what do return to the client.
2016-01-26 17:08:36 -07:00
Jason Wilder f58f0f5373 Fix cluster tests 2016-01-21 15:28:34 -05:00
Jason Wilder 0926b19e6b Prevent creating points with NaN float values
Float values are not supported in the existing engine and the tsm1
engines.  This changes NewPoint to return an error if a field value
contains a NaN field.  It also allows us to validate fields to prevent
other unsupported types from sneaking in through other input plugins.
2015-10-27 17:12:52 -06:00
Cory LaNou d19a510ad2 refactor Points and Rows to dedicated packages 2015-09-16 15:33:08 -05:00
Philip O'Toole a19cea36bd Rename cluster unit test function
Makes future tests, related to shard mapping, clearer.
2015-07-17 13:05:15 -07:00
Jason Wilder eb1cd44b8d Log write errors
Since the client only receives a "write failed" or "partial write" error
message, log more context in the logs.
2015-06-09 14:49:22 -06:00
Jason Wilder 1024965db7 Create shard received from cluster writer 2015-06-05 22:16:51 -06:00
Ben Johnson fb06549552 remove bind address from cluster config 2015-06-05 17:07:54 -06:00
Ben Johnson 5a5c077790 refactor cluster to use mux 2015-06-05 16:54:12 -06:00
Jason Wilder 3957e096f8 Remove ownerID from protobufs
Not needed since the node that processes the request is the owner.
2015-06-02 14:45:52 -06:00
Ben Johnson bf823d9887 Integrating cmd/influxd/run. 2015-05-30 14:06:36 -06:00
Ben Johnson c916256ac9 Rename cluster.Writer to cluster.ShardWriter. 2015-05-30 14:05:27 -06:00