Commit Graph

90 Commits (c5bdb5af86ea306b8d09ac7aa4246f0e2a02b043)

Author SHA1 Message Date
Ben Johnson 3404386a02 Merge pull request #2236 from influxdb/term-signal
Term signal
2015-04-10 17:02:13 -06:00
Ben Johnson eaf4bfca0a Fix term signal.
This commit changes raft so that term changes are made immediately and
term change signals are made afterward. Previously, election timeouts
were invalidated by incoming term changes which caused an election loop.

Stale term was also fixed and http/pprof was added too.
2015-04-10 13:52:20 -06:00
Jason Wilder a5e180ca31 Merge pull request #2229 from influxdb/jw-run
Close resources when stopping a node
2015-04-09 22:10:50 -06:00
Jason Wilder c22909f984 Shutdown idle http connections when closing client
When closing a node (and client) in the tests, idle connections
remain even after the client is closed.
2015-04-09 20:51:46 -06:00
Jason Wilder cb51ace768 Avoid spurious logging when stopping streamer
If the client is closed, we'd always log a reconnect error even
when trying to close.
2015-04-09 20:45:02 -06:00
Ben Johnson 47fa90fae2 Remove zero-length data panic from messaging.Conn.
This check is no longer necessary as the checksum added
to the messaging.Message will catch any data errors.
2015-04-09 10:53:43 -06:00
Ben Johnson cc83f2c39b Add checksum to message encoding
This commit changes the binary format of messaging.Message to encode
a 4-byte checksum at the beginning of it. This is used when reading
data back out to verify that it is not corrupt.

Corrupted messages are truncated on recovery so the broker can
restart from the previous message.
2015-04-07 14:24:22 -06:00
Ben Johnson 7aad1a5820 Remove dead code in messaging. 2015-04-07 12:36:07 -06:00
Jason Wilder 0f1fb3a5c3 Fix race on Topic.indexByUrl
applySetTopicMaxIndex() was updating the topics.indexByUrl w/o locking it.

WARNING: DATA RACE
Write by goroutine 1365:
  runtime.mapassign1()
      /usr/local/go/src/runtime/hashmap.go:376 +0x0
  github.com/influxdb/influxdb/messaging.(*Broker).applySetTopicMaxIndex()
      /home/ubuntu/.go_project/src/github.com/influxdb/influxdb/messaging/broker.go:496 +0x198
  github.com/influxdb/influxdb/messaging.(*Broker).Apply()
      /home/ubuntu/.go_project/src/github.com/influxdb/influxdb/messaging/broker.go:542 +0x33a
  github.com/influxdb/influxdb.(*Broker).Apply()
      <autogenerated>:1 +0x78
  github.com/influxdb/influxdb/messaging.(*RaftFSM).Apply()
      /home/ubuntu/.go_project/src/github.com/influxdb/influxdb/messaging/broker.go:614 +0x24f
  github.com/influxdb/influxdb/raft.(*Log).applyNextUnappliedEntry()
      /home/ubuntu/.go_project/src/github.com/influxdb/influxdb/raft/log.go:1431 +0x75c
  github.com/influxdb/influxdb/raft.(*Log).applier()
      /home/ubuntu/.go_project/src/github.com/influxdb/influxdb/raft/log.go:1369 +0x18f

Previous read by goroutine 1540:
  runtime.mapiterinit()
      /usr/local/go/src/runtime/hashmap.go:535 +0x0
  github.com/influxdb/influxdb/messaging.(*Topic).DataURLs()
      /home/ubuntu/.go_project/src/github.com/influxdb/influxdb/messaging/broker.go:681 +0x11d
  github.com/influxdb/influxdb/cmd/influxd.(*Handler).serveMetadata()
      /home/ubuntu/.go_project/src/github.com/influxdb/influxdb/cmd/influxd/handler.go:95 +0x3fd
  github.com/influxdb/influxdb/cmd/influxd.(*Handler).ServeHTTP()
      /home/ubuntu/.go_project/src/github.com/influxdb/influxdb/cmd/influxd/handler.go:45 +0x540
  net/http.serverHandler.ServeHTTP()
      /usr/local/go/src/net/http/server.go:1703 +0x1f6
  net/http.(*conn).serve()
      /usr/local/go/src/net/http/server.go:1204 +0x1087
2015-04-06 21:16:19 -06:00
Ben Johnson fd96e245cb Merge branch 'master' of https://github.com/influxdb/influxdb into broker-recovery
Conflicts:
	CHANGELOG.md
2015-04-04 08:09:06 -06:00
Ben Johnson ba3026f400 Broker log recovery.
This pull request adds recovery to the messaging.Topic when opening. If
any partial messages are found then the file is truncated at that point
and started from there. This can occur when ungracefully shutting down
a server. It can leave half written messages at the end of segments.
2015-04-04 08:06:35 -06:00
Jason Wilder 9ee0f6445e Fix broker connect race at startup
When a data node starts up, the broker URLs were not set before
they were actually being used.  The call to client.Open() in
turn triggers the raft streamer and heartbeat which try to connect
to the broker.   If those started before the subsequent client.SetURLs()
call, you would see the following error in the logs at startup:

[messaging] 2015/04/01 11:59:22 reconnecting to broker: url={  <nil>  /messaging/messages index=2&streaming=true&topicID=0 }, err=Get /messaging/messages?index=2&streaming=true&topicID=0: unsupported protocol scheme ""

Fixing this race uncovered another bug where the join urls would be
cleared the first time the broker was started.  In this case, the
join urls should be left alone since they were set properly w/ SetURLs.

Fixes #2152
2015-04-03 21:04:42 -06:00
Jason Wilder 6d4c7e9cd5 Handle broker and data node endpoints regardless of role
This is a pre-requisite for #1934.  When running separate
broker and data nodes, you currently need to know what role
a host is performing.  This complicates cluster setup in
that you must configure separate broker URLs and data node
URLs.

This change allows a broker only node to redirect data nodes endpoints
to a valid data node and a data only node to redirect broker
endpoints to a valid broker.
2015-04-03 21:00:43 -06:00
Todd Persen 82bf75c691 Merge pull request #2150 from runner-mei/connection_refused
Make InfluxDB win32 friendly, fix unit test is failed on the windows and it is for connection refused.
2015-04-03 16:31:51 -07:00
runner.mei e129c53f64 fix unit test is failed on the windows 2015-04-03 13:50:28 +08:00
runner.mei 6a7cb61f6d fix unit test is failed with connection refused on the win32 2015-04-03 13:48:52 +08:00
Jason Wilder 64f6900ce2 Change 1000ms to 1s 2015-04-02 11:27:59 -06:00
Jason Wilder 91fb7e3756 Track data node urls on brokers
This sends data node urls via the broker heartbeat from each data
node.  The urls are tracked on the broker to support simpler
cluster setup as well as distributed queries.
2015-04-02 11:27:53 -06:00
Ben Johnson 057309fc8e Simplify raft snapshotting, entry apply. 2015-03-26 20:32:39 -06:00
Philip O'Toole c461c7e7b4 Remove logging to specific file 2015-03-25 17:41:39 -07:00
Ben Johnson 11c808f55f Add restore and bootstrap.
This commit adds the "influxd restore" command to the CLI. This allows
a snapshot that has been produced by "influxd backup" to be restored
to a config location and the broker and raft directories will be
bootstrapped based on the state of the snapshot.
2015-03-22 15:31:49 -06:00
Ben Johnson 75450466a6 Fix leader id initialization. 2015-03-16 11:27:57 -06:00
Ben Johnson 41d357ac04 Fixes based on code review comments. 2015-03-14 14:07:09 -06:00
Ben Johnson b045ad5d92 Wrap open logic in anonymous functions. 2015-03-14 13:55:19 -06:00
Ben Johnson 96748cb217 Update file permissions. 2015-03-14 13:49:25 -06:00
Ben Johnson 53dbec8232 Add config notifications and increased test coverage. 2015-03-14 13:36:06 -06:00
Ben Johnson 12e8939647 Fix messaging client redirection. 2015-03-12 15:34:20 -06:00
Ben Johnson 7880bc2452 Add zero length data checks. 2015-03-12 12:12:26 -06:00
Ben Johnson 5f6bcf523f Fix broker integration bugs. 2015-03-11 12:00:45 -06:00
Ben Johnson 66115f95ca Merge branch 'master' of https://github.com/influxdb/influxdb into stateless-broker
Conflicts:
	cmd/influxd/run.go
	messaging/broker.go
	messaging/client.go
	messaging/client_test.go
	messaging/intg_test.go
	server.go
	tx_test.go
2015-03-10 16:27:37 -06:00
Ben Johnson 27e9132796 Integrate stateless broker into remaining packages. 2015-03-10 14:53:45 -06:00
Ben Johnson 4160d0b785 Add continuously streaming topic readers. 2015-03-09 22:32:12 -06:00
Ben Johnson 5f5c6ca297 Integrate stateless messaging into influxdb package. 2015-03-09 15:47:41 -06:00
Philip O'Toole c714f147e8 Merge pull request #1875 from influxdb/raft_log_trace
Support control of Raft debug logging
2015-03-09 08:31:06 -07:00
Ben Johnson 9b5aeb1a90 Refactor messaging client/conn. 2015-03-07 23:21:44 -07:00
Ben Johnson ef8658e0fc Continuing stateless broker refactor. 2015-03-08 15:28:43 -06:00
Philip O'Toole 710e0d7de9 Add unit test for client tracking leader URL 2015-03-06 15:19:18 -08:00
Philip O'Toole 96645c7b51 Only use scheme and host of redirected URL 2015-03-06 15:19:16 -08:00
Philip O'Toole fb618cb385 Support control of Raft debug logging 2015-03-06 15:13:33 -08:00
Philip O'Toole 317fe8335a Client should track current broker leader 2015-03-06 13:59:21 -08:00
Joseph Rothrock 9b63eed767 Merge branch 'master' of github.com:influxdb/influxdb into 1753
Conflicts:
	CHANGELOG.md
2015-03-06 12:37:15 -08:00
Joseph Rothrock 771ef1cec3 Don't Panic on Missing Dirs
Issue: 1753
Figure out what's missing, create it, and keep going.
2015-03-06 12:11:02 -08:00
David Norton ac8a062ab0 fix #1866: use RUnlock instead of Unlock 2015-03-06 12:53:19 -05:00
David Norton 8583260bc5 fix #1866: race accessing topic replicas map 2015-03-06 11:14:11 -05:00
Ben Johnson b937f06c1f Implementing stateless broker. 2015-03-05 19:32:20 -07:00
Ben Johnson 1bbf154f2a Removing replicas and subscriptions from broker. 2015-03-01 22:32:20 -07:00
Ben Johnson 85be4e1f6a Merge branch 'master' of https://github.com/influxdb/influxdb into broker-truncation 2015-03-01 09:47:39 -07:00
Ben Johnson 16dbe8ba2f Add Broker.Truncate(). 2015-03-01 07:06:25 -07:00
Cory LaNou ad16b1b1ed disabling test until fixed. Issue #1726 2015-02-25 11:20:37 -07:00
Ben Johnson 39ae8e6406 Add topic segmentation. 2015-02-24 20:32:20 -07:00