Commit Graph

5676 Commits (51dbb171ee46dc6f349111d4300b0259444e89b7)

Author SHA1 Message Date
Cory LaNou 51dbb171ee rearrange some tests, add some timeout testing 2015-05-27 15:13:31 -06:00
Cory LaNou 1ac46b56b8 adding timeouts and deadlines 2015-05-27 14:40:19 -06:00
Cory LaNou 372cb28023 Merge pull request #2661 from influxdb/tcp-connection-pool
ClusterWriter/ClusterListener with connection pooling.
2015-05-27 13:36:37 -06:00
Cory LaNou 5c1d407d5e pool map key is now nodeID, always get a fresh nodeInfo when dialing 2015-05-27 13:18:03 -06:00
Cory LaNou b699938bdb make the cluster listener a Opener 2015-05-27 10:30:52 -06:00
Cory LaNou 4da0e9a93c close client pool 2015-05-27 10:06:04 -06:00
Cory LaNou 1228de4e7c move tcp to cluster 2015-05-27 10:02:38 -06:00
Cory LaNou 3dc688cff2 check shutdown properly 2015-05-27 08:32:04 -06:00
Cory LaNou 00ce3e5cfc nil out channel so we can check that if needed 2015-05-27 08:32:04 -06:00
Cory LaNou 31593e5083 refactoring and deadlock fix 2015-05-27 08:32:04 -06:00
Cory LaNou ee0a67ae36 fix package naming 2015-05-27 08:32:03 -06:00
Cory LaNou 6346ab55e7 first pass at tcp connection pooling 2015-05-27 08:32:03 -06:00
Jason Wilder 5bc7e9815b Merge pull request #2665 from influxdb/jw-shard-store
Shard storage management
2015-05-26 18:23:01 -06:00
Jason Wilder 85f59d696b Create and open shards on-demand
Uses a structure like:

  /root/
      /db1/rp1/1
              /2
      /db2/rp2/3

If a write is assigned to a shard on the local node but the shard
has not been created, create it when the write returns an error
and retry the write.
2015-05-26 16:38:45 -06:00
Jason Wilder 7ace10f7e6 Load shards from filesystem when tsdb.Store is opened 2015-05-26 15:43:46 -06:00
Jason Wilder cb23a7a297 Fix breakage from rename of data to cluster package 2015-05-26 15:42:44 -06:00
Paul Dix 99c0b58473 Merge pull request #2658 from influxdb/jw-write-path
Wire up new write path
2015-05-26 15:57:14 -04:00
Paul Dix 01618dc143 Move data.Node to tsdb.Store. Move data to cluster. 2015-05-26 15:56:54 -04:00
Jason Wilder 21bfb150a1 Wire up new write path
This allows the new write path to be hooked up if you start the
server with `INFLUXDB_ALPHA1=1`.  When set, writes will go though
the coordinator and be stubbed out to write to a single local data
node with one shards.  The write will be logged and written to
disk .

The env var is used so that the current write path is not completely
broken which would break many of the tests that depend on writes.

Note that queries are not currently working w/ the this change.
2015-05-26 12:07:56 -06:00
Ben Johnson 79ce57ce30 Merge pull request #2654 from influxdb/meta
Implement meta.Store and meta.Data.
2015-05-25 16:32:15 -06:00
Ben Johnson 7258a9be3c Implement meta.Store and meta.Data. 2015-05-25 16:28:58 -06:00
Paul Dix f861159e8e Merge pull request #2646 from influxdb/shared-shard-index
Shared metadata index
2015-05-24 07:40:07 -04:00
Paul Dix 6c80108f63 Change Database to DatabaseIndex, remove leftover warn statement 2015-05-24 07:39:45 -04:00
Paul Dix c3ab88a715 Make the metadata index shared across shards while keeping field types and encoding local to each shard. 2015-05-23 18:06:07 -04:00
Jason Wilder e30e7f3c16 Fix broken merge
tcp client/server was added while Point was moved and converted
to an interface in two other PRs.
2015-05-22 22:23:01 -06:00
Jason Wilder 2a727dc8ce Merge pull request #2641 from influxdb/jw-points
Convert Points to interface
2015-05-22 22:08:32 -06:00
Cory LaNou 7a645698e2 Merge pull request #2632 from influxdb/tcp-client-server
first pass at tcp client/server with proto buffs
2015-05-22 16:21:32 -06:00
Cory LaNou 54bcc0d331 WriteShardRequest -> WriteShard 2015-05-22 16:19:29 -06:00
Jason Wilder 1076153a00 Convert Point to interface
Should be possible to replace the implementation with a more
optimized version now.
2015-05-22 15:39:55 -06:00
Cory LaNou afbb6e28bb test ErrBindAddressRequired 2015-05-22 15:33:37 -06:00
Cory LaNou 1098a8aea1 more refactoring 2015-05-22 15:28:22 -06:00
Jason Wilder 528f47e093 Convert Point.Fields to Point.Fields() 2015-05-22 15:22:03 -06:00
Jason Wilder f8d599cda9 Convert Point.Tags to Point.Tags() 2015-05-22 15:12:34 -06:00
Cory LaNou 742b9de3c2 refactor writeShardRequest 2015-05-22 15:12:07 -06:00
Cory LaNou cf02c342d9 make some methods private 2015-05-22 15:08:28 -06:00
Cory LaNou 888b2cbecb clarify error comment 2015-05-22 15:01:27 -06:00
Jason Wilder 5dcab443dc Move data.Point to tsdb.Point 2015-05-22 15:00:51 -06:00
Cory LaNou e0f13cb44c removing unused error 2015-05-22 14:58:34 -06:00
Cory LaNou 0b6f83871c acking with proper success/error messages 2015-05-22 14:53:42 -06:00
Jason Wilder d633d7683f Convert Point.Name to Point.Name()
Hiding state behind interface.
2015-05-22 14:39:33 -06:00
Jason Wilder 997020963a Convert Point.Time to Point.Time()
Make public state private so that Point can be converted to an
interface.
2015-05-22 14:39:15 -06:00
Paul Dix 8ea9da2703 Merge pull request #2638 from influxdb/metadata-in-shard
Metadata in shard
2015-05-22 16:34:44 -04:00
Paul Dix ae56dbb77a Fix rebase issues. 2015-05-22 16:15:36 -04:00
Paul Dix 8f937cae87 Initial implementation for writing data to a shard. 2015-05-22 16:11:18 -04:00
Jason Wilder a6ae533572 Allow coordinator to map points to multiple shards
Given a WritePointsRequest, MapShards will return a map
of shard ID to points.  This map can then be used to write
each point to the respective shard owners.
2015-05-22 16:11:18 -04:00
Cory LaNou 7ae8c76ff4 unskip tests 2015-05-21 16:38:58 -06:00
Cory LaNou 02cbf09afd first pass at tcp client/server with proto buffs 2015-05-21 16:26:20 -06:00
Jason Wilder 2bc299d808 Merge pull request #2628 from influxdb/jw-binary-write
Shard write protobufs
2015-05-21 11:44:11 -06:00
Jason Wilder da74f03104 Update CONTRIBUTING.md with protobuf setup 2015-05-21 11:29:34 -06:00
Jason Wilder b7b909798b Use go generate to build protobufs 2015-05-21 11:12:58 -06:00