Commit Graph

624 Commits (fd840f242c7a3afc90e9198d68b4a97225f78299)

Author SHA1 Message Date
Joe LeGasse dc8ed7953d Remove custom binary-conversion functions
Also cleaned up some excess allocations, and other cruft from the code
2016-02-18 13:56:35 -05:00
bastard a5cbc43afd Merge remote-tracking branch 'upstream/master' into conjoined-field-names-for-graphite-templates 2016-02-18 00:10:11 +00:00
Jason Wilder b4490788a0 Merge pull request #5512 from influxdata/ga-remove-json-write
Remove JSON write path
2016-02-17 16:52:22 -07:00
Mark Rushakoff 759538da2a Add httpd stats for durations, num active requests 2016-02-17 15:38:55 -08:00
Ben Johnson e3b4b71c13 refactor query executor
This commit moves the `QueryExecutor` to the `cluster` package
and provides an interface to it inside the `influxql` package.
2016-02-17 15:13:56 -07:00
Edd Robinson f315444f86 Merge pull request #5255 from titilambert/github4752
Workaround for #4752
2016-02-17 11:00:58 +00:00
gunnaraasen 426bc0aa3e Move admin UI assets to the admin service directory 2016-02-16 17:12:39 -08:00
gunnaraasen a7a8258b3e Add config option to enable JSON write path 2016-02-16 16:29:25 -08:00
Jonathan A. Sternberg 23da067593 Fix meta.Client CreateDatabaseWithRetentionPolicy RPC command
Previously, meta.Client would drop the default retention policy when
trying to create a database with a retention policy. The RPC has now
been modified to include the desired retention policy in the
CreateDatabase command and have it use that retention policy information
instead of the default configuration when provided.

This also lowers the number of RPC calls for
CreateDatabaseWithRetentionPolicy to only a single RPC call instead of
two.

Protections have also been included so creating a retention policy with
different parameters will return an error similar to if you tried to
modify the retention policy separately.

Fixes #5696.
2016-02-16 16:21:26 -05:00
Jason Wilder 8115cdf5bf Merge pull request #5682 from influxdata/mr-separate-diagnostics-package
Move monitor.Diagnostics to its own package
2016-02-16 09:38:15 -07:00
Ross McDonald 6bb1097d6b Merge pull request #5638 from influxdata/ross-http-version-handler
Added test coverage to httpd handler version header
2016-02-16 10:05:41 -06:00
Nicholas Katsaros a727266744 Add graphite and udp services to the default config generator
Fix #4752
2016-02-16 10:12:48 -05:00
Ross McDonald a724c90477 Changed body to io.Reader. 2016-02-16 09:01:09 -06:00
Jason Wilder 6fb00c1a9b Remove MetaServers from node.json
This removes the MetaServers property from node.json to eliminate one
of the four places those addresses are stored on disk.  We always use
the values that come through the config (via file, env var or -join arg).
2016-02-15 22:42:51 -07:00
Chris H (CruftMaster) 53cccd02be Merge remote-tracking branch 'upstream/master' into conjoined-field-names-for-graphite-templates 2016-02-14 22:45:15 +00:00
Chris H (CruftMaster) 7de2c7f233 Switching to mixedCaps 2016-02-14 22:36:02 +00:00
Mark Rushakoff e1af9fe596 Deregister diagnostics client in service.Close
The hinted handoff and graphite services called out
DeregisterDiagnosticsClient in the inlined Monitor interface but never
called it.
2016-02-13 15:04:05 -08:00
Mark Rushakoff 5a61bdad70 Move monitor.Diagnostics to its own package
I was trying to create a Diagnostics Client in the tsdb package, but
IIRC importing `monitor` caused an import cycle of:
tsdb -> monitor -> cluster -> tsdb.

Moving Diagnostics to its own package will allow further use of
diagnostics.Client without running into import cycles.
2016-02-13 14:56:15 -08:00
Jason Wilder b45b2c99d1 Use ClusterTracing config to control meta handler logging
Meta HTTP commands are cluster level requests and were showing up in
the main log creating a lot of noise.  Switch them to use the ClusterTracing
config option which is disabled by default.
2016-02-12 16:40:10 -07:00
Jason Wilder 24dd5ef0ee Fix go vet 2016-02-12 13:55:41 -07:00
Ross McDonald f1a5201e67 Condensed check for version header into one line. 2016-02-12 14:32:07 -06:00
Jason Wilder cd568548fd Fix race in peerStore 2016-02-12 12:06:23 -07:00
Cory LaNou 7ad31fa6ab address pr feedback 2016-02-12 11:35:20 -07:00
Jason Wilder ddcfac7e8e Remove peers.json
No longer needed now that peers are pull from the meta nodes.
2016-02-12 11:35:20 -07:00
Cory LaNou 1b25c0cb80 ask for a free port immediatly before using to prevent it being returned to the available pool 2016-02-12 11:35:19 -07:00
Cory LaNou 52077b2dfc fix race condition 2016-02-12 11:35:19 -07:00
Cory LaNou e9a2c33556 give less time to lose lease on random port for test 2016-02-12 11:35:19 -07:00
Cory LaNou df5d587105 make meta test suite less racy 2016-02-12 11:35:19 -07:00
Cory LaNou 360f40561e misc fixes and changelog 2016-02-12 11:35:19 -07:00
Cory LaNou 92e8516660 specify raft bind address with real random ports 2016-02-12 11:35:19 -07:00
Cory LaNou 7e62201793 specify bind address meta test 2016-02-12 11:35:19 -07:00
Cory LaNou b17293f75c fix data race 2016-02-12 11:35:19 -07:00
Cory LaNou f861d5811e fix adhoc joining of cluster 2016-02-12 11:35:19 -07:00
Cory LaNou 807354f195 passing test suite... hopefully 2016-02-12 11:32:53 -07:00
Cory LaNou d9f1df0ecf sane cluster starting with join args 2016-02-12 11:32:53 -07:00
Jason Wilder e1effa6601 fix build after rebase on master 2016-02-12 11:32:53 -07:00
Joe LeGasse 1852814612 Update authentication cache to work for clusters
Also removed unused code and duplicated test code
2016-02-12 07:16:57 -05:00
Ross McDonald e38b3e799c Added test to http handler for checking the version header information. 2016-02-11 13:09:39 -06:00
Todd Persen cb1b1859fe Fix conflicts and split out client/server version. 2016-02-10 22:30:19 -08:00
Edd Robinson 7ff67db776 Merge pull request #5613 from influxdata/er-cq-typo
Tidy up README
2016-02-10 23:38:48 +00:00
Ben Johnson d9a6a7340f add canonical paths 2016-02-10 11:30:52 -07:00
Ben Johnson 5a0d1ab7c1 rename influxdb/influxdb to influxdata/influxdb
This commit changes all the import and URL references from:

    github.com/influxdb/influxdb

to:

    github.com/influxdata/influxdb
2016-02-10 10:26:18 -07:00
Chris H (CruftMaster) b3fd0ef737 Rebase to new changes in master 2016-02-10 17:07:34 +00:00
Chris H (CruftMaster) abe29bfaf1 Warning in readme.md 2016-02-10 16:51:05 +00:00
Ben Johnson 5c33b9d786 remove Mapper test references 2016-02-10 09:40:30 -07:00
Ben Johnson 627cd9d486 add dedupe iterator 2016-02-10 09:40:29 -07:00
Ben Johnson 607750ab1b add SHOW MEASUREMENTS iterator 2016-02-10 09:40:28 -07:00
Ben Johnson 00806de9b8 refactor query engine 2016-02-10 09:40:25 -07:00
Ben Johnson cde973f409 refactor query engine 2016-02-10 09:40:24 -07:00
Chris H (CruftMaster) cbc9bb5718 Stop people from using both measurement* and field* in a template (plus tests) 2016-02-10 15:29:09 +00:00
Chris H (CruftMaster) 0bdceb33c7 Switching to t.separator 2016-02-10 15:27:30 +00:00
Chris H (CruftMaster) 7bdcfbfa41 Added some README 2016-02-10 15:27:30 +00:00
Chris H (CruftMaster) 2822b89481 Adding conjoined-field-names for graphite templates 2016-02-10 15:27:30 +00:00
Edd Robinson 4e1c3d0a56 Tidy up README 2016-02-10 11:50:42 +00:00
Edd Robinson bcc46171cf Address some lint issues 2016-02-10 08:52:08 +00:00
Edd Robinson cfbb219e49 Fixes #5545 2016-02-09 18:41:26 +00:00
Nic Grayson 3a12b2e392 fixed grammar/typos in collectd section of docs 2016-02-09 10:31:57 -06:00
Jason Wilder 83b96b35e9 Merge pull request #5419 from m4ce/master
Adding support for matching tags multiple times in Graphite parser
2016-02-09 09:23:54 -07:00
Tait Clarridge 63ff0ca488 Fix for lease redirect
Previously, the lease redirect was invalid causing anything relying
on a lease for execution (eg. continuous queries) to cease functioning.

The name/nodeid URL param parsing has been moved up to the top of the
handler so the options can be forwarded on to the real leader.

X-Github-Closes: #5592
2016-02-09 09:48:19 -05:00
Matteo Cerutti b26859a5db adding support for matching tags multiple times 2016-02-09 10:19:37 +01:00
Jason Wilder a9552fdd91 Merge pull request #5565 from tpitale/configure-udp-precision
Configurable precision on UDP services
2016-02-08 15:20:55 -07:00
Jason Wilder 4002733695 Merge pull request #5182 from flisky/master
Graphite: prefer more specific template over default template when possible
2016-02-08 12:08:43 -07:00
Edd Robinson b34699d222 Fix typo from a rename 2016-02-08 16:05:05 +00:00
Tony Pitale 6dd407c819 use go fmt 2016-02-07 10:59:31 -06:00
Tony Pitale 1e4eae4343 Configurable precision on UDP services
* pass configured precision string to point parsing
* add Precision configuration to UDP config
* default configured precision to match what it appears to be now (from ParsePoints)
2016-02-06 13:56:53 -06:00
runner.mei 7c58bf1970 don't panic while addr is invalid 2016-02-05 11:44:16 +08:00
runner.mei 978305a6dd fix unit test of influxd in the windows
fix unit test of influxd in the windows
2016-02-05 11:44:14 +08:00
Jason Wilder 6ff0da39aa Merge pull request #5487 from runner-mei/patch-8
fix cross platform error
2016-02-04 13:46:58 -07:00
David Norton efbac5fce2 fix #5505: clear authCache when pwd changes 2016-02-02 17:51:21 -05:00
Jason Wilder 1c19320e79 Merge pull request #5515 from influxdata/jw-config
Remove top-level dir config option
2016-02-02 13:26:11 -07:00
Jason Wilder 51da6e3d56 Remove top-level dir config option
This breaks backup/restore if it's not "meta" as well as breaks
upgrades from prior releases for some users.
2016-02-02 13:03:44 -07:00
Seif Lotfy 052f8d2a63 Return error if database is not found for setting the retention policy
Instead of returning nil, nil in case of no DB being found return
ErrDatabaseNotExists.
Fixes #5504
2016-02-02 21:02:40 +01:00
Seif Lotfy 343749e02d Add new ErrDatabaseNotExists error 2016-02-02 21:02:40 +01:00
runner 217095ad08 fix cross platform error
fix cross platform error
2016-01-31 11:02:49 +08:00
Jason Wilder 01193668cf Fix nil pointer panic when dropping collectd points
Fixes #5449
2016-01-28 21:36:59 -07:00
Cory LaNou 0f6c75ab7d make tests pass 2016-01-28 11:03:44 -06:00
Cory LaNou 51f6c64134 ake new and upgrade behavior the same for meta/data node numbering 2016-01-28 10:05:56 -06:00
Cory LaNou d70b694d7d fix misc meta statup bugs 2016-01-28 10:05:53 -06:00
Cory LaNou 31c2e7012a allow for node upgrade 2016-01-28 10:05:53 -06:00
Jason Wilder 716714364a Backup node.json with metastore backup 2016-01-27 17:39:54 -07:00
Todd Persen 66e6375973 Move status request metrics to their own label 2016-01-26 18:10:02 -08:00
Todd Persen 06e91dfca1 Remove MetaClient.Ping from `/ping` and move it to `/status` 2016-01-26 17:58:44 -08:00
Jason Wilder ac0c593d8d Prevent double-read locking meta client
Possible fix for #5437.  meta.Client.RetentionPolicy acquired a read-lock and
then called Database which called data() which acquired a read-lock again.
If a write lock was taken between these two read-locks (likely by Authenticate),
the write-lock would block, and the second read-lock would also block
causing a dead-lock.
2016-01-24 22:01:09 -07:00
Jason Wilder ca06755422 Fix merge breakage 2016-01-24 22:00:51 -07:00
David Norton 58e0eed9cb Merge pull request #5403 from influxdata/meta-service2
refactor meta into separate meta client & service
2016-01-22 20:06:51 -05:00
Jason Wilder 1696db1c40 Fixup default hostname and config 2016-01-22 17:05:25 -07:00
David Norton c0df09d544 make sure there are CQs before acquiring lease 2016-01-22 17:01:55 -05:00
Jonathan A. Sternberg 1429f4b4ea Teach the CQ runner how to deal with a resample interval higher than the query interval
Previously if you issued a CQ with a resample interval higher than the
query interval, such as the following:

    CREATE CONTINUOUS QUERY cq ON db
        RESAMPLE EVERY 4m
        BEGIN
            SELECT mean(value) INTO cpu_mean FROM cpu GROUP BY time(2m)
        END

This would result in strange behavior because the FOR value defaulted to
the GROUP BY interval and the minimum time passing before a CQ ran was
also the resample interval, so it wouldn't run the appropriate intervals
even if you set the resample duration to a higher value.

This tweaks the CQ runner to set the minimum interval before a bucket
becomes capable of running to the lower of the query interval or the
resample interval instead of always using the resample interval.

It also sets the default resample duration to be the higher value of the
query interval or the resample interval so the above query gets a
default of 4m instead of 2m and will execute 2 queries every 4 minutes.

If you manually set the resample duration to a lower value than the
resample interval, the old behavior will still happen and should be
considered an error.

This also makes trying to create a continuous query with a resample
duration of below the resample interval or query interval (whichever is
higher) as an error returned by the parser.

Fixes #5286.
2016-01-22 09:43:46 -05:00
Jason Wilder 8456169855 Update backup/restore to use MetaClient 2016-01-21 15:32:35 -05:00
David Norton 2e8cfce7be convert CQ service to new meta client 2016-01-21 15:32:03 -05:00
Cory LaNou 15314111cb buildable test suite 2016-01-21 15:31:27 -05:00
David Norton 177ce33b02 fix build break 2016-01-21 15:31:27 -05:00
Jason Wilder 19feed1d13 Fix go vet warnings 2016-01-21 15:30:44 -05:00
Jason Wilder b7e1ecf471 Close idle MetaClient http connections
Prevents too many open files when running tests
2016-01-21 15:30:09 -05:00
Jason Wilder ad52d0fbd9 Fix tests 2016-01-21 15:30:09 -05:00
Jason Wilder bbe13c2818 Ensure HTTP and TCP bind addresses have hostnames
If a bind-address of :8088 is used, cluster nodes cannot
connect to those nodes because there is no hostname portion
of the address.  When we see a bind-address without a hostname,
use the os hostname or localhost if that fails if it is not specified
in the config already.
2016-01-21 15:28:34 -05:00
David Norton 58c4474f53 retry AcquireLease on error 2016-01-21 15:28:34 -05:00
Cory LaNou 9fd651277b use local logger 2016-01-21 15:28:34 -05:00
Cory LaNou e36eaa0378 fix vet warnings 2016-01-21 15:28:34 -05:00
Paul Dix c99b214e87 Fix retention policy meta client interface 2016-01-21 15:28:34 -05:00
David Norton a2057415d8 meta lease retry 2016-01-21 15:28:34 -05:00
David Norton 38f09178ef add leases to meta service and client 2016-01-21 15:28:33 -05:00
Paul Dix 0341bc3532 Update meta client and retention service.
* Remove VisitRetentionPolicies from meta client.
* Update retention enforcer to run on every data node.
2016-01-21 15:28:33 -05:00
Paul Dix 70de1a7690 Update meta service/client and shard precreator.
* Wire up DataNode(id uint64).
* Remove IsLeader test on precreator.
* Clean up error in client if the server returns a non-200 on get snapshot.
2016-01-21 15:28:33 -05:00
Paul Dix 9ea8ff357e Wire up meta service and client delete data node 2016-01-21 15:28:33 -05:00
Paul Dix 7b71b66e31 Update meta service, meta client, and httpd handler
* Improve the ping endpoint so that it can optionally check for leader agreement across all meta servers
* Add Ping method to the meta client
* Fix ClusterID tests
* Remove WaitForLeader from meta client and remove unnecessary references to it
2016-01-21 15:28:33 -05:00
Paul Dix 101f93f1db Add meta service test to ensure cluster id persisted 2016-01-21 15:28:33 -05:00
Paul Dix 2f07fe88ca Update meta client to use data method to protect cache 2016-01-21 15:28:33 -05:00
Paul Dix 101ab32571 Fix meta-service for server integration tests
* Updated CreateShardGroup to not return an error if it already exists so it's idempotent
* Removed old test making sure you can't delete the default RP. You can delete it now, there was no reason to disallow it.
* Wired up the UpdateRetentionPolicy functionality
2016-01-21 15:28:33 -05:00
Cory LaNou 2715d5ef72 add clusterID and tests 2016-01-21 15:28:33 -05:00
Paul Dix fb9181d240 Fix meta-service build 2016-01-21 15:28:33 -05:00
Paul Dix bfcf5d63ce Clean up meta service close. 2016-01-21 15:28:33 -05:00
David Norton f23fea81b3 take rlock and grab ref to data 2016-01-21 15:28:33 -05:00
David Norton d1fcf1f7a1 wire up meta client shard methods 2016-01-21 15:28:33 -05:00
Paul Dix f385945058 Update Server to work with new metaservice/client 2016-01-21 15:28:33 -05:00
Cory LaNou d0cad8a022 add subscription meta client test 2016-01-21 15:28:33 -05:00
Cory LaNou 853f4bf70e add continuous query meta client tests 2016-01-21 15:28:33 -05:00
Cory LaNou a41222befb add continuous query/subscription methods to meta client 2016-01-21 15:28:33 -05:00
Cory LaNou 7c41c0e02f add user tests for meta client 2016-01-21 15:28:33 -05:00
Paul Dix 9fd9a666bf Add CreateDataNode to meta client/service 2016-01-21 15:28:33 -05:00
Cory LaNou 53042ac56d bringing back client user methods 2016-01-21 15:28:33 -05:00
Paul Dix 13e32f6880 Update close handling on meta service 2016-01-21 15:28:33 -05:00
Paul Dix d2e3cf519c Cleanup host/port in meta service 2016-01-21 15:28:33 -05:00
Paul Dix e906107bea Update meta service to handle host names
This ensures that the meta service will gracefully handle host name changes in a single server configuration.

It also changes the raft setup to use the user specified bind address (and thus hostname) instead of pulling it off the listener, which returns the IP. This will enable users to have hostnames listed instead of IPs in the megastore, making it easier to read. This also means that underlying IPs can change without causing problems in a cluster.
2016-01-21 15:28:32 -05:00
Paul Dix 0f36fbe5ce Add comment to client 2016-01-21 15:28:32 -05:00
Paul Dix 1632980eb8 Cleanup PrintLns in meta client 2016-01-21 15:28:32 -05:00
Paul Dix eda4a6eda0 Wire up meta service and client recovery.
* increase sleep on error in client exec in case a server went down so we don't max out retries before a new leader gets elected
* update and add close logic to service, handler, raft state, and the client
2016-01-21 15:28:32 -05:00
David Norton 5c20e16406 wire up some RP stuff in meta client / service 2016-01-21 15:28:32 -05:00
David Norton c84e9b38d0 fix unit tests after backing out proto change 2016-01-21 15:28:32 -05:00
David Norton f91fd0b8ae back out proto struct changes 2016-01-21 15:28:32 -05:00
Paul Dix e9e63b573b Cycle to next server on failure in meta client 2016-01-21 15:28:32 -05:00
Paul Dix 1e63fa4e2c Enforce max retries on meta client 2016-01-21 15:28:32 -05:00
Paul Dix 90a08154c5 Wire up redirects to execute against raft leader 2016-01-21 15:28:32 -05:00
David Norton c7721c8948 don't clone database infos in client 2016-01-21 15:28:32 -05:00
David Norton 6561b702b8 remove commented out test code 2016-01-21 15:28:32 -05:00
David Norton f80f860ee5 temporarily rename statement_executor_test.go 2016-01-21 15:28:32 -05:00
David Norton 1d6878c37c wire up some meta client funcs and tests 2016-01-21 15:28:32 -05:00
Paul Dix c9d82ad0ad Wire up meta service functionality
* Add dir, hostname, and bind address to top level config since it applies to services other than meta
* Add enabled flags to example toml for data and meta services
* Wire up add/remove raft peers and meta servers to meta service
* Update DROP SERVER to be either DROP META SERVER or DROP DATA SERVER
* Bring over statement executor from old meta package
* Start meta service client implementation
* Update meta service test to use the client
* Wire up node ID/meta server storage information
2016-01-21 15:28:32 -05:00
David Norton 688bc7a2f1 fix go vet error 2016-01-21 15:28:32 -05:00
Cory LaNou d69c5f853f set store peers when starting up from config 2016-01-21 15:28:32 -05:00
David Norton 79d81a2448 add meta service tests & bug fixes 2016-01-21 15:28:32 -05:00
Cory LaNou 9ec7a710c9 some misc refactoring on influxd startup 2016-01-21 15:28:32 -05:00
Cory LaNou 8d878fff91 buildable meta -> services/meta 2016-01-21 15:28:32 -05:00
David Norton bf0b477a0b set raftState on the store 2016-01-21 15:28:32 -05:00
Cory LaNou d3ab0b5ae6 buildable again. lot of wip 2016-01-21 15:28:32 -05:00
Cory LaNou b0d0668138 wip 2016-01-21 15:28:32 -05:00
David Norton 94b05404dc remove cors from handler 2016-01-21 15:28:31 -05:00
David Norton 169c6a5dfa store and handler to interface 2016-01-21 15:28:31 -05:00
David Norton 9f93f0b84a convert to AfterIndex 2016-01-21 15:28:31 -05:00
David Norton 05da43d9f6 rough out meta service 2016-01-21 15:28:31 -05:00