Commit Graph

9121 Commits (372302bcbd61cb83a4e844c7dc29473afcc0463d)

Author SHA1 Message Date
Jason Wilder 372302bcbd Reduce lock contention in Cache.WriteMulti
A write-lock was taken the whole time, but we only need the write
lock at the end.
2016-01-25 16:48:34 -07:00
Jason Wilder 5bee8880db Reduce lock content in engine.WritePoints
Writing the snapshot would deduplicate the snapshot points
while still holding the engine write-lock.  This can be expensive
under high load and cause writes to back up and OOM the server.

Instead, grab the snapshot under the lock and dedup it after releasing
the lock.

Possible fix for #5442
2016-01-25 15:37:34 -07:00
Jason Wilder 9d3c9329a6 Merge pull request #5438 from influxdata/jw-deadlock
Remove double read-lock in meta client
2016-01-24 23:38:56 -07:00
Jason Wilder ac0c593d8d Prevent double-read locking meta client
Possible fix for #5437.  meta.Client.RetentionPolicy acquired a read-lock and
then called Database which called data() which acquired a read-lock again.
If a write lock was taken between these two read-locks (likely by Authenticate),
the write-lock would block, and the second read-lock would also block
causing a dead-lock.
2016-01-24 22:01:09 -07:00
Jason Wilder ca06755422 Fix merge breakage 2016-01-24 22:00:51 -07:00
David Norton 58e0eed9cb Merge pull request #5403 from influxdata/meta-service2
refactor meta into separate meta client & service
2016-01-22 20:06:51 -05:00
Jason Wilder e5ac1b7464 Merge pull request #5428 from influxdata/jw-ms-config
Fixup default hostname and config
2016-01-22 17:51:10 -07:00
Jason Wilder 1696db1c40 Fixup default hostname and config 2016-01-22 17:05:25 -07:00
David Norton c0df09d544 make sure there are CQs before acquiring lease 2016-01-22 17:01:55 -05:00
Jonathan A. Sternberg 865b267eb0 Merge pull request #5368 from influxdata/js-5286-cq-every-greater-than-interval
Teach the CQ runner how to deal with a resample interval higher than the query interval
2016-01-22 10:09:24 -05:00
Jonathan A. Sternberg 1429f4b4ea Teach the CQ runner how to deal with a resample interval higher than the query interval
Previously if you issued a CQ with a resample interval higher than the
query interval, such as the following:

    CREATE CONTINUOUS QUERY cq ON db
        RESAMPLE EVERY 4m
        BEGIN
            SELECT mean(value) INTO cpu_mean FROM cpu GROUP BY time(2m)
        END

This would result in strange behavior because the FOR value defaulted to
the GROUP BY interval and the minimum time passing before a CQ ran was
also the resample interval, so it wouldn't run the appropriate intervals
even if you set the resample duration to a higher value.

This tweaks the CQ runner to set the minimum interval before a bucket
becomes capable of running to the lower of the query interval or the
resample interval instead of always using the resample interval.

It also sets the default resample duration to be the higher value of the
query interval or the resample interval so the above query gets a
default of 4m instead of 2m and will execute 2 queries every 4 minutes.

If you manually set the resample duration to a lower value than the
resample interval, the old behavior will still happen and should be
considered an error.

This also makes trying to create a continuous query with a resample
duration of below the resample interval or query interval (whichever is
higher) as an error returned by the parser.

Fixes #5286.
2016-01-22 09:43:46 -05:00
David Norton 914a9a1de6 fix build after rebase 2016-01-21 15:38:13 -05:00
Jason Wilder 8456169855 Update backup/restore to use MetaClient 2016-01-21 15:32:35 -05:00
David Norton 2e8cfce7be convert CQ service to new meta client 2016-01-21 15:32:03 -05:00
Cory LaNou ef596c6b6b skip backup/restore test until we update the code 2016-01-21 15:32:03 -05:00
Cory LaNou 15314111cb buildable test suite 2016-01-21 15:31:27 -05:00
David Norton 19c79af840 fix build break 2016-01-21 15:31:27 -05:00
David Norton 177ce33b02 fix build break 2016-01-21 15:31:27 -05:00
Jason Wilder 19feed1d13 Fix go vet warnings 2016-01-21 15:30:44 -05:00
Jason Wilder b7e1ecf471 Close idle MetaClient http connections
Prevents too many open files when running tests
2016-01-21 15:30:09 -05:00
Jason Wilder ad52d0fbd9 Fix tests 2016-01-21 15:30:09 -05:00
Jason Wilder ef524b0539 Fix monitor tests 2016-01-21 15:28:34 -05:00
Jason Wilder f58f0f5373 Fix cluster tests 2016-01-21 15:28:34 -05:00
Jason Wilder bbe13c2818 Ensure HTTP and TCP bind addresses have hostnames
If a bind-address of :8088 is used, cluster nodes cannot
connect to those nodes because there is no hostname portion
of the address.  When we see a bind-address without a hostname,
use the os hostname or localhost if that fails if it is not specified
in the config already.
2016-01-21 15:28:34 -05:00
Jason Wilder e901b648a6 Use TCPHost for writing and query to other nodes 2016-01-21 15:28:34 -05:00
David Norton 58c4474f53 retry AcquireLease on error 2016-01-21 15:28:34 -05:00
Ross McDonald 83cca53e9e Removed references to restore from influxd's main.go, which was causing compile issues. 2016-01-21 15:28:34 -05:00
Cory LaNou 9fd651277b use local logger 2016-01-21 15:28:34 -05:00
Cory LaNou e36eaa0378 fix vet warnings 2016-01-21 15:28:34 -05:00
Paul Dix c99b214e87 Fix retention policy meta client interface 2016-01-21 15:28:34 -05:00
David Norton a2057415d8 meta lease retry 2016-01-21 15:28:34 -05:00
David Norton 38f09178ef add leases to meta service and client 2016-01-21 15:28:33 -05:00
Paul Dix 0341bc3532 Update meta client and retention service.
* Remove VisitRetentionPolicies from meta client.
* Update retention enforcer to run on every data node.
2016-01-21 15:28:33 -05:00
Paul Dix 70de1a7690 Update meta service/client and shard precreator.
* Wire up DataNode(id uint64).
* Remove IsLeader test on precreator.
* Clean up error in client if the server returns a non-200 on get snapshot.
2016-01-21 15:28:33 -05:00
Paul Dix 9ea8ff357e Wire up meta service and client delete data node 2016-01-21 15:28:33 -05:00
Paul Dix 7b71b66e31 Update meta service, meta client, and httpd handler
* Improve the ping endpoint so that it can optionally check for leader agreement across all meta servers
* Add Ping method to the meta client
* Fix ClusterID tests
* Remove WaitForLeader from meta client and remove unnecessary references to it
2016-01-21 15:28:33 -05:00
Paul Dix 101f93f1db Add meta service test to ensure cluster id persisted 2016-01-21 15:28:33 -05:00
Paul Dix 2f07fe88ca Update meta client to use data method to protect cache 2016-01-21 15:28:33 -05:00
Paul Dix 101ab32571 Fix meta-service for server integration tests
* Updated CreateShardGroup to not return an error if it already exists so it's idempotent
* Removed old test making sure you can't delete the default RP. You can delete it now, there was no reason to disallow it.
* Wired up the UpdateRetentionPolicy functionality
2016-01-21 15:28:33 -05:00
Cory LaNou 2715d5ef72 add clusterID and tests 2016-01-21 15:28:33 -05:00
Paul Dix 68f33c93ce Finish wiring up meta service to server 2016-01-21 15:28:33 -05:00
Paul Dix fb9181d240 Fix meta-service build 2016-01-21 15:28:33 -05:00
Paul Dix bfcf5d63ce Clean up meta service close. 2016-01-21 15:28:33 -05:00
Paul Dix 88cf27e63b Remove debugging print lines. 2016-01-21 15:28:33 -05:00
David Norton f23fea81b3 take rlock and grab ref to data 2016-01-21 15:28:33 -05:00
David Norton d1fcf1f7a1 wire up meta client shard methods 2016-01-21 15:28:33 -05:00
Paul Dix f385945058 Update Server to work with new metaservice/client 2016-01-21 15:28:33 -05:00
Cory LaNou d0cad8a022 add subscription meta client test 2016-01-21 15:28:33 -05:00
Cory LaNou 853f4bf70e add continuous query meta client tests 2016-01-21 15:28:33 -05:00
Cory LaNou a41222befb add continuous query/subscription methods to meta client 2016-01-21 15:28:33 -05:00