Commit Graph

385 Commits (c9d82ad0adf32eb321ab1c25755a43a2a035a40d)

Author SHA1 Message Date
Paul Dix c9d82ad0ad Wire up meta service functionality
* Add dir, hostname, and bind address to top level config since it applies to services other than meta
* Add enabled flags to example toml for data and meta services
* Wire up add/remove raft peers and meta servers to meta service
* Update DROP SERVER to be either DROP META SERVER or DROP DATA SERVER
* Bring over statement executor from old meta package
* Start meta service client implementation
* Update meta service test to use the client
* Wire up node ID/meta server storage information
2016-01-21 15:28:32 -05:00
David Norton 688bc7a2f1 fix go vet error 2016-01-21 15:28:32 -05:00
Cory LaNou d69c5f853f set store peers when starting up from config 2016-01-21 15:28:32 -05:00
David Norton 79d81a2448 add meta service tests & bug fixes 2016-01-21 15:28:32 -05:00
Cory LaNou 9ec7a710c9 some misc refactoring on influxd startup 2016-01-21 15:28:32 -05:00
Cory LaNou 8d878fff91 buildable meta -> services/meta 2016-01-21 15:28:32 -05:00
David Norton bf0b477a0b set raftState on the store 2016-01-21 15:28:32 -05:00
Cory LaNou d3ab0b5ae6 buildable again. lot of wip 2016-01-21 15:28:32 -05:00
Cory LaNou b0d0668138 wip 2016-01-21 15:28:32 -05:00
David Norton 94b05404dc remove cors from handler 2016-01-21 15:28:31 -05:00
David Norton 169c6a5dfa store and handler to interface 2016-01-21 15:28:31 -05:00
David Norton 9f93f0b84a convert to AfterIndex 2016-01-21 15:28:31 -05:00
David Norton 05da43d9f6 rough out meta service 2016-01-21 15:28:31 -05:00
Paul Dix 59fbd371fc Implement backup/restore for TSM.
This changes backup and restore to work for TSM. It breaks it for b1 and bz1, but since those are getting removed it's ok.

The backup runs against any host that is specified and can backup either the metasstore, a database, specific retention policy, or a specific shard. It can also take incremental backups with the `since` flag, which will only backup TSM files that have been created since that timestamp.

The backup is safe to run online. However, for shards that are still hot for writes, they won't be able to create new TSM files while the backup for that single shard runs. If the backup isn't too large and the write throughput isn't too high this shouldn't be a problem since the writes will just go into the WAL cache.
2015-12-30 18:06:50 -05:00
Jonathan A. Sternberg 5d4ecf853c Add continuous query option for customizing resampling
This makes the following syntax possible:

    CREATE CONTINUOUS QUERY mycq ON mydb
        RESAMPLE EVERY 1m FOR 1h
        BEGIN
          SELECT mean(value) INTO cpu_mean FROM cpu GROUP BY time(5m)
        END

The RESAMPLE option customizes how often an interval will be sampled and
the duration. The interval is customized with EVERY. Any intervals
within the resampling duration on a multiple of the resample interval
will be updated with the new results from the query.

The duration is customized with FOR. This determines how long an
interval will participate in resampling.

Both options are optional. If RESAMPLE is in the syntax, at least one of
the two needs to be given. The default for both is the interval of the
continuous query.

The service also improves tracking of the last run time and the logic of
when a query for an interval should be run. When determining the oldest
interval to run for a query, the continuous query service determines
what would have been the optimal time to perform the next query based on
the last run time. It then uses this time to determine the oldest
interval that should be run using the resample duration and will
resample all intervals between this time and the current time as opposed
to potentially forgetting about the last run in an interval if the
continuous query service gets delayed for some reason.

This removes the previous config options for customizing continuous
queries since they are no longer relevant and adds a new option of
customizing the run interval. The run interval determines how often the
continuous query service polls for when it should execute a query. This
option defaults to 1s, but can be set to 1m if the least common factor
of all continuous queries' intervals is a higher value (like 1m).
2015-12-28 16:43:49 -05:00
Sebastian Borza d885f4156b removing golang reference 2015-12-22 12:54:43 -05:00
Sebastian Borza 3bf5d9c969 moving UDP payload size to config option, improve throughput performance
moving UDP payload size to optional config choice to imporove throughput performance
2015-12-22 09:14:50 -05:00
Nathaniel Cook 8214676363 fix panic when merging empty series 2015-12-15 11:32:51 -07:00
Tim Raymond 718fb92cf5 Remove registration, add 'usage-client'
Server registration and stats reporting has been removed from what was
once http://enterprise.influxdata.com. The app that lived there, now
runs at http://usage.influxdata.com, so that the subdomain can
eventually be repurposed. Because we also want to repurpose the
`enterprise-client` repo, we have also renamed that to `usage-client`.
InfluxDB no longer needs the `registration` service now, since all of
the endpoints it communicates with simply discard the data provided to
them.
2015-12-10 15:58:34 -05:00
Nathaniel Cook eb080785d4 add bounds checking for shard group timestamps
enforce strict valid time ranges on points

small
2015-12-09 13:07:13 -07:00
Philip O'Toole a5cb913035 Drop UDP point on bad parse and keep going
Fixes issue #4992.
2015-12-07 19:47:59 -08:00
Cory LaNou cf4e901078 silence registration logging for testing 2015-12-03 08:04:10 -06:00
Paul Dix 6e1d0f47d4 Merge pull request #4921 from pires/3170-db_not_found
Handler errors should be formatted as JSON.
2015-12-02 11:36:53 -05:00
Cory LaNou 3cd8056664 Merge pull request #4876 from e-dard/lint
Lint
2015-12-02 08:55:47 -06:00
Edd Robinson 8384ba3e84 Update comments. 2015-12-02 12:35:58 +00:00
Paulo Pires 4587a5387d Handler errors should be formatted as JSON. Fixes #3170 2015-11-27 21:12:05 +00:00
Cory LaNou be488b7d12 implement close notifier and timeout on executors 2015-11-24 21:07:18 -06:00
Данило Глинський (Danylo Hlynskyi) 8ba35c3b8c fix collectd manual typo 2015-11-23 18:54:47 +02:00
Edd Robinson ffbd6037e2 Initial lint for all packages under services. Supports #4098 2015-11-22 19:23:56 +00:00
Philip O'Toole b1c5a36b0d Merge pull request #4846 from jsternberg/feature/allow-nan-in-graphite-service
Allow NaN as a valid value on the graphite service
2015-11-20 08:09:51 -08:00
Jonathan A. Sternberg 5fa36639db Allow NaN as a valid value on the graphite service
The canonical graphite implementation will read and discard NaN values
instead of throwing an error when reading on the line receiver protocol.
Since this is the default behavior for graphite, InfluxDB should have
the same behavior for compatibility.

Previously, a NaN value would result in an error printed to the console.
When you have a large number of NaN values being sent every minute, this
results in the log file filling with useless messages.
2015-11-20 11:07:25 -05:00
Philip O'Toole d86a02eeab Fix minor typo in variable name 2015-11-20 07:48:42 -08:00
Paweł Kowalak 8c2f6eb7e0 Comment additions to services to satisfy golint 2015-11-19 13:25:07 +01:00
Philip O'Toole ba807a268b Merge pull request #4835 from orthogonous/master
Comment additions to conform to golint
2015-11-18 17:47:51 -08:00
orthogonous 65f4060de3 Comment additions to conform to golint
Also 1 slight code change that fixed an if statement
that golint didn't like.
2015-11-19 11:26:21 +10:00
Philip O'Toole 06cd9b1e4d Merge pull request #4411 from gobwas/master
Update handler.go
2015-11-18 16:12:18 -08:00
Sebastian Borza e17e6394c9 fixing typo for pointsWrittenOK 2015-11-18 14:28:39 -05:00
John Nguyen 63e958f06e fixed export lint issues in services/admin 2015-11-16 17:31:53 +08:00
Philip O'Toole 69803ddc6f Merge pull request #4791 from orthogonous/master
Added some comments to the udp service so golint passes. Ref #4098
2015-11-13 19:10:49 -08:00
Philip O'Toole 7ea9b3e49e Allow openTSDB point errors logging to be disabled 2015-11-13 14:54:54 -08:00
orthogonous 434d06008e Added some comments to the udp service so golint passes. Ref #4098 2015-11-14 08:46:26 +10:00
Philip O'Toole bc1091ec31 Revert "Merge pull request #4684 from nkatsaros/default-config"
This reverts commit 87441584f3, reversing
changes made to a14829bc1d.
2015-11-11 09:29:06 -08:00
Cory LaNou 6ecb62e4d2 Merge pull request #4737 from ch33hau/4283-hh-throws-error-even-if-disabled
Disable HintedHandoff if configuration is not set. #4283
2015-11-10 19:07:27 -06:00
Cameron Sparr 9625953d3e Add UDP OS buffer size recommendations 2015-11-10 11:00:11 -07:00
Cameron Sparr 727b9f6d8b Documentation updates for UDP, graphite, collectd listeners 2015-11-10 10:35:21 -07:00
Cameron Sparr 0a683144fb Update collectd and graphite UDP listeners with perf enhancements
closes #4678
2015-11-10 10:35:21 -07:00
ch33hau 8bfdfbda0b Disable HintedHandoff if configuration is not set. #4283 2015-11-11 01:12:34 +08:00
Philip O'Toole bece8fed2a Better retention enforcement logging
Fixes issue #4727.
2015-11-09 17:22:24 -08:00
Cameron Sparr 78e6979be5 Make parser/writer internal func names consistent 2015-11-06 16:43:35 -07:00
Cameron Sparr 5c28048132 UDP service listener performance enhancements 2015-11-06 15:53:47 -07:00