Commit Graph

216 Commits (be83e81102c523621c8e4d3ea8c8b228a6afd082)

Author SHA1 Message Date
Ross McDonald d405d81361 Remove cluster section from sample configuration. Adds coordinator
service to sample configuration.
2016-05-24 08:34:08 -05:00
Nathaniel Cook 73076fa0a3 update example config 2016-05-19 10:53:29 -06:00
Jonathan A. Sternberg 5d9eae61b0 Add https-private-key option to httpd config
The HTTPS configuration for the httpd service only had an option to
specify the certificate file and the same file would be used for both
the certificate and private key file (they could be concatenated
together).

This adds an additional option to specify the files differently from
each other while still allowing the previous behavior. If only
`https-certificate` is specified, the httpd service will try to load the
private key from the `https-certificate` file. If a separate
`https-private-key` file is specified, the private key will be loaded
from there instead.

Fixes #1310.
2016-05-18 21:05:57 -04:00
gunnaraasen eec8cc94c7 Support multiple OpenTSDB inputs 2016-04-11 08:47:38 -07:00
gunnaraasen fccd0943a8 Support multiple collectd inputs 2016-04-11 08:47:38 -07:00
Tehmasp Chaudhri 13b74f3a20 Fix config typo 2016-04-04 15:23:42 -06:00
Ben Johnson 6ec50975e5 remove old config options from sample config
This commit removes the entire `hinted-handoff` section and some
`meta` config options from the sample configuration file.

Fixes #6176
2016-03-31 11:46:00 -06:00
Jonathan A. Sternberg ad7480e64b Limit bucket count in selection
Fixes #6078.
2016-03-30 22:57:09 -04:00
Jonathan A. Sternberg 364dce36ab Support chunking in the middle of a series in the emitter
Limit the maximum size of the return value when chunking is not used to
prevent the server from going out of memory.

Fixes #6115.
2016-03-30 14:36:35 -04:00
Mark Rushakoff 7a2adfcc5d Remove unused WAL configuration variables/fields
These were all b1/bz1 settings that no longer have any effect:

- {Default,}MaxWALSize
- {Default,}WALFlushInterval
- {Default,}WALPartitionFlushDelay
- {Default,WAL}ReadySeriesSize
- {Default,WAL}CompactionThreshold
- {Default,WAL}MaxSeriesSize
- {Default,WAL}FlushColdInterval
- {Default,WAL}PartitionSizeThreshold
2016-03-20 13:16:52 -07:00
Ross McDonald e007a09b0c Updated sample configuration for new meta service options. 2016-02-09 09:34:31 -06:00
Jack Zampolin 611090412e Address PR comments 2016-02-05 14:08:24 -08:00
Jack Zampolin 26a878ff56 Fix [[graphite]] config section 2016-02-05 13:44:11 -08:00
Jason Wilder 51da6e3d56 Remove top-level dir config option
This breaks backup/restore if it's not "meta" as well as breaks
upgrades from prior releases for some users.
2016-02-02 13:03:44 -07:00
Paul Dix c9d82ad0ad Wire up meta service functionality
* Add dir, hostname, and bind address to top level config since it applies to services other than meta
* Add enabled flags to example toml for data and meta services
* Wire up add/remove raft peers and meta servers to meta service
* Update DROP SERVER to be either DROP META SERVER or DROP DATA SERVER
* Bring over statement executor from old meta package
* Start meta service client implementation
* Update meta service test to use the client
* Wire up node ID/meta server storage information
2016-01-21 15:28:32 -05:00
Jason Wilder 15d723dc77 Change default engine to tsm1
data engine config var is ignored now and you can only create tsm1
shards.  Exists shards will work as is until they are migrated to
tsm1 shards.
2016-01-11 12:02:36 -07:00
Jonathan A. Sternberg 5d4ecf853c Add continuous query option for customizing resampling
This makes the following syntax possible:

    CREATE CONTINUOUS QUERY mycq ON mydb
        RESAMPLE EVERY 1m FOR 1h
        BEGIN
          SELECT mean(value) INTO cpu_mean FROM cpu GROUP BY time(5m)
        END

The RESAMPLE option customizes how often an interval will be sampled and
the duration. The interval is customized with EVERY. Any intervals
within the resampling duration on a multiple of the resample interval
will be updated with the new results from the query.

The duration is customized with FOR. This determines how long an
interval will participate in resampling.

Both options are optional. If RESAMPLE is in the syntax, at least one of
the two needs to be given. The default for both is the interval of the
continuous query.

The service also improves tracking of the last run time and the logic of
when a query for an interval should be run. When determining the oldest
interval to run for a query, the continuous query service determines
what would have been the optimal time to perform the next query based on
the last run time. It then uses this time to determine the oldest
interval that should be run using the resample duration and will
resample all intervals between this time and the current time as opposed
to potentially forgetting about the last run in an interval if the
continuous query service gets delayed for some reason.

This removes the previous config options for customizing continuous
queries since they are no longer relevant and adds a new option of
customizing the run interval. The run interval determines how often the
continuous query service polls for when it should execute a query. This
option defaults to 1s, but can be set to 1m if the least common factor
of all continuous queries' intervals is a higher value (like 1m).
2015-12-28 16:43:49 -05:00
Sebastian Borza 8057d6f4ff adding language to config sample for udp-payload-size 2015-12-22 12:57:55 -05:00
Sebastian Borza 3bf5d9c969 moving UDP payload size to config option, improve throughput performance
moving UDP payload size to optional config choice to imporove throughput performance
2015-12-22 09:14:50 -05:00
Philip O'Toole 01ac0b3f23 Tweak compaction log messages 2015-12-15 10:33:13 -08:00
Alexandre Viau b8ea35ffe3 removed trailing space 2015-12-13 16:58:05 -05:00
Tim Raymond 120e9e8421 Remove Registration section from example config
Since the registration service has been removed, this section of the
example config is irrelevant.
2015-12-10 16:24:15 -05:00
Paul Dix 8096c6b845 Update TSM, address PR #5011 comments
* Moved TSM file extension to a constant
* Fixed typos
* Changed group.size() back to being a uint64 since it can have multiple files up to 4GB each.
2015-12-07 14:47:17 -05:00
Paul Dix 937233d988 Update TSM compaction planning logic
* Update Plan to do a full compaction if cold for writes
* Remove MaxFileSize as a config variable from Compactor. Should be a set constant
* Update Plan to keep track of if the last check was fully compacted so we can skip future planning calls
* Update compact min file count to 3 so that compactions run more frequently
2015-12-07 08:26:30 -05:00
Paul Dix 1bee7d1512 Update TSM, remove old version, add config
* remove rolloverTSMFileSize constant that is no longer used
* remove the maxGenerationFileCount since it is no longer a limitation that's necessary with the new compaction scheme. We no longer read WAL segments as part of the compaction so memory is only used as we read in each individual key
* remove minFileCount and switch to a user configurable variable
* remove the mutex from WALSegmentWriter. There's never more than one open in the WAL at one time and it's not exported through any function so the lock on the WAL should be used. This simplified keeping track of the last write time and removed a bunch of unnecessary locks.
* update WALSegmentWriter.Write to take the compressed bytes so that encoding and compression can occur before the call to write (while we don't hold the WAL lock)
* remove a bunch of unnecessary locking in WAL.writeToLog
* Add check for TSM file magic number and vesion
* Remove old tsm, log, and unused cursor code
* Remove references to tsm1dev everywhere except in the inspector
* Clean up config options for compaction and snapshotting
* Remove old TSM configuration options
* Update the config.sample.toml with TSM options
* Update WAL compact to force if it has been cold for writes for a configurable period of time (1h by default)
2015-12-06 18:50:39 -05:00
gunnaraasen fccd6a9399 Clarify engine comment about tsm1 2015-12-02 00:09:50 -08:00
Philip O'Toole 83b78fbc25 Correct reversed write timeouts
[ci skip]
2015-11-20 11:55:01 -08:00
Philip O'Toole b2419ca95d Clarify when precreation may happen
[ci skip]
2015-11-20 11:22:43 -08:00
Philip O'Toole 4281da6688 Correct WAL logging enabled flag
[ci skip]
2015-11-17 08:07:55 -08:00
Philip O'Toole 7ea9b3e49e Allow openTSDB point errors logging to be disabled 2015-11-13 14:54:54 -08:00
Ross McDonald bd186195db Restructured filesystem hierarchy, and updated pre/post installation scripts to ease upgrade progress. Also adding first iteration of new build script (build.py).
New package filesystem hierarchy:
	 - /var/lib/influxdb => Data directories
	 - /usr/lib/influxdb => Scripts and other miscellaneous files
	 - /etc/influxdb => Configuration
	 - /usr/bin/ => Where InfluxDB binaries will live
2015-11-12 10:42:41 -06:00
Cory LaNou a4c54cb0a8 clarify config comment 2015-11-11 11:16:40 -06:00
Cory LaNou 615024ac47 update sample config (was missing cluster-tracing as well). 2015-11-11 10:58:33 -06:00
Cameron Sparr 9625953d3e Add UDP OS buffer size recommendations 2015-11-10 11:00:11 -07:00
Philip O'Toole cb548a65ef Clearly mark tsm1 as experimental in sample config
[ci skip]
2015-11-04 08:27:17 -08:00
Philip O'Toole 3ee1f19316 Add precreation section to sample config 2015-10-28 08:19:37 -07:00
Philip O'Toole 7d22fc75a3 Support configurable purge interval 2015-10-26 13:07:25 -07:00
Philip O'Toole 667ad3342a Refactor registration as a service
Registration also involves statistics and diagnostics upload, for the
purposes of remote management. This means there will be long-running
goroutines in effect. Therefore move the code to a service model.
2015-10-19 15:01:14 -07:00
Philip O'Toole 73f18d5dc4 More information about Enterprise registration
[ci skip]
2015-10-15 14:04:57 -07:00
Philip O'Toole 7521b9a897 Add Enterprise registration configuration details
[ci skip]
2015-10-15 14:04:19 -07:00
Philip O'Toole f298e88b39 Auto-create UDP service database
All other services operate like this, so make UDP service consistent.
2015-10-14 08:30:09 -07:00
Paul Dix 40ff4f4a86 Change default to bz1 2015-10-06 15:30:34 -07:00
Philip O'Toole dbba6356df Whitespace clean-up
[ci skip]
2015-10-02 23:30:00 -07:00
Philip O'Toole 5e72fec32c Update sample config with HH retry-max-interval
[ci skip]
2015-10-01 12:20:35 -07:00
Philip O'Toole 66d3ef1ce0 shard-writer-timeout defaults to 2x write-timeout
[ci skip]
2015-09-30 16:56:23 -07:00
Philip O'Toole a196d3663a Allow configuration of UDP retention policy
Fixes issue #4529
2015-09-28 15:17:56 -07:00
Philip O'Toole b079d201a0 Document openTSDB defaults in sample config
[ci skip]
2015-09-25 14:04:37 -07:00
Philip O'Toole 1fc61a428c Add default Graphite database to sample config
[ci skip]
2015-09-24 18:50:24 -07:00
Philip O'Toole 620d3740ad More corrections of monitor sample config
[ci skip]
2015-09-18 15:31:44 -07:00
Philip O'Toole c5b6c72fdd Correct comments in monitor config block
[ci skip]
2015-09-18 15:30:36 -07:00
Philip O'Toole f73bc6ff19 Merge pull request #4140 from influxdb/engine_config
Make engine configurable
2015-09-16 21:17:43 -07:00
Philip O'Toole e49644ea94 Update CHANGELOG 2015-09-16 19:30:02 -07:00
Philip O'Toole 7b1a4e6700 Control whether each query should be logged
Fixes issue #4138
2015-09-16 19:26:23 -07:00
Philip O'Toole e4fde993f1 Make engine configurable 2015-09-16 19:09:25 -07:00
Philip O'Toole 3cb29d9241 By default write stats to default retention policy
This reverts to a previous design for storing stats, whereby if a
non-default retention policy is configured as the destination retention
policy, it is up the user to explicitly create it, with the desired
replication factor and duration.
2015-09-10 15:08:09 -07:00
Philip O'Toole 70607836e1 Set default monitor store interval to 10 seconds 2015-09-08 20:37:15 -07:00
Philip O'Toole 24aca5611a Add batch-pending control to openTSDB input 2015-09-08 19:35:42 -07:00
Philip O'Toole 5373f263a3 Add pending control to batcher
With this change, the generic batcher used by many inputs can now be
buffered. Testing shows that this performance of the Graphite input by
10-100%, with the biggest improvements at lower numbers of connections.
2015-09-08 19:32:00 -07:00
Philip O'Toole 349ba8b307 Add batching and stats to openTSDB input 2015-09-08 16:19:50 -07:00
Alexandre Viau f0c57066c5 removed trailing spaces 2015-09-06 22:16:00 -04:00
Philip O'Toole 1fdf361f00 Merge pull request #4015 from influxdb/configurable_internal_rp
Monitor retention policy is configurable
2015-09-05 08:53:43 -07:00
Philip O'Toole 406480f4b2 Retention enforcement period to 30 minutes
This was previously set in the code.

[ci skip]
2015-09-04 23:30:13 -07:00
Philip O'Toole 214cfea53c Monitor retention policy is configurable 2015-09-04 22:53:04 -07:00
Philip O'Toole c0d61740a4 Ensure monitor database exists
This requires a leader in the cluster, so wait for that before
attempting the database creation.
2015-09-02 16:26:16 -07:00
Philip O'Toole 53f8cb9714 Disable monitor store in config as well as code
[ci skip]
2015-09-02 08:09:17 -07:00
Philip O'Toole aeee50582b Add port to default store-address
[ci skip]
2015-09-02 07:47:19 -07:00
Philip O'Toole 366c0115f9 Serve expvar information from HTTP package 2015-09-01 15:22:37 -07:00
Philip O'Toole f05dc20b58 Hook new monitor service to server
u
2015-09-01 15:03:52 -07:00
Philip O'Toole d87e668c78 Remove obsolete monitoring code 2015-09-01 15:03:52 -07:00
Philip O'Toole 6193226ce8 Revert "Merge pull request #3771 from influxdb/tcp_graphite_timeout"
This reverts commit d7f646f7a4, reversing
changes made to d6f9903f10.

Conflicts:
	CHANGELOG.md

Fixes issue #3809
2015-08-24 10:53:14 -07:00
Philip 4930a6d8bb Start adding timeouts to TCP Graphite input 2015-08-20 15:10:22 -07:00
Paul Dix 9df3b7d828 Add WAL configuration options 2015-08-18 16:59:54 -04:00
Philip O'Toole 8dbcff833b Revert "Revert "Merge pull request #3599 from tpitale/multiple-udp-servers""
This reverts commit 5f83f038e1.
2015-08-10 16:21:27 -07:00
Philip O'Toole 5f83f038e1 Revert "Merge pull request #3599 from tpitale/multiple-udp-servers"
This reverts commit 7f35d65412, reversing
changes made to f828781544.
2015-08-10 14:44:14 -07:00
Tony Pitale e943afacdc Change (previously unused) single udp config to support multiple UDP listeners
do not use NewConfig for UDP

appendUDPService must return a value

udp service does not need to handle error

fix missing case of c.UDP in tests
2015-08-09 11:28:09 -04:00
Paul Dix fb76c34a79 Merge pull request #3426 from jhorwit2/jah/continuous-logging
Added additional logging to continuous queries
2015-07-27 17:43:55 -04:00
Todd Persen 381c300ac5 Merge pull request #3424 from influxdb/pd-update-sample-config
Update sample config with information on batching for UDP, Graphite, …
2015-07-23 17:09:28 -07:00
Josh Horwitz e722b4b4ad Added additional logging to continuous queries 2015-07-23 19:09:42 -04:00
Gunnar d9f16987fc Merge pull request #3439 from influxdb/ga-admin-https
Add HTTPS option and logger to admin service
2015-07-23 15:09:56 -07:00
gunnaraasen 54c4066a5f Remove cert.pem file 2015-07-23 14:55:56 -07:00
gunnaraasen e5ead383e5 Add HTTPS option and logger to admin service 2015-07-22 16:49:12 -07:00
Paul Dix 205a10dbe3 Update sample config with information on batching for UDP, Graphite, and CollectD 2015-07-21 17:48:52 -07:00
Todd Persen 47d5c2d65f First pass at re-enabling HTTPS. 2015-07-17 16:57:31 -07:00
Josh Horwitz 4529af1afd Fixed flush interval & flush delay parameters 2015-07-15 16:59:07 -04:00
Josh Horwitz 307154573c Updated wal & flush parameters
As you can see [here](b2b69c63fe/tsdb/config.go (L23)), it appears the configuration parameters were updated in code, but not the default configuration.
2015-07-11 12:36:00 -04:00
Philip O'Toole d09b0867f7 Add explanations of PointWriter configs 2015-07-02 12:50:15 -04:00
Philip O'Toole a0a891f0b1 Add explanatory note on WAL configuration 2015-07-02 10:40:29 -04:00
Can ZHANG c130efb5e2 Add fields to config metric name schema of graphite 2015-06-24 23:09:06 -06:00
Philip O'Toole d159239714 Update sample config to describe reporting 2015-06-11 00:21:15 -07:00
Philip O'Toole e84f661f7e Use defaults for Graphite input where necessary 2015-06-10 20:38:17 -07:00
Todd Persen f4c07d9d92 Add some section-level comments to the config. 2015-06-09 10:27:27 -07:00
Todd Persen f9188687de Clean up intervals and add graphite section. 2015-06-09 03:38:07 -07:00
Todd Persen f44f433c90 Create a valid default config for v0.9.0 2015-06-08 14:44:52 -07:00
Ben Johnson b925e1c1af Multi-node clustering.
This commit adds the ability to cluster multiple nodes together to share
the same metadata through raft consensus.
2015-06-05 14:41:19 -06:00
Renan Strauss 3b09a59540 Add UDP service back 2015-06-04 10:24:48 +02:00
Philip O'Toole 0c26fd67a6 Correct comments for batcher timeouts 2015-05-29 09:29:08 -07:00
Philip O'Toole eae313c4e2 Integrate batching with collectd 2015-05-28 15:22:36 -07:00
Philip O'Toole d231fc1290 Add Graphite batching params to example config 2015-05-28 12:17:56 -07:00
Chris Laws 6bb0dfdd21 Fixes typos in sample config file 2015-05-24 09:25:07 +09:30
Philip O'Toole f063ebfad2 Default Raft election timeout to 5 seconds 2015-05-12 13:57:28 -07:00