7b71b66e31
Update meta service, meta client, and httpd handler
...
* Improve the ping endpoint so that it can optionally check for leader agreement across all meta servers
* Add Ping method to the meta client
* Fix ClusterID tests
* Remove WaitForLeader from meta client and remove unnecessary references to it
2016-01-21 15:28:33 -05:00
101f93f1db
Add meta service test to ensure cluster id persisted
2016-01-21 15:28:33 -05:00
2f07fe88ca
Update meta client to use data method to protect cache
2016-01-21 15:28:33 -05:00
101ab32571
Fix meta-service for server integration tests
...
* Updated CreateShardGroup to not return an error if it already exists so it's idempotent
* Removed old test making sure you can't delete the default RP. You can delete it now, there was no reason to disallow it.
* Wired up the UpdateRetentionPolicy functionality
2016-01-21 15:28:33 -05:00
2715d5ef72
add clusterID and tests
2016-01-21 15:28:33 -05:00
fb9181d240
Fix meta-service build
2016-01-21 15:28:33 -05:00
bfcf5d63ce
Clean up meta service close.
2016-01-21 15:28:33 -05:00
f23fea81b3
take rlock and grab ref to data
2016-01-21 15:28:33 -05:00
d1fcf1f7a1
wire up meta client shard methods
2016-01-21 15:28:33 -05:00
f385945058
Update Server to work with new metaservice/client
2016-01-21 15:28:33 -05:00
d0cad8a022
add subscription meta client test
2016-01-21 15:28:33 -05:00
853f4bf70e
add continuous query meta client tests
2016-01-21 15:28:33 -05:00
a41222befb
add continuous query/subscription methods to meta client
2016-01-21 15:28:33 -05:00
7c41c0e02f
add user tests for meta client
2016-01-21 15:28:33 -05:00
9fd9a666bf
Add CreateDataNode to meta client/service
2016-01-21 15:28:33 -05:00
53042ac56d
bringing back client user methods
2016-01-21 15:28:33 -05:00
13e32f6880
Update close handling on meta service
2016-01-21 15:28:33 -05:00
d2e3cf519c
Cleanup host/port in meta service
2016-01-21 15:28:33 -05:00
e906107bea
Update meta service to handle host names
...
This ensures that the meta service will gracefully handle host name changes in a single server configuration.
It also changes the raft setup to use the user specified bind address (and thus hostname) instead of pulling it off the listener, which returns the IP. This will enable users to have hostnames listed instead of IPs in the megastore, making it easier to read. This also means that underlying IPs can change without causing problems in a cluster.
2016-01-21 15:28:32 -05:00
0f36fbe5ce
Add comment to client
2016-01-21 15:28:32 -05:00
1632980eb8
Cleanup PrintLns in meta client
2016-01-21 15:28:32 -05:00
eda4a6eda0
Wire up meta service and client recovery.
...
* increase sleep on error in client exec in case a server went down so we don't max out retries before a new leader gets elected
* update and add close logic to service, handler, raft state, and the client
2016-01-21 15:28:32 -05:00
5c20e16406
wire up some RP stuff in meta client / service
2016-01-21 15:28:32 -05:00
c84e9b38d0
fix unit tests after backing out proto change
2016-01-21 15:28:32 -05:00
f91fd0b8ae
back out proto struct changes
2016-01-21 15:28:32 -05:00
e9e63b573b
Cycle to next server on failure in meta client
2016-01-21 15:28:32 -05:00
1e63fa4e2c
Enforce max retries on meta client
2016-01-21 15:28:32 -05:00
90a08154c5
Wire up redirects to execute against raft leader
2016-01-21 15:28:32 -05:00
c7721c8948
don't clone database infos in client
2016-01-21 15:28:32 -05:00
6561b702b8
remove commented out test code
2016-01-21 15:28:32 -05:00
f80f860ee5
temporarily rename statement_executor_test.go
2016-01-21 15:28:32 -05:00
1d6878c37c
wire up some meta client funcs and tests
2016-01-21 15:28:32 -05:00
c9d82ad0ad
Wire up meta service functionality
...
* Add dir, hostname, and bind address to top level config since it applies to services other than meta
* Add enabled flags to example toml for data and meta services
* Wire up add/remove raft peers and meta servers to meta service
* Update DROP SERVER to be either DROP META SERVER or DROP DATA SERVER
* Bring over statement executor from old meta package
* Start meta service client implementation
* Update meta service test to use the client
* Wire up node ID/meta server storage information
2016-01-21 15:28:32 -05:00
688bc7a2f1
fix go vet error
2016-01-21 15:28:32 -05:00
d69c5f853f
set store peers when starting up from config
2016-01-21 15:28:32 -05:00
79d81a2448
add meta service tests & bug fixes
2016-01-21 15:28:32 -05:00
9ec7a710c9
some misc refactoring on influxd startup
2016-01-21 15:28:32 -05:00
8d878fff91
buildable meta -> services/meta
2016-01-21 15:28:32 -05:00
bf0b477a0b
set raftState on the store
2016-01-21 15:28:32 -05:00
d3ab0b5ae6
buildable again. lot of wip
2016-01-21 15:28:32 -05:00
b0d0668138
wip
2016-01-21 15:28:32 -05:00
94b05404dc
remove cors from handler
2016-01-21 15:28:31 -05:00
169c6a5dfa
store and handler to interface
2016-01-21 15:28:31 -05:00
9f93f0b84a
convert to AfterIndex
2016-01-21 15:28:31 -05:00
05da43d9f6
rough out meta service
2016-01-21 15:28:31 -05:00
59fbd371fc
Implement backup/restore for TSM.
...
This changes backup and restore to work for TSM. It breaks it for b1 and bz1, but since those are getting removed it's ok.
The backup runs against any host that is specified and can backup either the metasstore, a database, specific retention policy, or a specific shard. It can also take incremental backups with the `since` flag, which will only backup TSM files that have been created since that timestamp.
The backup is safe to run online. However, for shards that are still hot for writes, they won't be able to create new TSM files while the backup for that single shard runs. If the backup isn't too large and the write throughput isn't too high this shouldn't be a problem since the writes will just go into the WAL cache.
2015-12-30 18:06:50 -05:00
5d4ecf853c
Add continuous query option for customizing resampling
...
This makes the following syntax possible:
CREATE CONTINUOUS QUERY mycq ON mydb
RESAMPLE EVERY 1m FOR 1h
BEGIN
SELECT mean(value) INTO cpu_mean FROM cpu GROUP BY time(5m)
END
The RESAMPLE option customizes how often an interval will be sampled and
the duration. The interval is customized with EVERY. Any intervals
within the resampling duration on a multiple of the resample interval
will be updated with the new results from the query.
The duration is customized with FOR. This determines how long an
interval will participate in resampling.
Both options are optional. If RESAMPLE is in the syntax, at least one of
the two needs to be given. The default for both is the interval of the
continuous query.
The service also improves tracking of the last run time and the logic of
when a query for an interval should be run. When determining the oldest
interval to run for a query, the continuous query service determines
what would have been the optimal time to perform the next query based on
the last run time. It then uses this time to determine the oldest
interval that should be run using the resample duration and will
resample all intervals between this time and the current time as opposed
to potentially forgetting about the last run in an interval if the
continuous query service gets delayed for some reason.
This removes the previous config options for customizing continuous
queries since they are no longer relevant and adds a new option of
customizing the run interval. The run interval determines how often the
continuous query service polls for when it should execute a query. This
option defaults to 1s, but can be set to 1m if the least common factor
of all continuous queries' intervals is a higher value (like 1m).
2015-12-28 16:43:49 -05:00
d885f4156b
removing golang reference
2015-12-22 12:54:43 -05:00
3bf5d9c969
moving UDP payload size to config option, improve throughput performance
...
moving UDP payload size to optional config choice to imporove throughput performance
2015-12-22 09:14:50 -05:00
b3adfa6b76
Graphite: fix an edge case in Parser.ApplyTemplate
2015-12-21 15:09:10 +08:00
8214676363
fix panic when merging empty series
2015-12-15 11:32:51 -07:00
718fb92cf5
Remove registration, add 'usage-client'
...
Server registration and stats reporting has been removed from what was
once http://enterprise.influxdata.com . The app that lived there, now
runs at http://usage.influxdata.com , so that the subdomain can
eventually be repurposed. Because we also want to repurpose the
`enterprise-client` repo, we have also renamed that to `usage-client`.
InfluxDB no longer needs the `registration` service now, since all of
the endpoints it communicates with simply discard the data provided to
them.
2015-12-10 15:58:34 -05:00
eb080785d4
add bounds checking for shard group timestamps
...
enforce strict valid time ranges on points
small
2015-12-09 13:07:13 -07:00
a5cb913035
Drop UDP point on bad parse and keep going
...
Fixes issue #4992 .
2015-12-07 19:47:59 -08:00
cf4e901078
silence registration logging for testing
2015-12-03 08:04:10 -06:00
6e1d0f47d4
Merge pull request #4921 from pires/3170-db_not_found
...
Handler errors should be formatted as JSON.
2015-12-02 11:36:53 -05:00
3cd8056664
Merge pull request #4876 from e-dard/lint
...
Lint
2015-12-02 08:55:47 -06:00
8384ba3e84
Update comments.
2015-12-02 12:35:58 +00:00
4587a5387d
Handler errors should be formatted as JSON. Fixes #3170
2015-11-27 21:12:05 +00:00
be488b7d12
implement close notifier and timeout on executors
2015-11-24 21:07:18 -06:00
8ba35c3b8c
fix collectd manual typo
2015-11-23 18:54:47 +02:00
ffbd6037e2
Initial lint for all packages under services. Supports #4098
2015-11-22 19:23:56 +00:00
b1c5a36b0d
Merge pull request #4846 from jsternberg/feature/allow-nan-in-graphite-service
...
Allow NaN as a valid value on the graphite service
2015-11-20 08:09:51 -08:00
5fa36639db
Allow NaN as a valid value on the graphite service
...
The canonical graphite implementation will read and discard NaN values
instead of throwing an error when reading on the line receiver protocol.
Since this is the default behavior for graphite, InfluxDB should have
the same behavior for compatibility.
Previously, a NaN value would result in an error printed to the console.
When you have a large number of NaN values being sent every minute, this
results in the log file filling with useless messages.
2015-11-20 11:07:25 -05:00
d86a02eeab
Fix minor typo in variable name
2015-11-20 07:48:42 -08:00
8c2f6eb7e0
Comment additions to services to satisfy golint
2015-11-19 13:25:07 +01:00
ba807a268b
Merge pull request #4835 from orthogonous/master
...
Comment additions to conform to golint
2015-11-18 17:47:51 -08:00
65f4060de3
Comment additions to conform to golint
...
Also 1 slight code change that fixed an if statement
that golint didn't like.
2015-11-19 11:26:21 +10:00
06cd9b1e4d
Merge pull request #4411 from gobwas/master
...
Update handler.go
2015-11-18 16:12:18 -08:00
e17e6394c9
fixing typo for pointsWrittenOK
2015-11-18 14:28:39 -05:00
63e958f06e
fixed export lint issues in services/admin
2015-11-16 17:31:53 +08:00
69803ddc6f
Merge pull request #4791 from orthogonous/master
...
Added some comments to the udp service so golint passes. Ref #4098
2015-11-13 19:10:49 -08:00
7ea9b3e49e
Allow openTSDB point errors logging to be disabled
2015-11-13 14:54:54 -08:00
434d06008e
Added some comments to the udp service so golint passes. Ref #4098
2015-11-14 08:46:26 +10:00
bc1091ec31
Revert "Merge pull request #4684 from nkatsaros/default-config"
...
This reverts commit 87441584f3a14829bc1d
2015-11-11 09:29:06 -08:00
6ecb62e4d2
Merge pull request #4737 from ch33hau/4283-hh-throws-error-even-if-disabled
...
Disable HintedHandoff if configuration is not set. #4283
2015-11-10 19:07:27 -06:00
9625953d3e
Add UDP OS buffer size recommendations
2015-11-10 11:00:11 -07:00
727b9f6d8b
Documentation updates for UDP, graphite, collectd listeners
2015-11-10 10:35:21 -07:00
0a683144fb
Update collectd and graphite UDP listeners with perf enhancements
...
closes #4678
2015-11-10 10:35:21 -07:00
8bfdfbda0b
Disable HintedHandoff if configuration is not set. #4283
2015-11-11 01:12:34 +08:00
bece8fed2a
Better retention enforcement logging
...
Fixes issue #4727 .
2015-11-09 17:22:24 -08:00
78e6979be5
Make parser/writer internal func names consistent
2015-11-06 16:43:35 -07:00
5c28048132
UDP service listener performance enhancements
2015-11-06 15:53:47 -07:00
8ac958f8a0
Merge pull request #4683 from influxdb/enterprise-client
...
Updated to use the new Enterprise Client
2015-11-06 13:13:15 -05:00
377d926aba
Fixed a logging message posting to Enterprise
2015-11-06 10:21:18 -05:00
b1df4bd76e
Updated to use the new Enterprise Client
2015-11-05 16:46:45 -05:00
4f8b0aca9a
Add graphite and udp services to the default config generator
2015-11-05 16:42:21 -05:00
e2db577e0f
Implement a UDP client
...
closes #4647
2015-11-04 14:18:14 -07:00
3ea4366e4e
Unit test handling bad write requests
2015-10-30 11:05:47 -07:00
d9a60d7e4c
fix write points panic
2015-10-30 10:52:50 -07:00
5044b9d42c
Use HTTPS instead of Https in variable names
2015-10-30 08:49:37 +11:00
de7919240f
Migrate internal stats to consistent names
...
Go style -- and existing runtime stats -- do not use underscores, but
instead use camel case. This change makes the internal stats adhere to
that convention.
2015-10-28 21:07:45 -07:00
ef190b4013
Merge pull request #4600 from influxdb/wait_for_leader
...
ping endpoint can now wait for leader
2015-10-28 12:04:53 -07:00
1f6f9b35d1
Unit test ping endpoint
2015-10-28 11:37:25 -07:00
2046556b71
Ping endpoint supports waiting for leader
2015-10-28 11:37:07 -07:00
7d6d23e661
Write as many points as we can parse via line protocol
...
This changes the HTTP line protocol handler to behave similar to the other
handler in that they will write as many points as possible. Previously, we
would fail the entire batch if one point failed. This can happen more frequently
now with NaN being more explicitly unsupported. Now it will write as many points
that parse successfully and return a "partial write" error to the client with the
lines that failed to parse.
2015-10-27 17:12:57 -06:00
0926b19e6b
Prevent creating points with NaN float values
...
Float values are not supported in the existing engine and the tsm1
engines. This changes NewPoint to return an error if a field value
contains a NaN field. It also allows us to validate fields to prevent
other unsupported types from sneaking in through other input plugins.
2015-10-27 17:12:52 -06:00
335e4325d8
Merge pull request #4516 from influxdb/hh_processor_per_node
...
Refactor hinted-handoff service
2015-10-27 14:43:08 -07:00
f38c53695d
Add node's active state to diagnostic output
2015-10-26 18:59:58 -07:00
f703f58d22
Add HH diagnostics
2015-10-26 18:59:58 -07:00
87299caad1
Add HH statistics
2015-10-26 18:59:58 -07:00
9a73d26bfb
Implement NodeProcessor
...
A NodeProcessor wraps an on-disk queue and the goroutine that attempts
to drain that queue and send the data to the associated target node.
2015-10-26 18:59:55 -07:00
1f74b46bdb
Merge pull request #4565 from bwhaley/clarify_filters
...
Update README.md
2015-10-26 17:24:10 -07:00
7d22fc75a3
Support configurable purge interval
2015-10-26 13:07:25 -07:00
8aa79c551f
Update README.md
...
Clarify use of filters with multiple consecutive wildcards
2015-10-24 15:01:44 -07:00
90beaee803
silence subscriber service logging during testing
2015-10-23 14:48:32 -06:00
7c71ed53a4
fix subscriber logic when closed
2015-10-21 15:08:55 -06:00
956efaeb94
Merge pull request #4506 from influxdb/enterprise_stats
...
Enterprise registration as a service
2015-10-20 08:57:19 -07:00
f0c4501135
Add registration info to diagnostic output
2015-10-19 16:54:26 -07:00
83db5cdbf5
Update CHANGELOG
2015-10-19 15:30:40 -07:00
878663e1e3
Periodic upload of stats to Enterprise
2015-10-19 15:25:07 -07:00
667ad3342a
Refactor registration as a service
...
Registration also involves statistics and diagnostics upload, for the
purposes of remote management. This means there will be long-running
goroutines in effect. Therefore move the code to a service model.
2015-10-19 15:01:14 -07:00
ef72c3c64d
Fix typo in retention service comment
...
[ci skip]
2015-10-19 14:24:25 -07:00
ff18bf7213
Make Open() and Close() on Graphite sync'ed
...
This will ensure that these operations don't run concurrently. This
change also ensures nil batchers are not closed.
Fixes issue #4494 .
2015-10-19 11:13:31 -07:00
e73a8e423c
fix #4472:too many points in the GROUP BY interval
2015-10-16 07:17:14 -04:00
cb1aaa8e42
Merge pull request #4375 from influxdb/subscriptions
...
Feature add subscriber service for creating/dropping subscriptions
2015-10-15 09:17:26 -06:00
485c446e98
Correct typos in UDP README
...
[ci skip]
2015-10-15 07:48:34 -07:00
82f104a8b1
Merge pull request #4436 from influxdb/tag-names-to-keys
...
WIP tag name --> tag key, field name --> field key
2015-10-14 16:02:46 -07:00
8b31007aa7
Adds subscriber service for creating/dropping subscriptions to the
...
InfluxDB data stream.
2015-10-14 15:23:45 -06:00
25f957c5c6
Only call Stop on non-nil batchers
2015-10-14 08:55:06 -07:00
a938cd3dee
openTSDB Open should complete before Close runs
2015-10-14 08:55:06 -07:00
3907656cc2
Add README for UDP service
...
Fixes issue #4041 .
2015-10-14 08:30:10 -07:00
f298e88b39
Auto-create UDP service database
...
All other services operate like this, so make UDP service consistent.
2015-10-14 08:30:09 -07:00
5ab86f7578
Update README.md
2015-10-13 16:56:37 -07:00
ed7b9f7485
tag name --> tag key
2015-10-13 16:41:57 -07:00
822af73f88
implement continuous queries as regular execs of into queries.
...
Now that we have into queries, we can implement them as regular
queries that are just run on a timer.
2015-10-13 15:51:19 +00:00
d25e264009
Update handler.go
...
Add `Access-Control-Expose-Headers` for `ping` endpoint clients be able to retrieve `X-Influxdb-Version` and `Date` from the server.
2015-10-12 18:05:18 +03:00
37cf9a1610
Deletion while iterating is OK in Go
2015-10-09 16:30:20 -07:00
f12470a99e
If there are no HH segments, then nothing to purge
2015-10-09 14:29:21 -07:00
c06ac8f94c
Don't add a new segment every purge check
...
Everytime the purge check was running, a new segment was being added.
This meant the list of almost-empty files in the HH directories would
grow continually.
2015-10-09 14:26:47 -07:00
657aa5a134
Add README for collectd
2015-10-09 09:15:22 -07:00
b009f25e3d
Delete queues for inactive nodes
...
Deletion only takes place if all data in the queue is older than the
configured time.
2015-10-08 20:34:24 -07:00
5b0a8ed306
HH should not process dropped nodes
2015-10-08 18:23:12 -07:00
2add55107e
Fix graphite parser merge error, nargs
2015-10-08 11:15:03 -06:00
a9bf213076
Merge pull request #3484 from dawbs/dawbs-fix-3429
...
Bugfix for #3429 String representations of RegexLiterals generated in…
2015-10-08 13:12:10 -04:00
3bea25b428
graphite parser: apply tags from the Parser on the template
2015-10-08 10:56:13 -06:00
136dbef0e7
Formatting fixes
2015-10-08 19:41:36 +10:00
26f6d00668
Bugfix for #3429 String representations of RegexLiterals generated in influxql/ast.go add the / char as a start and end delimiter, but does not escape any / characters that may exist with the regex
2015-10-08 19:41:36 +10:00
73a630dfa6
graphite parser: apply tags from the Parser on the template
2015-10-07 23:19:29 -06:00
f3e3bf7a0e
typo
2015-10-07 21:25:49 +01:00
5815e0b0ee
updated documentation
2015-10-07 21:24:05 +01:00
d8ac746703
correct formatting
2015-10-07 20:35:05 +01:00
5fd8777c56
add tests
2015-10-07 20:32:10 +01:00
a27186fb7a
raise exception when field keyword is specified multiple times
2015-10-07 20:31:46 +01:00
bcd6c06173
Merge remote-tracking branch 'upstream/master' into graphite-template-custom-field
...
Conflicts:
services/graphite/parser.go
2015-10-07 17:48:34 +01:00
44d52ac138
Fully lock HH node queue creation
...
I believe this change address the issues with hinted-handoff not fully replicating all data to nodes that come back online after an outage.. A detailed explanation follows.
During testing of of hinted-handoff (HH) under various scenarios, HH stats showed that the HH Processor was occasionally encountering errors while unmarshalling hinted data. This error was not handled completely correctly, and in clusters with more than 3 nodes, this could cause the HH service to stall until the node was restarted. This was the high-level reason why HH data was not being replicated.
Furthermore by watching, at the byte-level, the hinted-handoff data it could be seen that HH segment block lengths were getting randomly set to 0, but the block data itself was fine (Block data contains hinted writes). This was the root cause of the unmarshalling errors outlined above. This, in turn, was tracked down to the HH system opening each segment file multiple times concurrently, which was not file-level thread-safe, so these mutiple open calls were corrupting the file.
Finally, the reason a segment file was being opened multiple times in parallel was because WriteShard on the HH Processor was checking for node queues in an unsafe manner. Since WriteShard can be called concurrently this was adding queues for the same node more than once, and each queue-addition results in opening segment files.
This change fixes the locking in WriteShard such the check for an existing HH queue for a given node is performed in a synchronized manner.
2015-10-07 02:33:43 -07:00
5b0767c30b
EOF is OK in HH processor
2015-10-07 01:56:55 -07:00
8b49c37120
Count HH errors
2015-10-06 20:49:40 -07:00
5d5515a497
If HH can't unmarshal a block, skip that block
2015-10-06 20:49:40 -07:00
883d32cfd0
Add public function to graphite parser to apply template
2015-10-06 17:42:36 -06:00
bb398daf75
Updates based on @otoolp's PR comments
2015-10-05 20:09:56 -04:00
5d9b89d601
Disable copier test
...
Not implemented for tsm1 engine
2015-10-05 20:09:56 -04:00
7555ccbd70
WIP: engine work
2015-10-05 20:06:21 -04:00
2ac0357406
Support dropping non-Raft nodes
2015-10-04 00:19:52 -07:00
d74e0690c7
Revert "Merge pull request #4233 from influxdb/drop-server"
...
This reverts commit 0bdb36f6dc3085fbc138
2015-10-02 08:39:57 -07:00
f50813460e
protobuf update.. :-(
2015-10-01 15:39:15 -05:00
8a1e5a9e53
Clamp initial value of HH retry interval
...
This could happen due to misconfiguration, so do something sensible in
that case.
2015-10-01 12:04:33 -07:00
878f776403
Exponential backoff if any hinted-handoff fails
2015-09-30 21:27:13 -07:00
4eba2c1725
Add config support for max HH retry interval
2015-09-30 21:10:03 -07:00
235714755c
HH processor-level stats
...
This change maintains stats on a per-shard and per-node basis.
2015-09-28 18:39:39 -07:00
14db3ce9f5
Add service-level stats for hinted-handoff
2015-09-28 18:08:35 -07:00
a196d3663a
Allow configuration of UDP retention policy
...
Fixes issue #4529
2015-09-28 15:17:56 -07:00
49a70d0fca
Merge pull request #4238 from influxdb/hh_control
...
Fully disable hinted-handoff service if requested
2015-09-28 12:11:18 -07:00
a4a8fa0ff0
Fully disable hinted-handoff service if requested
...
Without this change if hinted-handoff was disabled the service would
correctly reject writes, but it would process any data sitting in
hinted-handoff queues. With this change the service is completely
disabled.
2015-09-25 18:03:43 -07:00
9de3125f6b
Graphite TCP should not block system shutdown
...
With this change Graphite TCP connections are tracked on a per-service
basis. This allows a closing Graphite service to first shutdown any
active connections, thereby unblocking the rest of shutdowm.
This work exposed small shortcomings with the existing Diagnostics
system and that code has alse been tweaked.
Fixes issue #4017
2015-09-24 14:08:38 -07:00
49c0b6ea73
Fix go vet warnings
...
This patch fixes the following go vet warnings:
```
services/continuous_querier/service.go:326: influxql.Statements
composite literal uses unkeyed fields
exit status 1
services/httpd/handler_test.go:145: models.Rows composite literal uses
unkeyed fields
services/httpd/handler_test.go:146: models.Rows composite literal uses
unkeyed fields
services/httpd/handler_test.go:165: models.Rows composite literal uses
unkeyed fields
services/httpd/handler_test.go:166: models.Rows composite literal uses
unkeyed fields
services/httpd/handler_test.go:187: models.Rows composite literal uses
unkeyed fields
services/httpd/handler_test.go:188: models.Rows composite literal uses
unkeyed fields
exit status 1
```
Signed-off-by: Antonio Murdaca <runcom@linux.com>
2015-09-21 15:28:54 +02:00
ef35d6dcc2
formatting
2015-09-21 12:26:43 +01:00
27c1cc23fd
Working prototype..
2015-09-21 12:18:19 +01:00
9121b422f8
comment out tests for now..
2015-09-21 10:47:42 +01:00
20e4fdfa9a
allow specifying fieldname in graphite template
2015-09-20 21:17:50 +01:00
72f6f7d268
Merge pull request #4134 from influxdb/issue-3447
...
Refactor Points and Rows to dedicated packages
2015-09-17 15:27:48 -05:00
38cb7b49de
Mising defer in httpd recovery. fixes #4124
2015-09-17 09:37:27 -05:00
ba830be3b9
actually move influxql.Row* -> models.Row*
2015-09-16 16:32:50 -05:00
d19a510ad2
refactor Points and Rows to dedicated packages
2015-09-16 15:33:08 -05:00
22ea73d0c3
Figure out the InfluxDB version from /ping and show it on the admin page
2015-09-16 13:59:50 +00:00
d538829b4c
Enhance openTSDB logging and stats
2015-09-09 13:30:11 -07:00
fef20c77b2
Cleanly terminate openTSDB connection on EOF
...
This is not really an error, so don't log it.
2015-09-09 13:01:13 -07:00
02fcaf853d
Add node re Graphite configuration
...
[ci skip]
2015-09-08 23:22:34 -07:00
519a30a463
Add note on openTSDB batching
...
[ci skip]
2015-09-08 23:19:17 -07:00
24aca5611a
Add batch-pending control to openTSDB input
2015-09-08 19:35:42 -07:00
95530e1623
Set UDP input defaults if not set
2015-09-08 19:32:20 -07:00
5373f263a3
Add pending control to batcher
...
With this change, the generic batcher used by many inputs can now be
buffered. Testing shows that this performance of the Graphite input by
10-100%, with the biggest improvements at lower numbers of connections.
2015-09-08 19:32:00 -07:00
e38a204afc
Merge pull request #4043 from influxdb/opentsdb_batching
...
Add batching and stats to openTSDB input
2015-09-08 19:27:35 -07:00
1ce5187b66
Merge pull request #4049 from influxdb/udp_stats
...
Add stats to the UDP input
2015-09-08 19:18:17 -07:00
9677a0faab
Add collectd stats
2015-09-08 19:07:47 -07:00
27932409b0
Add stats to the UDP input
2015-09-08 18:48:35 -07:00
817328d378
Add basic stats to the CQ service
2015-09-08 18:17:20 -07:00
349ba8b307
Add batching and stats to openTSDB input
2015-09-08 16:19:50 -07:00
73510a0a68
Fix invalid time stamp in graphite metric causes panic
...
If a timestamp was larger than the max epoch value was sent via
graphite it would cause the timestamp to overflow when it was
marshaled/unmarshaled back from the raft log. The overflow cause
the shard group to get created with the wrong timestamp which cause
a panic when writing the point. The panic was caused because the
timestamp that were supposed to exists in a map created by MapShards
did not actually exist so a nil ShardGroup was used.
The change prevents creating the point with an invalid timestamp. Since
graphite using a timestamp in seconds, the maximum range is known and
can be prevented. This also adds a check for the minimum range as well.
Fixes #3785
2015-09-08 10:07:47 -06:00
332ce6481d
Removed unused Graphite NewConfig
...
This function is not helpful for sections of the config that support
multiple instances.
2015-09-08 08:32:19 -07:00
bbc103305b
Support multiple Graphite inputs
...
Fixes issue #3636
2015-09-06 21:33:46 -07:00
fa29e12222
Shutdown UDP Graphite on SIGTERM
...
Service.Close() had no way of closing the UDP Conn. This change makes
the UDP an attribute of the server, so Close() can access it.
2015-09-05 00:30:59 -07:00
579e2a250c
Add stats to httpd package
2015-09-04 12:37:59 -07:00
3df898bd90
Merge pull request #3987 from influxdb/global_expvar_hookup_diagnostics
...
Use expvar statistics directly
2015-09-04 11:13:17 -07:00
89bc392ec4
Access expvar directly from monitor
...
expvar map is already global so access it directly. This simplifies the
code and makes it much eaisier to use from other modules.
2015-09-04 09:45:24 -07:00
cf5a655249
Don't precreate shard groups entirely in past
...
Fixes issue #3722
2015-09-04 08:31:50 -07:00
6ad35e23e9
Integrate code review feedback
2015-09-03 20:50:54 -07:00
d58532d844
Add Graphite diagnostics
...
Graphite diagnostics currently show TCP connections.
2015-09-03 20:50:54 -07:00
e07432c59f
Implement diagnostics support
...
This change adds support for diagnostics by decomposing the existing
interface into two interfaces -- one for stats, and the other for
diags. It also adds some basic monitor of system, network, and the Go
runtime.
2015-09-03 20:50:54 -07:00
dce666e757
fix #3979 : fix race in CQ service
2015-09-03 19:55:40 -04:00
deff06f850
add copier service
...
This commit adds the copier service which allows one server to
copy shards from another server. This will be used for moving
shards in the cluster.
2015-09-03 13:07:35 -06:00
0cb9618d6d
fix CQ intoDB()
2015-09-03 09:07:57 -04:00
d466b19388
update CQ service unit tests
2015-09-03 07:12:15 -04:00
66001cfbb5
fix #2555 : add integration tests for CQs
2015-09-03 07:12:15 -04:00
021a6f5453
rename CQ tests
2015-09-03 07:12:15 -04:00
99a22c174b
fix #2555 : add backreference in CQs
...
Add new query syntax to allow the following in CQs:
INTO "1hPolicy".:MEASUREMENT
2015-09-03 07:12:15 -04:00
4e2ee1ea70
Rename MonitorService to just Monitor
...
monitor is not a service, it has more in common with meta, since it
provides functionality to the query layer. This names makes this
clearer.
2015-09-02 15:07:30 -07:00
366c0115f9
Serve expvar information from HTTP package
2015-09-01 15:22:37 -07:00
9df17409d3
Use monitor service with Graphite
2015-09-01 15:21:36 -07:00
d87e668c78
Remove obsolete monitoring code
2015-09-01 15:03:52 -07:00
d771612718
Set default retention check interval to 30 minutes
...
Since the minimum retention period is 1 hour, checking every 10 minutes
seems excessive and generates noise in the logs.
2015-08-27 16:08:03 -07:00
ae825fdf3d
Correct typo in retention service logs
2015-08-27 16:08:03 -07:00
74dad8c68c
fix collectd tests for float data
2015-08-25 09:14:38 -05:00
6193226ce8
Revert "Merge pull request #3771 from influxdb/tcp_graphite_timeout"
...
This reverts commit d7f646f7a4d6f9903f10#3809
2015-08-24 10:53:14 -07:00
d7f646f7a4
Merge pull request #3771 from influxdb/tcp_graphite_timeout
...
Close idle Graphite TCP connections
2015-08-20 17:08:17 -07:00
50b0f67290
Add Graphite TCP timeout tests
2015-08-20 15:46:08 -07:00
4930a6d8bb
Start adding timeouts to TCP Graphite input
2015-08-20 15:10:22 -07:00
afe1f598ca
Cache name and fields if requested
...
Through profiling of writes, point.Fields() and point.Name() were called
repeatedly in PointsWriter and the Shard. These calls are somewhat expensive
when writing large batches so we can cache them to avoid wasting CPU cycles.
Using influx_stress with default settings
Before:
Wrote 10000000 points at average rate of 202570
Average response time: 235.450355ms
After:
Wrote 10000000 points at average rate of 246120
Average response time: 182.881008ms
2015-08-20 15:48:38 -06:00
8e51064db1
Log Graphite batch size and timeout
2015-08-20 11:23:09 -07:00
cf5ac2603d
Fix Graphite README typo
...
Fixes #3727
2015-08-19 07:53:29 -07:00
409fe0afe3
Merge pull request #3686 from jonseymour/secure-options
...
Prevent 'p' parameter of OPTIONS requests being logged.
2015-08-18 17:19:39 -07:00
5bb699e9a9
Enhance precreation log messages
2015-08-18 16:20:55 -07:00
bdce79fe57
Merge branch 'secure-options-minimal' into secure-options
2015-08-19 09:15:58 +10:00
1d5ff55d76
Remove redaction logic from parseCredentials.
...
We now redact the credentials in the logger, so the function implemented
by the deleted lines now seems redudndant.
Signed-off-by: Jon Seymour <jon@wildducktheories.com>
2015-08-19 09:08:54 +10:00
28a6b1f3fd
Merge pull request #3697 from influxdb/chunking_10k
...
Merge same-series data if not chunking
2015-08-18 13:23:10 -07:00
2805c4a9b5
Ensure 'p' parameter is not logged, even on OPTIONS requests.
...
Previously password redaction only occurred inside the
authentication handler and the authentication handler is not on
the request path for OPTIONS requests and, in any case, would
not be invoked because of an early return on OPTIONS
requests by the CORS handler.
Now, we change the response logger to explictly replace any
occurrence of the 'p' parameter from the query string with
'[REDACTED]' prior to logging the response.
Signed-off-by: Jon Seymour <jon@wildducktheories.com>
2015-08-18 09:41:16 +10:00
6415944d01
Don't repeat retention policy log message
2015-08-17 16:15:51 -07:00
487c336571
Correctly merge rows for identical series
...
If no chunking was requested by the user, the co-ordinating node buffers all
results in RAM before emitting a single result. However buffering was not
merging results for rows which had data for the same series. This change fixes this.
Fixes issue #3242 .
2015-08-17 13:43:17 -07:00
7cf31a74cd
Prevent out of memory range slices from being created
...
If the hinted handoff segment is corrupt, the size read could be
invalid and attempting to create a slice using that size causes
a panic. Ideally, we'd have a checksum on the seqment record but
for now just return an error when the size is larger than the
segment file.
Fixes #3687
2015-08-17 10:48:01 -06:00
e5e782d13d
Merge pull request #3517 from dim/fix-cq-timeouts
...
Batch CQ writes to avoid timeouts
2015-08-14 10:52:17 -06:00
bb6de3b8f3
Merge pull request #3522 from dim/fix-cq-timeout-bug
...
Consume CQ results on request timeouts
2015-08-14 10:52:04 -06:00
0b05980ae2
silence snapshotter logger for testing
2015-08-13 20:53:40 -05:00
a7cb0df4af
Fix typos/spacing
2015-08-13 10:02:05 -06:00
668181d275
Make log statements more consistent
...
* Capitalize first letter of message
* Log all services staring consistently
* Remove some extraneous log statements in meta.Store
* Log data dirs for meta, data and hinted handoff
2015-08-13 10:01:42 -06:00
089d947bf3
Shutdown Graphite listener first during Close()
...
Without this the WaitGroup was not fully decremented as the Accept()
call on the listener never exited, and Wait() then never exited.
2015-08-12 12:49:58 -07:00
966dee7559
Set sensible Graphite batching defaults
2015-08-11 18:34:36 -07:00
7dc7389e96
Remove dump from client and handler
2015-08-07 11:56:30 -07:00
398ffabab7
Fix panic in hinted handoff processor
...
A short write has occurred and we do not have enough bytes to determine
the size of the payload. This is corrupted record that we should drop.
Instead of panicing, log the error and advance the queue since the error
at this location is unreoverable currently.
Fixes #3436
2015-08-06 14:06:41 -06:00
4f7df336f2
Fix go vet
2015-08-05 12:16:17 -06:00
2fac5471ce
add TLS support to the OpenTSDB plugin
2015-07-31 12:00:22 -06:00
762a1c69d5
Consume CQ results on request timeouts
2015-07-30 16:59:47 +01:00
642d6eba85
Batch CQ writes to avoid timeouts
2015-07-30 15:24:51 +01:00
2a02605ef6
Merge pull request #3474 from influxdb/ga-options
...
Respond to OPTIONS requests on /query endpoint
2015-07-28 16:32:26 -07:00
fb76c34a79
Merge pull request #3426 from jhorwit2/jah/continuous-logging
...
Added additional logging to continuous queries
2015-07-27 17:43:55 -04:00
5ef0be2d71
Respond to OPTIONS requests on /query endpoint
2015-07-27 11:58:44 -07:00
38387ba6b7
Fixed typo.
2015-07-24 19:41:46 -04:00
5b50002728
Fixed typo in README
2015-07-24 17:07:41 -04:00
e722b4b4ad
Added additional logging to continuous queries
2015-07-23 19:09:42 -04:00
d9f16987fc
Merge pull request #3439 from influxdb/ga-admin-https
...
Add HTTPS option and logger to admin service
2015-07-23 15:09:56 -07:00
d1fc0a3cc9
Merge pull request #3375 from influxdb/https
...
First pass at re-enabling HTTPS.
2015-07-23 15:04:29 -07:00
b30351f750
Remove redundant loggers and clean up logic
2015-07-23 15:01:48 -07:00
8ab9424295
Return error if HTTPS fails
2015-07-23 15:00:33 -07:00
614332bf17
Exit if HTTPS fails
2015-07-23 14:50:45 -07:00
96575e678a
Merge pull request #3427 from influxdb/ga-pw-log
...
Logging tweaks, sanitize passwords and note if authentication is enabled
2015-07-23 14:12:41 -07:00
e5ead383e5
Add HTTPS option and logger to admin service
2015-07-22 16:49:12 -07:00
17cc09259b
change for better code clarity
2015-07-21 19:40:43 -06:00
785a8b4d9a
Sanitize password from HTTP logs
2015-07-21 18:28:05 -07:00
0780cf6599
Add a config test for HTTPS.
2015-07-21 18:20:08 -07:00
20de2bc914
Log authentication enabled message
2015-07-21 17:53:12 -07:00
3e29d8821a
catch opentsdb malformed tags if they are missing keys or values
2015-07-21 16:11:58 -06:00
425a65fca1
RemoteShard mapping now performed over TCP
...
With this change remote mapping no longer uses HTTP, as the HTTP ports
exposed by nodes on the cluster are not known cluster wide. The TCP
ports exposed by the cluster service are, so this change uses that
functionality. Each RemoteMapper has its own dedicated connection pool
for each node, and remote mapping TCP connections are in no way coupled
with query TCP connections.
2015-07-20 10:44:38 -07:00
47d5c2d65f
First pass at re-enabling HTTPS.
2015-07-17 16:57:31 -07:00
f549910a18
Merge pull request #3279 from LK4D4/fix_style_else
...
Fix style issues with else
2015-07-17 11:53:42 -07:00
9ba37325f6
Fixes authorization.
...
Adds GRANT and REVOKE statements for admin privilege. Adds authorization to the query endpoint.
2015-07-17 11:33:06 -07:00
675eacbf2c
Fix style issues with else
...
In go it's better to just continue flow without "else", if it is return in
"if" statement.
Signed-off-by: Alexander Morozov <lk4d4@docker.com>
2015-07-17 11:10:23 -07:00
a0dba108a0
Unit test shard mapper handler error handling
2015-07-15 16:08:26 -07:00
65a580779b
Shard mapping handler returns on errors
2015-07-15 16:08:10 -07:00
7bdba556a1
Support pretty printing shard mapping
2015-07-15 13:06:16 -07:00
74cb96646c
Refactor query engine for distributed query support
...
With this change, the query engine code gathers information about
shards and tagsets by working with individual shards, collating the
information, and returning that to the client. It does not assume that any
particular shard is local, and accesses all shards through abstracted
Mappers, of which there are two types -- a Mapper type for Raw queries
and a second type for Aggregate queries. There are corresponding
Executors for each type of Mapper, but both types of Executors share the
same interface.
2015-07-15 12:54:55 -07:00
e4f2d8a6c4
Fixed httpd logger to get user from query params
2015-07-13 17:36:34 -04:00
72f52d44c9
making queries syntactically correct
2015-07-08 17:25:56 -06:00
6b8d3268e6
Fix code review comments
2015-07-07 11:41:12 -06:00
db63ada7db
Drop NaN and Inf values from graphite input
...
NaN is skipped by graphite. Inf is not a supported value for Influxdb.
2015-07-06 16:14:02 -06:00
b58df5344c
Use a single batcher for graphite service
...
Previously there was a batcher per connection and each batcher was
flushed when the connection was closed. This didn't have much of an
effect when multiple clients connected and disconnected since it would
flush the batch immediately. It also did not help UDP traffic.
Instead, there is now a shared batcher for the service so that multiple
connections will not cause frequent flushes.
2015-07-06 16:14:02 -06:00
4a71692b88
Allow extra tags using graphite default template
...
Fixes #3223
2015-07-06 16:14:02 -06:00
0b481d55f4
Fix graphite filter searching matching wrong template
...
Default template would get chosen in some cases when a matching filter
existed.
Fixes #3245
2015-07-06 16:14:02 -06:00
a3ab093996
Parse NaN as float
...
Fixes #3230
2015-07-06 16:14:01 -06:00
87c962d62d
Fix panics when collectd fails to start
...
If collectd fails to start, it can panic when close is called because
some variables have not be initialized yet.
2015-06-29 22:09:47 -06:00
5fccee3d16
Fix spelling errors in comments and strings.
2015-06-28 02:54:34 -04:00
ff61d88c10
Create UDP batcher before listening for UDP
...
Otherwise a packet may be received before the batcher has been
instantiated.
2015-06-26 10:17:33 -04:00
2c333c6c63
Fix markdown formatting
2015-06-25 22:08:13 -06:00
562d7cd37d
Handle timestamp special cases
...
If no timestamp is sent or the value -1 is sent, the current UTC
time is used.
2015-06-24 23:53:13 -06:00
c5a10cf93d
Use raw metric name when default template fails to match
2015-06-24 23:38:10 -06:00
fbfb90d66c
Code review fixes
2015-06-24 23:33:42 -06:00
320a951575
Fix default template being returned when partially matching
...
another filter.
2015-06-24 23:09:08 -06:00
ba7187f554
Add comments to graphite parser
2015-06-24 23:09:08 -06:00
a76e812b38
Add graphite parser benchmark
2015-06-24 23:09:08 -06:00
9ed71ad492
Add test for matching similar patterns
2015-06-24 23:09:08 -06:00
613b1d2eb7
Prevent duplicate filters in config
...
A filter should map directly to one template, allowing duplicate
filters is not supported.
2015-06-24 23:09:08 -06:00
f70eee6e7a
Add support for multiple measurement fields in templates
...
Provides a little more flexibility in controlling the parsed
metric names for metris like:
servers.localhost.cpu.cpu0.user
Previously, you could only use a single field like "cpu", "user"
or a wildcard to match "cpu.cpu0.user". You can now pull out "cpu"
and "user" and join them together in the metric name using a custom
separator character. By default this is ".".
2015-06-24 23:09:08 -06:00
a2a1956048
Use search tree for filter matching
...
This adds a sorted search tree for matchining filters to a template
more efficiently. Each filter is split on "." and each element is
added to the tree. Patterns with matching prefixes are added under
the same subtree.
2015-06-24 23:09:08 -06:00
98cbfdca51
Update tempalte format comment
2015-06-24 23:09:07 -06:00
b294930c95
Add graphite plugin readme
2015-06-24 23:09:07 -06:00
1ecf9b5d36
Fix validation failing when using a default template
2015-06-24 23:09:07 -06:00
dd0e6e5e02
Use strings.Fields to bef more forgiving of whitespace
2015-06-24 23:09:07 -06:00
fed8d67946
Add validation for graphite config templates and tags
2015-06-24 23:09:07 -06:00
b55981f090
Add support for per-template default tags
...
These are tags that can be added at the template level. They
will override any global tags and any parsed tags with the same
name from the metric will override these.
2015-06-24 23:09:07 -06:00
9cd82ae316
Add support for global tags
...
These are tags that can be add to all metrics.
2015-06-24 23:09:07 -06:00
cab9e36111
Add basic template filtering support
...
This filter implementation is fairly naive and won't scale well
to large numbers of templates and filters. It will be replaced
with a trie-based approach in the future.
2015-06-24 23:09:07 -06:00
e5fbf249ff
Remove DecodeNameAndTags func
2015-06-24 23:09:07 -06:00
46046c6d32
Add matcher for match a metric to a template
...
If no template matches, use a default template that just matches
the whole metric name as the measurement.
2015-06-24 23:09:07 -06:00
9bcbbd9875
Rename matcher to template
2015-06-24 23:09:07 -06:00
d69a21c4e4
Allow skipping fields in templates
2015-06-24 23:09:07 -06:00
a7d4d97743
Convert template matching to matcher struct
2015-06-24 23:09:06 -06:00
d132263f08
Add support for measurement*
2015-06-24 23:09:06 -06:00
d539b23817
Move graphite parser to separate file
2015-06-24 23:09:06 -06:00
2a383e6858
Fix unit tests for graphite
2015-06-24 23:09:06 -06:00
c130efb5e2
Add fields to config metric name schema of graphite
2015-06-24 23:09:06 -06:00
c76e904aa3
remove commented out code
2015-06-22 15:42:13 -04:00
6cbb605930
fix #2944 : update unit tests
2015-06-22 15:40:31 -04:00
0467aa8fec
fix #2944 : don't require WHERE time on create CQ
2015-06-22 15:39:22 -04:00
57e0879f84
Correct use of config write-tracing on the httpd handler
2015-06-17 09:55:34 -03:00
e165777509
Move doc on CQs to CQ services component
2015-06-15 13:55:11 -07:00
e0c26825d3
Don't log each UDP batch
...
Fixes issue #2989 .
2015-06-15 10:29:28 -07:00
d2da99049a
collectd test client defaults to no limit points
2015-06-12 15:28:31 -07:00
1eb0347670
Add infinite loop to collectd client rate-limiter
2015-06-12 15:25:46 -07:00
45802a5911
Remove binary committed in error
2015-06-12 15:21:51 -07:00
e02bb37565
collectd test client now configurable
2015-06-12 15:18:48 -07:00
9f2964d869
Add very basic initial test collectd client
2015-06-12 14:31:53 -07:00
7f96d0ed6a
Pull WithDefaults from collectd and openTSDB
...
This pattern, which is present in Graphite, is not needed for collectd
and openTSDB.
2015-06-12 14:17:18 -07:00
44033778f5
Ensure collectd uses defaults with partial config
2015-06-12 13:56:43 -07:00
b923ce0fc8
Minor updates to openTSDB README
2015-06-12 12:06:58 -07:00
27f0b082b2
Add basic documentation for the openTSDB input
2015-06-12 12:05:23 -07:00
916b4e2df9
Rename notes.md to README.md
...
This means it will be rendered automatically by github.
2015-06-12 09:33:22 -07:00
5e7ddc6a60
Merge pull request #2928 from neonstalwart/version-header
...
pass along version to httpd/handler
2015-06-12 09:46:33 -04:00
884472ff8c
Write a newline after writing error for line protocol
...
Errors generated via curl or CLI tools can be hard to read using
a shell.
2015-06-11 16:19:19 -06:00
4dff5f48aa
Export WaitForLeader on MetaStore
...
This exported function can then be used by Services and
server-reporting, so those components don't make progress until the
cluster is ready.
2015-06-11 11:10:07 -07:00
ce84047b44
pass along version to httpd/handler
2015-06-11 11:22:37 -05:00
595807e195
fix #2920 : remove commented out code
2015-06-11 09:45:50 -04:00
7c39ede6ba
fix #2920 : create collectd database on startup
2015-06-11 09:40:42 -04:00
bc31783a00
Refactor backup and restore
...
This commit updates the snapshot code as well as the "backup" and
"restore" command to work with the new architecture.
2015-06-10 22:07:01 -06:00
e294db3c49
Set openTSDB defaults in service
2015-06-10 20:42:44 -07:00
e84f661f7e
Use defaults for Graphite input where necessary
2015-06-10 20:38:17 -07:00
91b6964705
Ensure openTSDB target database exists
2015-06-10 20:05:34 -07:00
dddaf46b77
Update CHANGELOG
2015-06-10 20:02:37 -07:00
57ce67b97b
Ensure target Graphite database exists
2015-06-10 18:03:06 -07:00
01f2d7de75
Merge pull request #2878 from influxdb/shard_precreation
...
Shard precreation service
2015-06-10 16:40:39 -07:00
d942250c5e
Merge pull request #2862 from runner-mei/connection_closed2
...
Make InfluxDB win32 friendly, fix unit test is failed on the windows
2015-06-10 15:58:57 -07:00
5798d99692
Move shard precreation logic to meta/store
2015-06-10 15:37:00 -07:00
f679c81136
Simply use "precreator" for shard precreation
2015-06-10 15:27:38 -07:00
eb96698975
Unit-test shard precreation
2015-06-10 15:27:38 -07:00
dda073c19e
Add shard precreation service
2015-06-10 15:27:38 -07:00
3b74f753a8
Add short design notes on shard precreation
2015-06-10 15:27:38 -07:00
b5ad92be70
Create shard precreation service
...
Initial commit includes basic configuration control.
2015-06-10 15:27:38 -07:00
67d4ef0e28
Don't queue write failures that due to type conflicts
...
These will never succeed and will stay in the queue indefinitely.
2015-06-10 14:52:59 -06:00
a94cf8a4ca
Consolidate imports for retention service
...
Minor cosmetic change
2015-06-10 11:25:17 -07:00
8a5cf394d8
add ability to silence logging for testing
2015-06-10 10:27:57 -05:00
015adc949c
Make InfluxDB win32 friendly, fix unit test is failed on the windows
2015-06-10 07:29:08 +08:00
cbaf3cb0ed
Support setting openTSDB consistency level
2015-06-09 15:39:31 -07:00
e21d49e85f
Add openTSDB consistency level config param
2015-06-09 15:03:52 -07:00
8eea2b3092
Allow Server creation to return errors
2015-06-09 14:21:12 -07:00
952fb49368
Move parsing consistency levels to cluster package
...
Errors parsing the consistency level are indicated by returning a nil
Graphite input.
2015-06-09 14:21:12 -07:00
5dab8de7e0
Log hinted handoff processing duration
2015-06-09 14:49:18 -06:00
c9f9b37753
Add hinted handoff queue rate limiting
...
Basic throughput limiting to dynamically maintain a bytes/sec
limit if configured for hinted handoff retry queues. If a batch is
larger than the limit, the limit will slow the processing down to one
write per second.
By default, the limit is unbounded. It can be enabled to help prevent
retstarting nodes that have queued writes for them from being overloaded
at startup.
2015-06-09 14:46:13 -06:00
939e02a900
fix #2814 : return error instead of panic
2015-06-09 00:34:46 -04:00
88a7774a80
fix #2814 : remove unnecessary initialization
2015-06-09 00:34:46 -04:00
cef04db18f
fix #2814 : use toml.Duration where needed
2015-06-09 00:34:46 -04:00
2f03a07a32
fix #2814 : move repetative test data to global var
2015-06-09 00:34:46 -04:00
d5f52333a1
fix #2814 : hook collectd service back up
2015-06-09 00:34:46 -04:00
4769d65f23
Final Graphite test code fixes
2015-06-08 20:39:56 -07:00
9d628320c2
Rename test file to service_test.go
2015-06-08 20:39:42 -07:00
4d01c8ad41
Add test of Graphite UDP input
2015-06-08 20:39:42 -07:00
cd038ccddb
Add test of Graphite TCP input
2015-06-08 20:39:42 -07:00
b0d76913b0
Add new Service-style Graphite input support
2015-06-08 20:39:39 -07:00
1f8b04729f
Allow Graphite consistency control
...
Remove obsolete Graphite files
2015-06-08 20:00:05 -07:00
c1e093bc08
Add batching control to Graphite config
2015-06-08 20:00:05 -07:00
120b64fee8
Actually unit-test Graphite configuration
2015-06-08 20:00:05 -07:00
c1c8c5f0ca
Merge pull request #2835 from influxdb/jw-json-writes
...
Skip leading whitespace when detecting JSON writes without content-type
2015-06-08 17:49:44 -06:00
a9657289c4
Merge pull request #2823 from benbjohnson/opentsdb
...
Convert OpenTSDB to service
2015-06-08 17:32:55 -06:00
07d1aac50f
Fix comment in hinted handoff processor
2015-06-08 16:51:48 -06:00
48b98fea70
Skip leading whitespace when detecting JSON writes without content-type
...
Fixes #2834
2015-06-08 16:44:47 -06:00
b688eccb77
Refactor OpenTSDB to a service
...
This commit converts the OpenTSDB endpoint into a service.
2015-06-08 15:12:16 -06:00
ede254484d
Process hinted handoff queues concurrently
2015-06-08 15:05:44 -06:00
d86b15953c
Fix queue advance not writing updated position
...
The advance function was not writing the updated position in the
queue until after the next advance call was called. Resulted in
the very last message would get replayed on restart each time.
2015-06-08 12:09:31 -06:00
e75208f15c
Log remote write erros
2015-06-08 09:09:10 -06:00
2ccf97e6a0
Add wait group to hinted handoff service
2015-06-05 22:16:52 -06:00
4ec77d8b84
Fix comments
2015-06-05 22:16:52 -06:00
e2f3ff26a5
Fix data race when close hinted handoff service
2015-06-05 22:16:51 -06:00
f4cea559d3
Purge hinted handoff queue if entries stick around past a cutoff
2015-06-05 22:16:51 -06:00
9d67f9bf62
Add hinted handoff max age and throughput rate limit config options
2015-06-05 22:16:51 -06:00
809b9b8a83
Add basic hinted handoff support
...
If a remote write fails, it will be queued to a per-node, local disk
queue and retried later.
2015-06-05 22:16:51 -06:00
89c01cd37d
Add hinted handoff queue
...
This adds a disk backed queue that will be to persist hinted
handoff writes.
2015-06-05 22:16:51 -06:00
75b72c60fe
Add hinted handoff service
...
The hinted handoff service will queue a write to a remote node if
that write fails and periodically retry the write.
2015-06-05 22:16:51 -06:00
170c584b45
Add hinted handoff config and tests
2015-06-05 22:16:51 -06:00
fcf89c87dd
cr #2733 : uncomment defer in test
2015-06-05 22:30:57 -04:00
a235b4998d
fix #2733 : trying to fix CI race
2015-06-05 20:23:44 -04:00
e107a681c5
cr #2733 : make error message more detailed
2015-06-05 19:56:28 -04:00
d8f0445940
fix #2733 : fix race in tests
2015-06-05 19:56:28 -04:00
fb514f2302
fix #2733 : add endpoint to trigger CQ(s)
2015-06-05 19:56:28 -04:00
8699aa8f0b
fix #2733 : more tests
2015-06-05 19:56:28 -04:00
e0838cb705
fix #2733 : add tests for CQ service
2015-06-05 19:54:23 -04:00
092bc3fd2d
fix #2733 : hook CQs back in
2015-06-05 19:54:23 -04:00
041b31d6b3
Merge pull request #2783 from renan-/udp-refresh
...
Make UDP service code simpler
2015-06-05 16:23:16 -07:00
b925e1c1af
Multi-node clustering.
...
This commit adds the ability to cluster multiple nodes together to share
the same metadata through raft consensus.
2015-06-05 14:41:19 -06:00
c59a234e7b
Deleted shard groups are not counted as "expired"
2015-06-05 13:08:36 -07:00
a4a05241ba
Set retention enforcement default config
2015-06-05 11:27:30 -07:00
967ebe0568
Stop retention timers on service shutdown
2015-06-05 08:17:27 -07:00
e8ab17dbc5
[services/udp] Clean-up the code
2015-06-05 10:55:10 +02:00
5e5f2cd37d
Move expired and deleted shard logic to MetaStore
2015-06-04 22:18:52 -07:00
ae3e8c727a
Don't append retention service if not enabled
2015-06-04 21:47:06 -07:00
Paul Dix
Cory LaNou
David Norton
Jonathan A. Sternberg
Sebastian Borza
Yin Jifeng
Nathaniel Cook
Tim Raymond
Philip O'Toole
Edd Robinson
Paulo Pires
Данило Глинський (Danylo Hlynskyi)
Philip O'Toole
Jonathan A. Sternberg
Paweł Kowalak
orthogonous
John Nguyen
Cameron Sparr
ch33hau
Mark Bates
Nicholas Katsaros
oiooj
David Howden
Jason Wilder
bwhaley
Nathaniel Cook
Sean Beckett
Daniel Morsing
Sergey Kamardin
Nick Dawbarn
Rob Wilson
Antonio Murdaca
Alex Iribarren
Jason Wilder
Ben Johnson
Gunnar
Jon Seymour
gunnaraasen
Nathaniel Cook
Dimitrij Denissenko
Vorn Mom
Josh Horwitz
Todd Persen
Alexander Morozov
Joseph Crail
Can ZHANG
Vladimir Lopes
ben hockey
runner.mei
Renan Strauss