Commit Graph

473 Commits (34544d2c4b68158c0588d55a2d2d221ffb3b1a1b)

Author SHA1 Message Date
Nathaniel Cook 8b31007aa7 Adds subscriber service for creating/dropping subscriptions to the
InfluxDB data stream.
2015-10-14 15:23:45 -06:00
Philip O'Toole f298e88b39 Auto-create UDP service database
All other services operate like this, so make UDP service consistent.
2015-10-14 08:30:09 -07:00
Daniel Morsing 6d188d9703 Merge pull request #4409 from influxdb/intoq
wire up INTO queries.
2015-10-14 15:29:54 +01:00
Daniel Morsing 48223a90f9 Fix spelling nit.
Make it obvious that INTO is a keyword and not part of the sentence
2015-10-14 15:16:27 +01:00
Sean Beckett 8e65f0c84f Update server_test.go 2015-10-13 16:53:10 -07:00
Sean Beckett 9a9e04084f Update config.go 2015-10-13 16:49:09 -07:00
Daniel Morsing 822af73f88 implement continuous queries as regular execs of into queries.
Now that we have into queries, we can implement them as regular
queries that are just run on a timer.
2015-10-13 15:51:19 +00:00
Daniel Morsing 62dff895e2 wire up INTO queries.
Since INTO queries need to have absolute information about the database
to work, we need to create a loopback interface back to the cluster
in order to perform them.
2015-10-13 15:00:36 +00:00
Cory LaNou 6787525912 fixes multiple selectors overwriting each other. fixes #4360 2015-10-12 21:40:57 -05:00
David Norton 512d6ac050 fix #4280: only drop points matching WHERE clause 2015-10-09 18:34:32 -04:00
linearb 60d298936e return error when database rename conflicts with continuous queries 2015-10-09 13:57:02 -04:00
linearb 668b5b9bfb change syntax to ALTER DATABASE ... RENAME TO ... 2015-10-09 13:56:23 -04:00
linearb d2afd881e6 add integration test for RENAME DATABASE 2015-10-09 13:55:38 -04:00
linearb 7212bfce83 add RENAME DATABASE 2015-10-09 13:55:38 -04:00
Philip O'Toole 5b0a8ed306 HH should not process dropped nodes 2015-10-08 18:23:12 -07:00
Nick Dawbarn 26f6d00668 Bugfix for #3429 String representations of RegexLiterals generated in influxql/ast.go add the / char as a start and end delimiter, but does not escape any / characters that may exist with the regex 2015-10-08 19:41:36 +10:00
David Norton 4375545064 fix #4276: walk DropSeriesStatement 2015-10-05 19:56:30 -04:00
Philip O'Toole d74e0690c7 Revert "Merge pull request #4233 from influxdb/drop-server"
This reverts commit 0bdb36f6dc, reversing
changes made to 3085fbc138.
2015-10-02 08:39:57 -07:00
Cory LaNou 99da67007d no more shutdown 2015-10-01 15:39:15 -05:00
Cory LaNou 73372ed907 [Ee]xecuteShutdown -> [Mm]onitorShutdown 2015-10-01 15:39:15 -05:00
Cory LaNou 34bfb6b330 be sure to exit the go routine if we terminate normally 2015-10-01 15:39:15 -05:00
Cory LaNou 7a3e1f6b27 removing peer wip 2015-10-01 15:39:15 -05:00
Ben Johnson 343dd23ee7 refactor map functions to use list of values
This commit changes `tsdb.mapFunc` to use `tsdb.MapInput` instead
of an iterator. This will make it easier and faster to pass blocks
of values from the new storage engine into the engine.
2015-09-29 14:00:33 -06:00
Philip O'Toole 7cb8c2d2ec Add build timestamp to version data 2015-09-24 23:40:53 -07:00
Cory LaNou b1becfbcfc more tests, simplify mapFunc signatures 2015-09-23 09:27:19 -05:00
Cory LaNou 78bc740434 full support for min/max/first/last 2015-09-23 09:03:57 -05:00
Ben Johnson 8e27cf1fd8 Merge remote-tracking branch 'upstream/master' into refactor-select-mapper
Conflicts:
	tsdb/store.go
2015-09-22 13:58:24 -06:00
Ben Johnson 56cb2fae5d fix integration tests 2015-09-22 13:10:13 -06:00
David Norton 8bd2408320 convert SHOW TAG KEYS to distributed query 2015-09-21 11:30:51 -04:00
Philip O'Toole f73bc6ff19 Merge pull request #4140 from influxdb/engine_config
Make engine configurable
2015-09-16 21:17:43 -07:00
Philip O'Toole 7b1a4e6700 Control whether each query should be logged
Fixes issue #4138
2015-09-16 19:26:23 -07:00
Philip O'Toole e4fde993f1 Make engine configurable 2015-09-16 19:09:25 -07:00
Cory LaNou bf955c7c11 Merge pull request #4112 from influxdb/issue-3457
Period in field name that matches measurement fails.  Fixes #3457
2015-09-16 09:55:43 -05:00
Philip O'Toole afe5cc67f4 Repeat the identical time aggregate test more; 2015-09-15 16:22:25 -07:00
Cory LaNou f02d8eef4f unskip test 2015-09-15 16:36:11 -05:00
Cory LaNou 5395ac7634 Period in field name that matches measurement fails. Fixes #3457 2015-09-15 16:26:39 -05:00
Philip O'Toole 69c9f28e2d Fix compilation error in test code 2015-09-15 11:42:58 -07:00
Philip O'Toole 76e3552a95 Increase write-timeout to 30s during testing
This should hopefully reduce the rate of CI failures during testing.
2015-09-15 11:35:21 -07:00
Philip O'Toole ed917aa7f8 Log each query before execution
If a query causes a server to panic, the query is nowhere in the logs,
which makes debug very difficult.
2015-09-10 10:52:51 -07:00
Cory LaNou ef17dbb492 more tests 2015-09-10 11:32:30 -05:00
Cory LaNou fa04b0ab20 add server integration tests 2015-09-09 15:46:21 -05:00
Philip O'Toole 0ea0a3a71b Add build info to diagnostics 2015-09-09 12:43:51 -07:00
Philip O'Toole 402576bd93 Pass richer build information to Server 2015-09-09 12:36:32 -07:00
Philip O'Toole ca07b86254 Prohibit dropping the default retention policy
This is to prevents users from putting their system into an awkward
state. It is a policy that all databases must have at least a default
retention policy.

Fixes issue #3699.
2015-09-08 23:00:19 -07:00
Cory LaNou 62e9c24b25 fixes #3926 2015-09-08 14:15:48 -05:00
Jason Wilder 77d310337c Clarify comment 2015-09-08 11:05:39 -06:00
Jason Wilder dabb01691e Close server is reverse order that it was started
The server was closing by stopping the most depended on services first
which causes various panics while higher level services are still processing
task when the server closes.

Fixes #3881
2015-09-08 11:04:00 -06:00
Philip O'Toole bbc103305b Support multiple Graphite inputs
Fixes issue #3636
2015-09-06 21:33:46 -07:00
Jason Wilder b8da247b08 Open monitor server after metastore is ready 2015-09-05 09:06:51 -06:00
Philip O'Toole 267f8e6c11 Skipping race test
Issue #3997
2015-09-04 11:59:28 -07:00
Cory LaNou 3867fed0ff one additional test for top 2015-09-04 13:30:42 -05:00
Cory LaNou 9703467171 refactor validateAggregates 2015-09-04 13:30:41 -05:00
Cory LaNou 347ffc70b4 wire up advanced top sorting/slicing 2015-09-04 13:30:41 -05:00
Cory LaNou 8c4595b345 top is coming together. filling out fields properly 2015-09-04 13:30:41 -05:00
Cory LaNou ba79007960 wip 2015-09-04 13:30:41 -05:00
Cory LaNou f3e557d943 more top test scenarios 2015-09-04 13:30:41 -05:00
Cory LaNou 5a66725054 refactoring/adding top integration testing 2015-09-04 13:30:40 -05:00
Cory LaNou 35b9215aa9 refactor processTopBottom - wip 2015-09-04 13:30:40 -05:00
Cory LaNou 046282249a wip remapping top output 2015-09-04 13:30:40 -05:00
Cory LaNou 72fd115dc2 exposing tags on cursors, top/bottom are valid funcs now 2015-09-04 13:30:39 -05:00
dgnorton a3def8d3a7 Merge pull request #3993 from influxdb/dgn-fix-cq-test
fix CQ test
2015-09-04 13:12:29 -04:00
David Norton 0d43fdb94c fix CQ test 2015-09-04 12:35:41 -04:00
Jason Wilder df70a1c8ce Update tests to use Direction enum 2015-09-04 09:00:11 -06:00
Jason Wilder b7c0d47008 Add aggregate order by desc test 2015-09-03 22:31:58 -06:00
Jason Wilder 266bdc1c2b Support sort by time DESC in wal and bz1 engines 2015-09-03 22:28:36 -06:00
Cory LaNou 6592dcc699 EnableLogging -> LoggingEnabled 2015-09-03 16:56:07 -05:00
Cory LaNou 473c04dc0d silence wal logging during testing 2015-09-03 15:17:03 -05:00
Ben Johnson deff06f850 add copier service
This commit adds the copier service which allows one server to
copy shards from another server. This will be used for moving
shards in the cluster.
2015-09-03 13:07:35 -06:00
David Norton 4187fb4257 skip test instead of commenting out 2015-09-03 09:40:45 -04:00
David Norton 2739f5e688 temporarily disable a CQ test 2015-09-03 07:12:15 -04:00
David Norton 66001cfbb5 fix #2555: add integration tests for CQs 2015-09-03 07:12:15 -04:00
Philip O'Toole c0d61740a4 Ensure monitor database exists
This requires a leader in the cluster, so wait for that before
attempting the database creation.
2015-09-02 16:26:16 -07:00
Philip O'Toole d820fa7649 Add PointsWriter to monitor
This will allow the monitor to persistently store statistics
2015-09-02 15:45:11 -07:00
Philip O'Toole 9165a0cf4c Create statement_executor for monitor 2015-09-02 15:44:56 -07:00
Philip O'Toole 4e2ee1ea70 Rename MonitorService to just Monitor
monitor is not a service, it has more in common with meta, since it
provides functionality to the query layer. This names makes this
clearer.
2015-09-02 15:07:30 -07:00
Philip O'Toole 9df17409d3 Use monitor service with Graphite 2015-09-01 15:21:36 -07:00
Philip O'Toole f05dc20b58 Hook new monitor service to server
u
2015-09-01 15:03:52 -07:00
Philip O'Toole d87e668c78 Remove obsolete monitoring code 2015-09-01 15:03:52 -07:00
Philip O'Toole 1a55951f36 Backend support for database IF NOT EXISTS 2015-08-28 19:04:54 -07:00
Philip O'Toole e0066760ba Make the startup log message actually first 2015-08-28 15:14:20 -07:00
Philip O'Toole 191163c7c7 Move to Go 1.5 2015-08-27 09:58:15 -07:00
Philip O'Toole 94484067f3 Log Go runtime version 2015-08-26 14:47:59 -07:00
Philip O'Toole 10779c9634 Unit-test predictable first/last point ordering 2015-08-25 22:26:04 -07:00
Philip O'Toole ca52a3ffd7 Allow queries to be looped
Some queries may be subject to race conditions. This variable allows the
test to request that a query is executed multiple times, increasing the
chance that any race will be exposed.
2015-08-25 15:16:13 -07:00
Philip O'Toole 20e5d5f3a9 Test explicitly SELECTing time as a field 2015-08-21 12:14:53 -07:00
Philip O'Toole cd0ce4ac55 Re-enable chunking test post WAL updates
Fixes issue #3725.
2015-08-20 12:54:48 -07:00
Jason Wilder cd3ef17fc6 Merge pull request #3743 from influxdb/jw-env-config
Apply env overrideds for config command
2015-08-19 15:20:03 -06:00
Jason Wilder 0b2aa62172 Merge pull request #3737 from influxdb/jw-wal-dir
Ensure wal-dir has a default value
2015-08-19 13:53:37 -06:00
Jason Wilder 67c54ddc47 Apply env overrideds for config command
Matches the behavior of the run command and lets you test env
vars override config values more easily.
2015-08-19 13:52:28 -06:00
Jason Wilder 6bd9561de0 Ensure wal-dir has a default value 2015-08-19 13:39:22 -06:00
Philip O'Toole de95a42d4e Skip failing chunking test
Until https://github.com/influxdb/influxdb/issues/3725 is fixed.
2015-08-19 11:40:02 -07:00
Paul Dix c31b88de60 Merge pull request #3569 from influxdb/pd-wal
Add initial WAL implementation and tests
2015-08-18 20:45:32 -04:00
Paul Dix 9df3b7d828 Add WAL configuration options 2015-08-18 16:59:54 -04:00
Philip O'Toole 28a6b1f3fd Merge pull request #3697 from influxdb/chunking_10k
Merge same-series data if not chunking
2015-08-18 13:23:10 -07:00
Philip O'Toole 0eb93d675d Add some unit tests related to partner testing 2015-08-17 14:36:13 -07:00
Philip O'Toole 487c336571 Correctly merge rows for identical series
If no chunking was requested by the user, the co-ordinating node buffers all
results in RAM before emitting a single result. However buffering was not
merging results for rows which had data for the same series. This change fixes this.

Fixes issue #3242.
2015-08-17 13:43:17 -07:00
Daniel Morsing 29de960638 adjust granularity of block profile
32bit int platforms can't handle a constant this high, so cap it at
1 second instead.
2015-08-17 09:55:38 -07:00
Cory LaNou 014812d967 Merge pull request #3628 from influxdb/wildcard-expansion
Wildcard expansion
2015-08-14 15:29:14 -05:00
Daniel Morsing 72f51b0923 Merge pull request #3652 from influxdb/blockprof
turn on block profiling
2015-08-14 13:24:51 -07:00
Cory LaNou 4e04c069f9 error out if only tags are in the select clause 2015-08-14 15:24:37 -05:00
Cory LaNou b171ab6655 add many non GROUP BY * test scenarios. clean up comments for fixed tests 2015-08-14 15:05:27 -05:00
Cory LaNou d083ca463e add non group by host tests 2015-08-14 15:05:27 -05:00
Cory LaNou d58851a679 silence reporting in integration tests 2015-08-14 15:05:26 -05:00
Cory LaNou f507059ab4 more wildcard expansion tests 2015-08-14 15:05:26 -05:00
Cory LaNou 7fdb682966 first pass at raw queries with tags 2015-08-14 15:05:26 -05:00
Cory LaNou dc83c57d7e more wip 2015-08-14 15:05:26 -05:00
Cory LaNou 8ea3c47747 wip 2015-08-14 15:05:26 -05:00
Philip O'Toole 7b4879f0ce Fully remove a series when dropped
Fix issue #3226.
2015-08-14 10:50:35 -07:00
Jason Wilder f544113e8c Add test for CQ deadlock
Test script from #3517 that reproduces a CQ deadlock.  This is related
to #3522 as well.
2015-08-14 10:46:25 -06:00
Daniel Morsing 765eee74b1 turn on block profiling
This should help us figure out where exactly a database is hung when
it happens.
2015-08-13 15:28:53 -07:00
Philip O'Toole c81a0afb8f Merge pull request #3635 from influxdb/add_branch_to_version
Add build branch to version information
2015-08-13 14:41:14 -07:00
Philip O'Toole 717bce6899 Add build branch to version information 2015-08-13 13:40:51 -07:00
Philip O'Toole f74248c4e4 Add new server-level unit test
Added this as part of working on issue #3603.
2015-08-13 13:08:40 -07:00
Jason Wilder 668181d275 Make log statements more consistent
* Capitalize first letter of message
* Log all services staring consistently
* Remove some extraneous log statements in meta.Store
* Log data dirs for meta, data and hinted handoff
2015-08-13 10:01:42 -06:00
Jason Wilder 5dfd25269f Fix panic: reflect: call of reflect.Value.NumField on string Value
The env var overrides panic if we if peers was set in the config.
The type of that value is a []string and NumFields is not a valid
for that type.  We don't currently support this slice fields via
the env variable settings.  This particular one can be set with the
-join flag already so we just skip these fields for now.
2015-08-12 11:27:04 -06:00
Jason Wilder f07b8d4016 Make sure port is optional when passing hostanme flag 2015-08-12 10:41:43 -06:00
Philip O'Toole be792b926a Re-enable regex test
Test now passes after recent changes.
2015-08-10 17:17:17 -07:00
Philip O'Toole 5b39b96b9d Re-enable RP test
Test passes after latest changes.
2015-08-10 17:15:46 -07:00
Philip O'Toole f5c8280525 Fix multi UDP tests 2015-08-10 16:23:04 -07:00
Philip O'Toole 8dbcff833b Revert "Revert "Merge pull request #3599 from tpitale/multiple-udp-servers""
This reverts commit 5f83f038e1.
2015-08-10 16:21:27 -07:00
Philip O'Toole 5f83f038e1 Revert "Merge pull request #3599 from tpitale/multiple-udp-servers"
This reverts commit 7f35d65412, reversing
changes made to f828781544.
2015-08-10 14:44:14 -07:00
Philip O'Toole 7f35d65412 Merge pull request #3599 from tpitale/multiple-udp-servers
Change single udp config to support multiple UDP listeners
2015-08-10 14:22:27 -07:00
Philip O'Toole 84af826a0f Re-enable test "TestServer_Query_Now"
Recent changes must have fixed issue #2874.
2015-08-10 14:10:20 -07:00
Philip O'Toole a122e709ea Add system-level test for math 2015-08-10 13:13:04 -07:00
Jason Wilder 11f4ba82aa Add env override config test 2015-08-09 16:14:10 -06:00
Jason Wilder 775dadadf5 Use INFLUXDB as env var prefix 2015-08-09 16:06:38 -06:00
Tony Pitale e943afacdc Change (previously unused) single udp config to support multiple UDP listeners
do not use NewConfig for UDP

appendUDPService must return a value

udp service does not need to handle error

fix missing case of c.UDP in tests
2015-08-09 11:28:09 -04:00
Jason Wilder 6f9c18fe95 Allow overriding of configuration parameters using environment variables
This allows all config variable to be set via environment variables using
a similar naming convention for the toml config.  For example, to change the
HTTP API port using the config, you would set:

[http]
  bind-address = ":8086"

To change it with an environment variable, you would use:

  HTTP_BIND_ADDRESS=":8086" influxd

The section name is used as the env variable prefix and the config key
name is the suffix.  The only change to the config name is that "-" should
be replaced with "_" to avoid shell interpretation issues.

This makes it much easier to configure docker instances within a docker container
or adhoc instances at the command-line.

For slice config sections like graphite, you can currently only override the first
entry since the default config only has 1 entry.  To do that use, GRAPHITE_0 as the
prefix.  You cannot currently add new entries like GRAPHITE_1.  A future PR might
address this issue.

The environment variable values should be the same as the config values.

The order that configuration values are applied is as follows:
* Default config
* Config file
* Environment variables
* Command-line arguments

Fixes #3246
2015-08-07 14:27:38 -06:00
Jason Wilder 87376a1c35 Allow -hostname to also override remote addr port
When starting a influxd in a docker container, the processess needs to know
the hosts address and port in order to create its NodeInfo correctly.  -hostname
previously only allowed us to change the hostname and the port would always be 8088
which may not be correctly if running multiple containers on the same host.
2015-08-07 14:03:08 -06:00
Jason Wilder c90ffff855 Fallback to HOME env var to determine home dir
When building a static binary without cgo, user.Current() uses cgo
and always fails.  Fallback to to HOME env variable if it exists.
2015-08-07 14:03:08 -06:00
Cory LaNou 2d26c2c9fe Additional test coverage for non-existent fields 2015-08-06 15:01:34 -05:00
Cory LaNou fdaa809f9a fix some tests, add some tests, fix some test comments 2015-08-06 14:01:29 -05:00
Cory LaNou b19a8f3968 Fix alias, maintain column sort order 2015-08-06 14:01:03 -05:00
Jason Wilder 2b76dac479 Don't resolve hostname when creating node
Hostnames were always being resolved to an IP address and the IP
address was used as the host address and raft peer address.  There
was no way to use an actual hostname instead of an IP address.
2015-08-05 14:17:26 -06:00
Daniel Morsing 38b70b2cfd add test for evil identifiers that stress the parser 2015-08-05 19:41:00 +01:00
David Norton d661bf1a06 fix #3414: shard mappers perform query re-writing 2015-08-04 09:49:50 -04:00
David Norton 0b248e225c Fix aggregate queries and time precision on where clauses. 2015-07-29 15:54:55 -04:00
Cory LaNou 4bc7e11adb Continuous Query Tests 2015-07-28 14:58:44 -05:00
Jason Wilder b6c95925fb Add retry delay for report service loop when error occurs
There is a race when stopping servers where the meta.Store is closing
but the server has not signaled it is closing so the reporting goroutine
repeeatedly errors out in fast loop during this time.  It creates a lot
of noise in the logs.
2015-07-28 09:40:26 -06:00
Jason Wilder 514f36cf54 Exit report goroutine if server is closing 2015-07-28 09:40:25 -06:00
Jason Wilder 06d8ff7c13 Use config.Peers when passing -join flag
Removes the two separate variables in the meta.Config.  -join will
now override the Peers var.
2015-07-28 09:40:25 -06:00
Jason Wilder 47b8de7ce8 Hide Meta.Join from config command using toml skip annotation 2015-07-23 10:21:25 -06:00
Jason Wilder 84a8d7d24b Add cluster-tracing option to meta config
Useful for troubleshooting but too verbose for regular use.
2015-07-23 10:21:24 -06:00
Jason Wilder c1fc83e3d5 Make join private so it does not show up in config command 2015-07-23 10:21:24 -06:00
Jason Wilder e06f6f421b Add ability to join an existing cluster
This adds some basic ability to join a node to an existing cluster. It
uses a rpc layer to initiate a join request to an existing memeber. The
response indicates whether the joining node should take part in the raft
cluster and who it's peers should be.  If raft should not be started, the
peers are the addresses of the current raft members that it should delegate
consensus operations.

To keep the meta store implementation agnostic of whether it's running
a local raft or not, a consensusStrategy type was also added.
2015-07-23 10:21:22 -06:00
Jason Wilder abfd438a11 Implement remote meta data fetch
This adds some basic plumbing to make remote procedure calls to other cluster
members.  This first implementation allows a node to contact the raft leader
and fetch a copy of the meta data. This will be used by non-raft members to
pull down the latest metadata.
2015-07-23 10:21:22 -06:00
Ben Johnson a7f50ae03c refactor storage to engine 2015-07-22 11:08:10 -06:00
Philip O'Toole 112da5205b Add 2 passing test cases around multi-shard select 2015-07-21 07:28:20 -07:00
Philip O'Toole 29570ba563 Merge pull request #3406 from influxdb/math_tests
Basic mult and divide tests on aggregates
2015-07-20 19:09:29 -07:00
Philip O'Toole 178856e97e Basic mult and divide tests on aggregates 2015-07-20 19:02:03 -07:00
Philip O'Toole 7e0fa132c8 Raw query with no valid SELECT fields is not data 2015-07-20 15:59:52 -07:00
Philip O'Toole 10eecb441d Allow remote mapping to be forced
This is useful primarily for testing.
2015-07-20 10:44:45 -07:00
Philip O'Toole 425a65fca1 RemoteShard mapping now performed over TCP
With this change remote mapping no longer uses HTTP, as the HTTP ports
exposed by nodes on the cluster are not known cluster wide. The TCP
ports exposed by the cluster service are, so this change uses that
functionality. Each RemoteMapper has its own dedicated connection pool
for each node, and remote mapping TCP connections are in no way coupled
with query TCP connections.
2015-07-20 10:44:38 -07:00
Philip O'Toole 3e1b0fed78 Merge pull request #3345 from influxdb/ga-show-rp-syntax
Make `SHOW RETENTION POLICIES` consistent by requiring `ON`
2015-07-17 10:46:24 -07:00
Philip O'Toole 0d6c6bbe6f Correctly check if raw derivative is required
The multiple checks for Mapper and Executor type -- the lack of DRYness
in this code -- meant the same checks would need to be copied. Therefore
this change, as well as fixing the bug, improves the situation a little
bit by *asking* the Mappers what type of Executor is required. This code
is still not ideal.

Fixes #3355.
2015-07-16 23:28:38 -07:00
gunnaraasen ee8ba11c4f Add ON token to SHOW RETENTION POLICIES parser with tests 2015-07-16 13:45:50 -07:00
Philip O'Toole bd2f475b5d Merge pull request #3334 from mcastilho/master
Clean shutdown of influxd server
2015-07-16 13:00:13 -07:00
Philip O'Toole d8c31f0b3c Merge pull request #3320 from influxdb/streaming_dq
Support Distributed Queries
2015-07-16 12:33:21 -07:00
Philip O'Toole f41d2bab5d Start move to unified query executor 2015-07-15 19:31:13 -07:00
Philip O'Toole 74cb96646c Refactor query engine for distributed query support
With this change, the query engine code gathers information about
shards and tagsets by working with individual shards, collating the
information, and returning that to the client. It does not assume that any
particular shard is local, and accesses all shards through abstracted
Mappers, of which there are two types -- a Mapper type for Raw queries
and a second type for Aggregate queries. There are corresponding
Executors for each type of Mapper, but both types of Executors share the
same interface.
2015-07-15 12:54:55 -07:00
Philip O'Toole 21ea87432c Test that DROP DATABASE is isolated 2015-07-15 11:20:25 -07:00
Marcio Castilho e560eb19ee Clean shutdown of influxd server 2015-07-15 13:14:11 -04:00
Philip O'Toole a84c48bff6 Allow the PointWriter timeout to be configurable 2015-07-02 12:50:12 -04:00
Philip O'Toole ca86fa2633 Allow WAL inter-flush time to be configurable 2015-07-02 10:40:26 -04:00
Philip O'Toole c4a75ffef7 Merge pull request #3180 from influxdb/log_commit
Log GOMAXPROCS, version, and commit on start
2015-06-29 14:01:57 -04:00
Philip O'Toole e554115178 Log GOMAXPROCS, version, and commit on start 2015-06-29 13:52:03 -04:00
Joseph Crail 5fccee3d16 Fix spelling errors in comments and strings. 2015-06-28 02:54:34 -04:00
Ben Johnson b574e2f755 Add write ahead log
This commit adds a write ahead log to the shard. Entries are cached
in memory and periodically flushed back into the index. The WAL and
the cache are both partitioned into buckets so that flushing doesn't
stop the world as long.
2015-06-25 15:47:13 -06:00
Jason Wilder ea348ddc2c Add sample graphite config to default config 2015-06-24 23:09:07 -06:00
Jason Wilder fed8d67946 Add validation for graphite config templates and tags 2015-06-24 23:09:07 -06:00
Philip O'Toole e010daf74c Test GROUP BY when different tags have same value 2015-06-22 16:04:13 -07:00
Philip O'Toole 841a8d35ee Test queries after drop-and-create database with data 2015-06-22 11:55:44 -07:00
Philip O'Toole 8bff4c1c51 Unit test minimum retention policy duration 2015-06-17 11:02:38 -07:00
Philip O'Toole 60d338282e Unit test measurement recreation after DROP 2015-06-16 12:14:36 -07:00
Philip O'Toole 05bd0fc53d Check database existence during normalization
Statements were only being normalized if a default database was included
in the query (usually via the query param 'db'). However if no default
database was included, and none was an explicit part of the measurement
name, no database-existence check was run. This result in a later panic
with wildcard expansion.
2015-06-15 11:51:32 -07:00
Philip O'Toole b440c593a4 Merge pull request #2987 from neonstalwart/server_test-fix
fix version assertion in server_test
2015-06-15 10:26:11 -07:00
Cory LaNou a9d2e81772 More integration tests for retention policy 2015-06-15 11:00:41 -05:00
Cory LaNou e52acd3057 show tag field integration tests 2015-06-15 10:34:11 -05:00
ben hockey 90ea63baad fix version assertion in server_test 2015-06-15 09:44:49 -05:00
Philip O'Toole 85bb693119 Unit test version in HTTP response 2015-06-12 12:31:31 -07:00
Philip O'Toole d5fda01c2a Server now requires build information
Version is worth special-casing here, since the build information is
pretty important, and make components need it.

Fixes issue #2958.
2015-06-12 11:41:25 -07:00
Philip O'Toole 23a7896182 Server does not need Commit info, remove it 2015-06-12 11:28:32 -07:00
Cory LaNou 6cc9a2ee20 SHOW TAGS integration tests 2015-06-12 12:16:27 -05:00
dgnorton 0318b4562e Merge pull request #2936 from influxdb/fix-2935
fix #2935: hook CPU and memory profiling back up
2015-06-12 12:10:27 -04:00
David Norton b0b8d25752 fix #2935: remove delay on memprofile 2015-06-12 12:07:01 -04:00
Cory LaNou 69a20970ff SHOW MEASUREMENTS integration tests 2015-06-12 10:49:28 -05:00
David Norton fd2f1bb223 cr #2935: make code review changes 2015-06-12 11:47:59 -04:00
David Norton f717019d88 fix #2935: hook CPU and memory profiling back up 2015-06-12 11:47:59 -04:00
Cory LaNou 3686f2c496 Show series integration tests 2015-06-12 10:38:24 -05:00
Paul Dix 5e7ddc6a60 Merge pull request #2928 from neonstalwart/version-header
pass along version to httpd/handler
2015-06-12 09:46:33 -04:00
ben hockey aef4ba2e20 fix typos in server_test.md 2015-06-11 15:43:43 -05:00
Philip O'Toole e89d2d85a6 Merge pull request #2931 from influxdb/wait_for_leader
Export WaitForLeader on MetaStore
2015-06-11 13:26:00 -07:00
Cory LaNou 2373d9a68f Integration tests for drop measurement 2015-06-11 15:04:23 -05:00
Cory LaNou 224c6bb698 docs on integration testing 2015-06-11 14:17:07 -05:00
Philip O'Toole 4dff5f48aa Export WaitForLeader on MetaStore
This exported function can then be used by Services and
server-reporting, so those components don't make progress until the
cluster is ready.
2015-06-11 11:10:07 -07:00
ben hockey ce84047b44 pass along version to httpd/handler 2015-06-11 11:22:37 -05:00
Cory LaNou 2ecc2162cb add fill tests 2015-06-11 10:52:32 -05:00
Cory LaNou de17fcd6f6 missing parens to call func 2015-06-11 10:51:48 -05:00
David Norton a325071103 fix #2920: set MetaStore in collectd service 2015-06-11 09:53:16 -04:00
Philip O'Toole 431034c8f4 Wait for Raft leader even when cluster exists 2015-06-11 01:23:30 -07:00
Philip O'Toole b15a0df6d4 Wait for leader before reporting 2015-06-11 00:21:15 -07:00
Philip O'Toole 406509e93e Remove rogue ID field from reporting 2015-06-11 00:21:15 -07:00
Philip O'Toole 68fa173839 Report in goroutine so it outlasts return 2015-06-11 00:21:15 -07:00
Philip O'Toole 64af1b6241 Report number of measurements and series per node 2015-06-11 00:21:15 -07:00
Ben Johnson bc31783a00 Refactor backup and restore
This commit updates the snapshot code as well as the "backup" and
"restore" command to work with the new architecture.
2015-06-10 22:07:01 -06:00
Philip O'Toole 223bf672cf Update CHANGELOG 2015-06-10 20:05:59 -07:00
Philip O'Toole 57ce67b97b Ensure target Graphite database exists 2015-06-10 18:03:06 -07:00
Philip O'Toole f679c81136 Simply use "precreator" for shard precreation 2015-06-10 15:27:38 -07:00
Philip O'Toole 1ddab4627f Start the shard-precreation service 2015-06-10 15:27:38 -07:00
Cory LaNou f60f833123 more testing! 2015-06-10 15:50:33 -05:00
Cory LaNou 99a404eec9 more aggregation tests for ints, update skipped tests with issue numbers 2015-06-10 14:34:41 -05:00
Philip O'Toole cbaf3cb0ed Support setting openTSDB consistency level 2015-06-09 15:39:31 -07:00
Philip O'Toole 8eea2b3092 Allow Server creation to return errors 2015-06-09 14:21:12 -07:00
Cory LaNou 709a7ffd06 add where clause integration tests, add support for int64 comparisons 2015-06-09 15:45:00 -05:00
Cory LaNou a43054adce add test for querying across shards and fields 2015-06-09 12:51:59 -05:00
David Norton d5f52333a1 fix #2814: hook collectd service back up 2015-06-09 00:34:46 -04:00
Philip O'Toole b0d76913b0 Add new Service-style Graphite input support 2015-06-08 20:39:39 -07:00
Philip O'Toole 4b94a00644 Set auto-created retetion period to infinity
Fix issue #2817
2015-06-08 19:47:37 -07:00
Ben Johnson a9657289c4 Merge pull request #2823 from benbjohnson/opentsdb
Convert OpenTSDB to service
2015-06-08 17:32:55 -06:00
Ben Johnson b688eccb77 Refactor OpenTSDB to a service
This commit converts the OpenTSDB endpoint into a service.
2015-06-08 15:12:16 -06:00
Cory LaNou 393493fa98 more query tag tests 2015-06-08 14:58:55 -05:00
Cory LaNou 64b921580f enable database drop tests 2015-06-08 13:58:03 -05:00
Philip O'Toole 7fbc28963c Merge pull request #2816 from renan-/fix-udp
Fix UDP service, broken by 092bc3f and uncorrectly fixed in cc78fe4
2015-06-08 11:05:48 -07:00
Paul Dix 8f712eaf85 Ignore shards that don't have the selected measurement in them.
Fixes #2815 and fixes #2818.
2015-06-08 17:14:42 +02:00
Cory LaNou eaec3e42e4 add test for multiple measurements 2015-06-08 10:07:29 -05:00
Renan Strauss b44f6efdc2 Fix UDP service, broken by 092bc3f and uncorrectly fixed in cc78fe4 2015-06-08 10:49:07 +02:00
Philip O'Toole b19828fea7 Add test case for retention auto-create 2015-06-06 17:22:37 -07:00
Jason Wilder 3a19798968 Set metastore on cluster service 2015-06-05 22:29:52 -06:00
Jason Wilder 809b9b8a83 Add basic hinted handoff support
If a remote write fails, it will be queued to a per-node, local disk
queue and retried later.
2015-06-05 22:16:51 -06:00
Jason Wilder 75b72c60fe Add hinted handoff service
The hinted handoff service will queue a write to a remote node if
that write fails and periodically retry the write.
2015-06-05 22:16:51 -06:00
Ben Johnson 413da5cdc4 add mux.Serve() 2015-06-05 22:13:49 -06:00
Todd Persen 4bf024109f Merge pull request #2779 from influxdb/hook-CQs-back-2733
fix #2733: hook CQs back in
2015-06-05 20:40:31 -07:00
Ben Johnson 9ec6e4ff7e Merge pull request #2794 from benbjohnson/mux
Influxd mux integration
2015-06-05 21:15:18 -06:00
David Norton cc78fe4678 fix #2733: fix rebase issue...grrr 2015-06-05 20:05:31 -04:00
David Norton fb514f2302 fix #2733: add endpoint to trigger CQ(s) 2015-06-05 19:56:28 -04:00
David Norton 077159a126 fix #2733: check config before enabling CQ service 2015-06-05 19:54:23 -04:00
David Norton 092bc3fd2d fix #2733: hook CQs back in 2015-06-05 19:54:23 -04:00
Philip O'Toole 041b31d6b3 Merge pull request #2783 from renan-/udp-refresh
Make UDP service code simpler
2015-06-05 16:23:16 -07:00
Ben Johnson fb06549552 remove bind address from cluster config 2015-06-05 17:07:54 -06:00
Ben Johnson abbcf15bb2 integrate mux into influxd cluster service 2015-06-05 17:02:32 -06:00
Ben Johnson 6cf78206f0 refactor run.Server to use tcp.Mux 2015-06-05 16:31:58 -06:00
Ben Johnson b925e1c1af Multi-node clustering.
This commit adds the ability to cluster multiple nodes together to share
the same metadata through raft consensus.
2015-06-05 14:41:19 -06:00
Philip O'Toole a4a05241ba Set retention enforcement default config 2015-06-05 11:27:30 -07:00
Renan Strauss e8ab17dbc5 [services/udp] Clean-up the code 2015-06-05 10:55:10 +02:00
Philip O'Toole ae3e8c727a Don't append retention service if not enabled 2015-06-04 21:47:06 -07:00
Philip O'Toole fab9adb361 Hook in actual retention service 2015-06-04 17:02:22 -07:00
Cory LaNou 8d3f54172c wildcard and precision write integration tests 2015-06-04 12:58:03 -06:00
Cory LaNou 3f7cf15a8d not needed 2015-06-04 09:24:26 -06:00
Cory LaNou 08bec931c6 observe enabled flag when appending services 2015-06-04 09:24:08 -06:00
Renan Strauss 3b09a59540 Add UDP service back 2015-06-04 10:24:48 +02:00