Commit Graph

473 Commits (34544d2c4b68158c0588d55a2d2d221ffb3b1a1b)

Author SHA1 Message Date
Philip O'Toole 7e0fa132c8 Raw query with no valid SELECT fields is not data 2015-07-20 15:59:52 -07:00
Philip O'Toole 10eecb441d Allow remote mapping to be forced
This is useful primarily for testing.
2015-07-20 10:44:45 -07:00
Philip O'Toole 425a65fca1 RemoteShard mapping now performed over TCP
With this change remote mapping no longer uses HTTP, as the HTTP ports
exposed by nodes on the cluster are not known cluster wide. The TCP
ports exposed by the cluster service are, so this change uses that
functionality. Each RemoteMapper has its own dedicated connection pool
for each node, and remote mapping TCP connections are in no way coupled
with query TCP connections.
2015-07-20 10:44:38 -07:00
Philip O'Toole 3e1b0fed78 Merge pull request #3345 from influxdb/ga-show-rp-syntax
Make `SHOW RETENTION POLICIES` consistent by requiring `ON`
2015-07-17 10:46:24 -07:00
Philip O'Toole 0d6c6bbe6f Correctly check if raw derivative is required
The multiple checks for Mapper and Executor type -- the lack of DRYness
in this code -- meant the same checks would need to be copied. Therefore
this change, as well as fixing the bug, improves the situation a little
bit by *asking* the Mappers what type of Executor is required. This code
is still not ideal.

Fixes #3355.
2015-07-16 23:28:38 -07:00
gunnaraasen ee8ba11c4f Add ON token to SHOW RETENTION POLICIES parser with tests 2015-07-16 13:45:50 -07:00
Philip O'Toole bd2f475b5d Merge pull request #3334 from mcastilho/master
Clean shutdown of influxd server
2015-07-16 13:00:13 -07:00
Philip O'Toole d8c31f0b3c Merge pull request #3320 from influxdb/streaming_dq
Support Distributed Queries
2015-07-16 12:33:21 -07:00
Philip O'Toole f41d2bab5d Start move to unified query executor 2015-07-15 19:31:13 -07:00
Philip O'Toole 74cb96646c Refactor query engine for distributed query support
With this change, the query engine code gathers information about
shards and tagsets by working with individual shards, collating the
information, and returning that to the client. It does not assume that any
particular shard is local, and accesses all shards through abstracted
Mappers, of which there are two types -- a Mapper type for Raw queries
and a second type for Aggregate queries. There are corresponding
Executors for each type of Mapper, but both types of Executors share the
same interface.
2015-07-15 12:54:55 -07:00
Philip O'Toole 21ea87432c Test that DROP DATABASE is isolated 2015-07-15 11:20:25 -07:00
Marcio Castilho e560eb19ee Clean shutdown of influxd server 2015-07-15 13:14:11 -04:00
Philip O'Toole a84c48bff6 Allow the PointWriter timeout to be configurable 2015-07-02 12:50:12 -04:00
Philip O'Toole ca86fa2633 Allow WAL inter-flush time to be configurable 2015-07-02 10:40:26 -04:00
Philip O'Toole c4a75ffef7 Merge pull request #3180 from influxdb/log_commit
Log GOMAXPROCS, version, and commit on start
2015-06-29 14:01:57 -04:00
Philip O'Toole e554115178 Log GOMAXPROCS, version, and commit on start 2015-06-29 13:52:03 -04:00
Joseph Crail 5fccee3d16 Fix spelling errors in comments and strings. 2015-06-28 02:54:34 -04:00
Ben Johnson b574e2f755 Add write ahead log
This commit adds a write ahead log to the shard. Entries are cached
in memory and periodically flushed back into the index. The WAL and
the cache are both partitioned into buckets so that flushing doesn't
stop the world as long.
2015-06-25 15:47:13 -06:00
Jason Wilder ea348ddc2c Add sample graphite config to default config 2015-06-24 23:09:07 -06:00
Jason Wilder fed8d67946 Add validation for graphite config templates and tags 2015-06-24 23:09:07 -06:00
Philip O'Toole e010daf74c Test GROUP BY when different tags have same value 2015-06-22 16:04:13 -07:00
Philip O'Toole 841a8d35ee Test queries after drop-and-create database with data 2015-06-22 11:55:44 -07:00
Philip O'Toole 8bff4c1c51 Unit test minimum retention policy duration 2015-06-17 11:02:38 -07:00
Philip O'Toole 60d338282e Unit test measurement recreation after DROP 2015-06-16 12:14:36 -07:00
Philip O'Toole 05bd0fc53d Check database existence during normalization
Statements were only being normalized if a default database was included
in the query (usually via the query param 'db'). However if no default
database was included, and none was an explicit part of the measurement
name, no database-existence check was run. This result in a later panic
with wildcard expansion.
2015-06-15 11:51:32 -07:00
Philip O'Toole b440c593a4 Merge pull request #2987 from neonstalwart/server_test-fix
fix version assertion in server_test
2015-06-15 10:26:11 -07:00
Cory LaNou a9d2e81772 More integration tests for retention policy 2015-06-15 11:00:41 -05:00
Cory LaNou e52acd3057 show tag field integration tests 2015-06-15 10:34:11 -05:00
ben hockey 90ea63baad fix version assertion in server_test 2015-06-15 09:44:49 -05:00
Philip O'Toole 85bb693119 Unit test version in HTTP response 2015-06-12 12:31:31 -07:00
Philip O'Toole d5fda01c2a Server now requires build information
Version is worth special-casing here, since the build information is
pretty important, and make components need it.

Fixes issue #2958.
2015-06-12 11:41:25 -07:00
Philip O'Toole 23a7896182 Server does not need Commit info, remove it 2015-06-12 11:28:32 -07:00
Cory LaNou 6cc9a2ee20 SHOW TAGS integration tests 2015-06-12 12:16:27 -05:00
dgnorton 0318b4562e Merge pull request #2936 from influxdb/fix-2935
fix #2935: hook CPU and memory profiling back up
2015-06-12 12:10:27 -04:00
David Norton b0b8d25752 fix #2935: remove delay on memprofile 2015-06-12 12:07:01 -04:00
Cory LaNou 69a20970ff SHOW MEASUREMENTS integration tests 2015-06-12 10:49:28 -05:00
David Norton fd2f1bb223 cr #2935: make code review changes 2015-06-12 11:47:59 -04:00
David Norton f717019d88 fix #2935: hook CPU and memory profiling back up 2015-06-12 11:47:59 -04:00
Cory LaNou 3686f2c496 Show series integration tests 2015-06-12 10:38:24 -05:00
Paul Dix 5e7ddc6a60 Merge pull request #2928 from neonstalwart/version-header
pass along version to httpd/handler
2015-06-12 09:46:33 -04:00
ben hockey aef4ba2e20 fix typos in server_test.md 2015-06-11 15:43:43 -05:00
Philip O'Toole e89d2d85a6 Merge pull request #2931 from influxdb/wait_for_leader
Export WaitForLeader on MetaStore
2015-06-11 13:26:00 -07:00
Cory LaNou 2373d9a68f Integration tests for drop measurement 2015-06-11 15:04:23 -05:00
Cory LaNou 224c6bb698 docs on integration testing 2015-06-11 14:17:07 -05:00
Philip O'Toole 4dff5f48aa Export WaitForLeader on MetaStore
This exported function can then be used by Services and
server-reporting, so those components don't make progress until the
cluster is ready.
2015-06-11 11:10:07 -07:00
ben hockey ce84047b44 pass along version to httpd/handler 2015-06-11 11:22:37 -05:00
Cory LaNou 2ecc2162cb add fill tests 2015-06-11 10:52:32 -05:00
Cory LaNou de17fcd6f6 missing parens to call func 2015-06-11 10:51:48 -05:00
David Norton a325071103 fix #2920: set MetaStore in collectd service 2015-06-11 09:53:16 -04:00
Philip O'Toole 431034c8f4 Wait for Raft leader even when cluster exists 2015-06-11 01:23:30 -07:00
Philip O'Toole b15a0df6d4 Wait for leader before reporting 2015-06-11 00:21:15 -07:00
Philip O'Toole 406509e93e Remove rogue ID field from reporting 2015-06-11 00:21:15 -07:00
Philip O'Toole 68fa173839 Report in goroutine so it outlasts return 2015-06-11 00:21:15 -07:00
Philip O'Toole 64af1b6241 Report number of measurements and series per node 2015-06-11 00:21:15 -07:00
Ben Johnson bc31783a00 Refactor backup and restore
This commit updates the snapshot code as well as the "backup" and
"restore" command to work with the new architecture.
2015-06-10 22:07:01 -06:00
Philip O'Toole 223bf672cf Update CHANGELOG 2015-06-10 20:05:59 -07:00
Philip O'Toole 57ce67b97b Ensure target Graphite database exists 2015-06-10 18:03:06 -07:00
Philip O'Toole f679c81136 Simply use "precreator" for shard precreation 2015-06-10 15:27:38 -07:00
Philip O'Toole 1ddab4627f Start the shard-precreation service 2015-06-10 15:27:38 -07:00
Cory LaNou f60f833123 more testing! 2015-06-10 15:50:33 -05:00
Cory LaNou 99a404eec9 more aggregation tests for ints, update skipped tests with issue numbers 2015-06-10 14:34:41 -05:00
Philip O'Toole cbaf3cb0ed Support setting openTSDB consistency level 2015-06-09 15:39:31 -07:00
Philip O'Toole 8eea2b3092 Allow Server creation to return errors 2015-06-09 14:21:12 -07:00
Cory LaNou 709a7ffd06 add where clause integration tests, add support for int64 comparisons 2015-06-09 15:45:00 -05:00
Cory LaNou a43054adce add test for querying across shards and fields 2015-06-09 12:51:59 -05:00
David Norton d5f52333a1 fix #2814: hook collectd service back up 2015-06-09 00:34:46 -04:00
Philip O'Toole b0d76913b0 Add new Service-style Graphite input support 2015-06-08 20:39:39 -07:00
Philip O'Toole 4b94a00644 Set auto-created retetion period to infinity
Fix issue #2817
2015-06-08 19:47:37 -07:00
Ben Johnson a9657289c4 Merge pull request #2823 from benbjohnson/opentsdb
Convert OpenTSDB to service
2015-06-08 17:32:55 -06:00
Ben Johnson b688eccb77 Refactor OpenTSDB to a service
This commit converts the OpenTSDB endpoint into a service.
2015-06-08 15:12:16 -06:00
Cory LaNou 393493fa98 more query tag tests 2015-06-08 14:58:55 -05:00
Cory LaNou 64b921580f enable database drop tests 2015-06-08 13:58:03 -05:00
Philip O'Toole 7fbc28963c Merge pull request #2816 from renan-/fix-udp
Fix UDP service, broken by 092bc3f and uncorrectly fixed in cc78fe4
2015-06-08 11:05:48 -07:00
Paul Dix 8f712eaf85 Ignore shards that don't have the selected measurement in them.
Fixes #2815 and fixes #2818.
2015-06-08 17:14:42 +02:00
Cory LaNou eaec3e42e4 add test for multiple measurements 2015-06-08 10:07:29 -05:00
Renan Strauss b44f6efdc2 Fix UDP service, broken by 092bc3f and uncorrectly fixed in cc78fe4 2015-06-08 10:49:07 +02:00
Philip O'Toole b19828fea7 Add test case for retention auto-create 2015-06-06 17:22:37 -07:00
Jason Wilder 3a19798968 Set metastore on cluster service 2015-06-05 22:29:52 -06:00
Jason Wilder 809b9b8a83 Add basic hinted handoff support
If a remote write fails, it will be queued to a per-node, local disk
queue and retried later.
2015-06-05 22:16:51 -06:00
Jason Wilder 75b72c60fe Add hinted handoff service
The hinted handoff service will queue a write to a remote node if
that write fails and periodically retry the write.
2015-06-05 22:16:51 -06:00
Ben Johnson 413da5cdc4 add mux.Serve() 2015-06-05 22:13:49 -06:00
Todd Persen 4bf024109f Merge pull request #2779 from influxdb/hook-CQs-back-2733
fix #2733: hook CQs back in
2015-06-05 20:40:31 -07:00
Ben Johnson 9ec6e4ff7e Merge pull request #2794 from benbjohnson/mux
Influxd mux integration
2015-06-05 21:15:18 -06:00
David Norton cc78fe4678 fix #2733: fix rebase issue...grrr 2015-06-05 20:05:31 -04:00
David Norton fb514f2302 fix #2733: add endpoint to trigger CQ(s) 2015-06-05 19:56:28 -04:00
David Norton 077159a126 fix #2733: check config before enabling CQ service 2015-06-05 19:54:23 -04:00
David Norton 092bc3fd2d fix #2733: hook CQs back in 2015-06-05 19:54:23 -04:00
Philip O'Toole 041b31d6b3 Merge pull request #2783 from renan-/udp-refresh
Make UDP service code simpler
2015-06-05 16:23:16 -07:00
Ben Johnson fb06549552 remove bind address from cluster config 2015-06-05 17:07:54 -06:00
Ben Johnson abbcf15bb2 integrate mux into influxd cluster service 2015-06-05 17:02:32 -06:00
Ben Johnson 6cf78206f0 refactor run.Server to use tcp.Mux 2015-06-05 16:31:58 -06:00
Ben Johnson b925e1c1af Multi-node clustering.
This commit adds the ability to cluster multiple nodes together to share
the same metadata through raft consensus.
2015-06-05 14:41:19 -06:00
Philip O'Toole a4a05241ba Set retention enforcement default config 2015-06-05 11:27:30 -07:00
Renan Strauss e8ab17dbc5 [services/udp] Clean-up the code 2015-06-05 10:55:10 +02:00
Philip O'Toole ae3e8c727a Don't append retention service if not enabled 2015-06-04 21:47:06 -07:00
Philip O'Toole fab9adb361 Hook in actual retention service 2015-06-04 17:02:22 -07:00
Cory LaNou 8d3f54172c wildcard and precision write integration tests 2015-06-04 12:58:03 -06:00
Cory LaNou 3f7cf15a8d not needed 2015-06-04 09:24:26 -06:00
Cory LaNou 08bec931c6 observe enabled flag when appending services 2015-06-04 09:24:08 -06:00
Renan Strauss 3b09a59540 Add UDP service back 2015-06-04 10:24:48 +02:00
Cory LaNou 90d8f37a08 some refactoring, mostly aggregation tests 2015-06-03 16:07:40 -06:00
Cory LaNou 212bbd067f move helpers to their own file 2015-06-03 13:44:10 -06:00
Cory LaNou 2dac41a619 fix one test that now works 2015-06-03 11:54:45 -06:00
Cory LaNou e6f0a2f3ab do not use the word `fail` in test messages 2015-06-03 11:22:59 -06:00
Cory LaNou 621e2dc2f3 more test coverage, minor test refactorings 2015-06-03 11:15:20 -06:00
Cory LaNou 5c52c4cda1 add ability to set logger for testing 2015-06-03 09:58:39 -06:00
Cory LaNou 1ab8d58bb2 and then there were table tests! 2015-06-02 20:24:53 -06:00
Cory LaNou ec749c836f Merge pull request #2742 from influxdb/integration-tests
Integration tests
2015-06-02 16:42:16 -06:00
Cory LaNou ab8701721a more integration testing 2015-06-02 16:03:13 -06:00
Jason Wilder 156e7df346 Rename PointsWrite.Store to TSDBStore
Matches MetaStore naming convention better.
2015-06-02 14:47:59 -06:00
Jason Wilder 3957e096f8 Remove ownerID from protobufs
Not needed since the node that processes the request is the owner.
2015-06-02 14:45:52 -06:00
Ben Johnson 44d38cb430 Merge branch 'master' into alpha1 2015-06-02 10:40:52 -06:00
Cory LaNou d962283ae6 adding some integration tests 2015-06-01 16:04:20 -06:00
Cory LaNou 3597565955 reading and writing yo! 2015-06-01 11:59:58 -06:00
Ben Johnson bf823d9887 Integrating cmd/influxd/run. 2015-05-30 14:06:36 -06:00
Ben Johnson c916256ac9 Rename cluster.Writer to cluster.ShardWriter. 2015-05-30 14:05:27 -06:00
Ben Johnson 8c8a55a737 Removed 'failed' from test suite. 2015-05-30 08:59:27 -06:00
Ben Johnson 233c63029b Refactor influxd.Main. 2015-05-30 08:49:49 -06:00
Ben Johnson 4872e6db7c Move services to subpackage. 2015-05-30 08:20:12 -06:00
Ben Johnson 6677ea074f more services, more test fixes, getting closer... 2015-05-29 14:59:57 -06:00
Ben Johnson 9d4527071e Refactor run command. 2015-05-29 14:59:57 -06:00
Ben Johnson 1f294ce8de Add httpd.Handler.serveQuery() tests. 2015-05-29 14:59:57 -06:00
Ben Johnson df1aeee70a WIP 2015-05-29 14:56:30 -06:00