Commit Graph

416 Commits (659706e8749cfc6848b46071cdc1075125c7710c)

Author SHA1 Message Date
Dejan Golja e1b9982401 Add support for SET PASSWORD FOR user = 'PASSWORD'
Added support for the set password for user, so we can update user
password via the new server administration commands
2015-04-04 14:16:54 +11:00
Jason Wilder 6d4c7e9cd5 Handle broker and data node endpoints regardless of role
This is a pre-requisite for #1934.  When running separate
broker and data nodes, you currently need to know what role
a host is performing.  This complicates cluster setup in
that you must configure separate broker URLs and data node
URLs.

This change allows a broker only node to redirect data nodes endpoints
to a valid data node and a data only node to redirect broker
endpoints to a valid broker.
2015-04-03 21:00:43 -06:00
Philip O'Toole 8661155a7d Fix bad merge with master 2015-04-02 15:22:27 -07:00
Paul Dix 6c46a5c83b Add chunked responses and streaming of raw queries.
Refactored query engine to have different processing pipeline for raw queries. This enables queries that have a large offset to not keep everything in memory. It also makes it so that queries against raw data that have a limit will only p
rocess up to that limit and then bail out.

Raw data queries will only read up to a certain point in the map phase before yielding to the engine for further processing.

Fixes #2029 and fixes #2030
2015-04-02 15:22:27 -07:00
Paul Dix b13385f9ab Fix error after rebase 2015-04-02 15:22:27 -07:00
Paul Dix 332c42750b Update server and handler to work with streamed responses 2015-04-02 15:22:27 -07:00
Jason Wilder 91fb7e3756 Track data node urls on brokers
This sends data node urls via the broker heartbeat from each data
node.  The urls are tracked on the broker to support simpler
cluster setup as well as distributed queries.
2015-04-02 11:27:53 -06:00
Julius Volz 690f6f16c0 Fix all "golint" warnings in top-level package.
This fixes all "golint" warnings in the top-level "influxdb" package:

    database.go:316:2: can probably use "var tagStrings []string" instead
    database.go:1077:6: exported type RetentionPolicies should have comment or be unexported
    server.go:337:1: exported method Server.StartSelfMonitoring should have comment or be unexported
    server.go:345:3: can probably use "var points []Point" instead
    server.go:433:2: can probably use "var groups []group" instead
    server.go:498:2: can probably use "var groups []group" instead
    server.go:1417:1: comment on exported method Server.CreateRetentionPolicyIfNotExists should be of the form "CreateRetentionPolicyIfNotExists ..."
    server.go:1899:1: exported method Server.DropMeasurement should have comment or be unexported
    server.go:2153:10: if block ends with a return statement, so drop this else and outdent its block
    server.go:2695:2: can probably use "var rows []*influxql.Row" instead
    server.go:2876:11: if block ends with a return statement, so drop this else and outdent its block
    server.go:3804:1: exported method Server.StartReportingLoop should have comment or be unexported
    tx.go:133:4: can probably use "var mappers []influxql.Mapper" instead
    tx.go:231:1: exported method LocalMapper.Open should have comment or be unexported
    tx.go:254:1: exported method LocalMapper.Close should have comment or be unexported
    tx.go:348:1: exported method LocalMapper.Next should have comment or be unexported
2015-03-31 02:22:45 +02:00
Philip O'Toole a0e9daee63 SHOW DATABASES returns series name "databases" 2015-03-27 16:24:57 -07:00
Philip O'Toole 502ee0764f Saner names for diags and stats, and use tags 2015-03-26 16:27:13 -07:00
Philip O'Toole 70836bc87d Write stats and diags to different measurements
Also, some fixes to casts. This may need further work.
2015-03-26 16:27:13 -07:00
Philip O'Toole 4d8166c3b7 Write server diagnostics into database
This allows historical monitoring of all diagnostics. Of particular
interest will be GC performance, memory usage, number of shards and
shard groups on the cluster, index points, amongst other important data.
2015-03-26 16:27:13 -07:00
Philip O'Toole 4c7c87a7bb Refactor common diags into types
This will make it easier to use these common diagnostics function in
other components, like the Broker, as well as minimizing growth of the
server type, in terms of SLOC.

This new diags code can be converted to InfluxQL rows, for easy display
and re-writes back to the database for self-monitoring.
2015-03-26 16:27:13 -07:00
Philip O'Toole 0bcec78527 Merge pull request #2076 from influxdb/fix_up_logging
Fix up logging
2015-03-25 21:34:26 -07:00
Philip O'Toole 1660cb349e Merge pull request #2079 from influxdb/rename_report_server
Rename reporting function since it's not stats
2015-03-25 19:41:20 -07:00
Philip O'Toole 5e01e83e47 Rename reporting function since it's not stats 2015-03-25 19:39:27 -07:00
Philip O'Toole da25c94484 Move small utility function closer to its use 2015-03-25 19:35:27 -07:00
Philip O'Toole c461c7e7b4 Remove logging to specific file 2015-03-25 17:41:39 -07:00
Todd Persen 94a8019fc7 Wire up DROP CONTINUOUS QUERY 2015-03-25 11:58:33 -07:00
Ben Johnson 21782c0a67 Merge branch 'master' of https://github.com/influxdb/influxdb into backup-restore 2015-03-24 16:05:49 -06:00
Philip O'Toole bfad9a194d Merge pull request #2061 from influxdb/show_diags_command
SHOW DIAGNOSTICS command
2015-03-24 12:25:56 -07:00
Philip O'Toole 895b781e72 Creates "values" correctly for various diags 2015-03-24 09:27:54 -07:00
Philip O'Toole f6644d4de8 Add "shard groups" and "shards" to diags 2015-03-23 23:46:20 -07:00
Philip O'Toole e79d063f4f Count number of open shards 2015-03-23 23:45:57 -07:00
Philip O'Toole 1c440f8542 Implement "SHOW DIAGNOSTICS" 2015-03-23 23:45:57 -07:00
Philip O'Toole fba41df82b Push version and commit hash onto Server 2015-03-23 23:45:57 -07:00
Philip O'Toole a8f0dbcf1d Load shards on restart
Fix for #2012
2015-03-23 18:34:46 -07:00
Philip O'Toole 0a76c2bdfc Clear server maps on close
These maps are reloaded from the metastore on open.
2015-03-23 18:27:36 -07:00
Philip O'Toole 51e98b85ef Support sorting of retention policies
Sort them by name.
2015-03-23 17:27:15 -07:00
Philip O'Toole c239914249 Track number of queries received and executed 2015-03-23 16:48:25 -07:00
Philip O'Toole ab4e7b3047 Tighten up tag and ID choice for internal stats 2015-03-23 16:48:22 -07:00
Ben Johnson 3befa12fa3 Merge branch 'master' of https://github.com/influxdb/influxdb into backup-restore
Conflicts:
	cmd/influxd/main.go
	cmd/influxd/run.go
2015-03-22 15:38:41 -06:00
Ben Johnson 11c808f55f Add restore and bootstrap.
This commit adds the "influxd restore" command to the CLI. This allows
a snapshot that has been produced by "influxd backup" to be restored
to a config location and the broker and raft directories will be
bootstrapped based on the state of the snapshot.
2015-03-22 15:31:49 -06:00
Ben Johnson 0461f401f6 Add SnapshotWriter. 2015-03-19 22:23:52 -06:00
Philip O'Toole 05fb2842ba Use _id for Series IDs
Fix issue #2008
2015-03-19 19:04:07 -07:00
Philip O'Toole 62dc60d36d Merge pull request #2004 from influxdb/limit-group-by
Limit group by to MaxGroupByPoints (currently 100,000)
2015-03-19 16:45:24 -07:00
Philip O'Toole dc297e8012 Correctly use IsRawQuery 2015-03-19 16:32:25 -07:00
Philip O'Toole b045a0008e Remove redundant Aggregated() on select statement 2015-03-19 15:22:43 -07:00
Simon Kern 787d521ca5 removed unnecessary fmt.Sprintf 2015-03-19 22:17:15 +01:00
Cory LaNou c13bdd5e86 refactor, bug fix, move tests to integration 2015-03-19 14:33:47 -06:00
Cory LaNou e78e5f3e19 Limit group by to MaxGroupByPoints (currently 100,000) 2015-03-18 18:44:30 -06:00
Philip O'Toole 43a5747e6e Track shard-level stats 2015-03-18 15:26:49 -07:00
Philip O'Toole 61b5c4ee3c Set a timestamp when storing monitoring stats 2015-03-18 12:42:33 -07:00
David Norton 62ae32b0ce wire up regex in FROM clause 2015-03-16 19:40:59 -04:00
David Norton fd005a5f15 influxql: allow regex in FROM clause 2015-03-16 19:40:21 -04:00
Philip O'Toole 69530b7e8f Use a time.Ticker object for precise self-monitoring 2015-03-15 14:32:47 -07:00
Philip O'Toole eec2cf6bf6 Use a better ID for the server ID in stats 2015-03-15 14:32:46 -07:00
Philip O'Toole 50d2470041 Write totals, not diff, of internal stats 2015-03-15 14:32:46 -07:00
Philip O'Toole 032cfaa980 Launch self-monitoring in a go-routine 2015-03-15 14:32:46 -07:00
Philip O'Toole 43161bb5da Small chance that a database exists 2015-03-15 14:32:46 -07:00
Philip O'Toole d627634991 Ensure internal retention policy exists 2015-03-15 14:32:46 -07:00
Philip O'Toole ae3b3d5252 Rename "metrics" to "stats" 2015-03-15 14:32:46 -07:00
Philip O'Toole 5240151517 Actually start self-monitoring if requested 2015-03-15 14:32:11 -07:00
Philip O'Toole bf28720c6e Add "name" to stats object
This "name" becomes the Measurement name. In addition the implementation
for "SHOW STATS" has been re-instated.
2015-03-15 14:32:11 -07:00
Philip O'Toole 6867d806de Use Inc() shorthand for Stats 2015-03-15 14:32:11 -07:00
Philip O'Toole f730257ee4 Implement self-monitoring 2015-03-15 14:31:16 -07:00
Philip O'Toole 228e238436 Remove unnecessary err variable 2015-03-15 14:31:16 -07:00
Philip O'Toole 8825580500 Track server stats 2015-03-15 14:31:16 -07:00
Todd Persen b842bf2446 Merge pull request #1952 from dullgiulio/delete-no-panic
return error on delete statement, closes #1933
2015-03-15 10:16:42 -07:00
Ben Johnson 06d839223e Integration test delay. 2015-03-14 14:31:25 -06:00
Ben Johnson 41d357ac04 Fixes based on code review comments. 2015-03-14 14:07:09 -06:00
Ben Johnson 53dbec8232 Add config notifications and increased test coverage. 2015-03-14 13:36:06 -06:00
Giulio Iotti 2cc2077c3d do not accept empty database name in drop 2015-03-14 21:31:44 +02:00
Giulio Iotti 38bb2cf62e do not allow empty database name, closes #1950 2015-03-14 15:39:22 +02:00
Giulio Iotti a4f96a3b9f return error on delete statement, closes #1933 2015-03-14 11:14:32 +02:00
Ben Johnson 4b9a93d924 Merge branch 'master' of https://github.com/influxdb/influxdb into stateless-broker 2015-03-12 15:46:04 -06:00
Philip O'Toole d82d040e59 Create databases for Graphite at higher level
By doing this the SeriesWriter interface stays focused and just has
methods for writing data. Its name then remains coherent.
2015-03-12 13:33:25 -07:00
Ben Johnson 7880bc2452 Add zero length data checks. 2015-03-12 12:12:26 -06:00
Ben Johnson 7ab19b9f91 Merge branch 'master' of https://github.com/influxdb/influxdb into stateless-broker 2015-03-12 11:59:55 -06:00
Cory LaNou ddc6e119ab Add default column to SHOW RETENTION POLICIES 2015-03-12 11:07:51 -06:00
Cory LaNou 8655ed3d91 fixing default retention policy creation 2015-03-12 10:01:26 -06:00
Ben Johnson 5f6bcf523f Fix broker integration bugs. 2015-03-11 12:00:45 -06:00
Cory LaNou 982e59a416 Merge pull request #1906 from influxdb/show-servers
Add show servers to query language
2015-03-11 10:38:11 -06:00
Cory LaNou f6121a9b63 fixing edge case for infinite retention policy 2015-03-11 10:14:53 -06:00
Ben Johnson 66115f95ca Merge branch 'master' of https://github.com/influxdb/influxdb into stateless-broker
Conflicts:
	cmd/influxd/run.go
	messaging/broker.go
	messaging/client.go
	messaging/client_test.go
	messaging/intg_test.go
	server.go
	tx_test.go
2015-03-10 16:27:37 -06:00
Cory LaNou 5e65a07b31 retrieve list of data nodes 2015-03-10 15:47:34 -06:00
Ben Johnson 27e9132796 Integrate stateless broker into remaining packages. 2015-03-10 14:53:45 -06:00
Cory LaNou db067b6a2d move constant 2015-03-10 12:24:26 -06:00
Cory LaNou 91487cc8dd update comment 2015-03-10 10:52:23 -06:00
Cory LaNou b8fb723967 Enforce retention policy minimum (currently 1 hour) 2015-03-10 10:49:54 -06:00
Cory LaNou c150a8c8bc run all create shards groups through same methods 2015-03-09 18:54:37 -06:00
Cory LaNou 7495e6eb1c fix comments, some renaming, adding rlock 2015-03-09 18:05:39 -06:00
Cory LaNou b6e7210f47 close channel 2015-03-09 17:36:50 -06:00
Cory LaNou 67f03ef26b Pre-create shard groups
1) Pre create shard groups

2) Calculate and save the ShardGroupDuration for Retention Policies. This should never change even if they alter a retention policy in the future
2015-03-09 17:30:17 -06:00
Philip O'Toole abf664c18d Merge pull request #1895 from influxdb/create_default_rp
Create retention policy by default
2015-03-09 16:26:41 -07:00
Philip O'Toole f72313edd9 Support retention policy auto-creation 2015-03-09 16:24:03 -07:00
Ben Johnson 5f5c6ca297 Integrate stateless messaging into influxdb package. 2015-03-09 15:47:41 -06:00
Ben Johnson 713ca4b831 Merge branch 'master' into stateless-broker
Conflicts:
	messaging/broker.go
2015-03-09 12:27:14 -06:00
Philip O'Toole 1db4851e49 RLock Server when checking retention policies 2015-03-09 10:56:59 -07:00
Joseph Rothrock 9b63eed767 Merge branch 'master' of github.com:influxdb/influxdb into 1753
Conflicts:
	CHANGELOG.md
2015-03-06 12:37:15 -08:00
Joseph Rothrock 771ef1cec3 Don't Panic on Missing Dirs
Issue: 1753
Figure out what's missing, create it, and keep going.
2015-03-06 12:11:02 -08:00
Philip O'Toole 76c255a3c8 Add trace-level logging to server write path 2015-03-05 17:35:57 -08:00
Todd Persen 93b56ca058 Make sure queries for missing measurements return a `200 OK` even when using a `GROUP BY *`. 2015-03-05 17:27:46 -08:00
Philip O'Toole 77d8986626 Don't delete data with infinite retention 2015-03-05 15:22:36 -08:00
Ben Johnson 1bbf154f2a Removing replicas and subscriptions from broker. 2015-03-01 22:32:20 -07:00
Kai Zhang ef262171d6 trim command line before parse it
This can avoid some errors caused by an additional space at the end of
the command line.

For example: 'use db ' will be trimed to 'use db' and parsed correctly.

Signed-off-by: Kai Zhang <zhangk1985@gmail.com>
2015-03-01 10:44:20 +08:00
Philip O'Toole a7c0d71d9c Merge pull request #1770 from kylezh/dev
Fix error when alter retention policy
2015-02-27 20:55:14 -08:00
Paul Dix a26f11f489 fixes based on @dgnorton's feedback 2015-02-27 01:09:38 -05:00
Paul Dix 52f257e349 Make show series return IDs and work with limit and offset.
Fixes #1765
2015-02-27 01:09:38 -05:00
Kai Zhang 9f8639ded8 Fix error when alter retention policy
+ fix error when execute ALTER RETENTION POLICY bar ON foo DURATION 1h
+ add test to verify the fix

Signed-off-by: Kai Zhang <zhangk1985@gmail.com>
2015-02-27 09:54:39 +08:00