Commit Graph

184 Commits (6768c6ed3bba76d8b2098fd1b89c08f7268f666e)

Author SHA1 Message Date
Cory LaNou 572da8985c enforce minimum shard duration when creating retention policies 2016-12-20 09:11:43 -06:00
Jonathan A. Sternberg ec57108520 Use proper uber-go/zap import path
It looks like the real import path to the project is go.uber.org/zap
instead of github.com/uber-go/zap since the example in the project
references that path.
2016-12-15 08:54:14 -06:00
Jonathan A. Sternberg 21502a39e8 Switch logging to use structured logging everywhere
The logging library has been switched to use uber-go/zap. While the
logging has been changed to use structured logging, this commit does not
change any of the logging statements to take advantage of the new
structured log or new log levels. Those changes will come in future
commits.
2016-12-14 10:45:15 -06:00
Allen Petersen da9941b9a9 Reject invalid subscription urls
The url must have a scheme of udp,http,https and a port number.
CREATE SUBSCRIPTION will fail if there are invalid destinations.

Additionally Service.createSubscription fail invalid destinations are detected.

Fixes #7615
2016-12-07 12:54:41 -08:00
Edd Robinson aff2d7ce64 Ensure user privs dropped along with db (#7677) 2016-12-02 18:23:35 +00:00
Cory LaNou d54d32b9d4
prune shards in meta data 2016-12-01 11:22:16 -06:00
Cory LaNou 6e290040bb
remove SetDefaultRetentionPolicy method from meta.Client 2016-11-03 09:39:41 -05:00
Cory LaNou cd272ce6c3 fix retention policy creation inconsistencies 2016-11-03 09:09:43 -05:00
Jason Wilder ebf50d06ca Merge branch '1.0' into jw-merge-102 2016-10-06 09:51:35 -06:00
Joe LeGasse 9d26cc0732 Stablize the sorting of meta.ShardGroupInfos 2016-09-30 13:58:03 +01:00
Jonathan A. Sternberg bb8c4efa95 Do not automatically reset the shard duration when using ALTER RETENTION POLICY 2016-09-28 12:26:03 -05:00
Jason Wilder bc00490b92 Don't clone the cached meta store
The cacheData is read-only and does not need to be cloned.  Cloning
it for read operations can be expensive with lots of shards/database/RPs/etc.
2016-09-28 10:35:59 -06:00
joelegasse 1eda0277a7 Merge pull request #7365 from influxdata/jl-shardgroup-sort
Stablize the sorting of meta.ShardGroupInfos
2016-09-27 11:02:47 -04:00
Jonathan A. Sternberg 477d6231db Update source files to pass vet checks for go 1.7
The vet checks for some files did not pass for go 1.7. As part of a
preliminary start to making go 1.7 work with this software, go vet
should pass.

Also updated the gogo/protobuf dependency which fixed the code generator
to work with go 1.7 too. Ran `go generate` on the entire repository to
ensure every file was up to date.
2016-09-14 15:01:22 -05:00
Jonathan A. Sternberg 23f2d50ecb Use defaults from `meta` package for `CREATE DATABASE`
Instead of having the parser set the defaults, the command will set the
defaults so that the constants for that are actually used. This way we
can also identify which things the user provided and which ones we are
filling with default values.

This allows the meta client to be able to make smarter decisions when
determining if the user requested a conflict or if the requested
capabilities match with what is currently available. If you just say
`CREATE DATABASE WITH NAME myrp`, the user doesn't really care what the
duration of the retention policy is and just wants to use the default.
Now, we can use that information to determine if an existing retention
policy would conflict with what the user requested rather than returning
an error if a default value ever gets changed since the meta client
command can communicate intent more easily.
2016-08-30 13:23:49 -05:00
Jonathan A. Sternberg c05c7f6360 Revert "limit shard concurrency"
This reverts commit 6c7d56d4bc.
2016-08-29 12:39:52 -05:00
Jonathan A. Sternberg ed2f81357f go generate on every package to ensure they are generated with the correct dependency 2016-08-05 14:35:07 -05:00
Ben Johnson 6c7d56d4bc
limit shard concurrency
This commit limits queries to only process one shard at a time.
However, within a shard, multiple series can still be processed in
parallel. Shard iterators are lazily instantiated during query
execution to limit the amount of memory a given query uses.
2016-08-05 09:45:57 -06:00
David Norton e3328117a9 fix #7081: hardcode auto gen RP name 2016-07-27 16:49:46 -04:00
Joe LeGasse b3341e8f66 Stablize the sorting of meta.ShardGroupInfos 2016-07-27 16:17:52 -04:00
David Norton b1079c9080 revert previous DefaultRetentionPolicyName change 2016-07-27 11:38:42 -04:00
David Norton 8fbfdc5616 fix #7068: use DefaultRetentionPolicyName from cfg
Use DefaultRetentionPolicyName from the config instead of passing
through meta data.
2016-07-26 18:53:55 -04:00
Edd Robinson 2c85d498ef Remove unused errors 2016-07-06 15:04:39 +01:00
Jonathan A. Sternberg b8e52ce39a Merge pull request #6889 from influxdata/js-update-config-options
Update help and remove unused config options from the configuration file
2016-06-23 11:49:10 -05:00
kun 8aa2fe8ede fix ClusterID generation bug 2016-06-23 14:52:58 +08:00
Jonathan A. Sternberg 22173acb70 Update help and remove unused config options from the configuration file
Normalize the output for the various help options so they all follow the
same format and display all relevant options.

Removing some of the unused config options from the configuration file
and updating the help documentation. Removing some remaining references
to clustering within the open source version.
2016-06-21 14:06:05 -05:00
Jonathan A. Sternberg 497db2a6d3 Removing dead code from every package except influxql
The tsdb package had a substantial amount of dead code related to the
old query engine still in there. It is no longer used, so it was removed
since it was left unmaintained. There is likely still more code that is
the same, but wasn't found as part of this code cleanup.

influxql has dead code show up because of the code generation so it is
not included in this pruning.
2016-06-20 22:41:07 -05:00
Jonathan A. Sternberg 8e1b036b0a Modify the max nanosecond time to be one nanosecond less
The highest time represented by a nanosecond needs to be used for an
exclusive range, so the maximum time needs to be one less than the
possible maximum number of nanoseconds representable by an int64 so that
we don't lose a point at that one time.

Previously worked in the open source version because the timestamp used
for finding a shard would be truncated by the retention policy so the
lookup time didn't run into this edge case because it didn't rest on the
truncation boundary. Since that point didn't really belong in that shard
group and was placed there by mistake, it's best to fix this bug since
the timestamp used to create the shard group should be capable of
retrieving it.
2016-06-16 12:15:41 -05:00
Joe LeGasse 647210c57a Add TruncatedAt field to meta.ShardGroupInfo 2016-06-07 18:24:24 -04:00
Jonathan A. Sternberg b8e22d9d79 Merge pull request #6586 from influxdata/js-3733-rename-default-retention-policy
Modify the default retention policy name and make it configurable
2016-06-06 15:05:29 -05:00
Rubycut a849f42cc2 Lock auth properly, code suggested by @jwilder 2016-06-04 09:05:10 +02:00
Edd Robinson f4fc905fa9 Reject timestamps too far in future 2016-05-27 11:07:48 +01:00
Jonathan A. Sternberg 907c88d4b5 Truncate the shard group end time if it exceeds MaxNanoTime
Related to #6599.
2016-05-25 21:25:10 -04:00
Jonathan A. Sternberg baaa782c95 Modify the default retention policy name and make it configurable
The default retention policy name is changed to "autogen" instead of
"default" since it ends up being ambiguous when we tell a user to check
the default retention policy, it is uncertain if we are referring to the
default retention policy (which can be changed) or the retention policy
with the name "default".

Now the automatically generated retention policy name is "autogen".

The default retention policy is now also configurable through the
configuration file so an administrator can customize what they think
should be the default.

Fixes #3733.
2016-05-24 09:51:23 -04:00
David Norton 9d5c4a0f41 fix #6702: return correct required privileges
Change SelectStatement to return required privileges for all Sources.
2016-05-23 16:44:23 -04:00
David Norton b32208a850 fix #2048: check that RPs exist before creating CQ 2016-05-20 12:40:36 -04:00
Jason Wilder adddb0e6b8 Reduce lock contention when creating shard group 2016-05-09 16:10:38 -06:00
Edd Robinson d35fa1ec97 Remove redundant windows build tags 2016-05-03 14:22:02 +01:00
Seif Lotfy ced79acac5 Remove redundant error return from (Client) Database(name string) and (Client) Databases() 2016-04-30 17:04:38 -05:00
Stephen Gutekanst 9dc09c5257 Make logging output location more programmatically configurable (#6213)
This has various benefits:

- Users embedding InfluxDB within other Go programs can specify a different logger / prefix easily.
- More consistent with code used elsewhere in InfluxDB (e.g. services, other `run.Server.*` fields, etc).
- This is also more efficient, because it means `executeQuery` no longer allocates a single `*log.Logger` each time it is called.
2016-04-20 21:07:08 +01:00
David Norton 5e9ff7bc93 add WriteAuthorizer interface 2016-04-19 00:03:57 -04:00
Chris Ramón a4dd61a2c7 adds shard duration normalization before comparing retention policies 2016-04-13 13:09:20 -05:00
Cory LaNou 7eaf7f8855 CreateShardGroup was incrementing meta data index even when it was idempotent 2016-04-07 11:39:45 -05:00
Edd Robinson 5327a75a6f Merge pull request #6216 from influxdata/er-scope-proto
Change protobuf package names to avoid clashes
2016-04-07 16:38:21 +01:00
mvadu 94524a7699 Fix failure on Windows
Windows does not allow a file to be renamed while its in use. f.Sync does not close the handle. It needs to be explicitly closed before renaming.
2016-04-06 07:32:37 +05:30
Edd Robinson 184257a10d Scope all internal protobuf packages 2016-04-05 13:54:21 +01:00
Edd Robinson 712cc3f290 ShardGroup Duration is checked when recreating rp
Fixes #6178.
2016-04-01 11:59:41 +01:00
Cory LaNou 9bd60c3e07 remove node.json as well, change where we check for raft.db in startup 2016-03-31 17:15:58 -05:00
Cory LaNou 6c6fe87a40 add error message if detecting raft.db file 2016-03-31 17:15:42 -05:00
Cory LaNou a961ff9ebf minor restore fixes; fsync meta snapshots 2016-03-31 17:15:42 -05:00