Commit Graph

195 Commits (2980f5b2b4c642badbd9fce760a0ff5f847e699f)

Author SHA1 Message Date
Edd Robinson 4442cd7efa Fixes #7860.
Fix a regression introduced in #7449.

This commit ensures that create database with retention policy will work
correctly.
2017-01-23 17:57:32 +00:00
Joe LeGasse 2db0250b22 Add db/rp name validation
This change adds some very basic name validation with the following
plain-english description: names must be non-zero sequence of printable
characters that do not contain slashes ('/' or '\') and are not equal to
either "." or "..".

The intent is that, since we currently just use database and retention
policy names directly as path elements, these rules will hopefully leave
us with names that should be at least close to valid directory names.

Ideally, we would restrict names even further or not use them as path
elements directly, but this should be a step towards the former without
restricting names "too much"
2017-01-12 17:38:10 -05:00
Jason Wilder ccdb68841b Merge pull request #7820 from influxdata/cjl-7689-sgi-unix-epoch-zero-time
fix marshaling of zero time
2017-01-12 12:34:48 -07:00
Cory LaNou c6dc164537
fix marshaling of zero time 2017-01-11 12:00:18 -06:00
Vladimir Lopes f05df2a263 Fix panic when pruning shard groups
* Fix #7812 - Panic when pruning shard groups

* Update CHANGELOG.md
2017-01-11 14:56:40 +00:00
Mark Rushakoff a135906b43 Merge pull request #7747 from influxdata/mr-lint-cleanup
Miscellaneous lint cleanup
2017-01-10 08:22:00 -08:00
Jonathan A. Sternberg d7c8c7ca4f Support subquery execution in the query language
This adds query syntax support for subqueries and adds support to the
query engine to execute queries on subqueries.

Subqueries act as a source for another query. It is the equivalent of
writing the results of a query to a temporary database, executing
a query on that temporary database, and then deleting the database
(except this is all performed in-memory).

The syntax is like this:

    SELECT sum(derivative) FROM (SELECT derivative(mean(value)) FROM cpu GROUP BY *)

This will execute derivative and then sum the result of those derivatives.
Another example:

    SELECT max(min) FROM (SELECT min(value) FROM cpu GROUP BY host)

This would let you find the maximum minimum value of each host.

There is complete freedom to mix subqueries with auxiliary fields. The only
caveat is that the following two queries:

    SELECT mean(value) FROM cpu
    SELECT mean(value) FROM (SELECT value FROM cpu)

Have different performance characteristics. The first will calculate
`mean(value)` at the shard level and will be faster, especially when it comes to
clustered setups. The second will process the mean at the top level and will not
include that optimization.
2017-01-07 13:00:48 -06:00
Mark Rushakoff 6a94d200c8 Merge remote-tracking branch 'influx/master' into mr-godoc 2017-01-04 13:27:36 -08:00
Mark Rushakoff 07b87f2630 Miscellaneous lint cleanup 2017-01-03 09:47:32 -08:00
Mark Rushakoff 959c445a88 Fix broken return statements swallowing errors
There was no comment on either case specifying that the `return nil`
was deliberate instead of `return err`, so I'm assuming these were
typos. I added tests to conserve the error-returning behavior.
2017-01-03 08:50:34 -08:00
Mark Rushakoff 218fc3890d Update godoc for services
The admin service was deliberately skipped due to it being deprecated.
2016-12-30 18:03:01 -08:00
Cory LaNou 572da8985c enforce minimum shard duration when creating retention policies 2016-12-20 09:11:43 -06:00
Jonathan A. Sternberg ec57108520 Use proper uber-go/zap import path
It looks like the real import path to the project is go.uber.org/zap
instead of github.com/uber-go/zap since the example in the project
references that path.
2016-12-15 08:54:14 -06:00
Jonathan A. Sternberg 21502a39e8 Switch logging to use structured logging everywhere
The logging library has been switched to use uber-go/zap. While the
logging has been changed to use structured logging, this commit does not
change any of the logging statements to take advantage of the new
structured log or new log levels. Those changes will come in future
commits.
2016-12-14 10:45:15 -06:00
Allen Petersen da9941b9a9 Reject invalid subscription urls
The url must have a scheme of udp,http,https and a port number.
CREATE SUBSCRIPTION will fail if there are invalid destinations.

Additionally Service.createSubscription fail invalid destinations are detected.

Fixes #7615
2016-12-07 12:54:41 -08:00
Edd Robinson aff2d7ce64 Ensure user privs dropped along with db (#7677) 2016-12-02 18:23:35 +00:00
Cory LaNou d54d32b9d4
prune shards in meta data 2016-12-01 11:22:16 -06:00
Cory LaNou 6e290040bb
remove SetDefaultRetentionPolicy method from meta.Client 2016-11-03 09:39:41 -05:00
Cory LaNou cd272ce6c3 fix retention policy creation inconsistencies 2016-11-03 09:09:43 -05:00
Jason Wilder ebf50d06ca Merge branch '1.0' into jw-merge-102 2016-10-06 09:51:35 -06:00
Joe LeGasse 9d26cc0732 Stablize the sorting of meta.ShardGroupInfos 2016-09-30 13:58:03 +01:00
Jonathan A. Sternberg bb8c4efa95 Do not automatically reset the shard duration when using ALTER RETENTION POLICY 2016-09-28 12:26:03 -05:00
Jason Wilder bc00490b92 Don't clone the cached meta store
The cacheData is read-only and does not need to be cloned.  Cloning
it for read operations can be expensive with lots of shards/database/RPs/etc.
2016-09-28 10:35:59 -06:00
joelegasse 1eda0277a7 Merge pull request #7365 from influxdata/jl-shardgroup-sort
Stablize the sorting of meta.ShardGroupInfos
2016-09-27 11:02:47 -04:00
Jonathan A. Sternberg 477d6231db Update source files to pass vet checks for go 1.7
The vet checks for some files did not pass for go 1.7. As part of a
preliminary start to making go 1.7 work with this software, go vet
should pass.

Also updated the gogo/protobuf dependency which fixed the code generator
to work with go 1.7 too. Ran `go generate` on the entire repository to
ensure every file was up to date.
2016-09-14 15:01:22 -05:00
Jonathan A. Sternberg 23f2d50ecb Use defaults from `meta` package for `CREATE DATABASE`
Instead of having the parser set the defaults, the command will set the
defaults so that the constants for that are actually used. This way we
can also identify which things the user provided and which ones we are
filling with default values.

This allows the meta client to be able to make smarter decisions when
determining if the user requested a conflict or if the requested
capabilities match with what is currently available. If you just say
`CREATE DATABASE WITH NAME myrp`, the user doesn't really care what the
duration of the retention policy is and just wants to use the default.
Now, we can use that information to determine if an existing retention
policy would conflict with what the user requested rather than returning
an error if a default value ever gets changed since the meta client
command can communicate intent more easily.
2016-08-30 13:23:49 -05:00
Jonathan A. Sternberg c05c7f6360 Revert "limit shard concurrency"
This reverts commit 6c7d56d4bc.
2016-08-29 12:39:52 -05:00
Jonathan A. Sternberg ed2f81357f go generate on every package to ensure they are generated with the correct dependency 2016-08-05 14:35:07 -05:00
Ben Johnson 6c7d56d4bc
limit shard concurrency
This commit limits queries to only process one shard at a time.
However, within a shard, multiple series can still be processed in
parallel. Shard iterators are lazily instantiated during query
execution to limit the amount of memory a given query uses.
2016-08-05 09:45:57 -06:00
David Norton e3328117a9 fix #7081: hardcode auto gen RP name 2016-07-27 16:49:46 -04:00
Joe LeGasse b3341e8f66 Stablize the sorting of meta.ShardGroupInfos 2016-07-27 16:17:52 -04:00
David Norton b1079c9080 revert previous DefaultRetentionPolicyName change 2016-07-27 11:38:42 -04:00
David Norton 8fbfdc5616 fix #7068: use DefaultRetentionPolicyName from cfg
Use DefaultRetentionPolicyName from the config instead of passing
through meta data.
2016-07-26 18:53:55 -04:00
Edd Robinson 2c85d498ef Remove unused errors 2016-07-06 15:04:39 +01:00
Jonathan A. Sternberg b8e52ce39a Merge pull request #6889 from influxdata/js-update-config-options
Update help and remove unused config options from the configuration file
2016-06-23 11:49:10 -05:00
kun 8aa2fe8ede fix ClusterID generation bug 2016-06-23 14:52:58 +08:00
Jonathan A. Sternberg 22173acb70 Update help and remove unused config options from the configuration file
Normalize the output for the various help options so they all follow the
same format and display all relevant options.

Removing some of the unused config options from the configuration file
and updating the help documentation. Removing some remaining references
to clustering within the open source version.
2016-06-21 14:06:05 -05:00
Jonathan A. Sternberg 497db2a6d3 Removing dead code from every package except influxql
The tsdb package had a substantial amount of dead code related to the
old query engine still in there. It is no longer used, so it was removed
since it was left unmaintained. There is likely still more code that is
the same, but wasn't found as part of this code cleanup.

influxql has dead code show up because of the code generation so it is
not included in this pruning.
2016-06-20 22:41:07 -05:00
Jonathan A. Sternberg 8e1b036b0a Modify the max nanosecond time to be one nanosecond less
The highest time represented by a nanosecond needs to be used for an
exclusive range, so the maximum time needs to be one less than the
possible maximum number of nanoseconds representable by an int64 so that
we don't lose a point at that one time.

Previously worked in the open source version because the timestamp used
for finding a shard would be truncated by the retention policy so the
lookup time didn't run into this edge case because it didn't rest on the
truncation boundary. Since that point didn't really belong in that shard
group and was placed there by mistake, it's best to fix this bug since
the timestamp used to create the shard group should be capable of
retrieving it.
2016-06-16 12:15:41 -05:00
Joe LeGasse 647210c57a Add TruncatedAt field to meta.ShardGroupInfo 2016-06-07 18:24:24 -04:00
Jonathan A. Sternberg b8e22d9d79 Merge pull request #6586 from influxdata/js-3733-rename-default-retention-policy
Modify the default retention policy name and make it configurable
2016-06-06 15:05:29 -05:00
Rubycut a849f42cc2 Lock auth properly, code suggested by @jwilder 2016-06-04 09:05:10 +02:00
Edd Robinson f4fc905fa9 Reject timestamps too far in future 2016-05-27 11:07:48 +01:00
Jonathan A. Sternberg 907c88d4b5 Truncate the shard group end time if it exceeds MaxNanoTime
Related to #6599.
2016-05-25 21:25:10 -04:00
Jonathan A. Sternberg baaa782c95 Modify the default retention policy name and make it configurable
The default retention policy name is changed to "autogen" instead of
"default" since it ends up being ambiguous when we tell a user to check
the default retention policy, it is uncertain if we are referring to the
default retention policy (which can be changed) or the retention policy
with the name "default".

Now the automatically generated retention policy name is "autogen".

The default retention policy is now also configurable through the
configuration file so an administrator can customize what they think
should be the default.

Fixes #3733.
2016-05-24 09:51:23 -04:00
David Norton 9d5c4a0f41 fix #6702: return correct required privileges
Change SelectStatement to return required privileges for all Sources.
2016-05-23 16:44:23 -04:00
David Norton b32208a850 fix #2048: check that RPs exist before creating CQ 2016-05-20 12:40:36 -04:00
Jason Wilder adddb0e6b8 Reduce lock contention when creating shard group 2016-05-09 16:10:38 -06:00
Edd Robinson d35fa1ec97 Remove redundant windows build tags 2016-05-03 14:22:02 +01:00
Seif Lotfy ced79acac5 Remove redundant error return from (Client) Database(name string) and (Client) Databases() 2016-04-30 17:04:38 -05:00