The deprecated message is now attached to a new attribute returned with
the results. This message can then be read by clients to warn a user
about upcoming changes to the query engine.
The `influx` client has already been modified to read this message and
print it out for every format except CSV.
The first warning message is a deprecated message about removing `IF NOT
EXISTS` from `CREATE DATABASE`.
The message will also be printed to the server log.
Fixes#5707.
The QueryExecutor had a lot of dead code made obsolete by the query
engine refactor that has now been removed. The TSDBStore interface has
also been cleaned up so we can have multiple implementations of this
(such as a local and remote version).
A StatementExecutor interface has been created for adding custom
functionality to the QueryExecutor that may not be available in the open
source version. The QueryExecutor delegate all statement execution to
the StatementExecutor and the QueryExecutor will only keep track of
housekeeping. Implementing additional queries is as simple as wrapping
the cluster.StatementExecutor struct or replacing it with something
completely different.
The PointsWriter in the QueryExecutor has been changed to a simple
interface that implements the one method needed by the query executor.
This is to allow different PointsWriter implementations to be used by
the QueryExecutor. It has also been moved into the StatementExecutor
instead.
The TSDBStore interface has now been modified to contain the code for
creating an IteratorCreator. This is so the underlying TSDBStore can
implement different ways of accessing the underlying shards rather than
always having to access each shard individually (such as batch
requests).
Remove the show servers handling. This isn't a valid command in the open
source version of InfluxDB anymore.
The QueryManager interface is now built into QueryExecutor and is no
longer necessary. The StatementExecutor and QueryExecutor split allows
task management to much more easily be built into QueryExecutor rather
than as a separate struct.
A change to the admin UI prevented the success message being displayed
for empty results. This change restores the original behaviour for
this case.
Signed-off-by: Jon Seymour <jon@wildducktheories.com>
FormValue() would attempt to parse the body of a request when the
content-type is set to `application/x-www-form-urlencoded`. The write
handler never wants url-encoded forms, and should only ever check the
URL for query parameters.
Fixes#6061
It's possible for a single query to send multiple results that get
aggregated in the HTTP handler. If an earlier result passed in data and
a later result had an error, the error would be ignored.
Now an error for a statement will overwrite any previous results for
that statement.
Partially fixes#6094.
Prior to this when passing the same query and CQ name in a CREATE
CONTINUOUS QUERY command an error would be returned. This means the
command was not behaving in a similar way to other commands.a
Now when running the command with the same CQ name and query string no
error will be returned. Note, this change does not parse the query, it
simply compares a normalised query string to the existing one on the CQ.
Partially addresses #6094.
Previously, when creating a retention policy only the name was
considered when deciding if the policy already existed. This meant that
adding a second policy with the same name but different duration or
replica factor returned the original policy and no error.
This commit fixes that and ensures that name, duration and replica
factor are all considered.
Allows configuration of shard group duration at database creation, and retention
policy create/alter time.
Query examples:
```
CREATE DATABASE testdb WITH DURATION 90d SHARD DURATION 30m NAME rp_testdb
CREATE RETENTION POLICY rp_testdb2 ON testdb DURATION INF REPLICATION 1 SHARD DURATION 30m
ALTER RETENTION POLICY rp_testdb2 ON testdb SHARD DURATION 1h
```
This can be useful with long duration retention policies with lots of data, where
you can split into smaller shards to relieve memory pressure.
This seems to have been an oversight since all of the response writers
are supposed to implement this interface, but the gzipResponseWriter
didn't implement this interface for some reason.
The currently running queries can be listed with the command
`SHOW QUERIES` and it will display the current commands that have been
run, the database they were run against, and how long they have been
running.
Go 1.4.3 was a security release that also created a strange edge-case
that caused connections to not be kept alive and reused when Close()
is called on the Body of the request. Close() hasn't been required on
the Body of a request for some time, so there is no harm is not calling
it anymore.
... by extracting the db/rp from the given path.
Now that the code has "standardized" on extracting db/rp this way, the
ShardLocation struct is no longer necessary and thus has been removed.
We're back on the previous style of passing the path and walPath to
NewShard.
This commit updates tsdb.Shard to contain a ShardConfig and updates
tsdb.Store to directly reference a map of tsdb.Shard rather than the
previous tsdb.shardLocation abstraction.
Previously, CQs with the same name would be stored in the last run map
the same way. This caused only one of the CQs to run because after the
first one ran it would update the last run time for all CQs with the
same name.
Add the database name to the CQ ID in the last run map to differentiate
between CQs in different databases.
Fixes#5814.
Fixes#5612, #5573 and #5518.
Using the MetaExecuter, queries that need to run on both data nodes
and optionally the meta store will be executed across all data nodes
in the cluster.
Fixes#5680.
When dropping a data node, the following will now happen on the
Meta Store.
1) If any shards no longer have any owners (because the data node
being dropped is the only owner), they will be reassigned a
new owner from within their respective shard group.
2) If a shard group no longer has any shards/data nodes, they will
be marked as deleted.
When a shard is being assigned a new owner a data node with the fewest
number of shards in the shard group will be selected as the new owner.
Finally, checking the validity of a data node's ID now happens in the
Meta store, rather than in the state machine.
This fixes a couple of issues with starting meta-only nodes.
1. We were always calling CreateDataNode regardless of whether the the
node is running data services. We only call that now when node is
data enabled.
2. The node.json was created along-side creating the data node. Since
we are not creatinga a data node, this didn't happen anymore. There
wasn't a simple way to do this in one place so it's actually handle
for when creating a meta or a data node now. Since the ID assigned
to the node is the same regardless of role this works in all combinations
of roles.
3. The JoinMetaServer didn't return the ID of the joining node which
created some races when multiple nodes were joining. The join call now
returns that information to the caller.
Fixes#5754
The join option was incorrectly exposed on the meta config. It should
be at the top-level as a string and propogate down to the meta config
as a slice.
Fixes#5653 and #5394.
Previously dropping retention policies did not propogate to local TSDB
shards. Instead, the retention policiess would just be removed from the
Meta Store.
This PR adds ensures that data associated with retention policies is
removed, when the retention policy is dropped.
Also, it cleans up a couple of other methods in `tsdb`, including the
requirement to provide (redundant) shardIDs when deleting databases.
Dropping a meta node that had already been removed from the config
would fail because the raft.RemovePeers call would return an error
that the address was unknown. This change skips calling RemovePeer
if it doesn't exist.
Dropping a non-existing ID would hang for 10 seconds becuase the
meta.Client retryUntilExec didn't differentiate before command errors
and redirect errors. In this case, the command would return an error
but we'd try 10 more times and ultimately give up and return the error.
We now return immediately if the command returned and error because
retrying it will not succeed.
Finally, the join loop had no delay and would immediately try to join
the other nodes hundreds of times a second. We now pause a second if we've
tried every node at least once.
This fixes several issues related to the bind address and hostname:
* Allows bind addresses where a hostname or IP is not specified to
work correct and bind to all interfaces by default.
* Fixes the top-level "hostname" config option to allow overridding
all bind address hostnames. This allows a node to advertise a different
hostname than what is defined in the bind address setting.
* Adds the -hostname command-line option back to allow specifing
both -join and -hostname as command-line flags.
* Enforces a configuration precedence and overriding ability defined
as config file is overridden by env vars which are overriden by command-line
flags.
Fixes#5670#5671
Previously, meta.Client would drop the default retention policy when
trying to create a database with a retention policy. The RPC has now
been modified to include the desired retention policy in the
CreateDatabase command and have it use that retention policy information
instead of the default configuration when provided.
This also lowers the number of RPC calls for
CreateDatabaseWithRetentionPolicy to only a single RPC call instead of
two.
Protections have also been included so creating a retention policy with
different parameters will return an error similar to if you tried to
modify the retention policy separately.
Fixes#5696.
This removes the MetaServers property from node.json to eliminate one
of the four places those addresses are stored on disk. We always use
the values that come through the config (via file, env var or -join arg).
I was trying to create a Diagnostics Client in the tsdb package, but
IIRC importing `monitor` caused an import cycle of:
tsdb -> monitor -> cluster -> tsdb.
Moving Diagnostics to its own package will allow further use of
diagnostics.Client without running into import cycles.
Meta HTTP commands are cluster level requests and were showing up in
the main log creating a lot of noise. Switch them to use the ClusterTracing
config option which is disabled by default.
David Norton
Jonathan A. Sternberg
Aaron Knister
Chris Ramón
Todd Persen
Gunnar
Valeriy Simonov
Edd Robinson
Jonathan A. Sternberg
gunnaraasen
Cameron Sparr
Cory LaNou
Regan Kuchan
mvadu
Todd Persen
Jon Seymour
Jason Wilder
joelegasse
Tait Clarridge
Ben Johnson
Vidhuran Harichandra Babu
Mark Rushakoff
Mark Rushakoff
Jason Wilder
bastard
Ross McDonald
Nicholas Katsaros
Ross McDonald
Chris H (CruftMaster)
Chris H (CruftMaster)