Commit Graph

97 Commits (e7cce6953013e77ae78c761a38fe593c619b0f13)

Author SHA1 Message Date
Jason Wilder 9f41acba2f Move shard mapping logic into index 2016-03-29 12:59:27 -06:00
Jason Wilder 03ced4cc90 Load shards concurrently 2016-03-29 12:58:52 -06:00
Ben Johnson 7156c1f9bd add IteratorStats
This commit adds an `IteratorStats` that holds aggregate
iterator processing information. A method is also added to
`Iterator` to return the stats:

	Stats() influxql.IteratorStats

The remote iterators will also emit their stats in the point
stream upon first connection, on a given interval, and then
finally once the last point has been sent.
2016-03-21 16:25:19 -06:00
Ben Johnson beda072426 add support for remote expansion of regex
This commit moves the `tsdb.Store.ExpandSources()` function onto
the `influxql.IteratorCreator` and provides support for issuing
source expansion across a cluster.
2016-03-11 12:40:07 -07:00
Jason Wilder 992c78ee22 Remove period shard maintenance goroutine
This is no longer used in tsm and just peridocially locks everything
for no reason now.
2016-03-09 17:31:02 -07:00
Edd Robinson 7dbc0f49d3 Merge pull request #5818 from influxdata/er-upgrade-error
Highlight upgrade info for old shards
2016-03-09 19:39:59 +00:00
Edd Robinson 58c03448aa Merge pull request #5514 from influxdata/er-engine-panic
Ensure shards and engine are safely closed
2016-03-09 18:56:36 +00:00
Ben Johnson 41dde61226 SHOW SERIES 2016-03-08 11:47:57 -07:00
Jonathan A. Sternberg 2f0e246757 Implemented the tag values iterator for `SHOW TAG VALUES`
`SHOW TAG VALUES` output has been modified to print the measurement name
for every measurement and to return the output in two columns: key and
value. An example output might be:

    > SHOW TAG VALUES WITH KEY IN (host, region)
    name: cpu
    ---------
    key     value
    host    server01
    region  useast

    name: mem
    ---------
    key     value
    host    server02
    region  useast

`measurementsByExpr` has been taught how to handle reserved keys (ones
with an underscore at the beginning) to allow reusing that function and
skipping over expressions that don't matter to the call.

Fixes #5593.
2016-03-06 09:52:34 -05:00
Mark Rushakoff cdcb079769 Tag TSM stats with database, retention policy
... by extracting the db/rp from the given path.

Now that the code has "standardized" on extracting db/rp this way, the
ShardLocation struct is no longer necessary and thus has been removed.
We're back on the previous style of passing the path and walPath to
NewShard.
2016-02-29 09:17:34 -08:00
Jonathan A. Sternberg aa0b603938 Convert `SHOW FIELD KEYS` to the new query engine
Fixes #5579.
2016-02-25 18:31:02 -05:00
Mark Rushakoff 40a98e0d55 Add database, RP as tags on shard stats
This commit updates tsdb.Shard to contain a ShardConfig and updates
tsdb.Store to directly reference a map of tsdb.Shard rather than the
previous tsdb.shardLocation abstraction.
2016-02-25 13:41:55 -08:00
Ben Johnson 0dda9f6608 add remote execution
This commit adds remote execution to the query engine.
2016-02-25 08:41:20 -07:00
Edd Robinson aa845cec7e Check for shards needing conversion. Fixes #5723 2016-02-25 13:21:13 +00:00
Edd Robinson 16995b6c23 Add ShardError to provide context about shard that errored 2016-02-24 13:33:07 +00:00
Jonathan A. Sternberg 7a03df2af1 Remove the non-unreachable panics in the new query engine
The only panics left are ones that should be unreachable unless there is
a bug.

Fixes #5777.
2016-02-22 12:52:43 -05:00
Joe LeGasse dc8ed7953d Remove custom binary-conversion functions
Also cleaned up some excess allocations, and other cruft from the code
2016-02-18 13:56:35 -05:00
Mark Rushakoff fc9ab7a46f Miscellaneous cleanup in tsdb package
* When possible, initialize maps/slices to exact length/capacity
  * See slice benchmarks at
    https://gist.github.com/mark-rushakoff/b5650bd8f06bece0b9fd
* Fixed some typos
* Removed an unnecessary loop in stringset.intersect
2016-02-10 18:00:47 -08:00
Justin Nuß 82c276756a Lint tsdb and tsdb/engine package 2016-02-10 21:33:46 +01:00
Ben Johnson 5a0d1ab7c1 rename influxdb/influxdb to influxdata/influxdb
This commit changes all the import and URL references from:

    github.com/influxdb/influxdb

to:

    github.com/influxdata/influxdb
2016-02-10 10:26:18 -07:00
Jonathan A. Sternberg 2e7cf5328c Fix go vet issues on 1.4
go 1.5 was being used to develop the query engine branch, but we aren't
using 1.5 for master at the moment. This fixes issues that go vet brings
up in 1.4 that don't exist in 1.5.
2016-02-10 09:40:30 -07:00
Jonathan A. Sternberg d1f7c445e7 Modify iterators to work across shards
Aux iterators now ask the iterator creator what series will be returned
and determine which aux fields to create based on the results.

The `tsdb.Shards` struct also creates a call iterator around the
iterators returned from each shard.
2016-02-10 09:40:29 -07:00
Jonathan A. Sternberg c2d1206177 Implement the fill iterator
Fill requires an additional function for IteratorCreator to retrieve the
series that will be returned from the iterator. When fill is required
for an aggregate, the IteratorCreator will be asked what series will be
returned by the created iterator.
2016-02-10 09:40:29 -07:00
Ben Johnson 47c2bab74b add SHOW TAG KEYS support 2016-02-10 09:40:28 -07:00
Ben Johnson 607750ab1b add SHOW MEASUREMENTS iterator 2016-02-10 09:40:28 -07:00
Ben Johnson 2bdc9404ef revert meta execution 2016-02-10 09:40:28 -07:00
Ben Johnson 00806de9b8 refactor query engine 2016-02-10 09:40:25 -07:00
Ben Johnson cde973f409 refactor query engine 2016-02-10 09:40:24 -07:00
Edward Robinson c7bbe6ef17 Remove engine on close 2016-02-03 10:19:42 +00:00
Cory LaNou 7e1dc0231d fix data race in TSDB 2015-12-03 13:52:27 -06:00
Philip O'Toole de7919240f Migrate internal stats to consistent names
Go style -- and existing runtime stats -- do not use underscores, but
instead use camel case. This change makes the internal stats adhere to
that convention.
2015-10-28 21:07:45 -07:00
Paul Dix b11308133a Only limit field count for non-tsm engines 2015-10-06 15:49:37 -07:00
Paul Dix d47ddb5454 Cleanup after pd1 -> tsm1 name change. 2015-10-05 20:09:55 -04:00
Paul Dix 0b33a71bb7 Add recover to maintenance. Change snapshot writer to not use bolt on shard. 2015-10-05 20:06:22 -04:00
Paul Dix 1c8eac1523 Add PerformMaintenance to store for flushes and compactions.
Also fixed shard to work again with b1 and bz1 engines.
2015-10-05 20:06:22 -04:00
Paul Dix 2ba032b7a8 WIP: finish basics of PD1. IT WORKS! (kind of) 2015-10-05 20:06:21 -04:00
Paul Dix 7555ccbd70 WIP: engine work 2015-10-05 20:06:21 -04:00
Cory LaNou d19a510ad2 refactor Points and Rows to dedicated packages 2015-09-16 15:33:08 -05:00
Daniel Morsing 66fc270d1e Move aggregate funcs into tsdb
Pure move, No functional changes.
2015-09-11 14:27:47 +00:00
Philip O'Toole 3db9cc9b76 Basic instrumentation for shards 2015-09-08 19:56:42 -07:00
Jason Wilder 6b4926257a Add inspect tool
Start of a lower-level file inspection tool.  This currently dumps
summary statistics for the shards, index and WAL that can be used to
understand the shape of the data is in the local shards.  This util
operates on the shards itself and not through the server and is intended
more for debugging/troubleshooting.
2015-09-04 10:38:59 -06:00
Ben Johnson deff06f850 add copier service
This commit adds the copier service which allows one server to
copy shards from another server. This will be used for moving
shards in the cluster.
2015-09-03 13:07:35 -06:00
Daniel Morsing c4092d7fc3 Revert "move aggregate functions" 2015-09-02 10:47:58 -07:00
Daniel Morsing 65340a023b Move aggregate funcs into tsdb
Pure move, No functional changes.
2015-09-01 14:25:59 -07:00
Paul Dix a52a4be94c Merge pull request #3793 from influxdb/pd-fix-unsafe-series-shard-access
Fix map concurrent race with adding a shard to a series in the index.
2015-08-21 16:37:04 -04:00
Paul Dix 1a3074ed54 Fix map concurrent race with adding a shard to a series in the index. 2015-08-21 16:24:55 -04:00
Paul Dix 73f3dc1e14 Update store to properly manage WAL create/delete.
* Update the store to remove the WAL directories associated with a shard or database when they are deleted.
* Fix the Store so that it creates separate WAL directories for databases and retention policies.
2015-08-21 11:22:04 -04:00
Paul Dix 3348dab4e0 Fix bug with new shards not getting series data persisted. 2015-08-16 15:45:09 -04:00
Paul Dix 6776014047 Fix bug in stress script, remove extraneous printlns 2015-08-16 12:46:50 -04:00
Paul Dix a77a91933e WIP: fix bug with how bz writes index. fix bug with wal not having index set. 2015-08-16 12:46:50 -04:00