Go to file
Joe LeGasse 9af4894c1c influx_tsm: ignore series index and convert all points
A case (#5606) was found where a lot of data unexpectedly disappeared from a database
following a TSM conversion.

The proximate cause was an inconsistency between the root Bolt DB bucket list
and the meta data in the "series" bucket of the same shard. There were apparently valid
series in Bolt DB buckets that were no longer referenced by the meta data
in the "series" bucket - so-called orphaned series; since the conversion
process only iterated across the series found in the meta data, the conversion process
caused the orphaned series to be removed from the converted shards. This resulted in the
unexpected removal of data from the TSM shards that had previously been accessible
(despite the meta data inconsistency) in the b1 shards.

The root cause of the meta data inconsistency in the case above was a failure, in versions prior
to v0.9.3 (actually 3348dab) to update the "series" bucket with series that had been created in
previous shards during the life of the same influxd process instance.

This fix is required to avoid data loss during TSM conversions for shards that were created with
versions of influx that did not include 3348dab (e.g. prior to v0.9.3).

Analysis-by: Jon Seymour <jon@wildducktheories.com>
2016-02-16 13:46:01 -05:00
.hooks Use go1.4.3 2016-01-10 16:43:31 -07:00
client Switched 0.9 references to 0.10. 2016-02-11 08:44:45 -06:00
cluster add canonical paths 2016-02-10 11:30:52 -07:00
cmd influx_tsm: ignore series index and convert all points 2016-02-16 13:46:01 -05:00
etc Updated sample configuration for new meta service options. 2016-02-09 09:34:31 -06:00
importer Switched 0.9 references to 0.10. 2016-02-11 08:44:45 -06:00
influxql Pass the implicit end time from the query executor to the select call 2016-02-12 16:03:24 -05:00
models add canonical paths 2016-02-10 11:30:52 -07:00
monitor Move monitor.Diagnostics to its own package 2016-02-13 14:56:15 -08:00
pkg add canonical paths 2016-02-10 11:30:52 -07:00
scripts Merge pull request #5536 from mmattioli/cleanup 2016-02-08 08:57:44 -07:00
services Merge pull request #5682 from influxdata/mr-separate-diagnostics-package 2016-02-16 09:38:15 -07:00
shared/admin Switched 0.9 references to 0.10. 2016-02-11 08:44:45 -06:00
statik Switched 0.9 references to 0.10. 2016-02-11 08:44:45 -06:00
stress add canonical paths 2016-02-10 11:30:52 -07:00
tcp add canonical paths 2016-02-10 11:30:52 -07:00
tests rename influxdb/influxdb to influxdata/influxdb 2016-02-10 10:26:18 -07:00
toml add canonical paths 2016-02-10 11:30:52 -07:00
tsdb Remove redundant wal files 2016-02-16 20:45:13 +08:00
uuid add canonical paths 2016-02-10 11:30:52 -07:00
.dockerignore decouple circleci from tests using docker 2015-12-14 16:39:39 -07:00
.gitignore b*1 to tsm1 shard converter 2015-12-29 15:31:07 -05:00
CHANGELOG.md Merge pull request #5666 from influxdata/jw-gdm 2016-02-16 09:52:22 -07:00
CODING_GUIDELINES.md Fixes typo in CODING_GUIDELINES.md 2016-02-02 17:20:13 +05:30
CONTRIBUTING.md Manage dependencies with gdm 2016-02-12 16:27:17 -07:00
DOCKER.md Fix docker HTTP port example 2015-08-19 12:18:56 -06:00
Dockerfile Add instructions for building a minimal docker image 2015-08-07 14:27:33 -06:00
Dockerfile_build_ubuntu32 rename influxdb/influxdb to influxdata/influxdb 2016-02-10 10:26:18 -07:00
Dockerfile_build_ubuntu64 rename influxdb/influxdb to influxdata/influxdb 2016-02-10 10:26:18 -07:00
Dockerfile_build_ubuntu64_git rename influxdb/influxdb to influxdata/influxdb 2016-02-10 10:26:18 -07:00
Dockerfile_test_ubuntu32 rename influxdb/influxdb to influxdata/influxdb 2016-02-10 10:26:18 -07:00
Godeps Manage dependencies with gdm 2016-02-12 16:27:17 -07:00
LICENSE Update year to 2016 in LICENSE 2016-01-16 13:09:14 +05:30
LICENSE_OF_DEPENDENCIES.md updating Golang crypto 2015-11-04 15:30:32 -08:00
Makefile tsm1 meta lint 2015-10-15 15:03:10 -06:00
QUERIES.md Update QUERIES.md 2015-10-27 23:02:18 -07:00
README.md Update README for 0.10 2016-02-11 16:51:33 -08:00
build-docker.sh Modify shebang for Docker build script 2016-02-03 21:26:43 -05:00
build.py Substituting 'go get' for gdm commands. Slight restructure of commands. 2016-02-16 09:33:39 -06:00
build.sh rename influxdb/influxdb to influxdata/influxdb 2016-02-10 10:26:18 -07:00
circle-test.sh add docker rm flag so circleci can disable deletion of containers 2016-02-02 15:18:58 -07:00
circle.yml decouple circleci from tests using docker 2015-12-14 16:39:39 -07:00
errors.go tsm1 meta lint 2015-10-15 15:03:10 -06:00
gobuild.sh make tip container usable for any git revision, add go1.6beta1 test 2015-12-16 15:28:35 -07:00
influxdb.go add canonical paths 2016-02-10 11:30:52 -07:00
influxvar.go Fix minor typo in comment 2015-09-07 22:04:37 -07:00
nightly.sh Update packaing and build for new github org 2016-01-04 15:47:04 -08:00
node.go Remove MetaServers from node.json 2016-02-15 22:42:51 -07:00
package.sh Use go1.4.3 2016-01-10 16:43:31 -07:00
test.sh rename influxdb/influxdb to influxdata/influxdb 2016-02-10 10:26:18 -07:00

README.md

InfluxDB Circle CI

An Open-Source, Distributed, Time Series Database

InfluxDB is an open source distributed time series database with no external dependencies. It's useful for recording metrics, events, and performing analytics.

Features

  • Built-in HTTP API so you don't have to write any server side code to get up and running.
  • Data can be tagged, allowing very flexible querying.
  • SQL-like query language.
  • Clustering is supported out of the box, so that you can scale horizontally to handle your data. Clustering is currently in an alpha state.
  • Simple to install and manage, and fast to get data in and out.
  • It aims to answer queries in real-time. That means every data point is indexed as it comes in and is immediately available in queries that should return in < 100ms.

Installation

We recommend installing InfluxDB using one of the pre-built packages. Then start InfluxDB using:

  • service influxdb start if you have installed InfluxDB using an official Debian or RPM package.
  • systemctl start influxdb if you have installed InfluxDB using an official Debian or RPM package, and are running a distro with systemd. For example, Ubuntu 15 or later.
  • $GOPATH/bin/influxd if you have built InfluxDB from source.

Getting Started

Create your first database

curl -G 'http://localhost:8086/query' --data-urlencode "q=CREATE DATABASE mydb"

Insert some data

curl -XPOST 'http://localhost:8086/write?db=mydb' \
-d 'cpu,host=server01,region=uswest load=42 1434055562000000000'

curl -XPOST 'http://localhost:8086/write?db=mydb' \
-d 'cpu,host=server02,region=uswest load=78 1434055562000000000'

curl -XPOST 'http://localhost:8086/write?db=mydb' \
-d 'cpu,host=server03,region=useast load=15.4 1434055562000000000'

Query for the data

curl -G http://localhost:8086/query?pretty=true --data-urlencode "db=mydb" \
--data-urlencode "q=SELECT * FROM cpu WHERE host='server01' AND time < now() - 1d"

Analyze the data

curl -G http://localhost:8086/query?pretty=true --data-urlencode "db=mydb" \
--data-urlencode "q=SELECT mean(load) FROM cpu WHERE region='uswest'"

Documentation

Contributing

If you're feeling adventurous and want to contribute to InfluxDB, see our contributing doc for info on how to make feature requests, build from source, and run tests.

Looking for Support?

InfluxDB offers a number of services to help your project succeed. We offer Developer Support for organizations in active development, Managed Hosting to make it easy to move into production, and Enterprise Support for companies requiring the best response times, SLAs, and technical fixes. Visit our support page or contact sales@influxdb.com to learn how we can best help you succeed.