Commit Graph

28 Commits (d0688201ba013baea8a1a15f5ac2283db1981afe)

Author SHA1 Message Date
Edd Robinson 64171229e8
Add test for index sketches 2018-02-07 14:52:13 -07:00
Stuart Carnie 5dfe3b2645 inmem startup improvments
* only call ParseTags when necessary
* remove dependency on inmem.Series in tsdb test package
* Measurement and Series are no longer exported. Their use is restricted
  to the inmem package
* improve Measurement and Series types by exporting immutable
  fields and removing unnecessary APIs and locks

Reduced startup time from 28s to 17s. Overall improvement including
#9162 reduces startup from 46s to 17s for 1MM series across 14 shards.
2017-12-29 07:58:52 -07:00
Stuart Carnie c89d98dc02 gofmt 2017-05-25 16:00:23 -07:00
Stuart Carnie 386720b2e7 improvements to inmem/Measurement.TagSets API
```
benchmark                                             old ns/op     new ns/op     delta
BenchmarkMeasurement_TagSetsNoDimensions_1000-8       234054        117315        -49.88%
BenchmarkMeasurement_TagSetsDimensions_1000-8         996838        313313        -68.57%
BenchmarkMeasurement_TagSetsNoDimensions_100000-8     58940464      39452117      -33.06%
BenchmarkMeasurement_TagSetsDimensions_100000-8       175612060     70195562      -60.03%

benchmark                                             old allocs     new allocs     delta
BenchmarkMeasurement_TagSetsNoDimensions_1000-8       1026           26             -97.47%
BenchmarkMeasurement_TagSetsDimensions_1000-8         8026           2029           -74.72%
BenchmarkMeasurement_TagSetsNoDimensions_100000-8     100064         64             -99.94%
BenchmarkMeasurement_TagSetsDimensions_100000-8       800064         200067         -74.99%

benchmark                                             old bytes     new bytes     delta
BenchmarkMeasurement_TagSetsNoDimensions_1000-8       117080        69080         -41.00%
BenchmarkMeasurement_TagSetsDimensions_1000-8         549081        117176        -78.66%
BenchmarkMeasurement_TagSetsNoDimensions_100000-8     23298264      18498265      -20.60%
BenchmarkMeasurement_TagSetsDimensions_100000-8       66498276      23298360      -64.96%
```
2017-05-25 15:52:27 -07:00
Stuart Carnie 5c5bea2baa move Measurement and Series to inmem package 2017-05-19 08:17:09 -07:00
Jason Wilder 02e22f4a00 Fix deadlock in Measurement
The lazy sorting of series caused a deadlock since it can not take
a Lock when a caller may have already acquired an RLock.

filters should be called w/o any locks as the function already acquires
locks as needed.
2017-05-03 13:49:56 -06:00
Ben Johnson 358b1e0b05
Merge remote-tracking branch 'upstream/master' into tsi 2017-03-15 10:13:32 -06:00
Jason Wilder a024003f2c Merge branch '1.2' into jw-merge-12 2017-02-22 12:13:29 -07:00
Ben Johnson 78a9bb2527 Remove Tags.shouldCopy, replace with forceCopy on series creation.
Previously, tags had a `shouldCopy` flag to indicate if those tags
referenced an underlying buffer and should be copied to allow GC.
Unfortunately, this prevented tags from being copied that were
created and referenced the mmap which caused segfaults.

This change removes the `shouldCopy` flag and replaces it with a
`forceCopy` argument in `CreateSeriesIfNotExists()`. This allows
the write path to indicate that tags must be cloned on insert.
2017-02-21 11:13:35 -07:00
Edd Robinson 4fbba8234e Add Size to models.Tags 2017-02-08 18:44:48 +00:00
Edd Robinson 292b30b82b Fix subtle bugs and remove dead code from tsdb 2017-01-17 09:47:34 -08:00
Ben Johnson d13afa8f47
Iterator refactoring 2017-01-05 10:11:49 -07:00
Ben Johnson afce53e81b
Rebase fixes. 2017-01-05 10:00:44 -07:00
Ben Johnson 2a81351992
Implement tsdb.Index interface on tsi1.Index. 2017-01-05 10:00:43 -07:00
Edd Robinson 149b1cef1d
Fix 32bit overflow; limit capacity 2017-01-05 09:59:10 -07:00
Edd Robinson d19fbf5ab4
Wire in HLL estimator 2017-01-05 09:54:03 -07:00
Ben Johnson 9b62df23d2
Add MeasurementBlock. 2017-01-05 09:34:36 -07:00
Jason Wilder 2e473e9518 Fix panic in AppendSeriesKeyByID
Calling this function with a series ID that does not exist in
the measurement causes a panic.

Fixes #7334
2016-10-19 11:07:19 -06:00
Jason Wilder 6671ef00f0 Reduce allocations in idsForExpr 2016-09-26 08:36:59 -06:00
Ben Johnson 8aa224b22d
reduce memory allocations in index
This commit changes the index to point to index data in the shards
instead of keeping it in-memory on the heap.
2016-08-16 14:09:00 -06:00
Mark Rushakoff fb83374389 Track stats for number of series, measurements
Per database: track number of series and measurements
Per measurement: track number of series
2016-02-24 08:10:16 -08:00
Ben Johnson 5a0d1ab7c1 rename influxdb/influxdb to influxdata/influxdb
This commit changes all the import and URL references from:

    github.com/influxdb/influxdb

to:

    github.com/influxdata/influxdb
2016-02-10 10:26:18 -07:00
Jonathan A. Sternberg 2e7cf5328c Fix go vet issues on 1.4
go 1.5 was being used to develop the query engine branch, but we aren't
using 1.5 for master at the moment. This fixes issues that go vet brings
up in 1.4 that don't exist in 1.5.
2016-02-10 09:40:30 -07:00
Paul Dix 3348dab4e0 Fix bug with new shards not getting series data persisted. 2015-08-16 15:45:09 -04:00
Ben Johnson de1f9a3736 refactor tsdb tests into test package 2015-07-22 11:07:06 -06:00
Philip O'Toole 98d315b8d8 Ensure tags are always marshalled the same way
This added check ensures it's always alphabetically ordered when
marshalled.
2015-06-09 17:10:46 -07:00
David Norton 67464238ed add benchmarks for building in-mem series index 2015-06-02 17:00:25 -04:00
Paul Dix f660b1a3d3 Create QueryExecutor in tsdb. 2015-05-29 14:59:57 -06:00