Commit Graph

87 Commits (448b59883410db8ce3e8de8fdca5ad783511fa2d)

Author SHA1 Message Date
Jeff Wendling 26ca30e97a Ensure that cached series id sets are Go heap backed 2019-02-12 16:33:35 -07:00
Ben Johnson cf29b6bca4
Convert TagValueSeriesIDCache to use string fields 2019-02-12 14:45:38 -07:00
Edd Robinson 1188d75a99
Merge pull request #11202 from influxdata/er-tsi-times
Add skeleton TSI design doc
2019-01-28 11:54:59 -08:00
Edd Robinson 19a36e0dc7 Remove copy-on-write when caching bitmaps
In the case of caching TSI bitmaps belonging to immutable .tsi files,
the underlying bitset data can be mmapped. It is possible, though rare,
for this data to be unmapped (e.g., via a TSI compaction) but for the
cached bitmap to be subsequently read. This leads to a segfault.

This only happens when copy-on-write is set to true on the roaring
bitmap, because in that case only the internal pointers are cloned.

This change will reduce the TSI cache performance by around 10%, which I
have deemed to account for only a few microseconds typically.
2019-01-25 13:38:22 +00:00
Edd Robinson 045bb64c5e Add skeleton TSI design doc 2019-01-17 12:22:08 +00:00
Edd Robinson 810b5d9281 Bulk log file delete
This commit adds a method to delete many series ids from the LogFile in
bulk, reducing the number of fsyncs required.
2019-01-15 11:45:12 +00:00
Edd Robinson b025d9afa9 Improve efficiency of TSI index series drop
This commit improves the performance of a mass delete on the TSI index
by deleting at the measurement level instead of deleting each series
individually.
2019-01-14 12:46:55 +00:00
Edd Robinson 7ee4f499e1 Clarify best method of set difference 2019-01-14 12:46:53 +00:00
Edd Robinson 20a8528337 Ensure TSI bitset cache cleaned up on m drop 2019-01-14 11:23:13 +00:00
Mark Rushakoff d73d73c0d4 chore: rename imports from platform to influxdb
I did this with a dumb editor macro, so some comments changed too.

Also rename root package from platform to influxdb.

In interest of minimizing risk, anyone importing the root package has
now aliased it to "platform" so that no changes beyond imports were
necessary in those files.

Lastly, replace the old platform module to local path /dev/null so that
nobody can accidentally reintroduce a platform dependency while
migrating platform code to influxdb.
2019-01-09 20:51:47 -08:00
Mark Rushakoff f383e8337a test(tsdb/tsi1): test series id cache delete concurrently
Report the total number of gets, puts, and deletes at the end of the
test. I've found this kind of output to be a useful sanity check in
similar tests that exercise concurrency involving tasks.

Use a local random source in each goroutine. I unscientifically
eyeballed that to increase total operations by 5-10%.

Also call t.Parallel in a few more tests that involve disk access. This
shaves 1-2 seconds off the full tsi1 test suite on my machine.
2018-12-12 08:21:17 -08:00
Edd Robinson c4b42c72be Add option to disable TSI metrics 2018-12-10 15:02:26 +00:00
Edd Robinson e0cddadffd Allow TSI cache to be dynamically altered 2018-12-07 18:35:25 +00:00
Edd Robinson e13309ebbe Fix metric names 2018-12-07 16:37:17 +00:00
Edd Robinson b015757c06 Ensure all tsi1 metrics support multiple instances 2018-12-07 14:32:34 +00:00
Edd Robinson 79b108d174 Fix bug with slice reuse 2018-12-07 14:32:34 +00:00
Edd Robinson 2bb558a9d1 Ensure fileset protected by lock 2018-12-07 14:32:34 +00:00
Edd Robinson a1804d27be Fix race 2018-12-07 14:32:34 +00:00
Edd Robinson f9a2f7a017 go fmt 2018-12-07 14:32:34 +00:00
Edd Robinson 7960ccc320 Add TSI index metrics 2018-12-07 14:32:34 +00:00
Ben Johnson 98d24f7e3c
Merge pull request #1625 from influxdata/remove-influxdb-dependency
Remove influxdb dependency.
2018-11-29 14:23:00 -07:00
Ben Johnson 0084d4d824
Remove influxdb dependency. 2018-11-29 11:44:22 -07:00
zhulongcheng ed799a3d6c remove Index.CreateSeriesIfNotExists 2018-11-21 20:16:45 +08:00
Jeff Wendling 22e23d6e31 final touches
- move default directories to the storage package
- make the directory layout match before
- clean up some dead missed functions
2018-11-08 11:39:36 -07:00
Jeff Wendling 0d411023f2 config: clean up
- Breaks the weird cycle that existed with the EngineOptions
- Removes a bunch of useless parameters
- Moves around a bunch of defaults
2018-11-08 11:39:36 -07:00
Mark Rushakoff 985c260af7 chore(storage,tsdb): fix megacheck errors 2018-11-01 12:54:46 -07:00
Jeff Wendling 6830329ef4 review feedback 2018-10-31 15:41:39 -06:00
Jeff Wendling a7657ac409 tsdb: remove hll sketches
This keeps file compatability by just writing out zeros for the
sizes and offsets. Perhaps it's ok to just nuke everything and
remove the data.

It also keeps the hll package because it seems generally useful
even if it's not currently being used.
2018-10-31 15:41:39 -06:00
Edd Robinson 46a7b8155a
Merge pull request #1170 from zhulongcheng/rm-index
refactor(tsdb): remove tsdb.Index and tsdb.IndexSet
2018-10-30 11:10:54 +00:00
Jonathan A. Sternberg 67dc4d8cdd
fix: conform to logging style guide for initial log messages
These are the log messages that get printed immediately when starting
the application for the first time. This fixes the messages to conform
to the logging style guide.
2018-10-29 16:42:55 -05:00
zhulongcheng e739afb77d add tsi1.Index methods
These methods are copied from tsdb.IndexSet and modified slightly.
This fix is to remove tsdb.IndexSet to resolve #886.
2018-10-27 02:08:31 +08:00
Ben Johnson d856116b00
Add tsi1 measurement cardinality stats. 2018-10-17 08:38:41 -06:00
Jeff Wendling ce3e1fa90d Remove the fields index
This commit removes the remaining bits of the fields index. In doing
so, the buildCursor method on the engine would need to be updated.
It turns out, that code was statically dead, so delete it and anything
that depended on it. Additionally, delete anything as reported by
the unused tool in the tsdb package.
2018-10-15 10:22:07 -06:00
Jonathan A. Sternberg 8dee884038
refactor(toml): copy the toml utility package from influxdb to platform 2018-10-11 12:08:43 -05:00
Jonathan A. Sternberg e7e17d6972
refactor: move the logger package from influxdb to platform
We are moving the necessary code for 2.0 from the influxdb 1.X
repository to the platform 2.0 repository. The logger is an unnecessary
dependency on the old influxdb that is making life more complicated.
2018-10-10 15:49:07 -05:00
Chris Goller 90494a5309 chore(fmt): fix and simplify go formatting 2018-10-08 17:23:54 -05:00
Jeff Wendling 0360fc418c Remove empty index package for tsi1 2018-10-08 13:18:30 -06:00