Commit Graph

46 Commits (a3e66755ca017ea54f2cce9d4a3cbd8efe1dd770)

Author SHA1 Message Date
Jeff Wendling 8dbd98ccbe refactor: change the way the engine opens and closes and reload the cache
Open and Close now proceed as best as they are able to in the presence
of errors and clean up appropriately.
2019-02-04 10:32:52 -07:00
Jeff Wendling d2ddd48eea refactor: hook up metrics and wal to storage engine
It turns out that LastModified and DiskSize are unused, and so it
was easy to change to not care about the WAL.

This hooks up metrics and starts the WAL again.
2019-02-04 10:32:52 -07:00
Jeff Wendling 95de3d52b2 refactor: use concrete WAL in tsm1
At the cost of some nil checks, we don't have to have an interface, defend against
subtle bugs with nils in non-nil interfaces, an empty implementation, etc.

Also, the tsm1 engine is losing the WAL anyway.
2019-02-04 10:32:52 -07:00
Jeff Wendling c9bb55b889 refactor: move the tsm1/wal into the storage/wal package
Because the WAL relies on the tsm1.Value type, we move that into its own
tsm1/value package and set up some aliases forwarding them into tsm1. This
also required adding some methods and changing consumers to avoid the
unexported fields. I imagine this step will be useful one day when we make
the write path more efficient with respect to consuming points.

This commit additionally fixes some issues with generation. The iterator.tmpldata
and generation for array_cursor_* were removed accidentally when removing
iterators, making those generated files stale. Restore that and regenerate.

No change in functionality.
2019-02-04 10:32:52 -07:00
Edd Robinson b025d9afa9 Improve efficiency of TSI index series drop
This commit improves the performance of a mass delete on the TSI index
by deleting at the measurement level instead of deleting each series
individually.
2019-01-14 12:46:55 +00:00
Mark Rushakoff d73d73c0d4 chore: rename imports from platform to influxdb
I did this with a dumb editor macro, so some comments changed too.

Also rename root package from platform to influxdb.

In interest of minimizing risk, anyone importing the root package has
now aliased it to "platform" so that no changes beyond imports were
necessary in those files.

Lastly, replace the old platform module to local path /dev/null so that
nobody can accidentally reintroduce a platform dependency while
migrating platform code to influxdb.
2019-01-09 20:51:47 -08:00
Jeff Wendling 703c3c15ca Hook up DeleteBucket to the tsm1 engine 2019-01-09 15:24:26 -07:00
Edd Robinson 42ff769f1c Wire up storage.Engine to HTTP BucketService 2019-01-09 15:09:56 +00:00
Edd Robinson 5a12a3a72e Export access to bucket drop in engine 2019-01-09 13:35:07 +00:00
Edd Robinson c4b42c72be Add option to disable TSI metrics 2018-12-10 15:02:26 +00:00
Edd Robinson b015757c06 Ensure all tsi1 metrics support multiple instances 2018-12-07 14:32:34 +00:00
Edd Robinson bff655786f Ensure tsdb metrics properly registered 2018-12-07 14:32:34 +00:00
Edd Robinson aa936df138 Ensure all tsm1 metrics support multiple instances 2018-12-07 14:32:34 +00:00
Edd Robinson f9a2f7a017 go fmt 2018-12-07 14:32:34 +00:00
Edd Robinson 7960ccc320 Add TSI index metrics 2018-12-07 14:32:34 +00:00
Edd Robinson d1fe2bc188 Add series file metrics 2018-12-07 14:32:34 +00:00
Edd Robinson 4e67e37ca6 Fix bug with engineID 2018-12-07 14:32:34 +00:00
Edd Robinson 8ca637bd80 Refactor default labels and retention metrics 2018-12-07 14:32:34 +00:00
Edd Robinson 44e5fbae0a Convert WAL stats 2018-12-07 14:32:34 +00:00
Edd Robinson f56bc0853f Convert TSM compaction stats to Prom metrics
This commits converts all the 1.x TSM compaction statistics, which
previously were written to an _internal db, to Prometheus metrics.
2018-12-07 14:32:34 +00:00
zhulongcheng 36a2818caa fix(storage): missing `continue` 2018-11-13 00:08:43 +08:00
Jeff Wendling 39f4908946 fix(storage): allow disabling the WAL
We were passing a non-nil tsm1.Log containing a nil *tsm1.WAL which
would cause a panic when it was attempted to be used. Instead, always
pass a non-nil WAL.

We change the storage engine code to not pass in a nil WAL, and
additionally add a defensive check to change any nil WALs into a
NopWAL.
2018-11-09 10:45:24 -07:00
Jeff Wendling 4b504b84df respond to review feedback
- Add some documentation.
- Move compaction planner to an option instead of config.

The latter fits with the general theme of having config be things
that can be specified in a toml, and everything else being an
option.
2018-11-08 11:39:36 -07:00
Jeff Wendling a1b5b322bb some more refactoring
- add helpers to get directories out
- change FileStoreObserver to be an option rather than config.
2018-11-08 11:39:36 -07:00
Jeff Wendling 22e23d6e31 final touches
- move default directories to the storage package
- make the directory layout match before
- clean up some dead missed functions
2018-11-08 11:39:36 -07:00
Jeff Wendling 2cbc2ee896 refactor wal out, paths, and options 2018-11-08 11:39:36 -07:00
Jeff Wendling 0d411023f2 config: clean up
- Breaks the weird cycle that existed with the EngineOptions
- Removes a bunch of useless parameters
- Moves around a bunch of defaults
2018-11-08 11:39:36 -07:00
Edd Robinson 46a7b8155a
Merge pull request #1170 from zhulongcheng/rm-index
refactor(tsdb): remove tsdb.Index and tsdb.IndexSet
2018-10-30 11:10:54 +00:00
Jonathan A. Sternberg 67dc4d8cdd
fix: conform to logging style guide for initial log messages
These are the log messages that get printed immediately when starting
the application for the first time. This fixes the messages to conform
to the logging style guide.
2018-10-29 16:42:55 -05:00
zhulongcheng 9d29874e20 move SeriesFileDirectory constant to defaults package 2018-10-27 02:08:31 +08:00
zhulongcheng c89c79dc02 replace tsdb.Index interface with tsi1.Index instance
This fix is to remove tsdb.Index interface to resolve #886.
2018-10-27 02:08:31 +08:00
zhulongcheng 28fecc1f6f replace tsdb.IndexSet with tsi1.Index
This fix is to remove tsdb.IndexSet to resolve #886.
2018-10-27 02:08:31 +08:00
Jacob Marble f33e13a036 storage: Add MeasurementCardinalityStats and MeasurementStats to Engine 2018-10-18 11:10:21 -07:00
Jeff Wendling 30418ca2f5 fix(storage): WithEngineID and WithNodeID no longer panic
Both of these options would dereference a nil pointer when attempting
to apply. Instead, set the field to be the address of an integer
containing the right value.
2018-10-15 12:44:46 -06:00
Edd Robinson f815dcfeb7 Schedule retention enforcer from within engine 2018-10-10 15:45:26 +01:00
Edd Robinson d5ed17adb2 Rename retention service 2018-10-10 15:40:08 +01:00
Edd Robinson 1f2f03d93b Open/close Retention properly 2018-10-10 15:40:08 +01:00
Edd Robinson 031bb7f502 Add prom metrics 2018-10-10 15:40:08 +01:00
Edd Robinson 4597fdea48 Wire Retention service into engine 2018-10-10 15:40:08 +01:00
Edd Robinson 54d1ac09c1 Integrate Retention metrics 2018-10-10 15:40:08 +01:00
Edd Robinson 116456534a Initial Retention Service 2018-10-10 15:39:27 +01:00
Jeff Wendling 0360fc418c Remove empty index package for tsi1 2018-10-08 13:18:30 -06:00
Edd Robinson 6b23165146 Add Engine tests 2018-10-05 15:04:09 +01:00
Edd Robinson 81e0fbabeb Integrate WAL into engine 2018-10-05 12:44:27 +01:00
Edd Robinson cfb3df203a Working storage engine 2018-10-04 10:24:43 +01:00
Edd Robinson 981b2cdbea Skeleton storage engine 2018-10-04 10:24:43 +01:00