Commit Graph

82 Commits (b7a836197710a8ae6732d6fb8d81561711a664b8)

Author SHA1 Message Date
Stuart Carnie 8c296dd20a
fix(storage): Add unit tests to verify nil cursor
When matching series have no data, the NewGroupResultSet cursor should
return nil. Added tests to verify these conditions.
2018-12-12 14:27:11 -07:00
Stuart Carnie f2891c3f59
fix(storage): Fix panic when read request results in empty an group
Moves the check to determine if a series has data for the current time
range into the groupBySort function, which is consistent with the
groupNoneSort function.
2018-12-12 13:22:02 -07:00
Stuart Carnie 8f8311a3ae
chore(gen): Add ingen generator data structures to platform for reuse 2018-12-11 17:45:57 -07:00
Edd Robinson c4b42c72be Add option to disable TSI metrics 2018-12-10 15:02:26 +00:00
Edd Robinson b015757c06 Ensure all tsi1 metrics support multiple instances 2018-12-07 14:32:34 +00:00
Edd Robinson bff655786f Ensure tsdb metrics properly registered 2018-12-07 14:32:34 +00:00
Edd Robinson aa936df138 Ensure all tsm1 metrics support multiple instances 2018-12-07 14:32:34 +00:00
Edd Robinson f9a2f7a017 go fmt 2018-12-07 14:32:34 +00:00
Edd Robinson 7960ccc320 Add TSI index metrics 2018-12-07 14:32:34 +00:00
Edd Robinson d1fe2bc188 Add series file metrics 2018-12-07 14:32:34 +00:00
Edd Robinson 4e67e37ca6 Fix bug with engineID 2018-12-07 14:32:34 +00:00
Edd Robinson 8ca637bd80 Refactor default labels and retention metrics 2018-12-07 14:32:34 +00:00
Edd Robinson 44e5fbae0a Convert WAL stats 2018-12-07 14:32:34 +00:00
Edd Robinson f56bc0853f Convert TSM compaction stats to Prom metrics
This commits converts all the 1.x TSM compaction statistics, which
previously were written to an _internal db, to Prometheus metrics.
2018-12-07 14:32:34 +00:00
Jacob Marble b3e5e9fc31 feat(storage): return cost stats with read response via gRPC trailer 2018-12-05 15:32:41 -08:00
Edd Robinson 9403c1ec8e Ensure error strings not capitalised ST1005 2018-11-30 10:54:24 +00:00
Ben Johnson 0084d4d824
Remove influxdb dependency. 2018-11-29 11:44:22 -07:00
Ben Johnson 868fdbbe03
megacheck 2018-11-28 15:52:11 -07:00
Ben Johnson 1862b4421d
Integrate scanned values statistics tracking. 2018-11-28 15:32:06 -07:00
Mark Rushakoff a06d4dea16 refactor(query): explicitly shut down query controller
The flux query controller was updated to include a Shutdown method a
while ago. Explicitly handle query controller creation and shutdown
where applicable.

In influxd, this ensures that outstanding queries are handled before the
process dies. In tests, this ensures that query controller goroutines
aren't leaked, which drastically simplifies reading full stack traces.

This change also registers query controller metrics with the prometheus
registry in influxd.
2018-11-26 16:13:19 -08:00
Christopher M. Wolff a3ad02288b
Remove use of verbose from control.Config (#1573) 2018-11-26 16:02:41 -07:00
jlapacik 2b52224e84 update flux dependency - add missing Statistics() methods 2018-11-26 13:19:14 -08:00
Stuart Carnie f3d15be61b
chore: Add key order sanity check to StreamReaders 2018-11-21 12:01:36 -07:00
Stuart Carnie 8920dab57a
feedback: Changes per PR feedback 2018-11-21 12:01:36 -07:00
Stuart Carnie 831e2ab943
feat(reads): Implement merge operation for []reads.GroupResultSet
Specifically the currently supported GroupNone and GroupBy operations
2018-11-21 12:01:36 -07:00
Stuart Carnie 268a52148a
feat(reads): Implement merge operation for []reads.ResultSet 2018-11-21 12:01:36 -07:00
Stuart Carnie 7fb015cc7f
chore: Add GroupBy benchmark 2018-11-21 12:01:36 -07:00
Stuart Carnie 26408dc1fa
chore(reads): Document ResultSet and GroupResultSet APIs; add func Err
* add option to Group sort to specify precedence of `nil`
* improve unit tests for GroupResultSet implementation
2018-11-21 12:01:35 -07:00
Stuart Carnie 0237059d05
feat(reads): Implement ResultSet and GroupResultSet for protobuf streams 2018-11-21 12:01:35 -07:00
zhulongcheng 36a2818caa fix(storage): missing `continue` 2018-11-13 00:08:43 +08:00
Jeff Wendling 39f4908946 fix(storage): allow disabling the WAL
We were passing a non-nil tsm1.Log containing a nil *tsm1.WAL which
would cause a panic when it was attempted to be used. Instead, always
pass a non-nil WAL.

We change the storage engine code to not pass in a nil WAL, and
additionally add a defensive check to change any nil WALs into a
NopWAL.
2018-11-09 10:45:24 -07:00
Jeff Wendling 4b504b84df respond to review feedback
- Add some documentation.
- Move compaction planner to an option instead of config.

The latter fits with the general theme of having config be things
that can be specified in a toml, and everything else being an
option.
2018-11-08 11:39:36 -07:00
Jeff Wendling a1b5b322bb some more refactoring
- add helpers to get directories out
- change FileStoreObserver to be an option rather than config.
2018-11-08 11:39:36 -07:00
Jeff Wendling 22e23d6e31 final touches
- move default directories to the storage package
- make the directory layout match before
- clean up some dead missed functions
2018-11-08 11:39:36 -07:00
Jeff Wendling 2cbc2ee896 refactor wal out, paths, and options 2018-11-08 11:39:36 -07:00
Jeff Wendling c2a4b1ea7a clean up compat documentation some 2018-11-08 11:39:36 -07:00
Jessica Obermark 932b0bf01a compat: Package to convert old to new config 2018-11-08 11:39:36 -07:00
Jeff Wendling 0d411023f2 config: clean up
- Breaks the weird cycle that existed with the EngineOptions
- Removes a bunch of useless parameters
- Moves around a bunch of defaults
2018-11-08 11:39:36 -07:00
Christopher M. Wolff e95fb9cc9a
Fix error where _start or _stop where redundantly added to group key (#1308) 2018-11-05 14:06:08 -08:00
Stuart Carnie 56510ba0dc
Merge pull request #1262 from influxdata/sgc-fix-race
Fix race condition as `Close` was called by multiple goroutines
2018-11-01 14:22:32 -07:00
Mark Rushakoff 985c260af7 chore(storage,tsdb): fix megacheck errors 2018-11-01 12:54:46 -07:00
Adam 8a1fecd727
to() for influxdb (#1194)
* update to latest flux master incl. values interface refactor, type inference, planner.  

* add measurementColumn parameter, update function params spec
2018-11-01 14:44:56 -04:00
Stuart Carnie b7d9505ac1
fix(reads): Fix race condition when done channel was triggered
tl;dr
Previously, `Close` was being called concurrently by multiple
goroutines, resulting in a race condition. This commit resolves those
issues.

Background

The `Close` method was performing multiple duties, closing resources
and triggering that the table reading by the `Do` method was done.

Additionally, state to track whether more records existed and if the
table was empty, was ported from the more complicated gRPC
implementation. This logic has been simplified.

This new behavior:

* `table#Do` is responsible for triggering it is done, by closing the
  done channel
* The creator of the `table` is responsible for releasing the resources
  by calling the `table#Close` method
* The `table#Do` reading can be cancelled by calling the `Cancel`
  function, which is safe for concurrent use.
* the Do and Close methods are protected by a mutex to protect storage
  resources, such as cursors.
2018-11-01 11:35:36 -07:00
Nathaniel Cook 2a1d76eb2d chore: Update flux dependency for the type-inference changes 2018-10-31 17:11:11 -06:00
Stuart Carnie 869068de69
fix(reads): Ensure storage resources are cleaned
Move variable declarations outside the for loops and use nil
to test for resource ownership. defer function tests for nil and
will close resource.
2018-10-31 11:04:47 -07:00
Chris Goller d8548d41e1 chore(fmt): update formating with make fmt 2018-10-30 07:40:28 -05:00
Edd Robinson 46a7b8155a
Merge pull request #1170 from zhulongcheng/rm-index
refactor(tsdb): remove tsdb.Index and tsdb.IndexSet
2018-10-30 11:10:54 +00:00
Jonathan A. Sternberg 67dc4d8cdd
fix: conform to logging style guide for initial log messages
These are the log messages that get printed immediately when starting
the application for the first time. This fixes the messages to conform
to the logging style guide.
2018-10-29 16:42:55 -05:00
zhulongcheng 9d29874e20 move SeriesFileDirectory constant to defaults package 2018-10-27 02:08:31 +08:00
zhulongcheng c89c79dc02 replace tsdb.Index interface with tsi1.Index instance
This fix is to remove tsdb.Index interface to resolve #886.
2018-10-27 02:08:31 +08:00