Commit Graph

174 Commits (adbfee28b321990d024fe1242b957aee3346666d)

Author SHA1 Message Date
Stuart Carnie d135aecf02 Generate trace logs for a number of significant influx operations
* tsdb Store.Open traces all events related to opening files
    * op.name : tsdb.open
* retention policy shard deletions
    * op.name : retention.delete_check
* all TSM compaction strategies
    * op.name : tsm1.compact_group
* series file compactions
    * op.name : series_partition.compaction
* continuous query execution (if logging enabled)
    * op.name : continuous_querier.execute
* TSI log file compaction
    * op_name: index.tsi.compact_log_file
* TSI level compaction
    * op.name: index.tsi.compact_to_level
2018-02-21 15:08:49 -07:00
Jonathan A. Sternberg 2bbd96768d Update logging calls to take advantage of structured logging
Includes a style guide that details the basics of how to log.
2018-02-20 10:04:19 -06:00
Edd Robinson 6a66b5faf0 Cleanup services package 2018-01-21 10:52:37 -08:00
Jonathan A. Sternberg 0b7c56bcd8 Update the zap logger dependency
The previous sha was taken from a revision on a devel branch that I
thought would continue staying in the tree after it was merged. That
revision was rebased away and the API was changed for the logger.

This updates the usage of the logger and adds a simple package for
constructing the base logger.

The 1.0 version of zap changed the format of the default console logger
so this change moves over to this new logger instead of attempting to
retain backwards compatibility with the old format.
2017-11-10 16:27:16 -06:00
Jonathan A. Sternberg 83ecab6299 Prevent deadlock during collectd, graphite, opentsdb, and udp shutdown
All of these services start up goroutines and then wait for the
goroutines to finish. Each of them has a `tsdb.PointBatcher` that may
return a point during the shutdown sequence. During the shutdown
sequence, a lock was held. This lock may get accessed when attempting to
write the point that came back from the `tsdb.PointBatcher`. This caused
the read lock attempt to wait forever for the write lock to be unlocked
during `Close()`.

This modifies these methods so that the write lock is released while
waiting for goroutines to finish in these three services.
2017-10-19 15:57:05 -05:00
Joe LeGasse 815f740f4c initial fga work
wip

wip

fix tests / build
2017-05-26 13:16:27 -07:00
Jason Wilder 2cac46ebbc Convert usage of strings to []byte
Measurement name and field were converted between []byte and string
repetively causing lots of garbage.  This switches the code to use
[]byte in the write path.
2017-05-12 14:05:19 -06:00
Mark Rushakoff 535cf597f1 Report subset of config values in SHOW DIAGNOSTICS
This includes hand-selected config settings that are safe to expose and
not expected to include any kind of secrets.

Fixes #7821
2017-03-14 11:34:19 -07:00
Mark Rushakoff 601cbcd084 Merge branch '1.2' into mr-merge-12 2017-02-17 16:14:22 -08:00
Jonathan A. Sternberg 2fe48d6781 Rename zap import back to github.com/uber-go/zap
They rebased a revision we were previously relying upon that allowed us
to use the vanity name so we are reverting back to an older version with
the old import path.
2017-02-17 17:17:22 -06:00
Edd Robinson 9d30ee0a6b Fix subtle bugs and remove dead code from services 2017-01-17 09:47:34 -08:00
Mark Rushakoff 6a94d200c8 Merge remote-tracking branch 'influx/master' into mr-godoc 2017-01-04 13:27:36 -08:00
Cory LaNou 3c518f8927
panicing is bad -> error returns are good 2017-01-03 14:28:29 -06:00
Mark Rushakoff 218fc3890d Update godoc for services
The admin service was deliberately skipped due to it being deprecated.
2016-12-30 18:03:01 -08:00
Gustav Westling a8343dee99
Fix broken test. Change Fatalf to Fatal. 2016-12-29 20:41:00 +01:00
Gustav Westling 56d98325da
Removed ineffective assignments, and added checks for errors that previsouly was not checked 2016-12-29 20:26:15 +01:00
Jonathan A. Sternberg ec57108520 Use proper uber-go/zap import path
It looks like the real import path to the project is go.uber.org/zap
instead of github.com/uber-go/zap since the example in the project
references that path.
2016-12-15 08:54:14 -06:00
Mark Rushakoff eceecf4cfa Merge pull request #7721 from influxdata/mr-fix-flaky-port-tests
Bind to random port in service open/close tests
2016-12-14 10:03:26 -08:00
Jonathan A. Sternberg 21502a39e8 Switch logging to use structured logging everywhere
The logging library has been switched to use uber-go/zap. While the
logging has been changed to use structured logging, this commit does not
change any of the logging statements to take advantage of the new
structured log or new log levels. Those changes will come in future
commits.
2016-12-14 10:45:15 -06:00
Mark Rushakoff ede3144300 Bind to random port in service open/close tests
These tests have been flaky on Circle in the past due to those specific
ports being in use. Binding to port 0 will pick a random, available
port.
2016-12-14 08:29:29 -08:00
Cory LaNou cd272ce6c3 fix retention policy creation inconsistencies 2016-11-03 09:09:43 -05:00
Hung Nguyen Viet 2cb6c64787 Sections headers should be all capitals 2016-10-20 18:57:13 +07:00
Hung Nguyen Viet 1f26984181 Graphite 2016-10-20 18:57:13 +07:00
Hung Nguyen Viet cc7b61d385 fix grammar error 2016-10-20 18:57:13 +07:00
Hung Nguyen Viet 179f5b02cc TSM is the only engine now 2016-10-20 18:57:05 +07:00
Hung Nguyen Viet 1f31e8b707 clarify 2016-10-20 18:48:56 +07:00
Hung Nguyen Viet 4751ca1f43 use markdown 2016-10-20 18:47:54 +07:00
Hung Nguyen Viet dcb494ded5 use plural 2016-10-20 18:47:54 +07:00
Edd Robinson 168c91cc67 graphite service should retry database creation 2016-10-18 16:50:07 +01:00
Edd Robinson ad2d33f859 Ensure input services can be safely opened and closed 2016-10-18 10:35:38 +01:00
Edd Robinson 6cafdbc604 Ensure we don't mutate provided statistics tags 2016-08-24 11:40:13 +01:00
Edd Robinson 90ff713f21 Fix base64 encoding issue in stats
Fixes #7177.
2016-08-22 15:21:31 +01:00
Ben Johnson 8aa224b22d
reduce memory allocations in index
This commit changes the index to point to index data in the shards
instead of keeping it in-memory on the heap.
2016-08-16 14:09:00 -06:00
Jonathan A. Sternberg 530b00bd76 Use defaults from `meta` package for `CREATE DATABASE`
Instead of having the parser set the defaults, the command will set the
defaults so that the constants for that are actually used. This way we
can also identify which things the user provided and which ones we are
filling with default values.

This allows the meta client to be able to make smarter decisions when
determining if the user requested a conflict or if the requested
capabilities match with what is currently available. If you just say
`CREATE DATABASE WITH NAME myrp`, the user doesn't really care what the
duration of the retention policy is and just wants to use the default.
Now, we can use that information to determine if an existing retention
policy would conflict with what the user requested rather than returning
an error if a default value ever gets changed since the meta client
command can communicate intent more easily.
2016-08-09 12:00:06 -05:00
Jonathan A. Sternberg 837a9804cf Refactoring the monitor service to avoid expvar
Truncate the time interval output of the monitor service to be on even
time intervals rather than on every minute based on the start time. This
normalizes the output from the monitor service.
2016-07-07 11:13:58 -05:00
Jonathan A. Sternberg 497db2a6d3 Removing dead code from every package except influxql
The tsdb package had a substantial amount of dead code related to the
old query engine still in there. It is no longer used, so it was removed
since it was left unmaintained. There is likely still more code that is
the same, but wasn't found as part of this code cleanup.

influxql has dead code show up because of the code generation so it is
not included in this pruning.
2016-06-20 22:41:07 -05:00
Jack Zampolin f1db2f2ca5 Add port to all graphite log output to help with debugging multiple endpoints 2016-06-13 11:59:56 -07:00
Jonathan A. Sternberg 4f37bc5a40 Support specifying a retention policy for the graphite service
The graphite service will attempt to create the retention policy and use
it. If the retention policy doesn't exist, it will be created with the
default options.

Fixes #5655.
2016-06-08 17:01:44 -05:00
Joe Farrell bb8b08e2ca Fixed typo in docstring 2016-06-03 18:01:04 +01:00
Stephen Gutekanst 9dc09c5257 Make logging output location more programmatically configurable (#6213)
This has various benefits:

- Users embedding InfluxDB within other Go programs can specify a different logger / prefix easily.
- More consistent with code used elsewhere in InfluxDB (e.g. services, other `run.Server.*` fields, etc).
- This is also more efficient, because it means `executeQuery` no longer allocates a single `*log.Logger` each time it is called.
2016-04-20 21:07:08 +01:00
Cory LaNou d024ca2552 modify WritePoints function signature for p products 2016-03-14 16:55:54 +00:00
Cory LaNou 27cfaa4b7a in memory meta, single node configs, etc. 2016-03-14 16:55:54 +00:00
bastard a5cbc43afd Merge remote-tracking branch 'upstream/master' into conjoined-field-names-for-graphite-templates 2016-02-18 00:10:11 +00:00
Edd Robinson f315444f86 Merge pull request #5255 from titilambert/github4752
Workaround for #4752
2016-02-17 11:00:58 +00:00
Nicholas Katsaros a727266744 Add graphite and udp services to the default config generator
Fix #4752
2016-02-16 10:12:48 -05:00
Chris H (CruftMaster) 53cccd02be Merge remote-tracking branch 'upstream/master' into conjoined-field-names-for-graphite-templates 2016-02-14 22:45:15 +00:00
Chris H (CruftMaster) 7de2c7f233 Switching to mixedCaps 2016-02-14 22:36:02 +00:00
Mark Rushakoff e1af9fe596 Deregister diagnostics client in service.Close
The hinted handoff and graphite services called out
DeregisterDiagnosticsClient in the inlined Monitor interface but never
called it.
2016-02-13 15:04:05 -08:00
Mark Rushakoff 5a61bdad70 Move monitor.Diagnostics to its own package
I was trying to create a Diagnostics Client in the tsdb package, but
IIRC importing `monitor` caused an import cycle of:
tsdb -> monitor -> cluster -> tsdb.

Moving Diagnostics to its own package will allow further use of
diagnostics.Client without running into import cycles.
2016-02-13 14:56:15 -08:00
Ben Johnson d9a6a7340f add canonical paths 2016-02-10 11:30:52 -07:00