Commit Graph

62 Commits (cf81a3e66e3a42a46f18f49f6cb78658529c45dd)

Author SHA1 Message Date
Stuart Carnie d135aecf02 Generate trace logs for a number of significant influx operations
* tsdb Store.Open traces all events related to opening files
    * op.name : tsdb.open
* retention policy shard deletions
    * op.name : retention.delete_check
* all TSM compaction strategies
    * op.name : tsm1.compact_group
* series file compactions
    * op.name : series_partition.compaction
* continuous query execution (if logging enabled)
    * op.name : continuous_querier.execute
* TSI log file compaction
    * op_name: index.tsi.compact_log_file
* TSI level compaction
    * op.name: index.tsi.compact_to_level
2018-02-21 15:08:49 -07:00
Jonathan A. Sternberg 2bbd96768d Update logging calls to take advantage of structured logging
Includes a style guide that details the basics of how to log.
2018-02-20 10:04:19 -06:00
Edd Robinson 6a66b5faf0 Cleanup services package 2018-01-21 10:52:37 -08:00
Jonathan A. Sternberg 0b7c56bcd8 Update the zap logger dependency
The previous sha was taken from a revision on a devel branch that I
thought would continue staying in the tree after it was merged. That
revision was rebased away and the API was changed for the logger.

This updates the usage of the logger and adds a simple package for
constructing the base logger.

The 1.0 version of zap changed the format of the default console logger
so this change moves over to this new logger instead of attempting to
retain backwards compatibility with the old format.
2017-11-10 16:27:16 -06:00
Jonathan A. Sternberg 83ecab6299 Prevent deadlock during collectd, graphite, opentsdb, and udp shutdown
All of these services start up goroutines and then wait for the
goroutines to finish. Each of them has a `tsdb.PointBatcher` that may
return a point during the shutdown sequence. During the shutdown
sequence, a lock was held. This lock may get accessed when attempting to
write the point that came back from the `tsdb.PointBatcher`. This caused
the read lock attempt to wait forever for the write lock to be unlocked
during `Close()`.

This modifies these methods so that the write lock is released while
waiting for goroutines to finish in these three services.
2017-10-19 15:57:05 -05:00
Byungjin Park f7d8ad50e2 Update README.md 2017-10-18 22:46:44 +09:00
Byungjin Park f8dd2423fe Fix mis-typing in README.md of UDP Service 2017-10-08 03:09:01 +09:00
Jonathan Heusser c38d0de99d Update udp port to be in valid range
`80891` is not a valid port, changed to `8189`.
2017-07-27 13:26:37 +01:00
Joe LeGasse 815f740f4c initial fga work
wip

wip

fix tests / build
2017-05-26 13:16:27 -07:00
Karl 08b3dbcbc7 Make services/udp golintable 2017-04-03 18:32:36 -04:00
Mark Rushakoff 535cf597f1 Report subset of config values in SHOW DIAGNOSTICS
This includes hand-selected config settings that are safe to expose and
not expected to include any kind of secrets.

Fixes #7821
2017-03-14 11:34:19 -07:00
Mark Rushakoff 601cbcd084 Merge branch '1.2' into mr-merge-12 2017-02-17 16:14:22 -08:00
Jonathan A. Sternberg 2fe48d6781 Rename zap import back to github.com/uber-go/zap
They rebased a revision we were previously relying upon that allowed us
to use the vanity name so we are reverting back to an older version with
the old import path.
2017-02-17 17:17:22 -06:00
Edd Robinson 9d30ee0a6b Fix subtle bugs and remove dead code from services 2017-01-17 09:47:34 -08:00
Mark Rushakoff 218fc3890d Update godoc for services
The admin service was deliberately skipped due to it being deprecated.
2016-12-30 18:03:01 -08:00
Jonathan A. Sternberg ec57108520 Use proper uber-go/zap import path
It looks like the real import path to the project is go.uber.org/zap
instead of github.com/uber-go/zap since the example in the project
references that path.
2016-12-15 08:54:14 -06:00
Jonathan A. Sternberg 21502a39e8 Switch logging to use structured logging everywhere
The logging library has been switched to use uber-go/zap. While the
logging has been changed to use structured logging, this commit does not
change any of the logging statements to take advantage of the new
structured log or new log levels. Those changes will come in future
commits.
2016-12-14 10:45:15 -06:00
Edd Robinson 73c267abc1 udp service should retry database creation 2016-10-18 16:50:07 +01:00
Edd Robinson ad2d33f859 Ensure input services can be safely opened and closed 2016-10-18 10:35:38 +01:00
Edd Robinson 6cafdbc604 Ensure we don't mutate provided statistics tags 2016-08-24 11:40:13 +01:00
Edd Robinson 90ff713f21 Fix base64 encoding issue in stats
Fixes #7177.
2016-08-22 15:21:31 +01:00
Ben Johnson 8aa224b22d
reduce memory allocations in index
This commit changes the index to point to index data in the shards
instead of keeping it in-memory on the heap.
2016-08-16 14:09:00 -06:00
Cameron Sparr d688676266 Update UDP buffer setting doc to 25MB 2016-07-28 12:02:56 +01:00
Cameron Sparr 6afdcf9572 Fix incrementing pointsRx twice 2016-07-28 11:06:50 +01:00
Jonathan A. Sternberg 837a9804cf Refactoring the monitor service to avoid expvar
Truncate the time interval output of the monitor service to be on even
time intervals rather than on every minute based on the start time. This
normalizes the output from the monitor service.
2016-07-07 11:13:58 -05:00
Stephen Gutekanst 9dc09c5257 Make logging output location more programmatically configurable (#6213)
This has various benefits:

- Users embedding InfluxDB within other Go programs can specify a different logger / prefix easily.
- More consistent with code used elsewhere in InfluxDB (e.g. services, other `run.Server.*` fields, etc).
- This is also more efficient, because it means `executeQuery` no longer allocates a single `*log.Logger` each time it is called.
2016-04-20 21:07:08 +01:00
Cameron Sparr 23e2c40923 Reduce UDP service per-packet allocation size
This will reduce memory pressure and number of GC cycles, my results
sending 100,000 UDP points were:

- udp-payload-size=0: 242 GC cycles
- udp-payload-size=1500: 142 GC cycles
- udp-payload-size=0 (with change): 114 GC cycles
- udp-payload-size=1500 (with change): 112 GC cycles
2016-04-08 13:29:47 -06:00
Cory LaNou d024ca2552 modify WritePoints function signature for p products 2016-03-14 16:55:54 +00:00
Cory LaNou 27cfaa4b7a in memory meta, single node configs, etc. 2016-03-14 16:55:54 +00:00
Nicholas Katsaros a727266744 Add graphite and udp services to the default config generator
Fix #4752
2016-02-16 10:12:48 -05:00
Ben Johnson d9a6a7340f add canonical paths 2016-02-10 11:30:52 -07:00
Ben Johnson 5a0d1ab7c1 rename influxdb/influxdb to influxdata/influxdb
This commit changes all the import and URL references from:

    github.com/influxdb/influxdb

to:

    github.com/influxdata/influxdb
2016-02-10 10:26:18 -07:00
Tony Pitale 6dd407c819 use go fmt 2016-02-07 10:59:31 -06:00
Tony Pitale 1e4eae4343 Configurable precision on UDP services
* pass configured precision string to point parsing
* add Precision configuration to UDP config
* default configured precision to match what it appears to be now (from ParsePoints)
2016-02-06 13:56:53 -06:00
Paul Dix f385945058 Update Server to work with new metaservice/client 2016-01-21 15:28:33 -05:00
Cory LaNou 8d878fff91 buildable meta -> services/meta 2016-01-21 15:28:32 -05:00
Sebastian Borza d885f4156b removing golang reference 2015-12-22 12:54:43 -05:00
Sebastian Borza 3bf5d9c969 moving UDP payload size to config option, improve throughput performance
moving UDP payload size to optional config choice to imporove throughput performance
2015-12-22 09:14:50 -05:00
Philip O'Toole a5cb913035 Drop UDP point on bad parse and keep going
Fixes issue #4992.
2015-12-07 19:47:59 -08:00
orthogonous 434d06008e Added some comments to the udp service so golint passes. Ref #4098 2015-11-14 08:46:26 +10:00
Philip O'Toole bc1091ec31 Revert "Merge pull request #4684 from nkatsaros/default-config"
This reverts commit 87441584f3, reversing
changes made to a14829bc1d.
2015-11-11 09:29:06 -08:00
Cameron Sparr 9625953d3e Add UDP OS buffer size recommendations 2015-11-10 11:00:11 -07:00
Cameron Sparr 727b9f6d8b Documentation updates for UDP, graphite, collectd listeners 2015-11-10 10:35:21 -07:00
Cameron Sparr 78e6979be5 Make parser/writer internal func names consistent 2015-11-06 16:43:35 -07:00
Cameron Sparr 5c28048132 UDP service listener performance enhancements 2015-11-06 15:53:47 -07:00
Nicholas Katsaros 4f8b0aca9a Add graphite and udp services to the default config generator 2015-11-05 16:42:21 -05:00
Cameron Sparr e2db577e0f Implement a UDP client
closes #4647
2015-11-04 14:18:14 -07:00
Philip O'Toole de7919240f Migrate internal stats to consistent names
Go style -- and existing runtime stats -- do not use underscores, but
instead use camel case. This change makes the internal stats adhere to
that convention.
2015-10-28 21:07:45 -07:00
Philip O'Toole 485c446e98 Correct typos in UDP README
[ci skip]
2015-10-15 07:48:34 -07:00
Philip O'Toole 3907656cc2 Add README for UDP service
Fixes issue #4041.
2015-10-14 08:30:10 -07:00