Commit Graph

173 Commits (7f300dc248d767ee1a96642480dfa1daa28f18ba)

Author SHA1 Message Date
Sam Arnold db38959b7a
fix: fix help test for influx_inspect (#21052)
Closes #21007
2021-03-23 19:15:26 -04:00
davidby-influx 642726e898
feat: influx_inspect export to standard out (#20977)
Add a special value to the -out flag, a hyphen, to write to stdout.
While writing to stdout, send status messages to stderr instead of
stdout (the current behavior).

Closes https://github.com/influxdata/influxdb/issues/20974
2021-03-17 10:01:40 -07:00
Sam Arnold 21823db00b
feat: series creation ingress metrics (#20700)
After turning this on and testing locally, note the 'seriesCreated' metric

"localStore": {"name":"localStore","tags":null,"values":{"pointsWritten":2987,"seriesCreated":58,"valuesWritten":23754}},
"ingress": {"name":"ingress","tags":{"db":"_internal","login":"_systemuser_monitor","measurement":"cq","rp":"monitor"},"values":{"pointsWritten":2,"seriesCreated":1,"valuesWritten":4}},
"ingress:1": {"name":"ingress","tags":{"db":"_internal","login":"_systemuser_monitor","measurement":"database","rp":"monitor"},"values":{"pointsWritten":2,"seriesCreated":2,"valuesWritten":4}},
"ingress:2": {"name":"ingress","tags":{"db":"_internal","login":"_systemuser_monitor","measurement":"httpd","rp":"monitor"},"values":{"pointsWritten":2,"seriesCreated":1,"valuesWritten":46}},
"ingress:3": {"name":"ingress","tags":{"db":"_internal","login":"_systemuser_monitor","measurement":"ingress","rp":"monitor"},"values":{"pointsWritten":14,"seriesCreated":14,"valuesWritten":42}},
"ingress:4": {"name":"ingress","tags":{"db":"_internal","login":"_systemuser_monitor","measurement":"localStore","rp":"monitor"},"values":{"pointsWritten":2,"seriesCreated":1,"valuesWritten":6}},
"ingress:5": {"name":"ingress","tags":{"db":"_internal","login":"_systemuser_monitor","measurement":"queryExecutor","rp":"monitor"},"values":{"pointsWritten":2,"seriesCreated":1,"valuesWritten":10}},
"ingress:6": {"name":"ingress","tags":{"db":"_internal","login":"_systemuser_monitor","measurement":"runtime","rp":"monitor"},"values":{"pointsWritten":2,"seriesCreated":1,"valuesWritten":30}},
"ingress:7": {"name":"ingress","tags":{"db":"_internal","login":"_systemuser_monitor","measurement":"shard","rp":"monitor"},"values":{"pointsWritten":2,"seriesCreated":2,"valuesWritten":22}},
"ingress:8": {"name":"ingress","tags":{"db":"_internal","login":"_systemuser_monitor","measurement":"subscriber","rp":"monitor"},"values":{"pointsWritten":2,"seriesCreated":1,"valuesWritten":6}},
"ingress:9": {"name":"ingress","tags":{"db":"_internal","login":"_systemuser_monitor","measurement":"tsm1_cache","rp":"monitor"},"values":{"pointsWritten":2,"seriesCreated":2,"valuesWritten":18}},
"ingress:10": {"name":"ingress","tags":{"db":"_internal","login":"_systemuser_monitor","measurement":"tsm1_engine","rp":"monitor"},"values":{"pointsWritten":2,"seriesCreated":2,"valuesWritten":58}},
"ingress:11": {"name":"ingress","tags":{"db":"_internal","login":"_systemuser_monitor","measurement":"tsm1_filestore","rp":"monitor"},"values":{"pointsWritten":2,"seriesCreated":2,"valuesWritten":4}},
"ingress:12": {"name":"ingress","tags":{"db":"_internal","login":"_systemuser_monitor","measurement":"tsm1_wal","rp":"monitor"},"values":{"pointsWritten":2,"seriesCreated":2,"valuesWritten":8}},
"ingress:13": {"name":"ingress","tags":{"db":"_internal","login":"_systemuser_monitor","measurement":"write","rp":"monitor"},"values":{"pointsWritten":2,"seriesCreated":1,"valuesWritten":18}},
"ingress:14": {"name":"ingress","tags":{"db":"telegraf","login":"_systemuser_unknown","measurement":"cpu","rp":"autogen"},"values":{"pointsWritten":1342,"seriesCreated":13,"valuesWritten":13420}},
"ingress:15": {"name":"ingress","tags":{"db":"telegraf","login":"_systemuser_unknown","measurement":"disk","rp":"autogen"},"values":{"pointsWritten":642,"seriesCreated":6,"valuesWritten":4494}},
"ingress:16": {"name":"ingress","tags":{"db":"telegraf","login":"_systemuser_unknown","measurement":"diskio","rp":"autogen"},"values":{"pointsWritten":214,"seriesCreated":2,"valuesWritten":2354}},
"ingress:17": {"name":"ingress","tags":{"db":"telegraf","login":"_systemuser_unknown","measurement":"mem","rp":"autogen"},"values":{"pointsWritten":107,"seriesCreated":1,"valuesWritten":963}},
"ingress:18": {"name":"ingress","tags":{"db":"telegraf","login":"_systemuser_unknown","measurement":"processes","rp":"autogen"},"values":{"pointsWritten":107,"seriesCreated":1,"valuesWritten":856}},
"ingress:19": {"name":"ingress","tags":{"db":"telegraf","login":"_systemuser_unknown","measurement":"swap","rp":"autogen"},"values":{"pointsWritten":214,"seriesCreated":1,"valuesWritten":642}},
"ingress:20": {"name":"ingress","tags":{"db":"telegraf","login":"_systemuser_unknown","measurement":"system","rp":"autogen"},"values":{"pointsWritten":321,"seriesCreated":1,"valuesWritten":749}},

Closes: https://github.com/influxdata/influxdb/issues/20613
2021-02-05 14:52:43 -04:00
Sam Arnold ec40d5c380 chore: Fix spaces 2021-01-29 13:20:36 -05:00
Sam Arnold 8a16bf0531 chore: run goimports -w ./ 2021-01-29 11:40:02 -05:00
Sam Arnold 3a31e2370e fix(inspect): bad pattern matching 2021-01-29 10:55:20 -05:00
Sam Arnold a6152e8ac1 feat(inspect): Add report-disk for disk usage by measurement 2021-01-29 10:51:42 -05:00
Ayan George 431f073b9e
feat: Add -lponly flag to export sub-command (#19609)
When applied, this patch will add the -lponly flag to the export command
which instructs influx_inspect to only output line protocol without
comments and other out-of-band data.
2020-09-22 10:09:09 -04:00
Tristan Su 17d192e062
chore(dumptsm): clean up dead code (#17381) 2020-06-22 15:02:53 -07:00
David Norton 25381f97c8
Merge pull request #15952 from influxdata/er-verify-tombstone
feat(inspect): add influx_inspect verify-tombstone tool
2020-03-11 15:56:37 -04:00
David Norton fca2ee03a7 fix: update package comment 2020-03-11 15:37:02 -04:00
David Norton 0fcb240ebb fix: use highest verbosity level specified 2020-03-11 15:32:34 -04:00
David Norton b93bac197c fix: typo in error message 2020-03-11 15:28:45 -04:00
Ben Johnson 7a9eb1420c fix(tsdb): Fix -compact-series-file flag 2020-02-06 13:40:19 -07:00
Edd Robinson 34c0fdafc0 feat(storage): Offline series file compaction 2020-02-03 13:57:31 -07:00
Edd Robinson 92b9bdef14 feat(inspect): add verify-tombstone 2019-11-17 14:53:39 +00:00
Adam Perlin 23bb49981a fix(storage): replace panic with error in influx_inspect verify 2019-06-25 12:11:42 -07:00
Jeff Wendling 6833c4d082 fix(influx_inspect): verify-seriesfile with deleted index entries
The series index looks at a set of tombstones when querying the id for
a given key, but it does not look when asking for the offset for some
id, even if that id is deleted.

Update the verify tooling to check that the index agrees with the
deleted status of the id, but skip doing the extra checks if the
id is deleted.
2019-04-17 12:24:23 -06:00
Stuart Carnie 8aba6f73a5
feat(influx_inspect): Add -check-utf8 to verify sub-command 2018-12-21 10:44:43 -07:00
Jeff Wendling c4a3ecf021 fix verify seriesfile in presence of tombstones
tombstone entries don't contain a key, and parsing the empty
key panics.
2018-11-21 11:07:38 -07:00
Ben Johnson 76df4371ee
Merge pull request #9884 from influxdata/bj-dumptsi-series-file-help
Add -series-file flag to dumptsi command help.
2018-10-17 08:52:50 -06:00
Edd Robinson c57c1744ed Set cache size when building TSI index
If there is a significant amount of data in the WAL, then building the
TSI index can be problematic without being able to set the max cache
size to something larger.

This commit adds an option to se the maximum cache size.
2018-10-08 16:26:37 +01:00
Ben Johnson 2cb97146f0
Fix series file tombstoning.
This commit fixes an issue with the series file compaction process
where tombstones are lost after compaction and series existence
checks are not correct. This commit also fixes some smaller flushing
issues within the series file that mainly related to testing.
2018-10-05 08:23:25 -06:00
Jacob Marble 5c8f350dd8 influx_inspect: Make buildtsi code resusable 2018-08-07 11:33:21 -07:00
Ben Johnson fdfd038401
Add roaring bitmaps to TSI index files. 2018-07-24 17:59:23 +01:00
Edd Robinson ad388a8fd8 Address PR feedback 2018-07-09 11:51:48 +01:00
Edd Robinson 52e291eb62 uncomment profiling 2018-07-09 11:51:48 +01:00
Edd Robinson cd0294ea70 Enable concurrent index building 2018-07-09 11:51:48 +01:00
Edd Robinson ac0f125db9 Provide an API for re-using tags 2018-07-09 11:51:48 +01:00
Edd Robinson 9c2e7635d5 Avoid allocating a string for measurment names 2018-07-09 11:51:48 +01:00
Edd Robinson 3cf20823e9 Allow LogFile buffer size to be changed
When adding many series using offline tooling, it's likely that every
series involves an entry being appended to a LogFile. Typically an entry
is 11 or 12 bytes, but the default bufio.Writer buffer size is only 4K.

This means by default a write of 10,000 new series would involve ~30
buffer flushes.

This commit makes the buffer configurable, and sets the value in
`buildtsi` such that it reflects the number of series being written to
the LogFile.
2018-07-09 11:51:48 +01:00
Edd Robinson 4a90d142a7 Batch series being created 2018-07-09 11:51:15 +01:00
Edd Robinson 32ed165d9e Don't flush/sync when building index 2018-07-09 11:51:15 +01:00
Edd Robinson 2124b5e48e Logging format 2018-07-09 11:49:26 +01:00
Edd Robinson af88db51c1 Add profiling 2018-07-09 11:49:26 +01:00
Edd Robinson 08c3800a8a Address PR feedback 2018-07-04 10:57:40 +01:00
Edd Robinson cef4c399c5 Sort shard results 2018-07-03 14:20:10 +01:00
Edd Robinson e959fd49a9 Fix build 2018-07-03 14:15:32 +01:00
Edd Robinson 07ca0378bf Implementing -tag-keys is a TODO 2018-07-03 13:52:04 +01:00
Edd Robinson 89b1a49d38 Fine tune limit for when to use bitmap 2018-07-03 13:39:33 +01:00
Edd Robinson 5bce912a39 Fix panic when no series for measurement: 2018-07-03 12:39:20 +01:00
Edd Robinson 593d530525 Add total cardinality 2018-07-03 12:13:05 +01:00
Edd Robinson 33a60dfd85 Support for cardinality by measurement 2018-07-03 11:57:08 +01:00
Jacob Marble 0dc5393441 tsm/cache: Remove unused function parameter 2018-06-13 15:22:37 -07:00
Ben Johnson 1fc403337e
Add -series-file flag to dumptsi command help. 2018-05-22 10:36:00 -06:00
Jeff Wendling 6409b0a441 dumptsmwal: fix minor vet error 2018-05-18 18:21:19 -06:00
Jacob Marble 3f2ff742c0 Remove unused 'database' field 2018-05-18 09:22:43 -07:00
Ben Johnson 22ffb6acda
Add dumptsmwal command to influx_inspect. 2018-05-15 11:21:35 -06:00
Jacob Marble 87d73d405c tsdb/SeriesFile: remove unused function param 2018-05-04 11:22:12 -07:00
Jeff Wendling cf81a3e66e
Merge pull request #9770 from influxdata/jmw-inspect-series-file
Add verify-seriesfile to influx_inspect
2018-05-02 11:27:53 -06:00