23bb49981a
fix(storage): replace panic with error in influx_inspect verify
2019-06-25 12:11:42 -07:00
6833c4d082
fix(influx_inspect): verify-seriesfile with deleted index entries
...
The series index looks at a set of tombstones when querying the id for
a given key, but it does not look when asking for the offset for some
id, even if that id is deleted.
Update the verify tooling to check that the index agrees with the
deleted status of the id, but skip doing the extra checks if the
id is deleted.
2019-04-17 12:24:23 -06:00
8aba6f73a5
feat(influx_inspect): Add -check-utf8 to verify sub-command
2018-12-21 10:44:43 -07:00
c4a3ecf021
fix verify seriesfile in presence of tombstones
...
tombstone entries don't contain a key, and parsing the empty
key panics.
2018-11-21 11:07:38 -07:00
76df4371ee
Merge pull request #9884 from influxdata/bj-dumptsi-series-file-help
...
Add -series-file flag to dumptsi command help.
2018-10-17 08:52:50 -06:00
c57c1744ed
Set cache size when building TSI index
...
If there is a significant amount of data in the WAL, then building the
TSI index can be problematic without being able to set the max cache
size to something larger.
This commit adds an option to se the maximum cache size.
2018-10-08 16:26:37 +01:00
2cb97146f0
Fix series file tombstoning.
...
This commit fixes an issue with the series file compaction process
where tombstones are lost after compaction and series existence
checks are not correct. This commit also fixes some smaller flushing
issues within the series file that mainly related to testing.
2018-10-05 08:23:25 -06:00
5c8f350dd8
influx_inspect: Make buildtsi code resusable
2018-08-07 11:33:21 -07:00
fdfd038401
Add roaring bitmaps to TSI index files.
2018-07-24 17:59:23 +01:00
ad388a8fd8
Address PR feedback
2018-07-09 11:51:48 +01:00
52e291eb62
uncomment profiling
2018-07-09 11:51:48 +01:00
cd0294ea70
Enable concurrent index building
2018-07-09 11:51:48 +01:00
ac0f125db9
Provide an API for re-using tags
2018-07-09 11:51:48 +01:00
9c2e7635d5
Avoid allocating a string for measurment names
2018-07-09 11:51:48 +01:00
3cf20823e9
Allow LogFile buffer size to be changed
...
When adding many series using offline tooling, it's likely that every
series involves an entry being appended to a LogFile. Typically an entry
is 11 or 12 bytes, but the default bufio.Writer buffer size is only 4K.
This means by default a write of 10,000 new series would involve ~30
buffer flushes.
This commit makes the buffer configurable, and sets the value in
`buildtsi` such that it reflects the number of series being written to
the LogFile.
2018-07-09 11:51:48 +01:00
4a90d142a7
Batch series being created
2018-07-09 11:51:15 +01:00
32ed165d9e
Don't flush/sync when building index
2018-07-09 11:51:15 +01:00
2124b5e48e
Logging format
2018-07-09 11:49:26 +01:00
af88db51c1
Add profiling
2018-07-09 11:49:26 +01:00
08c3800a8a
Address PR feedback
2018-07-04 10:57:40 +01:00
cef4c399c5
Sort shard results
2018-07-03 14:20:10 +01:00
e959fd49a9
Fix build
2018-07-03 14:15:32 +01:00
07ca0378bf
Implementing -tag-keys is a TODO
2018-07-03 13:52:04 +01:00
89b1a49d38
Fine tune limit for when to use bitmap
2018-07-03 13:39:33 +01:00
5bce912a39
Fix panic when no series for measurement:
2018-07-03 12:39:20 +01:00
593d530525
Add total cardinality
2018-07-03 12:13:05 +01:00
33a60dfd85
Support for cardinality by measurement
2018-07-03 11:57:08 +01:00
0dc5393441
tsm/cache: Remove unused function parameter
2018-06-13 15:22:37 -07:00
1fc403337e
Add -series-file flag to dumptsi command help.
2018-05-22 10:36:00 -06:00
6409b0a441
dumptsmwal: fix minor vet error
2018-05-18 18:21:19 -06:00
3f2ff742c0
Remove unused 'database' field
2018-05-18 09:22:43 -07:00
22ffb6acda
Add dumptsmwal command to influx_inspect.
2018-05-15 11:21:35 -06:00
87d73d405c
tsdb/SeriesFile: remove unused function param
2018-05-04 11:22:12 -07:00
cf81a3e66e
Merge pull request #9770 from influxdata/jmw-inspect-series-file
...
Add verify-seriesfile to influx_inspect
2018-05-02 11:27:53 -06:00
2e470974ec
Merge pull request #9793 from influxdata/bj-delete-nonprintable-unicode
...
Add key sanitization to deletetsm
2018-04-30 15:38:08 -06:00
4cb36d5301
Add key sanitization to deletetsm
...
This commit adds the `-sanitize` flag to `influx_inspect deletetsm`
which will delete all keys that contain invalid, non-printable, or
replacement character unicode.
Usage:
```sh
$ influx_inspect deletetsm -sanitize PATH
```
2018-04-30 13:06:33 -06:00
108fa09439
Rename corrupt tsm files on load.
2018-04-27 14:27:44 -06:00
02b90d195e
verify-seriesfile: update verbose logging
2018-04-26 12:54:32 -06:00
bc2801c3fc
verify-seriesfile: some style and renaming fixes
2018-04-26 12:11:02 -06:00
c38108eb8a
fix some megacheck errors
2018-04-25 18:14:25 -06:00
388a664acc
verify-seriesfile: verify partitions in parallel
2018-04-25 18:10:27 -06:00
696ab40ddb
verify-seriesfile: reduce compaction threshold in tests
...
helps it run faster, and hopefully not OOM in teeny environments.
2018-04-25 18:10:23 -06:00
adeec62c0d
reorganize the verify commands into a folder
2018-04-25 18:10:22 -06:00
3e7a01b1db
refactor verify_seriesfile
...
- some style changes
- export and make usable the verify functions
- make smaller functions for verification
- change the flags to be more intuitive
2018-04-25 18:10:20 -06:00
9f09a3c1c2
add tests for verify_seriesfile
...
does some basic sanity checks. it's hard to be more exhaustive without
either taking a crazy amount of time, or being non-deterministic,
but at least this makes sure we barf in some cases.
2018-04-25 18:10:19 -06:00
e771040a7d
Add verify-seriesfile to influx_inspect
...
Fixes #9767
2018-04-25 18:10:17 -06:00
dff34939f8
Merge pull request #9028 from cbushko/lint-the-c-dirs
...
Fix linting on cmd files
2018-04-23 10:41:05 -05:00
8c9d179a94
Add 'influx_inspect deletetsm'
2018-04-19 10:36:08 -06:00
fd491b28a6
Check for root user when running buildtsi.
...
This commit also adds the `max-log-file-size` flag.
2018-04-04 09:58:02 -06:00
b3c2d9290f
Log error encountered when reading WAL files
...
Inspired by #9657 .
2018-03-30 09:40:58 -07:00
Adam Perlin
Jeff Wendling
Stuart Carnie
Jeff Wendling
Ben Johnson
Edd Robinson
Jacob Marble
Jeff Wendling
Jonathan A. Sternberg
Mark Rushakoff