f74c69c5e4
chore: update to go 1.20 ( #24088 )
...
* build: upgrade to go 1.19
* chore: bump go.mod
* chore: `gofmt` changes for doc comments
https://tip.golang.org/doc/comment
* test: update tests for new sort order
* chore: make generate-sources
* chore: make generate-sources
* chore: go 1.20
* chore: handle rand.Seed deprecation
* chore: handle rand.Seed deprecation in tests
---------
Co-authored-by: DStrand1 <dstrandboge@influxdata.com>
2023-02-09 14:14:35 -05:00
8ad6e17265
chore: add additional error logging when deleting shard ( #24038 )
...
* chore: add additional error logging when deleting shard
* chore: better logging message
2023-02-09 09:10:25 -05:00
7ad8fbad22
chore: fix trace message text ( #23918 )
2022-11-16 08:40:26 -05:00
4de89afd37
refactor: remove dead iterator code ( #23887 )
...
* fix: codegen without needing goimports
* refactor: remove dead code
2022-11-09 19:26:12 -05:00
2ad8995355
fix: improve delete speed when a measurement is part of the predicate ( #23786 )
...
* fix: improve delete speed when a measurement is part of the predicate
* test: add test for deleting measurement by predicate
* chore: improve error messaging and capturing
* chore: set goland to use the right formatting style
2022-10-14 15:09:32 -04:00
b72848d436
feat: optimize saving changes to fields.idx ( #23701 ) ( #23728 )
...
Instead of writing out the complete fields.idx
file when it changes, write out incremental
changes that will be applied to the file on
close and startup.
closes https://github.com/influxdata/influxdb/issues/23653
(cherry picked from commit 80c10c8c04https://github.com/influxdata/influxdb/issues/23703
2022-09-15 12:15:14 -07:00
c433342830
chore: remove duplicate word in comments ( #23685 )
...
Signed-off-by: Abirdcfly <fp544037857@gmail.com>
Signed-off-by: Abirdcfly <fp544037857@gmail.com>
2022-09-13 11:00:52 -05:00
619eb1cae6
fix: restore in-memory Manifest on write error ( #23552 ) ( #23578 )
...
Do not update the `FileSet` or `activeLogFile` field in the in-memory
Partition structure if the Manifest file is not correctly saved to
the disk.
closes https://github.com/influxdata/influxdb/issues/23553
(cherry picked from commit a8732dcf52https://github.com/influxdata/influxdb/issues/23554
2022-07-25 10:53:09 -07:00
f762346ecc
fix: add paths to tsi log and index file errors ( #23557 ) ( #23562 )
...
Add paths to various TSI errors on opening and unmarshaling files
to help poinpoint the corrupt files.
Closes https://github.com/influxdata/influxdb/issues/23556
(cherry picked from commit 25cea95bebhttps://github.com/influxdata/influxdb/issues/23558
2022-07-19 15:45:42 -07:00
00edb77253
fix: create TSI MANIFEST files atomically ( #23539 ) ( #23546 )
...
When a MANIFEST file is created in TSI, it
should be written to a temp file, then
atomically renamed, to avoid overwriting
the existing file only to fail on the
later write.
closes https://github.com/influxdata/influxdb/issues/23536
(cherry picked from commit 061cf55f2ahttps://github.com/influxdata/influxdb/issues/23538
2022-07-14 09:13:11 -07:00
4789d5402a
fix: improve error messages opening index partitions ( #23532 ) ( #23535 )
...
Where possible, add the file path path to any errors
on opening, reading, (un)marshaling, or validating
the various files comprising a partition
closes https://github.com/influxdata/influxdb/issues/23506
(cherry picked from commit a2dd708a26https://github.com/influxdata/influxdb/issues/23534
2022-07-13 13:20:47 -07:00
8bd4fc502d
fix: lost TSI reference / close TagValueSeriesIDIterator in error case ( #23461 )
2022-06-16 13:35:45 -05:00
53580ead1d
fix: remember shards that fail Open(), avoid repeated attempts ( #23437 ) ( #23455 )
...
If a shard cannot be opened, store its ID and last error.
Prevent future attempts to open during this invocation of
influxDB. This information is not persisted.
closes https://github.com/influxdata/influxdb/issues/23428
closes https://github.com/influxdata/influxdb/issues/23426
(cherry picked from commit 54ac7e54edhttps://github.com/influxdata/influxdb/issues/23434
closes https://github.com/influxdata/influxdb/issues/23436
2022-06-14 13:01:11 -07:00
a9df3f8a7c
fix: fully clean up partially opened TSI ( #23430 ) ( #23454 )
...
When one partition in a TSI fails to open, all previously opened
partitions should be cleaned up, and remaining partitions
should not be opened
closes https://github.com/influxdata/influxdb/issues/23427
(cherry picked from commit d3db48e93dhttps://github.com/influxdata/influxdb/issues/23432
2022-06-14 11:49:16 -07:00
8c9768cdb7
fix: replace unprintable and invalid characters in errors ( #23387 ) ( #23395 )
...
Replace unprintable and invalid characters with '?'
in logged errors. Truncate consecutive runs of them to
only 3 repeats of '?'
closes https://github.com/influxdata/influxdb/issues/23386
(cherry picked from commit 0ae0bd6e2ehttps://github.com/influxdata/influxdb/issues/23389
2022-06-01 14:42:51 -07:00
30a9fd43f6
fix: MeasurementsCardinality should not be less than 0 ( #23304 )
...
Clamp the value of Store.MeasurementsCardinality so that it can not be less
than 0. This primarily shows up as a negative numMeasurements value in
/debug/vars under some circumstances.
refs #23285
(cherry picked from commit 160cf678d5
2022-04-26 23:37:09 -05:00
82d1123e78
build: upgrade to Go 1.18.1 ( #23252 )
2022-04-13 15:24:27 -05:00
4e08604e48
feat: Add MeasurementNames method to MeasurementFieldSet ( #23173 )
2022-03-15 10:21:38 +00:00
e20b5e99a6
fix: remove nats for scraper processing ( #23107 )
...
* fix: remove nats for scraper processing
Scrapers now use go channels instead of NATS and interprocess communication.
This should fix #23085 .
Additionally, found and fixed #23106 .
* chore: fix formatting
* chore: fix static check and go.mod
* test: fix some flaky tests
* fix: mark NATS arguments as deprecated
2022-02-10 11:23:18 -05:00
b8ccf5bbbd
fix: correctly handle PartialWriteError ( #23098 )
...
Check for the correctly returned PartialWriteError
in (*shard).validateSeriesAndFields, allow partial
writes.
closes https://github.com/influxdata/influxdb/issues/23096
2022-02-02 14:43:23 -08:00
39eeb3e456
fix(restore): fix race condition which could cause restore command to fail ( #22965 )
...
Fixes a race condition in the restore code path that could cause shard data restores to fail. When the race condition occurs, `Error while freeing cold shard resources` appears in the log files.
This is port of PR 22796 from master-1.x to master. Attempts at creating a test case for master failed, so the fix has ported without a corresponding unit test.
fixes #22957
2021-12-06 14:50:40 -06:00
b970e359dc
feat: remaining storage metrics from OSS engine ( #22938 )
...
* fix: simplify disk size tracking
* refactor: EngineTags in tsdb package
* fix: fewer compaction buckets and dead code removal
* feat: shard metrics
* chore: formatting
* feat: tsdb store metrics
* feat: retention check metrics
* chore: fix go vet
* fix: review comments
2021-12-02 09:01:46 -05:00
799d349813
fix(tsi): sync index file before close ( #22927 )
...
(cherry picked from commit 5fd1b29d74
2021-11-24 15:52:45 -05:00
9308b6586a
chore: remove unused member from TagSet ( #22930 )
...
Co-authored-by: Tristan Su <suqing.sq@alibaba-inc.com>
2021-11-24 14:43:50 -05:00
edb21abe91
feat: metrics for wal subsystem ( #22918 )
...
https://github.com/influxdata/influxdb/issues/20026
2021-11-23 12:17:52 -05:00
feb459c785
feat: metrics for cache subsystem ( #22915 )
...
* fix: drop complicated cache metrics and document remaining
* feat: metrics for cache
2021-11-23 10:11:22 -05:00
a74e05177c
feat: disk size metrics per shard ( #22912 )
2021-11-22 16:53:55 -05:00
dece95d1dd
feat: tsm compaction metrics via prometheus ( #22904 )
...
* feat: tsm compaction metrics via prometheus
* chore: fix formatting
* chore: make activeCompactions a pointer
2021-11-19 14:51:22 -05:00
2bace7767d
fix: unhandled errors returned by Sketch.Merge ( #22858 )
2021-11-10 09:26:24 -05:00
f4e9ae94ca
build: upgrade protobuf library ( #22654 )
2021-11-02 16:00:54 -05:00
88afa9229b
fix: detect misquoted tag values and return an error ( #22754 ) ( #22785 )
...
SHOW TAG KEYS FROM "foo" where bar="misquoted" is
erroneous, because the tag value must be enclosed
in single, not double, quotes. Although this
correctly returns no tag keys, it is very
inefficient and has cause out-of-memory failures
at a customer. This fix short-circuits the query.
closes https://github.com/influxdata/influxdb/issues/22755
(cherry picked from commit af9e89a4d4https://github.com/influxdata/influxdb/issues/22757
2021-10-27 21:32:11 -07:00
5e6b0d539b
fix: extend snapshot copy to filesystems that cannot link ( #22703 ) ( #22732 )
...
If os.Link fails with syscall.ENOTSUP, then the file
system does not support links, and we must make copies
to snapshot files for backup. We also automatically make
copies instead of link on Windows, because although it
makes links, their semantics are different from Linux.
closes https://github.com/influxdata/influxdb/issues/16739
(cherry picked from commit d9b9e86db9https://github.com/influxdata/influxdb/issues/22701
2021-10-22 08:59:41 -07:00
ca992e9fff
chore: use io/os over ioutil ( #22656 )
2021-10-12 16:55:07 -05:00
2ecbb68fc3
test: fix DiskSizeBytes flakiness ( #22639 )
2021-10-08 09:46:58 -04:00
0f6614ffa7
fix(tsdb): sync series segment to disk after writing ( #22545 )
...
Co-authored-by: Tristan Su <suqing.sq@alibaba-inc.com>
2021-09-23 12:09:16 -05:00
47007f6988
fix: for Windows, copy snapshot files being backed up ( #22551 ) ( #22562 )
...
On Windows, make copies of files for snapshots, because
Go does not support the FILE_SHARE_DELETE flag which
allows files (and links) to be deleted while open. This
causes temporary directories to be left behind after
backups.
closes https://github.com/influxdata/influxdb/issues/16289
(cherry picked from commit 3702fe8e76https://github.com/influxdata/influxdb/issues/22557
2021-09-22 13:06:28 -07:00
e06e34aa56
test: sleep between 2nd and 3rd checks of engine last-modified time ( #22554 )
...
And rewrite the test to use testify
2021-09-22 14:14:46 -04:00
7dfd7de81f
feat: set X-Influxdb-Version and X-Influxdb-Build headers ( #22535 )
...
Closes #20224
Also a forward port of #22038 since I saw the same test failing on 2.x
2021-09-22 07:30:45 -04:00
233f277c07
test: check post-compaction series file sizes before reopening ( #22534 )
2021-09-21 16:48:44 -04:00
1f66b3110e
fix: upgrade influxql to latest version & fix predicate handling for show tag values metaqueries ( #22500 )
...
* feat: Add WITH KEY to show tag keys
* fix: add tests for multi measurement tag value queries
* chore: fix linter problems
* chore: revert influxql changes to keep WITH KEY disabled
* chore: add TODO for moving flux tests to flux repo
Co-authored-by: Sam Arnold <sarnold@influxdata.com>
2021-09-17 11:14:03 -06:00
ec7841b355
feat: support for flux cardinality query ( #22441 )
...
* feat: works with custom iterator
* feat: works with existing iterators
* chore: cleanup
* test: consistent assertions for tests
* fix: better log message if trying to filter on the value of a field key
* fix: comment for handling boolean literal; handle false boolean as well
* fix: make time range checking inclusive
2021-09-13 13:20:56 -06:00
5015297d40
fix: more expressive errors ( #22448 )
...
* fix: more expressive errors
Closes #22446
* fix: server only logging for untyped errors
* chore: fix formatting
2021-09-13 15:12:35 -04:00
3e275a123d
feat: multi-measurement query optimization ( #22301 )
...
* feat: multi-measurement query optimization
2021-09-13 13:00:08 -06:00
1482bdfa38
feat: partial support for series cardinality flux query ( #22378 )
2021-09-09 12:37:11 -06:00
af12281358
chore: delete pointless test for mixed-index shards ( #22405 )
2021-09-07 12:25:07 -04:00
7ad612b0d7
fix: discard excessive errors ( #22379 ) ( #22391 )
...
The tsmBatchKeyIterator discards excessive errors to avoid
out-of-memory crashes when compacting very corrupt files.
Any error beyond DefaultMaxSavedErrors (100) will be
discarded instead of appended to the error slice.
closes https://github.com/influxdata/influxdb/issues/22328
(cherry picked from commit e53f75e06dhttps://github.com/influxdata/influxdb/issues/22381
2021-09-03 14:57:36 -07:00
1c2d68b0cb
build: upgrade to go1.17 ( #22363 )
2021-09-01 16:09:02 -04:00
99cfbfe8cf
fix: repair bad port dropping return value names ( #22307 )
...
Named returns were not cherry-picked correctly from
commit a989f8f8b6https://github.com/influxdata/influxdb/issues/22303
2021-08-31 09:24:15 -07:00
12fff64760
fix: make TSI index compact old and too-large log files ( #22334 )
...
* TSI index should compact old or too-large log files
* Old tsl files should be compacted without new writes
* Add extra logging when disk size test fails
Co-authored-by: Sam Arnold <sarnold@influxdata.com>
2021-08-30 18:27:48 -04:00
37088e8f53
feat(influxql): Add hyper log log operators ( #22322 )
...
In addition to helping with normal queries, this can improve the 'SHOW CARDINALITY'
meta-queries.
Co-authored-by: Sam Arnold <sarnold@influxdata.com>
2021-08-30 15:46:46 -04:00
Jeffrey Smith II
davidby-influx
Sam Arnold
Abirdcfly
Dane Strandboge
Geoffrey Wossum
Andrew Charlton
CasMc
Dane Strandboge
Tristan Su
Daniel Moran
William Baker