* fix: prevent retention service creating orphaned shard files
Under certain circumstances, the retention service can fail to delete shards from
the store in a timely manner. When the shard groups are pruned based on age, this
leaves orphaned shard files on the disk. The retention service will then not attempt
to remove the obsolete shard files because the meta store does not know about them.
This can cause excessive disk space usage for some users.
This corrects that by requiring shards files be deleted before they can be removed
from the meta store.
fixes: #24529
Some series files which are smaller than the standard
sizes cause SIGBUS in influx_inspect and influxd, because
entry iteration walks onto mapped memory not backed by the
the file. Avoid walking off the end of the file while
iterating series entries in oddly sized files.
closes https://github.com/influxdata/influxdb/issues/24508
Co-authored-by: Geoffrey Wossum <gwossum@influxdata.com>
* chore: delete unused signing script
* fix: execute "compact-series-file" at startup
* fix: add "INFLUXD_OPTS" to "influxd config" line
* fix: use CONFIG to store influxd config path
Series file indices monotonically grew even
when series were deleted. Also stop
ignoring error in series index recovery
Partially closes https://github.com/influxdata/EAR/issues/3643
An empty slice in Sources causes an empty
WHERE clause to be generated by
Statement.String(). Use a nil in the field
to omit the WHERE clause completely
closes https://github.com/influxdata/influxdb/issues/24144
Setting the DB or RP in measurements extracted from
the predicate and used as sources causes Enterprise
/api/v2/delete to fail. The DB must explicitly be passed
to Store.Delete without being set in the measurement
struct passed as a source.
closes https://github.com/influxdata/influxdb/issues/24139
The CLI code used a single function from the `golang.org/x/crypto/ssh/terminal`
package (`terminal.IsTerminal`) which is just a wrapper around the
`golang.org/x/term` package's `term.IsTerminal` function. Replacing this
call prevents unnecessary and non-FIPS crypto functions from being pulled
into the binary.
closes: #24035
Absolute file paths in influx_inspect check-schema
cause an 'Invalid Argument' error. This was caused
fs.WalkDir using fs.ValidPath. Replacing with
filepath.WalkDir permits absolute paths.
closes https://github.com/influxdata/influxdb/issues/23987
* feat: include `crypto` diagnostics in `/debug/vars` output
Pulls `crypto` diagnostics and includes them in `/debug/vars` output.
If no `crypto` diagnostics are available, then OSS crypto information will
be shown instead.
closes: #23947
* chore: upgrade Go to 1.19.3
This re-runs ./generate.sh and ./checkfmt.sh to format and update
source code (this is primarily responsible for the huge diff.)
* fix: update tests to reflect sorting algorithm change
* feat: upgrade flux to 0.171.0
Tests failing, safety commit
First step in https://github.com/influxdata/influxdb/issues/23815
* fix: remove "org" parameter" from writeOptSource
I attempted to implement the "orgOpt" argument in a similar fashion
to f6669f7512. However, it looks like Flux doesn't accept "org" as
a parameter to "load". It responds with:
Error calling function \"load\" @113:16-113:30: error calling function \"to\" @6:19-6:47: unused arguments [org]
This brings us from 194 passing to 570 passing.
* fix: temporarily disable broken flux tests
These tests expect rows to be stored in a certain order. However,
nothing is specifying the sort order. This has been fixed in a
later update to flux: (see 3d6f47ded).
Temporarily disable these tests until we include a fixed
version of the flux tests.
* chore: add tests from a492993012
This fixes "test-flux.sh" so it runs tests within the "flux/"
directory. This uncovered some other issues with the tests
located within "flux/". These also needed to be updated
to match the newer flux API.
* feat: upgrade flux to 0.172.0
This includes changes made in "cbbf4b27da". Since "test.go" in 2.x
diverged from 1.x, some modifications were required to make this
compatible.
* feat: upgrade flux to 0.173.0
* feat: upgrade flux to v0.174.0
* fix: Update the condition when reseting cursor (#23522)
Filters that contain `or` may change between cursor resets so we must remember to update the condition in the read cursor.
```flux
|> filter(fn: (r) => ((r["_field"] == "field1" and r["_value"]==true) or (r["_field"] == "field2" and r["_value"] == false)))
```
Closes https://github.com/influxdata/flux/issues/4804
* feat: upgrade flux to 0.174.1
* feat: upgrade flux to 0.175.0
* chore: remove end-to-end tests
These were removed in a492993 for 2.x. These tests prevent "go test ./..."
from completing. As stated in the original commit, these tests should now be
handled by the "fluxtest" harness.
* feat: upgrade flux to 0.176.0
Some tests needed to be disabled within the flux harness. This is a
result of enabling "Optimize Aggregate Window" in flux@05a1065f.
These tests are not present in 2.x. Therefore, I am unsure if
the breakage is resolved in a later commit.
* feat: upgrade flux to 0.177.0
* feat: upgrade flux to 0.178.0
* feat: upgrade flux to v0.179.0
This removes all invocations of "flux.RegisterOpSpec". According
to flux@e39096d5, "flux.RegisterOpSpec" does nothing in the
current version of flux and was removed.
* chore: update fluxtest skip list (#23633)
* chore: manually backport 785a465e9a
This removes the reference to "flux.Spec".
* build(flux): update flux to v0.181.0 (#23682)
* build(flux): update flux to v0.184.2
* chore: skip more Flux acceptance tests
There are issues for each skip detailed in test-flux.sh.
* feat: upgrade flux to v0.185.0
This adds "FluxTesting" to the "HTTPD" configuration. This option is
hidden and disabled by default. When "FluxTesting" is set, it
enables the default testing flags for "Flux".
These flags allow the "vectorized float tests" and tests requiring
the "removeRedundantSortNodes" and "labelPolymorphism" flag
enabled to work. These changes are based off of d8553c002e.
flux@3d6f47ded is included within this version of Flux. Therefore
we can now include the "group_*" tests.
* feat: upgrade flux to 0.186.0
* feat: upgrade flux to 0.187.0
* feat: upgrade flux to 0.188.0
* fix: re-run ./generate.sh with updated protoc
* fix: restrict cores to match CircleCI documentation
Co-authored-by: davidby-influx <dbyrne@influxdata.com>
Co-authored-by: Markus Westerlind <marwes91@gmail.com>
Co-authored-by: Sean Brickley <sean@wabr.io>
Co-authored-by: Jonathan A. Sternberg <jonathan@influxdata.com>
Co-authored-by: Christopher M. Wolff <chris.wolff@influxdata.com>
This switches so that the message
skipped missing file: /path/to/tsm.tsm
is written to stdErr instead of stdout (or the output file if `-out` has been provided)
(cherry picked from commit a9bf1d54c1)
closes https://github.com/influxdata/influxdb/issues/23866
Co-authored-by: Ben Tasker <88340935+btasker@users.noreply.github.com>
New argument validation code for _fieldKeys system iterator
broke Enterprise tests because it is misused all over the
place. Back out the safety check.
When a file rename fails with EXDEV
(cross device or volume error), copy the
file and delete the original instead
Differs from master branch by overwriting
existing files instead of erring.
closes https://github.com/influxdata/influxdb/issues/22997
Instead of writing out the complete fields.idx
file when it changes, write out incremental
changes that will be applied to the file on
close and startup.
closes https://github.com/influxdata/influxdb/issues/23653
Jack
Geoffrey Wossum
davidby-influx
alespour
Brandon Pfeifer
Jamie Strandboge
Jeffrey Smith II
carlospeon
Sam Arnold