Add FutureWriteLimit and PastWriteLimit to retention
policies. Points which are outside of
now() + FutureWriteLimit
or
now() - PastWriteLimit
will be rejected on write with a PartialWriteError.
closes https://github.com/influxdata/influxdb/issues/25424
Fixes#10052
This commit fixes an issue where field keys would reappear in results
when querying previously dropped measurements.
The issue manifests itself when duplicates of a new series are inserted
into the `inmem` index. In this case, a map that tracks the number of
series belonging to a measurement was incorrectly incremented once for
each duplication of the series. Then, when it came time to drop the
measurement, the index assumed there were several series belonging to
the measurement left in the index (because the counter was higher than
it should be). The result of that was that the `fields.idx` file (which
stores a mapping between measurements and field keys) was not truncated
and rebuilt. This left old field keys in that file, which were then
returned in subsequent queries over all field keys.
use. However, because the reference counting was implemented via
mutexes, it was possible to double `RLock` the series file mutex. This
allows a `Lock` to arrive in-between the two `RLock`s, (such as when
deleting the database), causing deadlock.
This commit addresses this by ensuring that from within `IndexSet`
methods, when calling other `IndexSet` methods, that they're all
unexported, and that those unexported methods never take a lock on the
series file.
Keeping series file locking in exported `IndexSet` methods only, allows
one to see any future races more easily.
This commits adds a randomised deletion test, which aims to test the
correctness of deletions and drops in the index and series file.
The test works by maintaining its own index of points, series and
measurements, tracking the presence or absence or series.
There are four commands that can be randomly executed:
- insert a series at a random time
- drop an entire measurement
- drop an entire series
- delete series across a time range
After every invocation of one of these commands, `SHOW SERIES` is
executed on the server, and the results are compared to what the
test's series tracker believes should be present. If the results
differ then the test fails.
On failure, the last 100 queries executed, as well as the series
we expected, and the series returned by `SHOW SERIES`, are returned.
davidby-influx
Sam Arnold
Edd Robinson