From d758e265864c3fbb88fe55aa77a4412adf28bb1e Mon Sep 17 00:00:00 2001 From: Mark Herwege Date: Mon, 7 Apr 2025 18:07:39 +0200 Subject: [PATCH] Riemann sum documentation (#2477) * persistence extensions Signed-off-by: Mark Herwege * blockly Signed-off-by: Mark Herwege * fix image locations Signed-off-by: Mark Herwege * improvements Signed-off-by: Mark Herwege * clarify time dimension Signed-off-by: Mark Herwege --------- Signed-off-by: Mark Herwege --- .../blockly/rules-blockly-persistence.md | 6 ++ configuration/images/riemann-left.svg | 1 + configuration/images/riemann-midpoint.svg | 1 + configuration/images/riemann-right.svg | 1 + configuration/images/riemann-trapezoidal.svg | 1 + configuration/persistence.md | 76 ++++++++++++++++--- 6 files changed, 76 insertions(+), 10 deletions(-) create mode 100644 configuration/images/riemann-left.svg create mode 100644 configuration/images/riemann-midpoint.svg create mode 100644 configuration/images/riemann-right.svg create mode 100644 configuration/images/riemann-trapezoidal.svg diff --git a/configuration/blockly/rules-blockly-persistence.md b/configuration/blockly/rules-blockly-persistence.md index 357ef974c..8c52ea9a9 100644 --- a/configuration/blockly/rules-blockly-persistence.md +++ b/configuration/blockly/rules-blockly-persistence.md @@ -70,11 +70,17 @@ This method uses a time-weighted average calculation - evolution rate: gets the evolution rate of the state of the given Item in percent (may be positive or negative) - minimum: gets the minimum value of the State of the given Item - maximum: gets the maximum value of the State of the given Item +- Riemann sum: gets a Riemann sum of the states of the given Item. +This is an approximation of the integral of a continuous function represented by discrete values. - sum: gets the sum of the State of the given Item - state updates count: gets the count of State updates of the given Item - state changes count: gets the count of State changes of the given Item - all states: gets all States of the given Item +Some statistical values using time-weighted averages (Riemann sum, average, variance, deviation) have an extra parameter representing the approximation type for the calculation. +Possible values are `left`, `right`, `trapezoidal` or `midpoint`. +For more information, see: [Time-weighted calculations - Riemann Sums]({{base}}/configuration/persistence.html#time-weighted-calculations-riemann-sums) + In the case of the following two functions the block changes its appearance by replacing the time with an option to chose if the equal value should be skipped or not: ![previous-block](../images/blockly/blockly-persistence-get-previous.png) diff --git a/configuration/images/riemann-left.svg b/configuration/images/riemann-left.svg new file mode 100644 index 000000000..c548a6567 --- /dev/null +++ b/configuration/images/riemann-left.svg @@ -0,0 +1 @@ +VTTTVVUitdrukking 2Uitdrukking 3Uitdrukking 4Uitdrukking 5Uitdrukking 6Uitdrukking 7 \ No newline at end of file diff --git a/configuration/images/riemann-midpoint.svg b/configuration/images/riemann-midpoint.svg new file mode 100644 index 000000000..a23a5415a --- /dev/null +++ b/configuration/images/riemann-midpoint.svg @@ -0,0 +1 @@ +VTTTVVUitdrukking 1Uitdrukking 3Uitdrukking 4Uitdrukking 5Uitdrukking 6Uitdrukking 7Uitdrukking 8 \ No newline at end of file diff --git a/configuration/images/riemann-right.svg b/configuration/images/riemann-right.svg new file mode 100644 index 000000000..5d3a4affe --- /dev/null +++ b/configuration/images/riemann-right.svg @@ -0,0 +1 @@ +VTTTVVUitdrukking 2Uitdrukking 3Uitdrukking 4Uitdrukking 5Uitdrukking 6Uitdrukking 7 \ No newline at end of file diff --git a/configuration/images/riemann-trapezoidal.svg b/configuration/images/riemann-trapezoidal.svg new file mode 100644 index 000000000..91c0e7ccc --- /dev/null +++ b/configuration/images/riemann-trapezoidal.svg @@ -0,0 +1 @@ +VTTTVVUitdrukking 1Uitdrukking 3Uitdrukking 4Uitdrukking 5Uitdrukking 6Uitdrukking 7 \ No newline at end of file diff --git a/configuration/persistence.md b/configuration/persistence.md index 44eedbc38..be4e7beb3 100644 --- a/configuration/persistence.md +++ b/configuration/persistence.md @@ -286,9 +286,6 @@ Here is the full list of available persistence extensions: | `.minimumSince(ZonedDateTime)` | Gets the minimum value of the State of a persisted Item since a certain point in time (returns HistoricItem) | | `.minimumUntil(ZonedDateTime)` | Gets the minimum value of the State of a persisted Item until a certain future point in time (returns HistoricItem) | | `.minimumBetween(ZonedDateTime, ZonedDateTime)` | Gets the minimum value of the State of a persisted Item between certain points in time (returns HistoricItem) | -| `.averageSince(ZonedDateTime)` | Gets the average value of the State of a persisted Item since a certain point in time. This method uses a time-weighted average calculation (see example below) (returns State) | -| `.averageUntil(ZonedDateTime)` | Gets the average value of the State of a persisted Item until a certain point in time. This method uses a time-weighted average calculation (see example below) (returns State) | -| `.averageBetween(ZonedDateTime, ZonedDateTime)` | Gets the average value of the State of a persisted Item between certain points in time. This method uses a time-weighted average calculation (see example below) (returns State) | | `.medianSince(ZonedDateTime)` | Gets the median value of the State of a persisted Item since a certain point in time (returns State) | | `.medianUntil(ZonedDateTime)` | Gets the median value of the State of a persisted Item until a certain point in time (returns State) | | `.medianBetween(ZonedDateTime, ZonedDateTime)` | Gets the median value of the State of a persisted Item between certain points in time (returns State) | @@ -298,12 +295,6 @@ Here is the full list of available persistence extensions: | `.evolutionRateSince(ZonedDateTime)` | Gets the evolution rate of the state of a given Item since a certain point in time (returns DecimalType) | | `.evolutionRateUntil(ZonedDateTime)` | Gets the evolution rate of the state of a given Item until a certain point in time (returns DecimalType) | | `.evolutionRateBetween(ZonedDateTime, ZonedDateTime)` | Gets the evolution rate of the state of a given Item between certain points in time (returns DecimalType) | -| `.deviationSince(ZonedDateTime)` | Gets the standard deviation of the state of the given Item since a certain point in time (returns State) | -| `.deviationUntil(ZonedDateTime)` | Gets the standard deviation of the state of the given Item until a certain point in time (returns State) | -| `.deviationBetween(ZonedDateTime, ZonedDateTime)` | Gets the standard deviation of the state of the given Item between certain points in time (returns State) | -| `.varianceSince(ZonedDateTime)` | Gets the variance of the state of the given Item since a certain point in time (returns State) | -| `.varianceUntil(ZonedDateTime)` | Gets the variance of the state of the given Item until a certain future point in time (returns State) | -| `.varianceBetween(ZonedDateTime, ZonedDateTime)` | Gets the variance of the state of the given Item between certain points in time (returns State) | | `.sumSince(ZonedDateTime)` | Gets the sum of the previous States of a persisted Item since a certain point in time (returns State) | | `.sumUntil(ZonedDateTime)` | Gets the sum of the future States of a persisted Item until a certain point in time (returns State) | | `.sumBetween(ZonedDateTime, ZonedDateTime)` | Gets the sum of the previous States of a persisted Item between certain points in time (returns State) | @@ -313,6 +304,18 @@ Here is the full list of available persistence extensions: | `.countStateChangesSince(ZonedDateTime)` | Gets the number of changes in persisted States of an Item since a certain point in time | | `.countStateChangesUntil(ZonedDateTime)` | Gets the number changes in of persisted States of an Item until a certain point in time | | `.countStateChangesBetween(ZonedDateTime, ZonedDateTime)` | Gets the number of changes in persisted States of an Item between certain points in time | +| `.riemannSumSince(ZonedDateTime)` | Gets the Riemann sum of the States of a persisted Item since a certain point in time. This method uses a time-weighted calculation (see [below](#time-weighted-calculations-riemann-sums)) (returns State) | +| `.riemannSumUntil(ZonedDateTime)` | Gets the Riemann sum of the States of a persisted Item until a certain point in time. This method uses a time-weighted calculation (see [below](#time-weighted-calculations-riemann-sums)) (returns State) | +| `.riemannSumBetween(ZonedDateTime, ZonedDateTime)` | Gets the Riemann sum of the States of a persisted Item between certain points in time. This method uses a time-weighted calculation (see [below](#time-weighted-calculations-riemann-sums)) (returns State) | +| `.averageSince(ZonedDateTime)` | Gets the average value of the State of a persisted Item since a certain point in time. This method uses a time-weighted calculation (see [below](#time-weighted-calculations-riemann-sums)) (returns State) | +| `.averageUntil(ZonedDateTime)` | Gets the average value of the State of a persisted Item until a certain point in time. This method uses a time-weighted calculation (see [below](#time-weighted-calculations-riemann-sums)) (returns State) | +| `.averageBetween(ZonedDateTime, ZonedDateTime)` | Gets the average value of the State of a persisted Item between certain points in time. This method uses a time-weighted calculation (see [below](#time-weighted-calculations-riemann-sums)) (returns State) | +| `.varianceSince(ZonedDateTime)` | Gets the variance of the state of the given Item since a certain point in time. This method uses a time-weighted calculation (see [below](#time-weighted-calculations-riemann-sums)) (returns State) | +| `.varianceUntil(ZonedDateTime)` | Gets the variance of the state of the given Item until a certain future point in time. This method uses a time-weighted calculation (see [below](#time-weighted-calculations-riemann-sums)) (returns State) | +| `.varianceBetween(ZonedDateTime, ZonedDateTime)` | Gets the variance of the state of the given Item between certain points in time. This method uses a time-weighted calculation (see [below](#time-weighted-calculations-riemann-sums)) (returns State) | +| `.deviationSince(ZonedDateTime)` | Gets the standard deviation of the state of the given Item since a certain point in time. This method uses a time-weighted calculation (see [below](#time-weighted-calculations-riemann-sums)) (returns State) | +| `.deviationUntil(ZonedDateTime)` | Gets the standard deviation of the state of the given Item until a certain point in time. This method uses a time-weighted calculation (see [below](#time-weighted-calculations-riemann-sums)) (returns State) | +| `.deviationBetween(ZonedDateTime, ZonedDateTime)` | Gets the standard deviation of the state of the given Item between certain points in time. This method uses a time-weighted calculation (see [below](#time-weighted-calculations-riemann-sums)) (returns State) | | `.getAllStatesSince(ZonedDateTime)` | Gets all persisted State changes for an Item since a certain point in time (returns Iterable) | | `.getAllStatesUntil(ZonedDateTime)` | Gets all persisted State changes for an Item until a certain point in time (returns Iterable) | | `.getAllStatesBetween(ZonedDateTime, ZonedDateTime)` | Gets all persisted State changes for an Item between certain points in time (returns Iterable) | @@ -329,6 +332,45 @@ Some extensions return a HistoricItem object. It represents the state of a persisted item at a certain point in time. The most useful methods of the HistoricItem object returned by some queries, are `.getState()` and `.getTimestamp()` +### Time-weighted calculations - Riemann Sums + +A number of extensions (`riemannSum`, `average`, `variance` and `deviation`) use time-weighted calculations. +A Riemann Sum is an approximation of the integral of a continuous function represented by discrete values. +Each of the following extensions uses a Riemann Sum as a base for its calculation: + +- `average` = `riemannSum` / total duration +- `variance` = sum(value - `average`)^2 / count(values) +- `deviation` = sqrt(`variance`) + +The `riemannSum` calculation is especially useful when doing calculations such as electricity consumption or production over time (e.g. in kWh) from momentary power readings (e.g. in W). + +The `riemannSum` extension differs from the `sum` extension in that `sum` will just add values without considering the time interval between values. +It will therefore result in a sum with the same unit as the individual values. +Each value will have the same weight in the calculation. + +There are multiple types of approximation possible when calculating Riemann Sums. +The type can be explicitly set on all extensions doing time-weighted calculations (`riemannSum`, `average`, `variance` and `deviation`) with an extra parameter at the end (before the persistence service parameter). +Possible values for the type are: + +| `Riemann.LEFT` | `Riemann.RIGHT` | `Riemann.TRAPEZOIDAL` | `Riemann.MIDPOINT` | +|---------------------------------------------------|---------------------------------------------------|---------------------------------------------------|---------------------------------------------------| +| ![left](images/riemann-left.svg) | ![right](images/riemann-right.svg) | ![trapezoidal](images/riemann-trapezoidal.svg) | ![midpoint](images/riemann-midpoint.svg) | + +- `RiemannType.LEFT`: takes the persisted value at the start of the bucket to represent the value for the whole bucket. +This is most useful when there is a `everyChange` persistence strategy and the values represent a step function. An example would be dynamic electricity rates, as they will effectively be constant inside the bucket. +- `RiemannType.RIGHT`: takes the persisted value at the end of the bucket as an approximation for the value in the full preceding bucket. +- `RiemannType.TRAPEZOIDAL`: takes the average of the persisted value at the start end the end of the bucket, effectively making a linear interpolation to fit the curve. +This type is most useful when the real values change continuously. It can be used for any persistence strategy and any interval. +- `RiemannType.MIDPOINT`: uses 3 persisted values and uses the middle of the values as an approximation for the value halfway in the interval between the middle of point 1 and 2 and the middle of point 2 and 3. +This is the best approximation when the real values change continuously, the persistence intervals are short and the bucket sizes between persisted values are relatively constant. + +The default when no type is provided is `RiemannType.LEFT`. + +A Riemann sum is always calculated using seconds as unit for time. +As an example, the Riemann sum of power values in `kW` will result in an energy measurement in `kWs`. +You can rely on framework functionality to convert to the appropriate unit (e.g. `kWh`), or do an explicit conversion. +If you use plain `Number` items and don't use units, be aware of this time multiplication factor. + ### Examples To persist an Item called `Lights` in an rrd4j database, you would enter the following: @@ -342,7 +384,21 @@ lastChange = (lastChange !== null) ? lastChange : now ``` To get the average temperature over the last 5 minutes from the Item called `Temperature` in the influxdb persistence service, you would use: -`Temperature.averageSince(now.minusMinutes(5), "influxdb")` + +```java +Temperature.averageSince(now.minusMinutes(5), "influxdb")` +``` + +To calculate the total energy consumption over the last month from an Item called `Power` in the jdbc persistence service, you can do: + +```java +var today = now.truncatedTo(ChronoUnit.DAYS) +var currentMonthStart = today.withDayOfMonth(1) +var lastMonthStart = today.minusMonths(1).withDayOfMonth(1) +var consumption = Power.riemannSumBetween(lastMonthStart, currentMonthStart, RiemannType.TRAPEZOIDAL, "jdbc")` +``` + +If `Power` is a `QuantityType` Item, `consumption` will be of `QuantityType`. #### Time-weighted averages