Commit Graph

322 Commits (c7ff6eb5eee0659fb8b4b4d11ee204963ad3edc5)

Author SHA1 Message Date
Erik Montnemery b301ab25a3
Purge short term statistics (#58028)
* Purge short term statistics

* Less meep

* Add tests
2021-10-20 16:00:59 +02:00
Erik Montnemery e3534eec87
Report orphaned statistics in statistic validation (#57324) 2021-10-20 15:57:22 +02:00
Erik Montnemery 9a26a8cfd8
Add support for daily and monthly statistics (#57576)
* Add support for daily and monthly statistics

* Remove debug code

* Format code

* Don't use dateutil package

* Remove 2 TODOs

* Remove TODO

* Add comments
2021-10-19 08:29:23 +02:00
Marc Mueller aa7dc78a1e
Use assignment expressions 11 (#57792) 2021-10-17 20:15:48 +02:00
Erik Montnemery 8ef8838801
Correct detection of row_number support for MariaDB (#57663) 2021-10-14 11:19:39 -07:00
Erik Montnemery ee98849360
Always include start point for statistics (#57182) 2021-10-12 10:39:36 -07:00
Erik Montnemery 0139bfa749
Detect if mysql and sqlite support row_number (#57475) 2021-10-11 21:17:18 -07:00
Erik Montnemery be61009030
Correct SQL query generated by get_metadata_with_session (#57225)
Co-authored-by: Franck Nijhof <git@frenck.dev>
2021-10-07 13:23:23 -07:00
Erik Montnemery 947ae23749
Add home-assistant/core as codeowner for recorder (#57224) 2021-10-07 08:56:43 +02:00
Erik Montnemery 8337baa354
Correct migration to recorder schema 18 (#57165) 2021-10-06 13:29:42 +02:00
Erik Montnemery 8567aa9e13
Evict purged states from recorder's old_state cache (#56877)
Co-authored-by: J. Nick Koston <nick@koston.org>
2021-10-04 08:21:40 -07:00
Erik Montnemery dfb3a0c528
Correct database migration to schema version 22 (#56848) 2021-09-30 14:11:00 -07:00
Erik Montnemery 6af1a835e6
Optimize statistics generation (#56821)
* Optimize statistics generation

* pylint
2021-09-30 17:14:36 +02:00
Erik Montnemery a966714032
Minor cleanup of recorder statistics code (#55339) 2021-09-29 11:42:39 -07:00
Erik Montnemery 40ecf22bac
Remove automatic splitting of net meters from statistics (#56772) 2021-09-29 17:20:17 +02:00
Erik Montnemery 00651a4055
Optimize _get_states_with_session (#56734)
* Optimize _get_states_with_session

* Move custom filters to derived table

* Remove useless derived table

* Filter old states after grouping

* Split query

* Add comments

* Simplify state update period criteria

* Only apply custom filters if we didn't get an include list of entities

Co-authored-by: J. Nick Koston <nick@koston.org>
2021-09-29 17:08:27 +02:00
Marc Mueller 565a9fea6b
Import Callable from collections.abc (2) (#56776) 2021-09-29 14:06:51 +02:00
Erik Montnemery 52e9f76f94
Tweak DB migration to schema version 21 (#56767) 2021-09-29 11:25:50 +02:00
Erik Montnemery 552485bb05
Tweak list_statistic_ids (#55845)
Co-authored-by: Paulus Schoutsen <balloob@gmail.com>
2021-09-28 00:05:26 -07:00
Erik Montnemery 8ef123259e
Add WS API for updating unit_of_measurement in statistics metadata (#56184)
* Add WS API for updating statistics metadata

* Update homeassistant/components/recorder/websocket_api.py

Co-authored-by: Bram Kragten <mail@bramkragten.nl>

* Update homeassistant/components/recorder/websocket_api.py

Co-authored-by: Paulus Schoutsen <balloob@gmail.com>

* Fix typo

Co-authored-by: Bram Kragten <mail@bramkragten.nl>
Co-authored-by: Paulus Schoutsen <balloob@gmail.com>
2021-09-28 00:43:29 +02:00
Erik Montnemery 5976f898da
Add WS API for removing statistics for a list of statistic_ids (#55078)
* Add WS API for removing statistics for a list of statistic_ids

* Refactor according to code review, enable foreign keys support for sqlite

* Adjust tests

* Move clear_statistics WS API to recorder

* Adjust tests after rebase

* Update docstring

* Update homeassistant/components/recorder/websocket_api.py

Co-authored-by: Paulus Schoutsen <paulus@home-assistant.io>

* Adjust tests after rebase

Co-authored-by: Paulus Schoutsen <paulus@home-assistant.io>
2021-09-27 23:30:13 +02:00
Erik Montnemery fe66d6295c
Improve migration to recorder schema version 21 (#56204) 2021-09-27 19:31:40 +02:00
Erik Montnemery 64393b462d
Add migration for 5-minute statistics (#56585)
* Add migration for 5-minute statistics

* Tweaks
2021-09-24 09:19:22 +02:00
Erik Montnemery f0a4a89d21
Add comments to recorder statistics code (#56545)
* Add comments to recorder statistics code

* Revert accidental change of list_statistic_ids
2021-09-23 13:14:45 +02:00
Erik Montnemery 92253f5192
Minor refactoring of periodic statistics (#56492) 2021-09-22 13:31:33 -07:00
Marcin Ciupak b9ffd74db5
Fix recorder Oracle DB models (#55564)
* Fix recorder models for Oracle DB

* Fix StatisticsRuns

* Update migration for Oracle Identity columns.

* Update migration logic

Migration to schema version 22 done only for engine dialect oracle

* Add missing table check in schema 22 migration
2021-09-21 00:38:42 +02:00
Erik Montnemery 8c5efafdd8
Add 5-minute statistics for sensors (#56006)
* Add 5-minute statistics for sensors

* Address pylint issues

* Black

* Apply suggestion from code review

* Apply suggestions from code review

* Improve tests
2021-09-16 10:57:15 +02:00
Erik Montnemery d899d15a1e
Add statistics validation (#56020)
* Add statistics validation

* Remove redundant None-check

* Move validate_statistics WS API to recorder

* Apply suggestion from code review
2021-09-13 13:44:22 +02:00
Erik Montnemery d2a9f7904a
Include end time of statistics data points in API response (#56063)
* Include end time of statistics data points in API response

* Correct typing

* Update tests
2021-09-13 10:02:24 +02:00
Erik Montnemery 2a8121bdcd
Really change character set of statistics_meta table to utf8 (#56029) 2021-09-09 22:55:51 +02:00
Erik Montnemery 88dbc6373f
Make sure character set of events, states tables is utf8 (#56012)
* Make sure character set of events, states tables is utf8

* Pylint

* Apply suggestions from code review
2021-09-09 19:26:28 +02:00
Erik Montnemery a47532c69b
Change character set of statistics_meta table to utf8 (#56011) 2021-09-09 17:24:20 +02:00
Erik Montnemery 80fd330479
Add sum_decrease and sum_increase statistics (#55850) 2021-09-08 23:35:53 -07:00
Erik Montnemery 27764e9985
Fix handling of imperial units in long term statistics (#55959) 2021-09-08 08:08:48 -07:00
Erik Montnemery 22e6ddf8df
Do not let one bad statistic spoil the bunch (#55942) 2021-09-08 07:55:40 -07:00
Erik Montnemery 7e70252de5
Handle statistics for sensor with changing state class (#55316) 2021-08-27 16:18:49 +02:00
Erik Montnemery 6d4a47a53d
Fix double precision float for postgresql (#55249) 2021-08-26 10:06:53 +02:00
Erik Montnemery ffbd2d79c8
Generate statistics for all sensors with a supported state_class (#54882)
* Generate statistics for all sensors

* Fix bugs, add tests

* Address review comments

* Cleanup warnings

* Simplify tests

* Simplify selection of statistics

* Fix tests
2021-08-25 13:00:35 +02:00
Erik Montnemery 8877f37da0
Fix statistics for sensors setting last_reset (#55136)
* Re-add state_class total to sensor

* Make energy cost sensor enforce state_class total_increasing

* Drop state_class total

* Only report energy sensor issues once
2021-08-24 17:02:34 +02:00
Erik Montnemery 0624859bf4
Set statistics columns to double precision (#55053) 2021-08-24 11:18:59 +02:00
J. Nick Koston 8d69475d71
Fix recorder shutdown race and i/o in event loop (#54979) 2021-08-21 21:38:02 +02:00
J. Nick Koston aa7c72a8b5
Bump sqlalchemy to 1.4.23 (#54980)
- Changelog: https://docs.sqlalchemy.org/en/14/changelog/changelog_14.html#change-1.4.23
2021-08-21 13:00:39 -04:00
Erik Montnemery 32e297f4a0
Compile missing statistics (#54690) 2021-08-19 22:10:45 -07:00
Paulus Schoutsen 2f77b5025c
Add energy validation (#54567) 2021-08-18 11:21:51 -07:00
Erik Montnemery e1926caeb9
Remove STATE_CLASS_TOTAL and last_reset from sensor (#54755)
* Remove STATE_CLASS_TOTAL

* Update mill sensor

* Update tests

* Kill last_reset

* Return ATTR_LAST_RESET to utility_meter

* Update energy cost sensor

* Restore last_reset for backwards compatibility

* Re-add and update deprecation warning

* Update tests

* Fix utility_meter

* Update EnergyCostSensor

* Tweak

* Fix rebase mistake

* Fix test
2021-08-18 10:03:27 +02:00
Bram Kragten e23750b2a4
Add device class `gas` and enable statistics for it (#54110)
Co-authored-by: Martin Hjelmare <marhje52@gmail.com>
Co-authored-by: Erik Montnemery <erik@montnemery.com>
2021-08-11 09:58:19 -07:00
Paulus Schoutsen 557cc792e9
Fix SQLAlchemy test warnings (#54116) 2021-08-08 20:33:47 -07:00
Franck Nijhof c875ff8648
Store JSON in database in compact format (#53364)
* Store JSON in database in compact format

* Fix logbook
2021-07-22 12:05:59 -10:00
Ville Skyttä 470f2dd73f
Upgrade pyupgrade to 2.21.2, apply its changes (#52987) 2021-07-19 10:46:09 +02:00
Erik Montnemery e541bcd54d
Update statistics meta data on entity_id change (#52755) 2021-07-14 13:23:11 +02:00