Commit Graph

374 Commits (073fb40b79cf8aa06790fdceb23b6857db888c99)

Author SHA1 Message Date
J. Nick Koston 3c10ac308d
Fix migration to schema v25 with Postgresql (#68426) 2022-03-20 21:49:11 +01:00
J. Nick Koston 6ffef8373b
Add shutdown guard to Recorder pool in case there is no connection (#68407) 2022-03-20 01:28:44 -10:00
J. Nick Koston 3150915cb7
Convert unindexed domain queries to entity_id queries (#68404) 2022-03-20 01:28:17 -10:00
J. Nick Koston 816695cc96
Avoid selecting attributes in the history api when `no_attributes` is passed (#68352) 2022-03-19 23:47:22 -10:00
J. Nick Koston 0c0df07c52
Avoid hashing attributes when they are already in the cache (#68395) 2022-03-19 23:33:37 -07:00
J. Nick Koston 7ee647cc78
Fix FOREIGN KEY constraint failed when removing state_attributes (#68364) 2022-03-19 14:21:28 -07:00
J. Nick Koston 9864090e0b
Cache parsing attr in LazyState (#68232) 2022-03-18 07:44:37 -07:00
J. Nick Koston 9215702388
Separate attrs into another table (reduces database size) (#68224) 2022-03-18 00:23:13 -10:00
J. Nick Koston bc862e97ed
Use a dedicated executor pool for database operations (#68105)
Co-authored-by: Erik Montnemery <erik@montnemery.com>
Co-authored-by: Franck Nijhof <git@frenck.dev>
2022-03-17 23:09:01 -10:00
J. Nick Koston 95f20500ca
Commit any pending changes before running non-EventTasks in the recorder (#68287) 2022-03-17 21:33:22 -07:00
J. Nick Koston b18096fc54
Remove unused columns from states/events tables (#68078) 2022-03-13 23:44:55 -07:00
Franck Nijhof 36acb09cae
Update sqlalchemy to 1.4.32 (#68075) 2022-03-12 22:52:47 -10:00
Marc Mueller 596644d715
Fix typo [recorder] (#66879) 2022-02-19 16:04:20 +01:00
Marc Mueller cb736eaeaf
Add type ignore error codes [recorder] (#66780) 2022-02-18 10:37:38 +01:00
Erik Montnemery 39ed628cca
Suppress unwanted error messages during recorder migration (#66004) 2022-02-07 15:46:40 -08:00
Erik Montnemery 910b1f1ec8
Speed up deletion of duplicated statistics (#66014) 2022-02-07 09:11:52 -08:00
Christopher Masto 6d4df93bc7
Correct description of entity_globs (#65805) 2022-02-06 18:36:38 +01:00
Erik Montnemery 1f8e8926fe
Only remove duplicated statistics on error (#65653) 2022-02-04 11:31:12 -08:00
Erik Montnemery 020953e943
Improve recorder migration for PostgreSQL when columns already exist (#65680) 2022-02-04 10:55:28 -08:00
Erik Montnemery 41ab12cb88
Don't use shared session during recorder migration (#65672) 2022-02-04 09:55:11 -08:00
Erik Montnemery ac7662c82d
Remove limit of amount of duplicated statistics (#65641) 2022-02-04 14:49:45 +01:00
Franck Nijhof 7a2b699371
Update codespell to 2.1.0 (#64661) 2022-01-21 22:05:24 +01:00
Stefan Agner 0a9927d18e
Avoid locking the database for non-SQLite backends (#63847)
* Avoid locking the database for non-SQLite backends

Currently we only have a lock implementation for SQLite. Just return
success for all other databases as they are not expected to store data
in the config directory and the caller can assume that a backup can
be safely taken.

This fixes `RuntimeError: generator didn't yield` errors when creating
a backup with the current Supervisor dev builds.
2022-01-11 16:17:56 +01:00
epenet a19c95e4bd
Ensure service calls are typed [o-r] (#62920)
Co-authored-by: epenet <epenet@users.noreply.github.com>
2021-12-28 13:10:17 +01:00
Erik Montnemery 2f8e65a9b0
Store deleted duplicated statistics in .storage (#62574) 2021-12-22 14:27:56 +01:00
Erik Montnemery 48b3d6e1c0
Save original + duplicate pairs when deleting duplicated statistics (#62498) 2021-12-21 15:24:36 +01:00
Erik Montnemery cab2a74b5f
Don't pollute config dir with deleted duplicated statistics (#62489) 2021-12-21 14:27:35 +01:00
Paulus Schoutsen 77829e397b
Don't log DB connection string on error (#61927) 2021-12-15 20:54:57 +01:00
Erik Montnemery bceeaec2f8
Remove duplicated statistics rows (#61146)
* Remove duplicated statistics

* Fix misleading docstring

* Pylint knows best

* Correct test

* Oops

* Prevent insertion of duplicated statistics

* Tweak

* pylint

* Add models_schema_23.py

* Tweak
2021-12-13 14:15:36 +01:00
Michael 83989d7b40
Use relative imports [L-R] (#61575) 2021-12-13 00:24:46 +01:00
Erik Montnemery dc5888ab4a
Correct recorder.statistics.get_last_statistics (#61421) 2021-12-10 10:09:29 -08:00
Stefan Agner a13ae85982
Introduce only_supervisor for @websocket_api.ws_require_user() (#61298) 2021-12-08 16:49:35 -08:00
Erik Montnemery f30eb05870
Refactor recorder queue handling (#61161)
* Refactor recorder queue handling

* Address pylint's concerns

* Implement workaround for mypy bug

* Address review comments
2021-12-08 16:54:26 +01:00
Stefan Agner f0006b92be
Allow to lock SQLite database during backup (#60874)
* Allow to set CONF_DB_URL

This is useful for test which need a custom DB path.

* Introduce write_lock_db helper to lock SQLite database

* Introduce Websocket API which allows to lock database during backup

* Fix isort

* Avoid mutable default arguments

* Address pylint issues

* Avoid holding executor thread

* Set unlock event in case timeout occures

This makes sure the database is left unlocked even in case of a race
condition.

* Add more unit tests

* Address new pylint errors

* Lower timeout to speedup tests

* Introduce queue overflow test

* Unlock database if necessary

This makes sure that the test runs through in case locking actually
succeeds (and the test fails).

* Make DB_LOCK_TIMEOUT a global

There is no good reason for this to be an argument. The recorder needs
to pick a sensible value.

* Add Websocket Timeout test

* Test lock_database() return

* Update homeassistant/components/recorder/__init__.py

Co-authored-by: Erik Montnemery <erik@montnemery.com>

* Fix format

Co-authored-by: J. Nick Koston <nick@koston.org>
Co-authored-by: Erik Montnemery <erik@montnemery.com>
2021-12-07 13:16:24 +01:00
Erik Montnemery d980ca7e04
Correct recorder migration._add_columns for PostgreSQL (#60547) 2021-11-29 18:33:25 +01:00
J. Nick Koston 18e822b7b6
Bump sqlalchemy to 1.4.27 (#60383) 2021-11-25 19:55:17 -08:00
Erik Montnemery 4a5238efa5
Add support for calculating daily and monthly fossil energy consumption (#59588) 2021-11-22 09:10:54 -08:00
Erik Montnemery bcd9f3c05f
Correct end time for monthly statistics summary (#59551)
* Correct end time for monthly statistics summary

* Add tests
2021-11-12 15:01:40 +01:00
Erik Montnemery 65b1f0d9eb
Minor refactor of energy validator (#58209) 2021-11-10 22:38:15 -08:00
J. Nick Koston 751098c220
Bump sqlalchemy to 1.4.26 (#59527) 2021-11-10 20:01:59 -08:00
Erik Montnemery 470b01e4ce
Move async_migration_in_progress (#59087) 2021-11-05 10:40:56 +01:00
Erik Montnemery 491e62792b
Correct rescheduling of ExternalStatisticsTask (#59076) 2021-11-04 18:35:43 +01:00
Erik Montnemery 4c5aca93df
Add recorder status WS API (#58989)
* Add recorder status WS API

* Rename recorder/status to recorder/info

* Silence pylint

* Improve tests

* Address review comments

* Tweak

* Try to fix tests

* Try to debug flaky tests

* Try to fix tests

* Revert changes to async_migration_in_progress

* Try to fix tests

* Remove debug prints

* Apply suggestions from code review
2021-11-04 16:46:45 +01:00
Erik Montnemery be4e9f91b6
Change minimum supported SQLite version to 3.31.0 (#59073) 2021-11-04 16:34:35 +01:00
Erik Montnemery 2f3dea30e2
Correct migration to recorder schema 22 (#59048) 2021-11-04 10:29:10 +01:00
Erik Montnemery 34e5596375
Use async_track_utc_time_change to schedule short term statistics (#58903) 2021-11-01 10:49:54 -07:00
Marc Mueller 84618fa831
Use assignment expressions 30 (#58714) 2021-10-30 16:33:42 +02:00
Erik Montnemery dfa50a842a
Simplify recorder PgSQL version checks (#58533) 2021-10-27 16:05:40 +02:00
Erik Montnemery ac5e32d648
Corrections for external statistics (#58469) 2021-10-26 14:05:45 +02:00
Erik Montnemery e9ba5f3b4b
Warn when recorder connects to an unsupported database (#58161) 2021-10-26 13:41:59 +02:00