Commit Graph

249 Commits (b4bb7c38ceffbfff53fa2da590e3a946b9063c71)

Author SHA1 Message Date
Erik Montnemery e16a8063a5
Compile statistics for energy sensors (#50829)
* Compile statistics for energy sensors

* Update tests

* Rename abs_value to state

* Tweak

* Recreate statistics table

* Pylint

* Try to fix test

* Fix statistics for multiple energy sensors

* Fix energy statistics when last_reset is not set
2021-05-20 13:05:15 +02:00
J. Nick Koston e7f7e61e88
Ensure a wal checkpoint is scheduled nightly (#50746) 2021-05-17 16:27:51 -05:00
J. Nick Koston 72288710ca
Increase the sqlite cache size from ~2MiB to 8MiB (#50747) 2021-05-17 19:42:12 +02:00
Erik Montnemery 89dd3292ba
Initial draft of statistics (#49852) 2021-05-16 10:23:37 -07:00
tkdrob 2d5f5bfa9f
Add targets and selectors for services (P-R) (#50628) 2021-05-14 22:07:17 -07:00
Erik Montnemery 973f59e423
Refactor history component (#50287)
* Refactor history component

* Update tests

* Address review comments

* Correct deprecated functions
2021-05-11 09:21:57 +02:00
J. Nick Koston 906de23087
Bump sqlalchemy to 1.4.13 (#50138) 2021-05-05 20:08:48 -07:00
J. Nick Koston 1c8d9ca68b
Check exception causes for matching strings during recorder migration (#49999) 2021-05-02 20:57:42 -07:00
Paulus Schoutsen 1bd9826684
Handle different entity_id formats (#49969) 2021-05-01 20:30:28 -07:00
J. Nick Koston b27e9e376d
Use StaticPool for recorder and NullPool for all other threads with sqlite3 (#49693) 2021-04-26 19:20:31 -10:00
J. Nick Koston 376b787e4d
Skip recorder commit if there is nothing to do (#49614) 2021-04-25 12:05:49 +02:00
jan iversen c351098f04
HomeAssistantType —> HomeAssistant for Integrations p* - s* (#49558) 2021-04-22 19:58:02 +02:00
J. Nick Koston c10836fcee
Upgrade to sqlalchemy 1.4.11 (#49538) 2021-04-21 20:29:36 -10:00
J. Nick Koston 6d137d2316
Increase recorder test coverage (#49362)
Co-authored-by: Martin Hjelmare <marhje52@gmail.com>
2021-04-19 08:22:38 -07:00
J. Nick Koston e24f5831a2
Force recorder shutdown at final write event (#49145)
* Force recorder shutdown at EVENT_HOMEASSISTANT_FINAL_WRITE

* remove unreachable

* remove unreachable

* simplify

* cancel in async
2021-04-19 11:24:17 +02:00
Franck Nijhof 055cdc64c0
Add support for IoT class in manifest (#46935) 2021-04-15 10:21:38 +02:00
J. Nick Koston 53853f035d
Prevent calling stop or restart services during db upgrade (#49098) 2021-04-12 17:18:38 -07:00
J. Nick Koston 9368891b1b
Live db migrations and recovery (#49036)
Co-authored-by: Paulus Schoutsen <paulus@home-assistant.io>
2021-04-11 20:43:54 -10:00
Raman Gupta 3ca69f5568
Raise an exception when event_type exceeds the max length (#48115)
* raise an exception when event_type exceeds the max length that the recorder supports

* add test

* use max length constant in recorder

* update config entry reloaded service name

* remove exception string function because it's not needed

* increase limit to 64 and revert event name change

* fix test

* assert exception args

* fix test

* add comment about migration
2021-04-08 20:46:28 +02:00
Laszlo Magyar c7e4857d2c
Let recorder deal with event names longer than 32 chars (#47748) 2021-04-08 07:08:49 -10:00
Stefan Agner 815db999da
Use microsecond precision for datetime values on MariaDB/MySQL (#48749)
Co-authored-by: Paulus Schoutsen <balloob@gmail.com>
2021-04-07 09:13:55 +02:00
Franck Nijhof 3aed84560f
Merge of nested IF-IF cases - O-R (#48371) 2021-03-27 10:38:57 +01:00
J. Nick Koston c820dd4cb5
Have pylint warn when user visible log messages do not start with capital letter or end with a period (#48064)
Co-authored-by: Martin Hjelmare <marhje52@gmail.com>
2021-03-19 09:26:36 -05:00
Marc Mueller a3cd1854f6
Update typing 12 (#48073) 2021-03-18 14:31:38 +01:00
J. Nick Koston 61a2460c87
Improve error reporting in recorder purge test (#47929) 2021-03-14 19:46:21 -07:00
J. Nick Koston 33c4eb3434
Log the full exception when the recorder fails to setup (#47770) 2021-03-11 21:52:04 -08:00
Marc Mueller 92852b9c10
Add apply_filter attribute to recorder.purge service (#45826) 2021-03-11 17:03:30 -10:00
Marc Mueller 10848b9bdf
Recorder improvements (#47739) 2021-03-11 07:52:07 -10:00
CurrentThread 7c8851264f
Use LONGTEXT column instead of TEXT for MySQL/MariaDB and migrate existing databases (#47026) 2021-03-10 08:12:58 -10:00
J. Nick Koston a060acc2b1
Fix recorder with MSSQL (#46678)
Co-authored-by: Marc Mueller <30130371+cdce8p@users.noreply.github.com>
2021-03-09 11:16:19 -10:00
adrian-vlad 44293a3738
Add enable and disable services for recorder (#45778) 2021-02-24 07:26:05 -06:00
Franck Nijhof ea4bbd771f
Add service names to previously enriched services (#46929)
Co-authored-by: Tobias Sauerwein <cgtobi@users.noreply.github.com>
2021-02-23 14:10:13 +01:00
J. Nick Koston 9b69549f73
Recover and restart the recorder if the sqlite database encounters corruption while running (#46612) 2021-02-19 21:26:24 -10:00
J. Nick Koston 22dbac259b
Ensure recorder shuts down cleanly on restart before startup is finished (#46604) 2021-02-19 22:18:21 -08:00
Franck Nijhof 8b69608242
Add selectors to Browser, Recorder, Shopping List service definitions (#46749) 2021-02-18 17:06:25 +01:00
J. Nick Koston c9df42b69a
Add support for pre-filtering events to the event bus (#46371) 2021-02-14 09:42:55 -10:00
J. Nick Koston dc26fd5149
Ensure creating an index that already exists is forgiving for postgresql (#46185)
Unlikely sqlite and mysql, postgresql throws ProgrammingError instead
of InternalError or OperationalError when trying to create an index
that already exists.
2021-02-08 23:22:38 +01:00
Fabian Affolter 60d4dadcb6
Upgrade sqlalchemy to 1.3.23 (#45845) 2021-02-01 16:03:51 -10:00
J. Nick Koston 12af87bc6e
Add index to old_state_id column for postgres and older databases (#44757)
* Add index to old_state_id column for older databases

The schema was updated in #43610 but the index was not
added on migration.

* Handle postgresql missing ondelete

* create index first
2021-01-04 10:51:44 +01:00
Fabian Affolter 12b7b2098d
Upgrade sqlalchemy to 1.3.22 (#44698) 2020-12-31 14:28:15 -10:00
Ville Skyttä de04a1ed67
Enable more Bandit tests (#44307)
https://bandit.readthedocs.io/en/latest/plugins/index.html#complete-test-plugin-listing
2020-12-19 12:35:13 +01:00
J. Nick Koston 2cbb93be43
Always keep the current recorder run when purging (#43733) 2020-11-30 09:13:50 +01:00
J. Nick Koston 1162d9a752
Create tables with a charset that can hold all expected data under mysql (#43732)
By default these tables are created with utf8 which can only hold 3 bytes. This
meant that all emjoi would trigger a MySQLdb._exceptions.OperationalError because
they are 4 bytes.

This will only fix the issue for users who recreate their tables.
2020-11-28 23:54:05 +01:00
moinmoin-sh 337b8d279e
Ensure MariaDB/MySQL can be purged and handle states being deleted out from under the recorder (#43610)
* MariaDB doesn't purge #42402

This addresses  home-assistant#42402
Relationships within table "states" and between tables "states" and "events " home-assistant#40467 prevent the purge from working correctly. The database increases w/o any purge.
This proposal sets related indices to NULL and permits deleting of rows.
Further explanations can be found here home-assistant#42402
This proposal also allows to purge the tables "events" and "states" in any order.

* Update models.py

Corrected for Black style requirements

* Update homeassistant/components/recorder/models.py

Co-authored-by: J. Nick Koston <nick@koston.org>

* Add the options to foreign key constraints

* purge old states when database gets deleted out from under us

* pylint

Co-authored-by: J. Nick Koston <nick@koston.org>
2020-11-28 08:42:29 -10:00
Franck Nijhof 5dbb5f12eb
Upgrade sqlalchemy to 1.3.20 (#41765) 2020-10-13 08:43:58 -05:00
J. Nick Koston 33b548e247
Reduce ORM overhead when the old state was already written to the database (#41736)
We can avoid processing the relationship when the old state was already
written to the database which will common case with a commit interval
of 1s. Since we already know the value for old_state_id we can use it
instead of asking sqlalchemy to process the relationship at flush/commit
time which can significantly speed up sqlalchemy's _emit_insert_statements
implementation.
2020-10-13 15:34:46 +02:00
Matthias Alphart 93a9a11065
Add config validator helper positive_float (#41640) 2020-10-11 22:04:49 +02:00
J. Nick Koston 1c431aa7bd
Set created field when creating db events and states (#41523)
Avoids thousands of utcnow calls when the session
is commited on busy systems to fill in the field
default.
2020-10-09 09:31:17 +02:00
J. Nick Koston 4ab02cb9bc
Ensure recorder commit can retry after encountering invalid data (#41426) 2020-10-08 09:15:25 +02:00
J. Nick Koston 113d738fa2
Reduce orm overhead by grouping object expiration (#41394) 2020-10-07 15:35:48 +02:00