Commit Graph

238 Commits (1aef60c81cfdc73231f9c6ddfed33e62377da6eb)

Author SHA1 Message Date
Erik Montnemery fd6ffef52f
Support non-live database migration (#72433)
* Support non-live database migration

* Tweak startup order, add test

* Address review comments

* Fix typo

* Clarify comment about promoting dependencies

* Tweak

* Fix merge mistake

* Fix some tests

* Fix additional test

* Fix additional test

* Adjust tests

* Improve test coverage
2022-07-22 15:11:34 +02:00
Erik Montnemery 606d544157
Use recorder get_instance function to improve typing (#75567) 2022-07-22 11:58:26 +02:00
Thibault Cohen 1d7d2875e1
Add websocket command recorder/import_statistics (#73937)
* Expose ws_add_external_statistics in websocket API

* Refactor

* Add tests

* Improve test coverage

Co-authored-by: Thibault Cohen <titilambert@users.noreply.github.com>
Co-authored-by: Erik <erik@montnemery.com>
2022-07-21 12:36:49 +02:00
J. Nick Koston a8349a4866
Adjust entity filters to make includes stronger than excludes (#74080)
* Adjust entity filters to make includes stronger than excludes

Fixes #59080

* adjust test for stronger entity glob includes

* sync with docs
2022-06-28 11:42:51 -05:00
J. Nick Koston 00a79635c1
Revert "Remove sqlalchemy lambda_stmt usage from history, logbook, and statistics (#73191)" (#73917) 2022-06-23 13:59:55 -05:00
J. Nick Koston 7ae8bd5137
Remove sqlalchemy lambda_stmt usage from history, logbook, and statistics (#73191) 2022-06-07 16:15:50 -07:00
Erik Montnemery 5f2b4001f3
Separate recorder database schema from other classes (#72977)
* Separate recorder database schema from other classes

* fix logbook imports

* migrate new tests

* few more

* last one

* fix merge

Co-authored-by: J. Nick Koston <nick@koston.org>
2022-06-07 14:41:43 +02:00
J. Nick Koston de2e9b6d77
Fix state_changes_during_period history query when no entities are passed (#73139) 2022-06-06 12:50:52 -07:00
J. Nick Koston c66b000d34
Reduce branching in generated lambda_stmts (#73042) 2022-06-05 21:13:31 -07:00
J. Nick Koston 5b31414225
Fix misalignments between sql based filtering with the entityfilter based filtering (#72936) 2022-06-02 20:52:53 -07:00
J. Nick Koston f79e5e002b
Ensure recorder shuts down when its startup future is canceled out from under it (#72866) 2022-06-01 22:13:09 -07:00
J. Nick Koston c6e56c26b3
Fix logbook not setting up with an recorder filter that has empty fields (#72869) 2022-06-01 22:12:00 -07:00
J. Nick Koston d8b0376942
Fix purge of legacy database events that are not state changed (#72815) 2022-06-01 11:56:06 +02:00
J. Nick Koston 587fd05603
Make logbook inherit the recorder filter (#72728) 2022-05-30 17:34:32 -07:00
J. Nick Koston 040e120101
Fix recorder system health when the db_url is lacking a hostname (#72612) 2022-05-27 10:32:26 -07:00
Erik Montnemery 23bd64b7a2
Prevent duplication of statistics metadata (#71637)
* Prevent duplication of statistics metadata

* Add models_schema_28.py

* Handle entity renaming as a recorder job

* Improve tests
2022-05-24 15:34:46 +02:00
J. Nick Koston 4dbc1ed7a7
Add dialect (database engine) and version to recorder system health data (#72339) 2022-05-22 21:29:30 -07:00
J. Nick Koston 9c3f949165
Add live streaming logbook websocket endpoint (#72258)
Co-authored-by: Paulus Schoutsen <paulus@home-assistant.io>
2022-05-22 14:57:54 -05:00
J. Nick Koston 1001f9e39a
Fix last state in history minimal respones when all the states at the end are skipped (#72203) 2022-05-19 20:58:32 -07:00
Erik Montnemery edd7a3427c
Remove support for databases without ROW_NUMBER (#72092) 2022-05-18 21:52:38 -05:00
Erik Montnemery 037f6947d8
Fail recorder setup with unsupported dialect or version (#70888) 2022-05-18 16:52:46 +02:00
J. Nick Koston a4c1bcefb9
Tune sqlite based on configured settings (#72016) 2022-05-17 18:12:15 -05:00
J. Nick Koston a70e2a33dc
Fixing purging legacy rows and improve performance (#71916) 2022-05-15 21:25:07 -07:00
J. Nick Koston 98809675ff
Convert history queries to use lambda_stmt (#71870)
Co-authored-by: Paulus Schoutsen <paulus@home-assistant.io>
2022-05-15 10:47:29 -05:00
J. Nick Koston 8c2743bb67
Avoid storing last_changed in the database if its the same as last_updated (#71843) 2022-05-14 12:06:31 -07:00
J. Nick Koston a8f1dda004
Use ciso8601 for parsing MySQLdb datetimes (#71818)
* Use ciso8601 for parsing MySQLDB datetimes

The default parser is this:

5340191feb/MySQLdb/times.py (L66)

* tweak

* tweak

* add coverage for building the MySQLdb connect conv param
2022-05-13 19:26:09 -05:00
J. Nick Koston 7ab4960b1e
Restore v23 stats migration tests (#71743) 2022-05-12 21:14:02 -04:00
J. Nick Koston 1d9fb4bca8
Fix process_datetime_to_timestamp and add test coverage (#71755) 2022-05-12 17:12:50 -07:00
J. Nick Koston e2cef55162
Add history/history_during_period websocket endpoint (#71688) 2022-05-11 17:52:22 -05:00
J. Nick Koston 222baa53dd
Make database access in the eventloop raise an exception (#71547) 2022-05-09 15:22:08 -05:00
J. Nick Koston 7c9c0e911a
Move do_adhoc_statistics to recorder test helpers (#71544) 2022-05-08 12:45:57 -07:00
J. Nick Koston a8aa0e1cca
Add Estimated Database Size to the recorder system health (#71463) 2022-05-07 21:02:54 -07:00
J. Nick Koston eb77f8db85
Complete strict typing for recorder (#71274)
* Complete strict typing for recorder

* update tests

* Update tests/components/recorder/test_migrate.py

Co-authored-by: Martin Hjelmare <marhje52@gmail.com>

* Update tests/components/recorder/test_migrate.py

Co-authored-by: Martin Hjelmare <marhje52@gmail.com>

* Remove the asserts

* remove ignore comments

Co-authored-by: Martin Hjelmare <marhje52@gmail.com>
2022-05-04 12:22:50 -05:00
J. Nick Koston 13ce0a7d6a
Fix history using pre v25 queries during v26 migration (#71295) 2022-05-04 08:56:50 -07:00
J. Nick Koston e30940ef2a
Move processing of recorder service call arguments into services.py (#71260) 2022-05-03 13:56:22 -07:00
J. Nick Koston e1be6dd34f
Move recorder services to services.py (#71249) 2022-05-03 10:38:44 -07:00
J. Nick Koston 29bda196b5
Break apart recorder into tasks and core modules (#71222) 2022-05-02 21:53:56 -07:00
J. Nick Koston 75026f9fed
Separate recorder logic for state_changed and non-state_changed events (#71204) 2022-05-02 17:22:53 -05:00
J. Nick Koston 188040b8bb
Use lambda_stmt for recorder queries and migrate them to queries module (#71219) 2022-05-02 15:17:21 -07:00
J. Nick Koston 5db014666c
Avoid recording state_changed events in the events table (#71165)
* squash

fix mypy

* Update homeassistant/components/recorder/models.py

Co-authored-by: Paulus Schoutsen <paulus@home-assistant.io>

* pass all columns

* fix commented out code

* reduce logbook query complexity

* merge

* comment

Co-authored-by: Paulus Schoutsen <paulus@home-assistant.io>
2022-05-02 02:10:34 -05:00
J. Nick Koston c23866e5e5
De-duplicate event data into a new event_data table (#71135) 2022-05-01 21:01:17 -05:00
J. Nick Koston ff48720c6a
Remove db schema v23 stats migration tests (#71137) 2022-05-01 16:11:11 -07:00
J. Nick Koston 66a21e0bc3
Add basic system health data to the recorder (#71086) 2022-04-30 11:10:20 -05:00
J. Nick Koston 195811843b
Remove get_state and get_states history api calls (#70830) 2022-04-26 16:05:43 -07:00
J. Nick Koston 6ce768465f
Improve purge performance for non-sqlite databases (#70576) 2022-04-26 13:11:57 -07:00
J. Nick Koston 1c4a785fb3
Prevent autoflush from happening during attrs lookup (#70768) 2022-04-26 13:04:58 -07:00
J. Nick Koston f073f17040
Refactor tracking of the recorder run history (#70456)
Co-authored-by: Erik Montnemery <erik@montnemery.com>
2022-04-26 09:59:43 -10:00
Erik Montnemery 3016b5fbfc
Use recorder fixtures and helpers in tests (#70773) 2022-04-26 06:08:00 -10:00
J. Nick Koston 89e3a48751
Use database executor in migration test (#70774) 2022-04-26 07:42:28 -07:00
Erik Montnemery d045e8678d
Use recorder test fixtures in tests (#70652) 2022-04-25 14:23:52 +02:00