Commit Graph

371 Commits (9cab05c4b91262c34e4d3317ee1ce23aef73adb6)

Author SHA1 Message Date
Erik Montnemery 607a0e7697
Reduce size of get_statistics_during_period WS API response (#82131) 2022-11-16 17:36:30 +01:00
Erik Montnemery 9b8f94363c
Fix statistic_during_period for data with holes (#81847) 2022-11-16 12:46:29 +01:00
Erik Montnemery 68346599d2
Add WS API recorder/statistic_during_period (#80663) 2022-10-27 15:51:09 -04:00
Michael 8175dab7ab
Add week period to recorder statistics api (#80784)
* add week period to get statistics api

* add test
2022-10-25 20:07:28 +02:00
epenet e84e5f134e
Use US_CUSTOMARY_SYSTEM in tests (#80658)
* Use US_CUSTOMARY_SYSTEM in tests

* Don't update test_unit_system
2022-10-20 15:42:23 +02:00
Erik Montnemery f4951a4f31
Add CI job which runs recorder tests on MariaDB (#80586)
Co-authored-by: Franck Nijhof <git@frenck.dev>
2022-10-19 13:04:28 +02:00
Erik Montnemery 31a787558f
Ensure recorder test fixture is setup before hass fixture (#80528)
* Ensure recorder test fixture is setup before hass fixture

* Adjust more tests
2022-10-19 07:58:47 +02:00
Erik Montnemery 2c206ad050
Fix flaky recorder test (#80246)
* Fix flaky recorder test

* Update tests/components/recorder/test_init.py

Co-authored-by: epenet <6771947+epenet@users.noreply.github.com>

Co-authored-by: epenet <6771947+epenet@users.noreply.github.com>
2022-10-14 09:10:38 +02:00
Erik Montnemery 04cc2ae264
Correct initialization of new databases (#80234) 2022-10-13 13:01:27 +02:00
Erik Montnemery acb1477673
Avoid time traveling in recorder tests (#80247) 2022-10-13 11:51:27 +02:00
Erik Montnemery 4462f2fc46
Fix recorder tests related to mysql (#80238) 2022-10-13 11:44:48 +02:00
Erik Montnemery 466c4656ca
Refactor recorder migration (#80175)
* Refactor recorder migration

* Improve test coverage
2022-10-13 08:11:54 +02:00
Erik 4a1c40f09b Revert "Refactor recorder migration"
This reverts commit 69e10e5982.
2022-10-12 15:12:12 +02:00
Erik 69e10e5982 Refactor recorder migration 2022-10-12 15:00:28 +02:00
Erik Montnemery 9aa6043255
Set character set to utf8mb4 when connecting to MySQL or MariaDB databases (#79755) 2022-10-11 14:01:46 +02:00
Erik Montnemery e8af007124
Disable echo for non SQLite databases (#80032)
* Disable echo for non SQLite databases

* Add test
2022-10-10 22:42:38 +03:00
Erik Montnemery a2e3978d53
Don't normalize units of long term statistics (#79320)
* Don't normalize units of long term statistics

* Update statistics.py
2022-10-03 21:42:44 +02:00
Erik Montnemery 2de273500e
Remove state_unit_of_measurement from metadata DB table (#79370)
* Remove state_unit_of_measurement from metadata DB table

* Adjust test
2022-10-01 18:55:00 +02:00
Erik Montnemery bc2dffabc4
Improve naming of units used in statistics (#79276) 2022-09-30 08:38:44 +02:00
epenet 12e4d18038
Add volume to SensorDeviceClass (#77960)
* Add volume to SensorDeviceClass

* Adjust recorder

* Adjust tests

* Adjust sensor UNIT_CONVERTERS

* Adjust recorder

* Update strings.json
2022-09-27 19:37:52 +02:00
epenet 7c448416e1
Add speed to SensorDeviceClass (#77953)
* Add speed to SensorDeviceClass

* Adjust recorder

* Adjust tests

* Adjust sensor UNIT_CONVERTERS

* Add tests

* Add websocket tests

* Update strings.json
2022-09-27 17:19:34 +01:00
epenet bfcc18e5b8
Add distance to SensorDeviceClass (#77951)
* Add distance to SensorDeviceClass

* Adjust recorder

* Adjust tests

* Adjust recorder

* Update __init__.py

* Update test_websocket_api.py

* Update test_websocket_api.py

* Update test_websocket_api.py

* Update strings.json

* Fix tests

* Adjust docstring
2022-09-27 15:34:00 +01:00
Erik Montnemery c52d0f7495
Support converting statistics to another unit (#79117) 2022-09-27 08:44:58 +02:00
epenet df47fda2a0
Remove parametrization in recorder websocket api tests (#78864)
Remove parametrization in websocket api tests
2022-09-27 08:27:26 +02:00
epenet 08ebb9f31a
Test sum AND mean in recorder tests (#78998) 2022-09-23 09:55:17 -04:00
Erik Montnemery cd6697615f
Validate units when importing statistics (#78891) 2022-09-21 12:08:53 -04:00
Erik Montnemery dae00c70de
Allow selecting display unit when fetching statistics (#78578) 2022-09-20 17:43:57 -04:00
epenet 019d297ff0
Apply hass-relative-import to recorder tests (#78734) 2022-09-19 12:35:03 +02:00
Erik Montnemery dd20a7ea62
Display statistics in the source's unit (#78031) 2022-09-15 12:01:24 -04:00
Erik Montnemery 0a1fd36e03
Catch up with statistics after DB migration is done (#78469)
* Catch up with statistics after DB migration is done

* Don't access the database from the event loop

* Fix deadlocking test

* Fix test
2022-09-15 07:40:56 +02:00
Erik Montnemery 7937bfeedb
Deprecate history integration's statistics API (#78056) 2022-09-08 16:03:43 -04:00
Erik Montnemery 008ac8d10d
Improve statistics metadata WS API (#77209) 2022-08-31 11:30:45 +02:00
Erik Montnemery f61edf0778
Fix updating of statistics metadata name (#77207)
* Fix updating of statistics metadata name

* Fix test

* Test renaming
2022-08-23 17:51:17 +02:00
Erik Montnemery fd6ffef52f
Support non-live database migration (#72433)
* Support non-live database migration

* Tweak startup order, add test

* Address review comments

* Fix typo

* Clarify comment about promoting dependencies

* Tweak

* Fix merge mistake

* Fix some tests

* Fix additional test

* Fix additional test

* Adjust tests

* Improve test coverage
2022-07-22 15:11:34 +02:00
Erik Montnemery 606d544157
Use recorder get_instance function to improve typing (#75567) 2022-07-22 11:58:26 +02:00
Thibault Cohen 1d7d2875e1
Add websocket command recorder/import_statistics (#73937)
* Expose ws_add_external_statistics in websocket API

* Refactor

* Add tests

* Improve test coverage

Co-authored-by: Thibault Cohen <titilambert@users.noreply.github.com>
Co-authored-by: Erik <erik@montnemery.com>
2022-07-21 12:36:49 +02:00
J. Nick Koston a8349a4866
Adjust entity filters to make includes stronger than excludes (#74080)
* Adjust entity filters to make includes stronger than excludes

Fixes #59080

* adjust test for stronger entity glob includes

* sync with docs
2022-06-28 11:42:51 -05:00
J. Nick Koston 00a79635c1
Revert "Remove sqlalchemy lambda_stmt usage from history, logbook, and statistics (#73191)" (#73917) 2022-06-23 13:59:55 -05:00
J. Nick Koston 7ae8bd5137
Remove sqlalchemy lambda_stmt usage from history, logbook, and statistics (#73191) 2022-06-07 16:15:50 -07:00
Erik Montnemery 5f2b4001f3
Separate recorder database schema from other classes (#72977)
* Separate recorder database schema from other classes

* fix logbook imports

* migrate new tests

* few more

* last one

* fix merge

Co-authored-by: J. Nick Koston <nick@koston.org>
2022-06-07 14:41:43 +02:00
J. Nick Koston de2e9b6d77
Fix state_changes_during_period history query when no entities are passed (#73139) 2022-06-06 12:50:52 -07:00
J. Nick Koston c66b000d34
Reduce branching in generated lambda_stmts (#73042) 2022-06-05 21:13:31 -07:00
J. Nick Koston 5b31414225
Fix misalignments between sql based filtering with the entityfilter based filtering (#72936) 2022-06-02 20:52:53 -07:00
J. Nick Koston f79e5e002b
Ensure recorder shuts down when its startup future is canceled out from under it (#72866) 2022-06-01 22:13:09 -07:00
J. Nick Koston c6e56c26b3
Fix logbook not setting up with an recorder filter that has empty fields (#72869) 2022-06-01 22:12:00 -07:00
J. Nick Koston d8b0376942
Fix purge of legacy database events that are not state changed (#72815) 2022-06-01 11:56:06 +02:00
J. Nick Koston 587fd05603
Make logbook inherit the recorder filter (#72728) 2022-05-30 17:34:32 -07:00
J. Nick Koston 040e120101
Fix recorder system health when the db_url is lacking a hostname (#72612) 2022-05-27 10:32:26 -07:00
Erik Montnemery 23bd64b7a2
Prevent duplication of statistics metadata (#71637)
* Prevent duplication of statistics metadata

* Add models_schema_28.py

* Handle entity renaming as a recorder job

* Improve tests
2022-05-24 15:34:46 +02:00
J. Nick Koston 4dbc1ed7a7
Add dialect (database engine) and version to recorder system health data (#72339) 2022-05-22 21:29:30 -07:00
J. Nick Koston 9c3f949165
Add live streaming logbook websocket endpoint (#72258)
Co-authored-by: Paulus Schoutsen <paulus@home-assistant.io>
2022-05-22 14:57:54 -05:00
J. Nick Koston 1001f9e39a
Fix last state in history minimal respones when all the states at the end are skipped (#72203) 2022-05-19 20:58:32 -07:00
Erik Montnemery edd7a3427c
Remove support for databases without ROW_NUMBER (#72092) 2022-05-18 21:52:38 -05:00
Erik Montnemery 037f6947d8
Fail recorder setup with unsupported dialect or version (#70888) 2022-05-18 16:52:46 +02:00
J. Nick Koston a4c1bcefb9
Tune sqlite based on configured settings (#72016) 2022-05-17 18:12:15 -05:00
J. Nick Koston a70e2a33dc
Fixing purging legacy rows and improve performance (#71916) 2022-05-15 21:25:07 -07:00
J. Nick Koston 98809675ff
Convert history queries to use lambda_stmt (#71870)
Co-authored-by: Paulus Schoutsen <paulus@home-assistant.io>
2022-05-15 10:47:29 -05:00
J. Nick Koston 8c2743bb67
Avoid storing last_changed in the database if its the same as last_updated (#71843) 2022-05-14 12:06:31 -07:00
J. Nick Koston a8f1dda004
Use ciso8601 for parsing MySQLdb datetimes (#71818)
* Use ciso8601 for parsing MySQLDB datetimes

The default parser is this:

5340191feb/MySQLdb/times.py (L66)

* tweak

* tweak

* add coverage for building the MySQLdb connect conv param
2022-05-13 19:26:09 -05:00
J. Nick Koston 7ab4960b1e
Restore v23 stats migration tests (#71743) 2022-05-12 21:14:02 -04:00
J. Nick Koston 1d9fb4bca8
Fix process_datetime_to_timestamp and add test coverage (#71755) 2022-05-12 17:12:50 -07:00
J. Nick Koston e2cef55162
Add history/history_during_period websocket endpoint (#71688) 2022-05-11 17:52:22 -05:00
J. Nick Koston 222baa53dd
Make database access in the eventloop raise an exception (#71547) 2022-05-09 15:22:08 -05:00
J. Nick Koston 7c9c0e911a
Move do_adhoc_statistics to recorder test helpers (#71544) 2022-05-08 12:45:57 -07:00
J. Nick Koston a8aa0e1cca
Add Estimated Database Size to the recorder system health (#71463) 2022-05-07 21:02:54 -07:00
J. Nick Koston eb77f8db85
Complete strict typing for recorder (#71274)
* Complete strict typing for recorder

* update tests

* Update tests/components/recorder/test_migrate.py

Co-authored-by: Martin Hjelmare <marhje52@gmail.com>

* Update tests/components/recorder/test_migrate.py

Co-authored-by: Martin Hjelmare <marhje52@gmail.com>

* Remove the asserts

* remove ignore comments

Co-authored-by: Martin Hjelmare <marhje52@gmail.com>
2022-05-04 12:22:50 -05:00
J. Nick Koston 13ce0a7d6a
Fix history using pre v25 queries during v26 migration (#71295) 2022-05-04 08:56:50 -07:00
J. Nick Koston e30940ef2a
Move processing of recorder service call arguments into services.py (#71260) 2022-05-03 13:56:22 -07:00
J. Nick Koston e1be6dd34f
Move recorder services to services.py (#71249) 2022-05-03 10:38:44 -07:00
J. Nick Koston 29bda196b5
Break apart recorder into tasks and core modules (#71222) 2022-05-02 21:53:56 -07:00
J. Nick Koston 75026f9fed
Separate recorder logic for state_changed and non-state_changed events (#71204) 2022-05-02 17:22:53 -05:00
J. Nick Koston 188040b8bb
Use lambda_stmt for recorder queries and migrate them to queries module (#71219) 2022-05-02 15:17:21 -07:00
J. Nick Koston 5db014666c
Avoid recording state_changed events in the events table (#71165)
* squash

fix mypy

* Update homeassistant/components/recorder/models.py

Co-authored-by: Paulus Schoutsen <paulus@home-assistant.io>

* pass all columns

* fix commented out code

* reduce logbook query complexity

* merge

* comment

Co-authored-by: Paulus Schoutsen <paulus@home-assistant.io>
2022-05-02 02:10:34 -05:00
J. Nick Koston c23866e5e5
De-duplicate event data into a new event_data table (#71135) 2022-05-01 21:01:17 -05:00
J. Nick Koston ff48720c6a
Remove db schema v23 stats migration tests (#71137) 2022-05-01 16:11:11 -07:00
J. Nick Koston 66a21e0bc3
Add basic system health data to the recorder (#71086) 2022-04-30 11:10:20 -05:00
J. Nick Koston 195811843b
Remove get_state and get_states history api calls (#70830) 2022-04-26 16:05:43 -07:00
J. Nick Koston 6ce768465f
Improve purge performance for non-sqlite databases (#70576) 2022-04-26 13:11:57 -07:00
J. Nick Koston 1c4a785fb3
Prevent autoflush from happening during attrs lookup (#70768) 2022-04-26 13:04:58 -07:00
J. Nick Koston f073f17040
Refactor tracking of the recorder run history (#70456)
Co-authored-by: Erik Montnemery <erik@montnemery.com>
2022-04-26 09:59:43 -10:00
Erik Montnemery 3016b5fbfc
Use recorder fixtures and helpers in tests (#70773) 2022-04-26 06:08:00 -10:00
J. Nick Koston 89e3a48751
Use database executor in migration test (#70774) 2022-04-26 07:42:28 -07:00
Erik Montnemery d045e8678d
Use recorder test fixtures in tests (#70652) 2022-04-25 14:23:52 +02:00
Erik Montnemery 28ebab9c5a
Simplify waiting for recorder in tests (#70647) 2022-04-25 12:04:47 +02:00
J. Nick Koston 9f11063724
Fix state_changes_during_period bakery caching for limit and descending (#70610) 2022-04-24 21:47:21 -07:00
Erik Montnemery 982e314de6
Use recorder_mock in tests (#70363)
Co-authored-by: Paulus Schoutsen <balloob@gmail.com>
2022-04-22 19:29:44 -10:00
J. Nick Koston 3737b58e85
Avoid fetching metadata multiple times during stat compile (#70397) 2022-04-22 00:25:42 -10:00
J. Nick Koston 3142a106fc
Additional prep work for sqlalchemy 2.0 (#70358) 2022-04-21 12:39:18 -10:00
J. Nick Koston 89807f0d2d
Reduce the number of queries needed to compile statistics (#69731)
Co-authored-by: Erik Montnemery <erik@montnemery.com>
2022-04-15 09:13:29 -07:00
J. Nick Koston c85387290a
Increase recorder queue max backlog to 40k, improve message (#70065) 2022-04-14 11:45:07 -10:00
Erik Montnemery eb3458a3d2
Add MutexPool for recorder tests (#69410)
* Add MutexPool for recorder tests

* Fix get_schema_version

* Update test test_last_run_was_recently_clean

* Update test test_shutdown_before_startup_finishes

* Revert comments in test_write_lock_db

* Make the MutexPool lock a class variable

* Remove stale comment

* Move MutexPool

* Tweak debug prints
2022-04-12 07:41:46 -10:00
J. Nick Koston 72fffde77a
Fix spelling of periodic in recorder (#69658)
Co-authored-by: Martin Hjelmare <marhje52@gmail.com>
2022-04-07 23:37:02 -10:00
J. Nick Koston 66f0a3816a
Reduce memory pressure during database migration (#69628) 2022-04-07 18:29:31 -10:00
Erik Montnemery 949b0e1b65
Don't allow in-memory SQLite database (#69616) 2022-04-08 00:43:09 +02:00
Erik Montnemery 4a7c978f69
Speedup recorder test test_write_lock_db (#69578) 2022-04-07 15:36:58 +02:00
J. Nick Koston 5c7c09726a
Cleanup recorder history typing (#69408) 2022-04-07 00:09:05 -10:00
J. Nick Koston bde3646293
Always use a commit interval of 0 for the in memory db in tests (#69330) 2022-04-05 12:06:06 -10:00
J. Nick Koston ec131d685e
Auto repack the database on the second sunday of the month (#69314) 2022-04-04 21:39:12 -10:00
J. Nick Koston f5a13fc51b
Add ability to exclude attributes from being recorded by entity domain (#68824) 2022-03-29 20:13:08 -07:00
Joakim Sørensen 32b2d1e5c9
Add backup platform to recorder (#68229) 2022-03-25 23:17:11 -07:00
J. Nick Koston a566d3943c
Fix history queries while the database migration is in progress (#68598) 2022-03-24 09:49:13 -10:00
J. Nick Koston e911936a0d
Remove direct usage of concurrent.futures from recorder (#68593) 2022-03-24 09:48:49 -10:00
Erik Montnemery 61cc8e32f3
Include has_mean + has_sum in statistics metadata WS response (#68546)
* Include has_mean + has_sum in statistics metadata WS response

* Don't include has_mean/has_sum in history/list_statistic_ids

* Adjust tests

* Do include has_mean/has_sum in history/list_statistic_ids
2022-03-24 10:12:01 +01:00
Erik Montnemery b5c5da96ac
Add WS API to adjust incorrect energy statistics (#65147)
Co-authored-by: Paulus Schoutsen <balloob@gmail.com>
2022-03-22 15:18:30 -07:00
Pawel bcb8c7ec3c
Add API endpoint get_statistics_metadata (#68471)
Co-authored-by: Erik Montnemery <erik@montnemery.com>
2022-03-21 21:14:47 -07:00
J. Nick Koston 3150915cb7
Convert unindexed domain queries to entity_id queries (#68404) 2022-03-20 01:28:17 -10:00
J. Nick Koston 816695cc96
Avoid selecting attributes in the history api when `no_attributes` is passed (#68352) 2022-03-19 23:47:22 -10:00
Erik Montnemery cf4033b1bc
Simplify time zone setting in tests (#68330)
* Simplify timezone setting in tests

* Fix typo

* Adjust caldav tests

* Adjust input_datetime tests

* Adjust time_date tests

* Adjust tod tests

* Adjust helper tests

* Adjust recorder tests

* Adjust risco tests

* Adjust aemet tests

* Adjust flux tests

* Adjust forecast_solar tests

* Revert unnecessary change in forecast_solar test

* Adjust climacell tests

* Adjust google tests

* Adjust sensor tests

* Adjust sonarr tests

* Adjust template tests

* Adjust zodiac tests

Co-authored-by: Martin Hjelmare <marhje52@gmail.com>
2022-03-20 10:25:15 +01:00
J. Nick Koston 7ee647cc78
Fix FOREIGN KEY constraint failed when removing state_attributes (#68364) 2022-03-19 14:21:28 -07:00
J. Nick Koston 9215702388
Separate attrs into another table (reduces database size) (#68224) 2022-03-18 00:23:13 -10:00
J. Nick Koston bc862e97ed
Use a dedicated executor pool for database operations (#68105)
Co-authored-by: Erik Montnemery <erik@montnemery.com>
Co-authored-by: Franck Nijhof <git@frenck.dev>
2022-03-17 23:09:01 -10:00
J. Nick Koston 95f20500ca
Commit any pending changes before running non-EventTasks in the recorder (#68287) 2022-03-17 21:33:22 -07:00
Erik Montnemery d360ac91ca
Disable recorder nightly jobs in tests (#68188) 2022-03-15 09:23:52 -07:00
J. Nick Koston b18096fc54
Remove unused columns from states/events tables (#68078) 2022-03-13 23:44:55 -07:00
Erik Montnemery 910b1f1ec8
Speed up deletion of duplicated statistics (#66014) 2022-02-07 09:11:52 -08:00
Erik Montnemery 41ab12cb88
Don't use shared session during recorder migration (#65672) 2022-02-04 09:55:11 -08:00
Erik Montnemery ac7662c82d
Remove limit of amount of duplicated statistics (#65641) 2022-02-04 14:49:45 +01:00
Marc Mueller d057850971
Update typing - collections.abc (3) (#63947) 2022-01-12 08:04:17 +01:00
Stefan Agner 0a9927d18e
Avoid locking the database for non-SQLite backends (#63847)
* Avoid locking the database for non-SQLite backends

Currently we only have a lock implementation for SQLite. Just return
success for all other databases as they are not expected to store data
in the config directory and the caller can assume that a backup can
be safely taken.

This fixes `RuntimeError: generator didn't yield` errors when creating
a backup with the current Supervisor dev builds.
2022-01-11 16:17:56 +01:00
Erik Montnemery 2f8e65a9b0
Store deleted duplicated statistics in .storage (#62574) 2021-12-22 14:27:56 +01:00
Erik Montnemery 48b3d6e1c0
Save original + duplicate pairs when deleting duplicated statistics (#62498) 2021-12-21 15:24:36 +01:00
Erik Montnemery cab2a74b5f
Don't pollute config dir with deleted duplicated statistics (#62489) 2021-12-21 14:27:35 +01:00
Erik Montnemery 474ef54477
Fix threading error in recorder tests (#62187) 2021-12-17 13:17:48 +01:00
Erik Montnemery bceeaec2f8
Remove duplicated statistics rows (#61146)
* Remove duplicated statistics

* Fix misleading docstring

* Pylint knows best

* Correct test

* Oops

* Prevent insertion of duplicated statistics

* Tweak

* pylint

* Add models_schema_23.py

* Tweak
2021-12-13 14:15:36 +01:00
Erik Montnemery dc5888ab4a
Correct recorder.statistics.get_last_statistics (#61421) 2021-12-10 10:09:29 -08:00
Stefan Agner a13ae85982
Introduce only_supervisor for @websocket_api.ws_require_user() (#61298) 2021-12-08 16:49:35 -08:00
Erik Montnemery f30eb05870
Refactor recorder queue handling (#61161)
* Refactor recorder queue handling

* Address pylint's concerns

* Implement workaround for mypy bug

* Address review comments
2021-12-08 16:54:26 +01:00
Stefan Agner f0006b92be
Allow to lock SQLite database during backup (#60874)
* Allow to set CONF_DB_URL

This is useful for test which need a custom DB path.

* Introduce write_lock_db helper to lock SQLite database

* Introduce Websocket API which allows to lock database during backup

* Fix isort

* Avoid mutable default arguments

* Address pylint issues

* Avoid holding executor thread

* Set unlock event in case timeout occures

This makes sure the database is left unlocked even in case of a race
condition.

* Add more unit tests

* Address new pylint errors

* Lower timeout to speedup tests

* Introduce queue overflow test

* Unlock database if necessary

This makes sure that the test runs through in case locking actually
succeeds (and the test fails).

* Make DB_LOCK_TIMEOUT a global

There is no good reason for this to be an argument. The recorder needs
to pick a sensible value.

* Add Websocket Timeout test

* Test lock_database() return

* Update homeassistant/components/recorder/__init__.py

Co-authored-by: Erik Montnemery <erik@montnemery.com>

* Fix format

Co-authored-by: J. Nick Koston <nick@koston.org>
Co-authored-by: Erik Montnemery <erik@montnemery.com>
2021-12-07 13:16:24 +01:00
Erik Montnemery bcd9f3c05f
Correct end time for monthly statistics summary (#59551)
* Correct end time for monthly statistics summary

* Add tests
2021-11-12 15:01:40 +01:00
Erik Montnemery e5ee38d3b0
Adjust recorder tests according to #59087 (#59143) 2021-11-05 10:49:58 +01:00
Erik Montnemery 470b01e4ce
Move async_migration_in_progress (#59087) 2021-11-05 10:40:56 +01:00
Erik Montnemery 185f7beafc
Improve recorder migration tests (#59075) 2021-11-04 20:21:38 -07:00
Erik Montnemery 4c5aca93df
Add recorder status WS API (#58989)
* Add recorder status WS API

* Rename recorder/status to recorder/info

* Silence pylint

* Improve tests

* Address review comments

* Tweak

* Try to fix tests

* Try to debug flaky tests

* Try to fix tests

* Revert changes to async_migration_in_progress

* Try to fix tests

* Remove debug prints

* Apply suggestions from code review
2021-11-04 16:46:45 +01:00
Erik Montnemery be4e9f91b6
Change minimum supported SQLite version to 3.31.0 (#59073) 2021-11-04 16:34:35 +01:00
Erik Montnemery dfa50a842a
Simplify recorder PgSQL version checks (#58533) 2021-10-27 16:05:40 +02:00
Erik Montnemery ac5e32d648
Corrections for external statistics (#58469) 2021-10-26 14:05:45 +02:00
Erik Montnemery e9ba5f3b4b
Warn when recorder connects to an unsupported database (#58161) 2021-10-26 13:41:59 +02:00
Erik Montnemery f594bc353b
Add support for external statistics (#56607)
* Support external statistics

* Update tests

* Lint

* Adjust code after rebase

* Separate external statistic_id with :, add name to metadata

* Adjust tests

* Simplify get_metadata_with_session

* Address review comments

* Allow updating external statistics

* Validate input

* Adjust tests after rebase

* Pylint

* Adjust tests

* Improve test coverage
2021-10-26 10:26:50 +02:00
Erik Montnemery b301ab25a3
Purge short term statistics (#58028)
* Purge short term statistics

* Less meep

* Add tests
2021-10-20 16:00:59 +02:00
Erik Montnemery 8ef8838801
Correct detection of row_number support for MariaDB (#57663) 2021-10-14 11:19:39 -07:00
Erik Montnemery 0139bfa749
Detect if mysql and sqlite support row_number (#57475) 2021-10-11 21:17:18 -07:00
Paulus Schoutsen a4d9019ffc
Refactor persistent notification to no longer route all data via a service (#57157)
* Convert persistent notification tests to async

* Create/dismiss persistent notifications in exposed functions, not service calls

* Fix notify persistent_notification

* Remove setting up persistent_notification

* Drop more setups

* Empty methods

* Undeprecate sync methods because too big task

* Fix setup clearing notifications

* Fix a bunch of tests

* Fix more tests

* Uno mas

* Test persistent notification events

* Clean up stale comment

Co-authored-by: Martin Hjelmare <marhje52@gmail.com>
2021-10-07 12:58:00 +02:00
Erik Montnemery 69875cbd11
Improve sensor statistics validation (#56892) 2021-10-04 09:47:44 -07:00
Erik Montnemery 8567aa9e13
Evict purged states from recorder's old_state cache (#56877)
Co-authored-by: J. Nick Koston <nick@koston.org>
2021-10-04 08:21:40 -07:00
Erik Montnemery 40ecf22bac
Remove automatic splitting of net meters from statistics (#56772) 2021-09-29 17:20:17 +02:00
Erik Montnemery 00651a4055
Optimize _get_states_with_session (#56734)
* Optimize _get_states_with_session

* Move custom filters to derived table

* Remove useless derived table

* Filter old states after grouping

* Split query

* Add comments

* Simplify state update period criteria

* Only apply custom filters if we didn't get an include list of entities

Co-authored-by: J. Nick Koston <nick@koston.org>
2021-09-29 17:08:27 +02:00
Erik Montnemery 8ef123259e
Add WS API for updating unit_of_measurement in statistics metadata (#56184)
* Add WS API for updating statistics metadata

* Update homeassistant/components/recorder/websocket_api.py

Co-authored-by: Bram Kragten <mail@bramkragten.nl>

* Update homeassistant/components/recorder/websocket_api.py

Co-authored-by: Paulus Schoutsen <balloob@gmail.com>

* Fix typo

Co-authored-by: Bram Kragten <mail@bramkragten.nl>
Co-authored-by: Paulus Schoutsen <balloob@gmail.com>
2021-09-28 00:43:29 +02:00
Erik Montnemery 5976f898da
Add WS API for removing statistics for a list of statistic_ids (#55078)
* Add WS API for removing statistics for a list of statistic_ids

* Refactor according to code review, enable foreign keys support for sqlite

* Adjust tests

* Move clear_statistics WS API to recorder

* Adjust tests after rebase

* Update docstring

* Update homeassistant/components/recorder/websocket_api.py

Co-authored-by: Paulus Schoutsen <paulus@home-assistant.io>

* Adjust tests after rebase

Co-authored-by: Paulus Schoutsen <paulus@home-assistant.io>
2021-09-27 23:30:13 +02:00
Erik Montnemery 64393b462d
Add migration for 5-minute statistics (#56585)
* Add migration for 5-minute statistics

* Tweaks
2021-09-24 09:19:22 +02:00
Erik Montnemery 92253f5192
Minor refactoring of periodic statistics (#56492) 2021-09-22 13:31:33 -07:00