fd6ffef52f
Support non-live database migration ( #72433 )
...
* Support non-live database migration
* Tweak startup order, add test
* Address review comments
* Fix typo
* Clarify comment about promoting dependencies
* Tweak
* Fix merge mistake
* Fix some tests
* Fix additional test
* Fix additional test
* Adjust tests
* Improve test coverage
2022-07-22 15:11:34 +02:00
606d544157
Use recorder get_instance function to improve typing ( #75567 )
2022-07-22 11:58:26 +02:00
5f2b4001f3
Separate recorder database schema from other classes ( #72977 )
...
* Separate recorder database schema from other classes
* fix logbook imports
* migrate new tests
* few more
* last one
* fix merge
Co-authored-by: J. Nick Koston <nick@koston.org>
2022-06-07 14:41:43 +02:00
f79e5e002b
Ensure recorder shuts down when its startup future is canceled out from under it ( #72866 )
2022-06-01 22:13:09 -07:00
9c3f949165
Add live streaming logbook websocket endpoint ( #72258 )
...
Co-authored-by: Paulus Schoutsen <paulus@home-assistant.io>
2022-05-22 14:57:54 -05:00
222baa53dd
Make database access in the eventloop raise an exception ( #71547 )
2022-05-09 15:22:08 -05:00
a8aa0e1cca
Add Estimated Database Size to the recorder system health ( #71463 )
2022-05-07 21:02:54 -07:00
eb77f8db85
Complete strict typing for recorder ( #71274 )
...
* Complete strict typing for recorder
* update tests
* Update tests/components/recorder/test_migrate.py
Co-authored-by: Martin Hjelmare <marhje52@gmail.com>
* Update tests/components/recorder/test_migrate.py
Co-authored-by: Martin Hjelmare <marhje52@gmail.com>
* Remove the asserts
* remove ignore comments
Co-authored-by: Martin Hjelmare <marhje52@gmail.com>
2022-05-04 12:22:50 -05:00
13ce0a7d6a
Fix history using pre v25 queries during v26 migration ( #71295 )
2022-05-04 08:56:50 -07:00
e30940ef2a
Move processing of recorder service call arguments into services.py ( #71260 )
2022-05-03 13:56:22 -07:00
e1be6dd34f
Move recorder services to services.py ( #71249 )
2022-05-03 10:38:44 -07:00
29bda196b5
Break apart recorder into tasks and core modules ( #71222 )
2022-05-02 21:53:56 -07:00
75026f9fed
Separate recorder logic for state_changed and non-state_changed events ( #71204 )
2022-05-02 17:22:53 -05:00
188040b8bb
Use lambda_stmt for recorder queries and migrate them to queries module ( #71219 )
2022-05-02 15:17:21 -07:00
5db014666c
Avoid recording state_changed events in the events table ( #71165 )
...
* squash
fix mypy
* Update homeassistant/components/recorder/models.py
Co-authored-by: Paulus Schoutsen <paulus@home-assistant.io>
* pass all columns
* fix commented out code
* reduce logbook query complexity
* merge
* comment
Co-authored-by: Paulus Schoutsen <paulus@home-assistant.io>
2022-05-02 02:10:34 -05:00
c23866e5e5
De-duplicate event data into a new event_data table ( #71135 )
2022-05-01 21:01:17 -05:00
6ce768465f
Improve purge performance for non-sqlite databases ( #70576 )
2022-04-26 13:11:57 -07:00
1c4a785fb3
Prevent autoflush from happening during attrs lookup ( #70768 )
2022-04-26 13:04:58 -07:00
f073f17040
Refactor tracking of the recorder run history ( #70456 )
...
Co-authored-by: Erik Montnemery <erik@montnemery.com>
2022-04-26 09:59:43 -10:00
d045e8678d
Use recorder test fixtures in tests ( #70652 )
2022-04-25 14:23:52 +02:00
28ebab9c5a
Simplify waiting for recorder in tests ( #70647 )
2022-04-25 12:04:47 +02:00
982e314de6
Use recorder_mock in tests ( #70363 )
...
Co-authored-by: Paulus Schoutsen <balloob@gmail.com>
2022-04-22 19:29:44 -10:00
eb3458a3d2
Add MutexPool for recorder tests ( #69410 )
...
* Add MutexPool for recorder tests
* Fix get_schema_version
* Update test test_last_run_was_recently_clean
* Update test test_shutdown_before_startup_finishes
* Revert comments in test_write_lock_db
* Make the MutexPool lock a class variable
* Remove stale comment
* Move MutexPool
* Tweak debug prints
2022-04-12 07:41:46 -10:00
72fffde77a
Fix spelling of periodic in recorder ( #69658 )
...
Co-authored-by: Martin Hjelmare <marhje52@gmail.com>
2022-04-07 23:37:02 -10:00
66f0a3816a
Reduce memory pressure during database migration ( #69628 )
2022-04-07 18:29:31 -10:00
949b0e1b65
Don't allow in-memory SQLite database ( #69616 )
2022-04-08 00:43:09 +02:00
bde3646293
Always use a commit interval of 0 for the in memory db in tests ( #69330 )
2022-04-05 12:06:06 -10:00
ec131d685e
Auto repack the database on the second sunday of the month ( #69314 )
2022-04-04 21:39:12 -10:00
f5a13fc51b
Add ability to exclude attributes from being recorded by entity domain ( #68824 )
2022-03-29 20:13:08 -07:00
9215702388
Separate attrs into another table (reduces database size) ( #68224 )
2022-03-18 00:23:13 -10:00
bc862e97ed
Use a dedicated executor pool for database operations ( #68105 )
...
Co-authored-by: Erik Montnemery <erik@montnemery.com>
Co-authored-by: Franck Nijhof <git@frenck.dev>
2022-03-17 23:09:01 -10:00
95f20500ca
Commit any pending changes before running non-EventTasks in the recorder ( #68287 )
2022-03-17 21:33:22 -07:00
d360ac91ca
Disable recorder nightly jobs in tests ( #68188 )
2022-03-15 09:23:52 -07:00
0a9927d18e
Avoid locking the database for non-SQLite backends ( #63847 )
...
* Avoid locking the database for non-SQLite backends
Currently we only have a lock implementation for SQLite. Just return
success for all other databases as they are not expected to store data
in the config directory and the caller can assume that a backup can
be safely taken.
This fixes `RuntimeError: generator didn't yield` errors when creating
a backup with the current Supervisor dev builds.
2022-01-11 16:17:56 +01:00
474ef54477
Fix threading error in recorder tests ( #62187 )
2021-12-17 13:17:48 +01:00
f30eb05870
Refactor recorder queue handling ( #61161 )
...
* Refactor recorder queue handling
* Address pylint's concerns
* Implement workaround for mypy bug
* Address review comments
2021-12-08 16:54:26 +01:00
f0006b92be
Allow to lock SQLite database during backup ( #60874 )
...
* Allow to set CONF_DB_URL
This is useful for test which need a custom DB path.
* Introduce write_lock_db helper to lock SQLite database
* Introduce Websocket API which allows to lock database during backup
* Fix isort
* Avoid mutable default arguments
* Address pylint issues
* Avoid holding executor thread
* Set unlock event in case timeout occures
This makes sure the database is left unlocked even in case of a race
condition.
* Add more unit tests
* Address new pylint errors
* Lower timeout to speedup tests
* Introduce queue overflow test
* Unlock database if necessary
This makes sure that the test runs through in case locking actually
succeeds (and the test fails).
* Make DB_LOCK_TIMEOUT a global
There is no good reason for this to be an argument. The recorder needs
to pick a sensible value.
* Add Websocket Timeout test
* Test lock_database() return
* Update homeassistant/components/recorder/__init__.py
Co-authored-by: Erik Montnemery <erik@montnemery.com>
* Fix format
Co-authored-by: J. Nick Koston <nick@koston.org>
Co-authored-by: Erik Montnemery <erik@montnemery.com>
2021-12-07 13:16:24 +01:00
8c5efafdd8
Add 5-minute statistics for sensors ( #56006 )
...
* Add 5-minute statistics for sensors
* Address pylint issues
* Black
* Apply suggestion from code review
* Apply suggestions from code review
* Improve tests
2021-09-16 10:57:15 +02:00
32e297f4a0
Compile missing statistics ( #54690 )
2021-08-19 22:10:45 -07:00
aa9b99713c
Add purge_entities service call to recorder ( #48069 )
2021-05-22 10:30:05 -05:00
e16a8063a5
Compile statistics for energy sensors ( #50829 )
...
* Compile statistics for energy sensors
* Update tests
* Rename abs_value to state
* Tweak
* Recreate statistics table
* Pylint
* Try to fix test
* Fix statistics for multiple energy sensors
* Fix energy statistics when last_reset is not set
2021-05-20 13:05:15 +02:00
e7f7e61e88
Ensure a wal checkpoint is scheduled nightly ( #50746 )
2021-05-17 16:27:51 -05:00
89dd3292ba
Initial draft of statistics ( #49852 )
2021-05-16 10:23:37 -07:00
bf2d40adfe
Migrate from pytz to python-dateutil ( #49643 )
...
Co-authored-by: Paulus Schoutsen <balloob@gmail.com>
2021-05-07 22:46:26 -07:00
1bd9826684
Handle different entity_id formats ( #49969 )
2021-05-01 20:30:28 -07:00
b27e9e376d
Use StaticPool for recorder and NullPool for all other threads with sqlite3 ( #49693 )
2021-04-26 19:20:31 -10:00
c351098f04
HomeAssistantType —> HomeAssistant for Integrations p* - s* ( #49558 )
2021-04-22 19:58:02 +02:00
e24f5831a2
Force recorder shutdown at final write event ( #49145 )
...
* Force recorder shutdown at EVENT_HOMEASSISTANT_FINAL_WRITE
* remove unreachable
* remove unreachable
* simplify
* cancel in async
2021-04-19 11:24:17 +02:00
9368891b1b
Live db migrations and recovery ( #49036 )
...
Co-authored-by: Paulus Schoutsen <paulus@home-assistant.io>
2021-04-11 20:43:54 -10:00
10848b9bdf
Recorder improvements ( #47739 )
2021-03-11 07:52:07 -10:00
Erik Montnemery
J. Nick Koston
Stefan Agner
PeteBa
Paulus Schoutsen
jan iversen
Marc Mueller