Commit Graph

547 Commits (3322fa0294d3f4abcb8bf1b1803075b9674e1af1)

Author SHA1 Message Date
Erik Montnemery e050d187c4
Clarify SQLite can't drop foreign key constraints (#123898) 2024-08-14 07:04:53 -05:00
Erik Montnemery 80f5683cd6
Raise on database error in recorder.migration._add_constraint (#123646)
* Raise on database error in recorder.migration._add_constraint

* Fix test
2024-08-14 13:59:06 +02:00
Erik Montnemery b7bbc938d3
Drop violating rows before adding foreign constraints in DB schema 44 migration (#123454)
* Drop violating rows before adding foreign constraints

* Don't delete rows with null-references

* Only delete rows when integrityerror is caught

* Move restore of dropped foreign key constraints to a separate migration step

* Use aliases for tables

* Update homeassistant/components/recorder/migration.py

* Update test

* Don't use alias for table we're deleting from, improve test

* Fix MySQL

* Update instead of deleting in case of self references

* Improve log messages

* Batch updates

* Add workaround for unsupported LIMIT in PostgreSQL

* Simplify

---------

Co-authored-by: J. Nick Koston <nick@koston.org>
2024-08-14 09:31:37 +02:00
J. Nick Koston 00c1a3fd4e
Ensure legacy event foreign key is removed from the states table when a previous rebuild failed (#123388)
* Ensure legacy event foreign key is removed from the states table

If the system ran out of disk space removing the FK, it would
fail. #121938 fixed that to try again, however that PR was made
ineffective by #122069 since it will never reach the check.

To solve this, the migration version is incremented to 2, and
the migration is no longer marked as done unless the rebuild
/fk removal is successful.

* fix logic for mysql

* fix test

* asserts

* coverage

* coverage

* narrow test

* fixes

* split tests

* should have skipped

* fixture must be used
2024-08-09 06:19:12 +02:00
J. Nick Koston a35fa0e95a
Warn that the minimum SQLite version will change to 3.40.1 as of 2025.2 (#104298)
Co-authored-by: Robert Resch <robert@resch.dev>
2024-07-31 08:13:04 -05:00
Erik Montnemery ea75c8864f
Remove support for live schema migration of old recorder databases (#122399)
* Remove support for live schema migration of old recorder databases

* Update test
2024-07-29 15:52:18 +02:00
Erik Montnemery 8a84addc54
Add test of recorder platform with statistics support (#122754)
* Add test of recorder platform with statistics support

* Remove excessive line breaks
2024-07-29 11:57:53 +02:00
Erik Montnemery 81983d66f4
Avoid nesting sessions in recorder auto repairs tests (#122596) 2024-07-25 12:52:13 -05:00
Erik Montnemery ec957e4a94
Run statistics on 5-minute intervals in tests (#122592)
* Run statistics on 5-minute intervals in tests

* Fix test failing when mysql does not return rows in insert order
2024-07-25 17:32:49 +02:00
Erik Montnemery 1f2c54f112
Avoid nesting sessions in recorder purge tests (#122581) 2024-07-25 13:12:10 +02:00
Erik Montnemery 33d5ed52e6
Avoid nesting sessions in recorder statistics tests (#122582) 2024-07-25 12:26:44 +02:00
Erik Montnemery 108dc3795e
Remove incorrect use of Mock.assert_has_calls from recorder tests (#122439)
* Remove incorrect use of Mock.assert_has_calls from recorder tests

* Fix test
2024-07-23 09:39:27 +02:00
Erik Montnemery 96de0a4c94
Correct off-by-one bug in recorder non live schema migration (#122428)
* Correct off-by-one bug in recorder non live schema migration

* Remove change from the future
2024-07-23 00:30:31 +02:00
Erik Montnemery 42716723e6
Register WS command recorder/info early (#122425) 2024-07-22 16:26:52 -05:00
Erik Montnemery 3dc36cf068
Improve error handling when creating new SQLite database (#122406)
Co-authored-by: J. Nick Koston <nick@koston.org>
2024-07-22 14:16:11 -05:00
Erik Montnemery 20fc5233a1
Add recorder data migrator class to clean up states table (#122069) 2024-07-22 13:04:01 -05:00
Erik Montnemery 76cd53a864
Improve error handling when recorder schema migration fails (#122397) 2024-07-22 18:55:12 +02:00
Erik Montnemery e8b88557ee
Refactor recorder schema migration (#122372)
* Refactor recorder schema migration

* Simplify

* Remove unused imports

* Refactor _migrate_schema according to review comments

* Add comment
2024-07-22 16:53:54 +02:00
Erik Montnemery 2f47312eeb
Fix recorder setup hanging if non live schema migration fails (#122242) 2024-07-20 13:10:23 +02:00
Erik Montnemery 436a38c1d2
Revert "Fix recorder setup hanging if non live schema migration fails" (#122232) 2024-07-20 12:29:08 +02:00
Erik Montnemery 153b69c971
Fix recorder setup hanging if non live schema migration fails (#122207) 2024-07-20 11:17:40 +02:00
Marc Mueller a6068dcdf2
Update import locations in tests (#122216) 2024-07-20 11:16:04 +02:00
Marc Mueller 768d20c645
Fix recorder datetime annotations (#122214) 2024-07-20 11:10:25 +02:00
Erik Montnemery a0332d049b
Fix flaky recorder test (#122205) 2024-07-20 11:09:52 +02:00
Marc Mueller 2b486c3bd5
Replace unnecessary typing_extensions imports for Generator (#122169) 2024-07-19 12:56:27 +02:00
Marc Mueller ec937781ca
Update pylint to 3.2.5 (#122126)
* Update pylint to 3.2.5

* Remove unused pylint disable comment
2024-07-18 15:54:54 +02:00
Erik Montnemery 9970b7eece
Refactor recorder data migration (#121009)
* Refactor recorder data migration

* Fix stale docstrings

* Don't store a session object in BaseRunTimeMigration instances

* Simplify logic in EntityIDMigration.migration_done

* Fix tests
2024-07-16 21:50:19 +02:00
Erik Montnemery d8440e809a
Avoid mutating database schema definitions during schema migration (#122012)
* Avoid mutating database schema definitions during schema migration

* Adjust test when using mysql

* Address review comment
2024-07-16 20:27:49 +02:00
Erik Montnemery 336740dbfa
Add recorder migration test starting with schema version 9 (#122011) 2024-07-16 15:12:23 +02:00
J. Nick Koston 73f6e3c07b
Narrow sqlite database corruption check to ensure disk image is malformed (#121947)
* Narrow sqlite database corruption check to ensure disk image is malformed

The database corruption check would also replace the database when it
locked externally instead of only when its malformed.

This was discovered in https://github.com/home-assistant/core/issues/121909#issuecomment-2227409124
when a user did a manual index creation while HA was online

* tweak

* tweak

* fix

* fix
2024-07-14 23:23:07 +02:00
J. Nick Koston 19d2d023ab
Ensure states table rebuild still happens if the event_id index was removed (#121938)
* Ensure states table rebuild still happens if the event_id index was removed

If ix_states_event_id was removed by the foreign key still
exists, the states table would not get rebuilt. This should
not happen under normal circumstances and seems to only be
possible if the index was removed manually or Home
Assistant was restarted forcefully in the middle
of a previous migration from years ago.

* cover

* fix tests

* mysql wont allow at that point but thats ok as long as its gone at the end
2024-07-14 23:20:52 +02:00
J. Nick Koston 43596f22a4
Fix recorder max backlog calculation being too small (#121739) 2024-07-11 09:09:35 +02:00
J. Nick Koston c81d5a1ac2
Handle empty entity_id in the recorder filter (#121681)
fixes #111745
2024-07-10 09:53:38 -05:00
epenet bff663b3c1
Improve type hints in recorder result_processor (#121593) 2024-07-09 21:12:03 +02:00
J. Nick Koston 89ffee9ad5
Update ulid-transform to 0.10.1 (#121321) 2024-07-05 14:07:39 -05:00
Erik Montnemery 6eeb70151d
Use BIGINT SQL type for ID columns (#121025) 2024-07-05 10:42:39 -05:00
Erik Montnemery e47cbf3cf7
Use async_setup_recorder_instance fixture in recorder v32_migration tests (#121081)
Co-authored-by: J. Nick Koston <nick@koston.org>
2024-07-04 17:49:51 -05:00
Erik Montnemery 6df15ad8fc
Drop use of async_setup_recorder_instance fixture in recorder purge tests (#121193) 2024-07-04 10:39:24 -05:00
Erik Montnemery a1e6f8c2ec
Drop use of async_setup_recorder_instance fixture in recorder migration tests (#121196) 2024-07-04 10:39:13 -05:00
Erik Montnemery f1d6ad9073
Add test fixture to control recorder migration (#121180)
* Add test fixture to control recorder migration

* Update tests/components/recorder/conftest.py

Co-authored-by: J. Nick Koston <nick@koston.org>

* Update tests/components/recorder/conftest.py

---------

Co-authored-by: J. Nick Koston <nick@koston.org>
2024-07-04 13:10:08 +02:00
Erik Montnemery 255778d0c7
Use recorder test fixtures in recorder init tests (#121176) 2024-07-04 04:54:40 -05:00
Erik Montnemery d55d02623a
Add recorder test fixture to enable persistent SQLite database (#121137)
* Add recorder test fixture to enable persistent SQLite database

* Fix tests directly using async_test_home_assistant context manager
2024-07-04 09:59:37 +02:00
Erik Montnemery cc2782edc7
Use async_test_recorder fixture in recorder auto_repairs tests (#121125) 2024-07-04 01:38:01 +02:00
Erik Montnemery 408e524551
Add recorder test fixture for skipping tests by DB engine (#121118)
* Add recorder test fixture for skipping tests by DB engine

* Fix mistake
2024-07-04 00:02:46 +02:00
Erik Montnemery 5029da6919
Make the async_setup_recorder_instance fixture a context manager (#121086) 2024-07-03 14:05:34 -05:00
J. Nick Koston 44c89e6c3b
Cleanup v32 recorder migration test (#121083) 2024-07-03 10:45:19 -05:00
Erik Montnemery 46a488d871
Use async_setup_recorder_instance fixture in recorder auto_repairs tests (#121077) 2024-07-03 17:04:39 +02:00
Erik Montnemery 87f7703f3c
Use async_setup_recorder_instance fixture in recorder migration tests (#121050) 2024-07-03 08:56:05 -05:00
Marc Mueller f11b316dac
Import Generator from collections.abc (4) (#120917) 2024-07-01 11:54:42 +02:00
J. Nick Koston c5804d362c
Remove legacy foreign key constraint from sqlite states table (#120779) 2024-06-29 07:50:53 -05:00