Co-authored-by: Sid <27780930+autinerd@users.noreply.github.com>
Co-authored-by: Marc Mueller <30130371+cdce8p@users.noreply.github.com>
Co-authored-by: J. Nick Koston <nick@koston.org>
* Reduce latancy to load storage by making the task eager
This changes the semantics a bit under the hood because it
can raise sooner which means we do not store the task
as _load_task if it raises right away. That means
concurrent calls that result in failure are likely to try
again now which will be a tiny performance hit for this
case.
* fix
* will now finish in time
* Avoid circular import in Storage.async_delay_save
We call Storage.async_delay_save for every entity being added or removed
from the registry. The late import took more time than everything else
in the function.
* Avoid reschedule churn in Storage.async_delay_save
When we are adding or removing entities we will call async_delay_save
quite often which has to add and remove a TimerHandle on the event loop
which can add up when there are a lot of registry items changing.
If the timer handle still has 80% of the time remaining on it
we will avoid resceduling and let it fire at the time the
original async_delay_save call was made. This ensures we
do not force the event loop to rebuild its heapq because
too many timer handlers were cancelled at once
* div0
* add coverage for 0 since we had none
* fix bad conflict
* tweaks
* tweaks
* tweaks
* tweaks
* tweaks
* tweaks
* more test fixes
* mqtt tests rely on event loop overhead
* Avoid scheduling registry loads as tasks in tests
Since we patch out async_load in Store, these will not yield
to the event loop so it makes sense to await them instead
of creating tasks
This reduced my local test run times ~2.5% on average
* mock out save as well so we do not schedule tasks to save empty data
* tweaks
* fix lingering files
* another one
* too much for one PR, reduce
* fix targets
* Don't blow up if validators can't access the issue registry
* Make the check_config script open issue_registry read only
* Update tests/helpers/test_issue_registry.py
Co-authored-by: Martin Hjelmare <marhje52@gmail.com>
---------
Co-authored-by: Martin Hjelmare <marhje52@gmail.com>
* Improve handling of unrecoverable storage corruption
fixes#96574
If something in storage gets corrupted core can boot loop
or if its integration specific, the integration will fail to
start.
We now complainly loudly in the log, move away the corrupt data
and start fresh to allow startup to proceed so the user can
get to the UI and restore from backup without having to attach
a console (or otherwise login to the OS and manually modify files).
* test for corruption
* ensure OSError is still fatal
* one more case
* create an issue for corrupt storage
* fix key
* persist
* feedback
* feedback
* better to give the full path
* tweaks
* grammar
* add time
* feedback
* adjust
* try to get issue_domain from storage key
* coverage
* tweak wording some more
* Don't write storage to disk while stopping
* rework change
* lint
* remove delay save and schedule final write at stop
* update tests
* fix test component using private methods
* cleanup
* always listen
* use stop in restore state again
* whitelist JSON exceptions for later
* review comment
* make zwave tests use mock storage
* add third stage to hass shutdown
* use 3rd stage in storage
* update core state
* add writing data to multi stop ignore
* update core test
* review comment
* update name based on feedback
* Restore states through a JSON store
* Accept entity_id directly in restore state helper
* Keep states stored between runs for a limited time
* Remove warning
* Fixed file corruption bugs in private storage code.
* Restoring fixed test case.
* Implemented test suite for utils/json.py
* Added new unit test cases for util/json.py
* Dixed formatting nags
* Fixed more nags from the Hound
* Added doc strings to some very short functions
* Fixing lint's complains about my choice of parts of speach. Sigh.
* Moved atomic save operations down into util/json.py so that all benefit.
Added extra clean-up code to ensure that temporary files are removed in
case of errors.
Updated emulated_hue unit tests to avoid errors.
* Apparently 'e' is not allows as a variable name for an exception...
* Add storage helper
* Migrate config entries to use the storage helper
* Make sure tests do not do I/O
* Lint
* Add versions to stored data
* Add more instance variables
* Make migrator load config if nothing to migrate
* Address comments
epenet
J. Nick Koston
Joost Lekkerkerker
Marc Mueller
Erik Montnemery
Ville Skyttä
Franck Nijhof
Paulus Schoutsen
David F. Mulcahey
Bas Nijholt
Adam Mills
Nicko van Someren
Paulus Schoutsen