* Migrate issue registry to use singleton helper
The other registries were already migrated, but since this
one had a read only flag, it required a slightly different
solution since it uses the same hass.data key
* refactor
* Add thread safety checks to async_create_task
Calling async_create_task from a thread almost always results in an
fast crash. Since most internals are using async_create_background_task
or other task APIs, and this is the one integrations seem to get wrong
the most, add a thread safety check here
* Add thread safety checks to async_create_task
Calling async_create_task from a thread almost always results in an
fast crash. Since most internals are using async_create_background_task
or other task APIs, and this is the one integrations seem to get wrong
the most, add a thread safety check here
* missed one
* Update homeassistant/core.py
* fix mocks
* one more internal
* more places where internal can be used
* more places where internal can be used
* more places where internal can be used
* internal one more place since this is high volume and was already eager_start
Construct storage data in the executor
Constructing storage data can be expensive for large files and can block the event
loop. While ideally we optimize the construction of the data, there are some places
we cannot make it any faster. To avoid blocking the loop, the construction of
the data is now done in the executor by running the data_func in the executor.
2024-03-14 11:28:20.178 WARNING (MainThread) [asyncio] Executing <TimerHandle cancelled when=2319925.760294916 Store._async_schedule_callback_delayed_write() created at /Users/bdraco/home-assistant/homeassistant/helpers/storage.py:328> took 0.159 seconds
There is some risk that the data_func is not thread-safe and needs to be run in
the event loop, but I could not find any cases in our existing code where it
would be a problem
* Reduce latancy to load storage by making the task eager
This changes the semantics a bit under the hood because it
can raise sooner which means we do not store the task
as _load_task if it raises right away. That means
concurrent calls that result in failure are likely to try
again now which will be a tiny performance hit for this
case.
* fix
* will now finish in time
* Avoid circular import in Storage.async_delay_save
We call Storage.async_delay_save for every entity being added or removed
from the registry. The late import took more time than everything else
in the function.
* Avoid reschedule churn in Storage.async_delay_save
When we are adding or removing entities we will call async_delay_save
quite often which has to add and remove a TimerHandle on the event loop
which can add up when there are a lot of registry items changing.
If the timer handle still has 80% of the time remaining on it
we will avoid resceduling and let it fire at the time the
original async_delay_save call was made. This ensures we
do not force the event loop to rebuild its heapq because
too many timer handlers were cancelled at once
* div0
* add coverage for 0 since we had none
* fix bad conflict
* tweaks
* tweaks
* tweaks
* tweaks
* tweaks
* tweaks
* more test fixes
* mqtt tests rely on event loop overhead
* Don't blow up if validators can't access the issue registry
* Make the check_config script open issue_registry read only
* Update tests/helpers/test_issue_registry.py
Co-authored-by: Martin Hjelmare <marhje52@gmail.com>
---------
Co-authored-by: Martin Hjelmare <marhje52@gmail.com>
* Improve handling of unrecoverable storage corruption
fixes#96574
If something in storage gets corrupted core can boot loop
or if its integration specific, the integration will fail to
start.
We now complainly loudly in the log, move away the corrupt data
and start fresh to allow startup to proceed so the user can
get to the UI and restore from backup without having to attach
a console (or otherwise login to the OS and manually modify files).
* test for corruption
* ensure OSError is still fatal
* one more case
* create an issue for corrupt storage
* fix key
* persist
* feedback
* feedback
* better to give the full path
* tweaks
* grammar
* add time
* feedback
* adjust
* try to get issue_domain from storage key
* coverage
* tweak wording some more
* Prevent storage loads from monopolizing the executor pool
- At startup there is an increasing demand to load data
from storage. Similar to #49451 and #43085, we now prevent
the thread pool from being monopolized by storage loads and
allow other consumers that are doing network I/O to proceed
without having to wait for a free executor thread.
* Only create Semaphore instance when one is not already there
J. Nick Koston
Marc Mueller
Sid
Ville Skyttä
Erik Montnemery
epenet
Franck Nijhof
Joakim Plate
Joakim Sørensen
Ruslan Sayfutdinov
Paulus Schoutsen
tkdrob