* Cache is_official_image and is_docker_env in bootstrap to fix blocking I/O
These do blocking I/O later in the startup process
discovered in https://github.com/home-assistant/core/pull/120273
* comment
* Speed up async_get_loaded_integrations
Use a setcomp and difference to find the components to split
to avoid the loop. A setcomp is inlined in python3.12 so its
much faster
* Speed up async_get_loaded_integrations
Use a setcomp and difference to find the components to split
to avoid the loop. A setcomp is inlined in python3.12 so its
much faster
* simplify
* fix compat
* bootstrap
* fix tests
* Small speed up to setting up integration and config entries
When profiling tests, I noticed many calls to get_running_loop. In the places
where we are already in a coro, pass the existing loop so it does not have to
be looked up. I did not do this for places were we are not in a coro since there
is risk that an integration could be doing a non-thread-safe call and its better
that the code raises when trying to fetch the running loop vs the performance
improvement for these cases.
* fix merge
* missed some
* Turn on thread safety checks in async_dispatcher_send
We keep seeing issues where async_dispatcher_send is called from
a thread which means we call the callback function on the other
side in the thread as well which usually leads to a crash
* Turn on thread safety checks in async_dispatcher_send
We keep seeing issues where async_dispatcher_send is called from
a thread which means we call the callback function on the other
side in the thread as well which usually leads to a crash
* adjust
* Add thread safety checks to async_create_task
Calling async_create_task from a thread almost always results in an
fast crash. Since most internals are using async_create_background_task
or other task APIs, and this is the one integrations seem to get wrong
the most, add a thread safety check here
* Add thread safety checks to async_create_task
Calling async_create_task from a thread almost always results in an
fast crash. Since most internals are using async_create_background_task
or other task APIs, and this is the one integrations seem to get wrong
the most, add a thread safety check here
* missed one
* Update homeassistant/core.py
* fix mocks
* one more internal
* more places where internal can be used
* more places where internal can be used
* more places where internal can be used
* internal one more place since this is high volume and was already eager_start
* Avoid useless stat() syscalls for every logger record
shouldRollover will always return False since we do
not use maxBytes as we are only using RotatingFileHandler
for the backupCount option. Since every log record will
stat the log file to see if it exists and if its a normal
file, we can override the shouldRollover to reduce the
logging overhead quite a bit
1d3225ae05/Lib/logging/handlers.py (L189)
* assert False is False
* Speed up logger setup
Preload core.logger and avoid saving it
until after startup
* add comment about 180s
* Adjust grammar
---------
Co-authored-by: Martin Hjelmare <marhje52@gmail.com>
* Fix late load of anyio doing blocking I/O in the event loop
httpx loads anyio which loads the asyncio backend in the event loop
as soon as httpx makes the first request
* tweak
* Refactor setup time tracking to exclude time waiting on other operations
We now exclude the import time and th time waiting on
base platforms to setup from the setup times
* tweak
* tweak
* tweak
* tweak
* adjust
* fixes
* fixes
* preen
* preen
* tweak
* tweak
* adjust
* tweak
* reduce
* do not count integrtion platforms against their parent integration
* handle legacy tts platforms
* stt as well
* one more wait
* use the same pattern in all the legacy
* fix tts and stt legacy
* fix
* fix
* reduce
* preen
* entity comp does not wait for platforms
* scene blocks as well
* fix test
* test fixes
* coverage
* coverage
* coverage
* fix test
* Update tests/test_setup.py
Co-authored-by: Martin Hjelmare <marhje52@gmail.com>
* Update tests/test_setup.py
Co-authored-by: Martin Hjelmare <marhje52@gmail.com>
* Update homeassistant/setup.py
Co-authored-by: Martin Hjelmare <marhje52@gmail.com>
* strip
* strip WAIT_PLATFORM_INTEGRATION
* strip WAIT_PLATFORM_INTEGRATION
* strip WAIT_PLATFORM_INTEGRATION
* strip WAIT_PLATFORM_INTEGRATION
* remove complexity
* Apply suggestions from code review
* no longer works that way
* fixes
* fixes
* fixes
---------
Co-authored-by: Martin Hjelmare <marhje52@gmail.com>
* Replace pylint pointless-statement with ruff B018
* fix occurrences of B018
* disable pylint expression-not-assigned as well
---------
Co-authored-by: J. Nick Koston <nick@koston.org>
* Add category registry
* Add entity registry support
* Update homeassistant/components/config/entity_registry.py
Co-authored-by: Martin Hjelmare <marhje52@gmail.com>
* Use ulid instead
* Add tests for adding same name in different scopes
* Handle keyerror on update
* Lookup tweak
* Omit categories from entity registry snapshots
* Use base registry
* Update snapshots
* Update snapshots
---------
Co-authored-by: Martin Hjelmare <marhje52@gmail.com>
This does not make default_config a requirement, it only preloads
the python code for the integration so it does not have to be loaded
when the import executor is busy. While its a tiny init file, it always
ends up at the end of the line and delays startup
* Phase out periodic tasks
* false by default or some tests will block forever, will need to fix each one manually
* kwarg works
* kwarg works
* kwarg works
* fixes
* fix more tests
* fix more tests
* fix lifx
* opensky
* pvpc_hourly_pricing
* adjust more
* adjust more
* smarttub
* adjust more
* adjust more
* adjust more
* adjust more
* adjust
* no eager executor
* zha
* qnap_qsw
* fix more
* fix fix
* docs
* its a wrapper now
* add more coverage
* coverage
* cover all combos
* more fixes
* more fixes
* more fixes
* remaining issues are legit bugs in tests
* make tplink test more predictable
* more fixes
* feedreader
* grind out some more
* make test race safe
* one more
* Schedule periodic coordinator updates as background tasks.
Currently, the coordinator's periodic refreshes delay startup because they are not scheduled as background tasks. We will wait if the startup takes long enough for the first planned refresh. Another coordinator's scheduled refresh will be fired on busy systems, further delaying the startup. This chain of events results in the startup taking a long time and hitting the safety timeout because too many coordinators are refreshing.
This case can also happen with scheduled entity refreshes, but it's less common. A future PR will address that case.
* periodic_tasks
* periodic_tasks
* periodic_tasks
* merge
* merge
* merge
* merge
* merge
* fix test that call the sync api from async
* one more place
* cannot chain
* async_run_periodic_hass_job
* sun and pattern time changes from automations also block startup
* Revert "sun and pattern time changes from automations also block startup"
This reverts commit 6de2defa05.
* make sure polling is cancelled when config entry is unloaded
* Revert "Revert "sun and pattern time changes from automations also block startup""
This reverts commit e8f12aad55.
* remove DisabledError from homewizard test as it relies on a race
* fix race
* direct coverage
* Pre import the rest of the recorder platforms before asyncio starts
I removed these from #112131 since I had trouble with the weather
tests passing due to a race that I could not figure out. The race
seems to have gone away now (at least locally) so hopefully the CI
will pass now
* Avoid importing counter and proximity integrations in logbook
* Avoid importing counter and proximity integrations in logbook
Move analytics setup to stage 1 to avoid delaying frontend startup
analytics was only needed in the frontend startup phase for onboarding.
Its very unlikely the user will be able to complete the onboarding
steps and get to the analytics screen before analytics is done loading
so we can delay loading it until stage 1. To be absolutely sure that
it is ready, the core_config step in onboarding will wait to proceed
if it is some how still being setup
This one ends in stage 1 and other components have to wait
for it to be imported. Its cheap to import but it ends up
at the end of the line which means other end up waiting for
it which is time we could be doing startup work
`2024-03-04 23:13:04.347 INFO (MainThread) [homeassistant.bootstrap] Setting up stage 1: {usb, websocket_api, webhook, zeroconf, bluetooth, ssdp, dhcp, cloud, network, api, http, hassio}`
It currently always has a wait time for the import executor
`2024-03-04 23:13:04.496 DEBUG (MainThread) [homeassistant.loader] Component webhook import took 0.146 seconds (loaded_executor=True)`
image_upload will always be setup because its a dep of person
and since person is a dep of onboarding which is a dep of
frontend its already a base requirement for homeassistant.
Pillow is now listed as a requirement for homeassistant
so we can be sure it installed by the time bootstrap is
loaded
image_upload loading is currently a bottleneck to
get the frontend loaded because it has to load in the
import executor when everything is busy early in startup
Erik Montnemery
J. Nick Koston
Marc Mueller
Sid
Franck Nijhof
Paulus Schoutsen