* Always reload after a succesfull reauth-flow
* Add test, fix CI failures
* Add kwarg to prevent reloading and tests
* Do not reload entry for bond if it exists
* Remove mocks on internals
* Rename kwarg to always_reload
* Update tests/components/weatherflow_cloud/test_config_flow.py
* Update tests/components/homeworks/test_config_flow.py
* Update tests/components/homeworks/test_config_flow.py
* Rename to option to reload_even_if_entry_is_unchanged
ConfigEntries.async_initialize was trigger asyncio warnings because of the CPU time
to call __setattr__ for every variable for each ConfigEntry being loaded at startup
Co-authored-by: Sid <27780930+autinerd@users.noreply.github.com>
Co-authored-by: Marc Mueller <30130371+cdce8p@users.noreply.github.com>
Co-authored-by: J. Nick Koston <nick@koston.org>
* Move setup time logging into the context manager
We were fetching the time twice but since the context
manager already has the timing, move it there
* remove log setup assertions from integration test
* tweak logging to give us better data for tracking issues
* redundant
* adjust
* preen
* fixes
* adjust
* make api change internal so nobody uses it
* coverage
* fix test
* fix more tests
* coverage
* more tests assuming internal calls
* fix more
* adjust
* adjust
* fix axis tests
* fix broadlink -- it does not call async_forward_entry_setup
* missed some
* remove useless patch
* rename, detect it both ways
* clear
* debug
* try to fix
* handle phase finishing out while paused
* where its set does not need to know its late as that is an implemenation detail of setup
* where its set does not need to know its late as that is an implemenation detail of setup
* tweak
* simplify
* reduce complexity
* revert order change as it makes review harder
* revert naming changes as it makes review harder
* improve comment
* improve debug
* late dispatch test
* test the other way as well
* Update setup.py
* Update setup.py
* Update setup.py
* simplify
* reduce
* Refactor setup time tracking to exclude time waiting on other operations
We now exclude the import time and th time waiting on
base platforms to setup from the setup times
* tweak
* tweak
* tweak
* tweak
* adjust
* fixes
* fixes
* preen
* preen
* tweak
* tweak
* adjust
* tweak
* reduce
* do not count integrtion platforms against their parent integration
* handle legacy tts platforms
* stt as well
* one more wait
* use the same pattern in all the legacy
* fix tts and stt legacy
* fix
* fix
* reduce
* preen
* entity comp does not wait for platforms
* scene blocks as well
* fix test
* test fixes
* coverage
* coverage
* coverage
* fix test
* Update tests/test_setup.py
Co-authored-by: Martin Hjelmare <marhje52@gmail.com>
* Update tests/test_setup.py
Co-authored-by: Martin Hjelmare <marhje52@gmail.com>
* Update homeassistant/setup.py
Co-authored-by: Martin Hjelmare <marhje52@gmail.com>
* strip
* strip WAIT_PLATFORM_INTEGRATION
* strip WAIT_PLATFORM_INTEGRATION
* strip WAIT_PLATFORM_INTEGRATION
* strip WAIT_PLATFORM_INTEGRATION
* remove complexity
* Apply suggestions from code review
* no longer works that way
* fixes
* fixes
* fixes
---------
Co-authored-by: Martin Hjelmare <marhje52@gmail.com>
* Avoid pre-importing config_flows if the integration does support migration
Currently we pre-import the config flow module if it exists since
setting up the config entry required comparing the versions found
in the config_flow.py. We can avoid the pre-import if the integration
does not support async_migrate_entry which means we avoid loading
many config flows in memory at startup.
* cover
* fix missing block
* do not call directly
* its too fast now, the test gets more along
* Update homeassistant/loader.py
* Phase out periodic tasks
* false by default or some tests will block forever, will need to fix each one manually
* kwarg works
* kwarg works
* kwarg works
* fixes
* fix more tests
* fix more tests
* fix lifx
* opensky
* pvpc_hourly_pricing
* adjust more
* adjust more
* smarttub
* adjust more
* adjust more
* adjust more
* adjust more
* adjust
* no eager executor
* zha
* qnap_qsw
* fix more
* fix fix
* docs
* its a wrapper now
* add more coverage
* coverage
* cover all combos
* more fixes
* more fixes
* more fixes
* remaining issues are legit bugs in tests
* make tplink test more predictable
* more fixes
* feedreader
* grind out some more
* make test race safe
* one more
* Schedule periodic coordinator updates as background tasks.
Currently, the coordinator's periodic refreshes delay startup because they are not scheduled as background tasks. We will wait if the startup takes long enough for the first planned refresh. Another coordinator's scheduled refresh will be fired on busy systems, further delaying the startup. This chain of events results in the startup taking a long time and hitting the safety timeout because too many coordinators are refreshing.
This case can also happen with scheduled entity refreshes, but it's less common. A future PR will address that case.
* periodic_tasks
* periodic_tasks
* periodic_tasks
* merge
* merge
* merge
* merge
* merge
* fix test that call the sync api from async
* one more place
* cannot chain
* async_run_periodic_hass_job
* sun and pattern time changes from automations also block startup
* Revert "sun and pattern time changes from automations also block startup"
This reverts commit 6de2defa05.
* make sure polling is cancelled when config entry is unloaded
* Revert "Revert "sun and pattern time changes from automations also block startup""
This reverts commit e8f12aad55.
* remove DisabledError from homewizard test as it relies on a race
* fix race
* direct coverage
* Migrate remaining calls in config modules to async_get_component
There were a few cases that were still using get_component that
could have done blocking I/O in the event loop, although it
was unlikely.
The caching check in async_get_component has been moved
up to avoid creating the future if the module is already in
the cache
* fix one more
Some of the data we had to search for was already available
in a dict or underlying data structure. Make it available
instead of having to build it every time.
There are more places these can be used, but I only did
the device registry cleanup for now
Because the setup again was scheduled as a task, it would
not unset self._async_cancel_retry_setup in time and we would
try to unsub self._async_cancel_retry_setup after it had already
fired. Change it to call a callback that runs right away so it
unsets self._async_cancel_retry_setup as soon as its called
so there is no race
fixes#111796
* Always allow ignore and unignore flows for single config entry integrations
* Update tests/test_config_entries.py
Co-authored-by: Martin Hjelmare <marhje52@gmail.com>
---------
Co-authored-by: Martin Hjelmare <marhje52@gmail.com>
* Add support for pre-imports at setup time
alternative solution to #111331
* refactor
* refactor
* refactor
* mark >1.0s integrations
* no point in executor if already loaded
* no point in executor if already loaded
* cleanup
* cleanup
* two more
* one more
* analytics loads a lot more integrations
* cloud
* debug
* psutil, hardwre
* try zha
* Update homeassistant/setup.py
* await
* comments
* coverage
* coverage
* coverage
* move logic to loader
* move logic to loader
* preserve comments
* Allow setting if we support multiple config entries in config flow
* Move property to config flow instead of flow handler
* Move marking an integration as single instance only to manifest
* Revert line remove
* Avoid init a config flow or adding a new entry on a single instance with an entry
* Revert changes in test
* Process code review comments
* Apply code review suggestion
* Use discovery flow helper for hardware integrations
The discovery flow helper defers loading discovered integrations until after startup
to improve startup reliability.
* Use discovery flow helper for hardware integrations
The discovery flow helper defers loading discovered integrations until after startup
to improve startup reliability. Since hardware was not listed in as a
discovery integration, the notification for new discoveries was missing.
* Convert debouncer async_shutdown to be a normal function
nothing was being awaited here and the shutdown call was only used
in integrations marked internal and other internals. Its possible
that a custom component might have been using the method but it
seemed uncommon enough that it did not warrent marking as a breaking
change. The update coordinator is no longer awaiting anything in
async_shutdown either now but it seemed likely that this use
would get subclassed.
* fix
* Add async_schedule_reload helper to the ConfigEntries manager
We have cases where the the setup retry kicks in right before
the reload happens causing the reload to fail with
OperationNotAllowed. The async_schedule_reload will
cancel the setup retry before the async_reload task
is created to avoid this problem.
I updated a few integrations that were most likely
to have this problem. Future PRs will do a more
extensive audit
* coverage
* revert for now since this needs more refactoring in a followup
* cover
* cleanup and fixes
To wait for after dependencies we created a task to wait an asyncio.Event object,
instead of using an Event we can use an asyncio.Future instead and avoid the need
for a task wrapper
* Small performance improvements to config entry setup retry
- cache some properties that never change
- avoid loader.async_get_integration when we already have it
- avoid multiple integration.domain checks
* tweaks
* Make ConfigEntryItems responsible for updating unique ids
* Make ConfigEntryItems responsible for updating unique ids
* Make ConfigEntryItems responsible for updating unique ids
* Make ConfigEntryItems responsible for updating unique ids
* Make ConfigEntryItems responsible for updating unique ids
* Don't blow up if config entries have unhashable unique IDs
* Add test
* Add comment on when we remove the guard
* Don't stringify hashable non string unique_id
* Add helper function to update and reload config entry to config flow
* Use async_create_task
* Remove await
* Reload only when update & add task name
* Rename function
We should have been checking for the module in hass.data[DATA_COMPONENTS]
and not hass.config.components as the check was ineffective if there were
no existing integrations instances for the domain which is the case for
discovery or when the integration is ignored
* Improve performance of abort_entries_match
In #90406 a ChainMap was added which called __iter__
and __contains__ which ends up creating temp dicts
for matching
174e9da083/Lib/collections/__init__.py (L1022)
We can avoid this by removing the ChainMap since there
are only two mappings to match on.
This also means options no longer obscures data
* adjust comment
I added this in #77803 but I never designed it to be called
externally. External usage may break at any time because the
class is not designed for this. I should have made it protected
in the original PR but I did not think it would get called
externally (my mistake)
* Debounce discoveries to improve event loop stability at the started event
The first one is immediate and anything that fires within the next
second will be debounced to only happen once every second
* fix mock
* Reduce config flow matching overhead
Much of the config flow matching is happening on the context data
after converting via a series of functions. Avoid the conversions
by passing the context matcher deeper into the stack so only
relvant entries need to be processed.
The goal is to reduce the overhead and reduce the chance
the event loop falls behind at the started event when
all the discoveries are processed
* Reduce config flow matching overhead
Much of the config flow matching is happening on the context data
after converting via a series of functions. Avoid the conversions
by passing the context matcher deeper into the stack so only
relvant entries need to be processed.
The goal is to reduce the overhead and reduce the chance
the event loop falls behind at the started event when
all the discoveries are processed
* Reduce config flow matching overhead
Much of the config flow matching is happening on the context data
after converting via a series of functions. Avoid the conversions
by passing the context matcher deeper into the stack so only
relvant entries need to be processed.
The goal is to reduce the overhead and reduce the chance
the event loop falls behind at the started event when
all the discoveries are processed
* augment cover
* Make abort_entries_match available in options flow
* Add tests
* Exclude ignore entries and add test
* Move to OptionsFlow
* Adjust tests
* Use mock_config_flow
* Use AbortFlow
* Remove duplicate code
* Use a set for config entries task tracking
* Allow adding background tasks to config entries
* Add tests for config entry add tasks
* Update docstrings on core create task
* Migrate roon and august
* Use in more places
* Guard for None
* Only wait for import flows to initialize at setup
* Update hassio tests
* Update hassio tests
* Apply suggestions from code review
Co-authored-by: Martin Hjelmare <marhje52@gmail.com>
Co-authored-by: Martin Hjelmare <marhje52@gmail.com>
* Add OptionsFlow helper classes
* More integrations
* Adjust SchemaOptionsFlowHandler
* Use single class
* Simplify access to options
* Reduce PR
* Make _options private
* Add test
* Revert "Address late review of config entry wait for states tests (#81801)"
This reverts commit 12d76a8a4f.
* Revert "Implement ConfigEntry async_wait_for_states (#81771)"
This reverts commit 3cc9ecf1dc.
We only expect this exception to be raised as a result of an
implementation problem. When it is raised during production
it is currently hard to trace down why its happening
See #75835
* Avoid converting BluetoothServiceInfo to a dict for default discovery
Fixes
```
2022-07-19 09:46:48.303 ERROR (MainThread) [homeassistant] Error doing job: Task exception was never retrieved
Traceback (most recent call last):
File "/Users/bdraco/home-assistant/homeassistant/helpers/discovery_flow.py", line 74, in _async_process_pending_flows
await gather_with_concurrency(
File "/Users/bdraco/home-assistant/homeassistant/util/async_.py", line 201, in gather_with_concurrency
return await gather(
File "/Users/bdraco/home-assistant/homeassistant/util/async_.py", line 199, in sem_task
return await task
File "/Users/bdraco/home-assistant/homeassistant/data_entry_flow.py", line 222, in async_init
flow, result = await task
File "/Users/bdraco/home-assistant/homeassistant/data_entry_flow.py", line 249, in _async_init
result = await self._async_handle_step(flow, flow.init_step, data, init_done)
File "/Users/bdraco/home-assistant/homeassistant/data_entry_flow.py", line 359, in _async_handle_step
result: FlowResult = await getattr(flow, method)(user_input)
File "/Users/bdraco/home-assistant/homeassistant/config_entries.py", line 1484, in async_step_bluetooth
return await self.async_step_discovery(dataclasses.asdict(discovery_info))
File "/opt/homebrew/Cellar/python@3.10/3.10.5/Frameworks/Python.framework/Versions/3.10/lib/python3.10/dataclasses.py", line 1239, in asdict
return _asdict_inner(obj, dict_factory)
File "/opt/homebrew/Cellar/python@3.10/3.10.5/Frameworks/Python.framework/Versions/3.10/lib/python3.10/dataclasses.py", line 1246, in _asdict_inner
value = _asdict_inner(getattr(obj, f.name), dict_factory)
File "/opt/homebrew/Cellar/python@3.10/3.10.5/Frameworks/Python.framework/Versions/3.10/lib/python3.10/dataclasses.py", line 1280, in _asdict_inner
return copy.deepcopy(obj)
File "/opt/homebrew/Cellar/python@3.10/3.10.5/Frameworks/Python.framework/Versions/3.10/lib/python3.10/copy.py", line 172, in deepcopy
y = _reconstruct(x, memo, *rv)
File "/opt/homebrew/Cellar/python@3.10/3.10.5/Frameworks/Python.framework/Versions/3.10/lib/python3.10/copy.py", line 271, in _reconstruct
state = deepcopy(state, memo)
File "/opt/homebrew/Cellar/python@3.10/3.10.5/Frameworks/Python.framework/Versions/3.10/lib/python3.10/copy.py", line 146, in deepcopy
y = copier(x, memo)
File "/opt/homebrew/Cellar/python@3.10/3.10.5/Frameworks/Python.framework/Versions/3.10/lib/python3.10/copy.py", line 231, in _deepcopy_dict
y[deepcopy(key, memo)] = deepcopy(value, memo)
File "/opt/homebrew/Cellar/python@3.10/3.10.5/Frameworks/Python.framework/Versions/3.10/lib/python3.10/copy.py", line 161, in deepcopy
rv = reductor(4)
TypeError: Cannot pickle Objective-C objects
```
* Avoid converting BluetoothServiceInfo to a dict for default discovery
Fixes
```
2022-07-19 09:46:48.303 ERROR (MainThread) [homeassistant] Error doing job: Task exception was never retrieved
Traceback (most recent call last):
File "/Users/bdraco/home-assistant/homeassistant/helpers/discovery_flow.py", line 74, in _async_process_pending_flows
await gather_with_concurrency(
File "/Users/bdraco/home-assistant/homeassistant/util/async_.py", line 201, in gather_with_concurrency
return await gather(
File "/Users/bdraco/home-assistant/homeassistant/util/async_.py", line 199, in sem_task
return await task
File "/Users/bdraco/home-assistant/homeassistant/data_entry_flow.py", line 222, in async_init
flow, result = await task
File "/Users/bdraco/home-assistant/homeassistant/data_entry_flow.py", line 249, in _async_init
result = await self._async_handle_step(flow, flow.init_step, data, init_done)
File "/Users/bdraco/home-assistant/homeassistant/data_entry_flow.py", line 359, in _async_handle_step
result: FlowResult = await getattr(flow, method)(user_input)
File "/Users/bdraco/home-assistant/homeassistant/config_entries.py", line 1484, in async_step_bluetooth
return await self.async_step_discovery(dataclasses.asdict(discovery_info))
File "/opt/homebrew/Cellar/python@3.10/3.10.5/Frameworks/Python.framework/Versions/3.10/lib/python3.10/dataclasses.py", line 1239, in asdict
return _asdict_inner(obj, dict_factory)
File "/opt/homebrew/Cellar/python@3.10/3.10.5/Frameworks/Python.framework/Versions/3.10/lib/python3.10/dataclasses.py", line 1246, in _asdict_inner
value = _asdict_inner(getattr(obj, f.name), dict_factory)
File "/opt/homebrew/Cellar/python@3.10/3.10.5/Frameworks/Python.framework/Versions/3.10/lib/python3.10/dataclasses.py", line 1280, in _asdict_inner
return copy.deepcopy(obj)
File "/opt/homebrew/Cellar/python@3.10/3.10.5/Frameworks/Python.framework/Versions/3.10/lib/python3.10/copy.py", line 172, in deepcopy
y = _reconstruct(x, memo, *rv)
File "/opt/homebrew/Cellar/python@3.10/3.10.5/Frameworks/Python.framework/Versions/3.10/lib/python3.10/copy.py", line 271, in _reconstruct
state = deepcopy(state, memo)
File "/opt/homebrew/Cellar/python@3.10/3.10.5/Frameworks/Python.framework/Versions/3.10/lib/python3.10/copy.py", line 146, in deepcopy
y = copier(x, memo)
File "/opt/homebrew/Cellar/python@3.10/3.10.5/Frameworks/Python.framework/Versions/3.10/lib/python3.10/copy.py", line 231, in _deepcopy_dict
y[deepcopy(key, memo)] = deepcopy(value, memo)
File "/opt/homebrew/Cellar/python@3.10/3.10.5/Frameworks/Python.framework/Versions/3.10/lib/python3.10/copy.py", line 161, in deepcopy
rv = reductor(4)
TypeError: Cannot pickle Objective-C objects
```
Jan Bouwhuis
J. Nick Koston
Marc Mueller
Sid
G Johansson
Joost Lekkerkerker
Erik Montnemery
Jan-Philipp Benecke
Aidan Timson
Franck Nijhof
Paulus Schoutsen
epenet
Jesse Moody
Ville Skyttä
Michael
Aaron Bach
Zach Berger
Jc2k