* Schedule periodic coordinator updates as background tasks.
Currently, the coordinator's periodic refreshes delay startup because they are not scheduled as background tasks. We will wait if the startup takes long enough for the first planned refresh. Another coordinator's scheduled refresh will be fired on busy systems, further delaying the startup. This chain of events results in the startup taking a long time and hitting the safety timeout because too many coordinators are refreshing.
This case can also happen with scheduled entity refreshes, but it's less common. A future PR will address that case.
* periodic_tasks
* periodic_tasks
* periodic_tasks
* merge
* merge
* merge
* merge
* merge
* fix test that call the sync api from async
* one more place
* cannot chain
* async_run_periodic_hass_job
* sun and pattern time changes from automations also block startup
* Revert "sun and pattern time changes from automations also block startup"
This reverts commit 6de2defa05.
* make sure polling is cancelled when config entry is unloaded
* Revert "Revert "sun and pattern time changes from automations also block startup""
This reverts commit e8f12aad55.
* remove DisabledError from homewizard test as it relies on a race
* fix race
* direct coverage
Move analytics setup to stage 1 to avoid delaying frontend startup
analytics was only needed in the frontend startup phase for onboarding.
Its very unlikely the user will be able to complete the onboarding
steps and get to the analytics screen before analytics is done loading
so we can delay loading it until stage 1. To be absolutely sure that
it is ready, the core_config step in onboarding will wait to proceed
if it is some how still being setup
* Avoid errors when no internet connection
* Add error
* Create task in HA
* change from matter to automower
* tests
* Update homeassistant/components/husqvarna_automower/coordinator.py
Co-authored-by: Martin Hjelmare <marhje52@gmail.com>
* address review
* Make websocket optional
* fix aioautomower version
* Fix tests
* Use stored websocket
* reset reconnect time after sucessful connection
* Typo
* Remove comment
* Add test
* Address review
---------
Co-authored-by: Martin Hjelmare <marhje52@gmail.com>
* Dont remove api key during migration
* Fix return
* Fix test
* Make lambda more readable
* Move fixtures to init, move migration test to test_init.py
* Refactor config_entry data assignment
* Refactor system_bridge migration tests
* Fix type for debug message
* Fix type for debug message
* Remove duplicated unused code (rebase error)
* Refactor test_migration_minor_2_to_1 to test_migration_minor_future_to_2
* Fix version check in async_migrate_entry
* Update migration logic to handle future minor version
* Add ConfigEntryState assertion in test_init.py
* Change condition to minor_version < 2
Co-authored-by: Martin Hjelmare <marhje52@gmail.com>
* Refactor system bridge migration tests
* Remove minor downgrade code
* Update tests/components/system_bridge/test_init.py
Co-authored-by: Martin Hjelmare <marhje52@gmail.com>
* Update tests/components/system_bridge/test_init.py
Co-authored-by: Martin Hjelmare <marhje52@gmail.com>
* Apply suggestions from code review
Co-authored-by: Martin Hjelmare <marhje52@gmail.com>
* Move dataclass to store requested data to data.py
* Use dataclass in config flow
* Move media player and sensor onto data.py dataclass
* Move data and handler inside validate
---------
Co-authored-by: Martin Hjelmare <marhje52@gmail.com>
These were very likely to be cached so they were low on the
list to migrate, but since they are called in the event loop
its best to be sure we do no blocking I/O
discovered in https://github.com/home-assistant/core/pull/112295#discussion_r1513505710
We only checked if the last language had files to load
instead of all of them. The checks for each language
are the same because the only reason we would skip
a language is a missing/broken integration or the integration
is a single file. Both of these loop conditions are always
the same reguardless of the language so the check worked
* Add system_health to the hassio pre-imports to avoid a late executor job
`2024-03-05 17:01:33.034 DEBUG (MainThread) [homeassistant.loader] Importing platforms for hassio executor=[system_health] loop=[] took 0.12s`
This one does not take that much time but it happens at a time where
the import executor is the most busy during startup
* key
* move patch as its too early now
* Fix config modules being imported in the event loop
There was a late import in this integration because of the circular import.
The code has been rearranged to avoid the circular imports
* fixes
* fixes
* fix patching
* make eager
* remove unrelated change from this branch
We previously checked Integration.config_flow to see if we should
pre-import the config flow, but this is now always set for some
integration like `homeassistant_green`, `hassio`, etc. Instead
we can add it to the rest of the platforms since we already know
which files exist. This simplifies the logic and ensures the pre-import
still happens if the file is there even if its not listed in the
manifest
`2024-03-04 22:54:31.906 DEBUG (MainThread) [homeassistant.loader] Importing platforms for homeassistant_green executor=[config_flow] loop=[] took 2.74s`
* Migrate remaining calls in config modules to async_get_component
There were a few cases that were still using get_component that
could have done blocking I/O in the event loop, although it
was unlikely.
The caching check in async_get_component has been moved
up to avoid creating the future if the module is already in
the cache
* fix one more
* Handle case where deCONZ light entity might not report a supported color mode
* If in an unknown color mode set ColorMode.UNKNOWN
* Fix comment from external discussion
* add deprecaction
* Deprecate old config options for mqtt json light
* Do not deprecate brightness flag
* Enable brightness support by default
* Keep `false` as default for brightness flag
* Add warnings and register issue
* log warning and register on use of color_mode flag
* Remove redundant cv.deprecated logging + follow up comments
Some of the data we had to search for was already available
in a dict or underlying data structure. Make it available
instead of having to build it every time.
There are more places these can be used, but I only did
the device registry cleanup for now
* Adding Energy
* Adding Energy
* Work in progress
* Add fixtures
* Add product info
* Add sensors
* Add icons
* Update metadata
* Use SensorEntityDescription for Energy
* Use ENERGY_STORAGE
* Add tests
* Fix coverage
* Update wall connector precision and units
* Change devices
* Fix serial number
* Add icons and VIN to wall connector
* Fix serial number again
* Update snapshots
* Use timestamp for minutes to arrival
* Cleanup snapshot
* Improvements
* Update fixture
* Add "code" to translations
* Add "code" to snapshot
* Use async_add_entities once
* Disable a bunch of sensors
* Ruff
* Improve fixture and test coverage
* Regen Snapshots
* Add init to coordinator
* Add 'issues' template function for listing active issues.
* Add issue template function test
* Add 'issue' template function for getting specific issue by domain and issue_id
* Remove comment
* Fix function description
* Remove reduntant function,
Fix tests
* remove pass_context
* remove issues filter
Co-authored-by: Erik Montnemery <erik@montnemery.com>
---------
Co-authored-by: Erik Montnemery <erik@montnemery.com>
* Import in the executor by default for core integration
* merge correct branch in
* Group loading of platforms in the import executor
* adjust test
* remove other pr
* Fix async_prepare_setup_platform test
The message changed because the order changed but was not caught before
merge because it required the combination of PRs to change
the error message
* fix
* tweak
* fix
* self review
* review
* fix hue mocking
* Update homeassistant/loader.py
Co-authored-by: Paulus Schoutsen <balloob@gmail.com>
* lint
* Fix async_get_component loading in the executor when the module is already loaded
The sys.modules check was incorrect (only on dev)
* fix
* Avoid multiple executor jobs with concurrent calls to async_get_component
Return a future that can be awaited if the component is
curently being loaded
* adjust
* coverage
* coverage
* concurrent platforms load test
* doc strings
* coverage
---------
Co-authored-by: Paulus Schoutsen <balloob@gmail.com>
* Add support for local (lan) panel integration
* Fix merge conflicts
* Remove executable flag from non-executable files
* Fix tests
* Update homeassistant/components/elmax/__init__.py
Shorten comment
Co-authored-by: Erik Montnemery <erik@montnemery.com>
* Fix typehint
* Rename DummyPanel into DirectPanel
* Update homeassistant/components/elmax/__init__.py
Rewording
Co-authored-by: Erik Montnemery <erik@montnemery.com>
* Update homeassistant/components/elmax/__init__.py
Rewording
Co-authored-by: Erik Montnemery <erik@montnemery.com>
* Refactor option step into menu step
* Change requirement statement
* Refactor dictionary key entries to use existing constants
* Align step names to new constants
* Align step names to new constants amd align tests
* Align step names to new constants amd align tests
* Align step names to new constants
* Simplify logic to handle entire entry instead of a portion of the state
* Simplify working mode checks
* Add data_description dictionary to better explain SSL and FOLLOW_MDSN options
* Add support for local (lan) panel integration
* Fix merge conflicts
* Remove executable flag from non-executable files
* Fix tests
* Update homeassistant/components/elmax/__init__.py
Shorten comment
Co-authored-by: Erik Montnemery <erik@montnemery.com>
* Fix typehint
* Rename DummyPanel into DirectPanel
* Update homeassistant/components/elmax/__init__.py
Rewording
Co-authored-by: Erik Montnemery <erik@montnemery.com>
* Update homeassistant/components/elmax/__init__.py
Rewording
Co-authored-by: Erik Montnemery <erik@montnemery.com>
* Refactor option step into menu step
* Change requirement statement
* Refactor dictionary key entries to use existing constants
* Align step names to new constants
* Align step names to new constants amd align tests
* Align step names to new constants amd align tests
* Align step names to new constants
* Simplify logic to handle entire entry instead of a portion of the state
* Simplify working mode checks
* Add data_description dictionary to better explain SSL and FOLLOW_MDSN options
* Add newline at end of file
* Remove CONF_ELMAX_MODE_DIRECT_FOLLOW_MDNS option
* Fix Ruff pre-check
---------
Co-authored-by: Erik Montnemery <erik@montnemery.com>
* Update System Bridge to support version 4.x.x and above
Update systembridgeconnector to version 4.0.0.dev4
Update system_bridgeconnector version to 4.0.0.dev6
Refactor WebSocket client handling in config_flow.py
Update strings
Update data handling
Add default field values to SystemBridgeCoordinatorData
Add version check and issue creation for unsupported System Bridge versions
Update coordinator.py to set disks and memory to None
Update system bridge coordinator to use token instead of API key
Update systembridgeconnector version to 4.0.0.dev7
Update systembridgeconnector version to 4.0.0.dev8
Update systembridgeconnector version to 4.0.0.dev9
Changes
Update units
Fix GPU memory calculation in sensor.py
Update GPU memory unit of measurement
Add translation keys for binary sensor names
Cleanup
Add async_migrate_entry function for entry migration
Update systembridgeconnector version to 4.0.0.dev10
Update systembridgeconnector version to 4.0.0.dev11
Add version check and authentication handling
Update token description in strings.json
Fix skipping partitions without data in system_bridge sensor
Update systembridgeconnector version to 4.0.0.dev12
Update systembridgeconnector version to 4.0.0
Add check for unsupported version of System Bridge
Update systembridgeconnector version to 4.0.1
Update debug log message in async_setup_entry function
Remove debug log statement
Fixes
Update key to token
Update tests
Update tests
Remove unused import in test_config_flow.py
Remove added missing translations for another PR
Refactor CPU power per CPU calculation
Make one liner into lambda
Refactors
Fix exception type in async_setup_entry function
Move checks to class and set minor version
Remove unnecessary comment in gpu_memory_free function
Remove translation_key for memory_used_percentage sensor
Reverse string change
Update token placeholder in strings.json
Remove suggested_display_precision from sensor descriptions
Remove suggested_display_precision from GPU sensor setup
Refactor sensor code
* Update migrate entry
* Refactor GPU-related functions to use a decorator
* Move per cpu functions to use decorator
* Refactor functions to use decorators for data availability
* Remove CONF_API_KEY from config entry data
* Add test for migration
* Refactor import statement in test_config_flow.py
Since config_entries always requires the config_flow to be loaded
to check for migrations, load it if we know it exists when loading
the underlying integration
* Split up hassio coordinator and data into new file to allow preload
Since we cannot mark hassio as having a config_flow, it will
not get preloaded and since cloud will almost always load right
after it and block the import executor, we want to preload
the hassio config_flow and platform modules so the other
dependants can continue on while cloud is being imported
to not delay startup
* tweak
* tweak
* Add tests for number of devices created
* Change default fixture to 2 devices in one system
* Update snapshot
* Change deviuce to test
* Merge tests to one module
* Avoid trying to import platforms that do not exist
* adjust
* fixes
* cleanup
* cleanup
* cleanup
* Apply suggestions from code review
* docs
* fixes
* fixes
* comment
* coverage
* coverage
* coverage
* Switch config to use async_get_component
This was another path where integrations that were marked to load in the executor
would be loaded in the loop
* Switch config to use async_get_component/async_get_platform
This was another path where integrations that were marked to load in the executor
would be loaded in the loop
* merge
* refactor
* refactor
* coverage
* preen
* preen
* Avoid expensive inspect calls in config validators
inspect has a performance problem https://github.com/python/cpython/issues/92041
We now avoid calling inspect unless we are going to log
* remove unused
* reduce
* get_integration_logger
* Rename device.py to hub.py
* Rename AxisNetworkDevice to AxisHub
* Move hub.py into hub package
* Rename get_axis_device to get_axis_api
* Split out get_axis_api to its own module
* Rename device object to hub
* Rename device to api in config flow
* Convenience method to get hub
* Fix utility meter reset without tarrifs
When using a utility_meter helper set up with a "Meter reset cycle" of
"No cycle" it is impossible to properly reset the meter to 0 and utilize
the "last reset" and "last period" attributes on the helper.
With these changes the service call can now directly reset a meter in
such a configuration and the UI selector has been updated to include
utility meters in the selection list.
Fixes#96920
* Undo UI target selection modes
As requested to not cause confusion by being able to reset individual
meters when using multiple tariffs.
* Adjust logic to only reset standalone sensors
The original logic would allow resetting meters with multiple tariffs
which was an unintended side effect.
* Vallox: reset filter button
* Better names
* Change from button to date platform
* Review
* Fix
* Drop ValloxDateEntityDescription
* Stale docstrings
* Stale docstring
* Move HassSetPosition to homeassistant domain
* Add test for unsupported domain with HassSetPosition
* Split service intent handler
* cleanup
---------
Co-authored-by: Paulus Schoutsen <balloob@gmail.com>
* Use correct service name with satellite + local wake word detection
* Don't load platforms for satellite services
* Update homeassistant/components/wyoming/data.py
Co-authored-by: Paulus Schoutsen <balloob@gmail.com>
* Fix ruff error
---------
Co-authored-by: Paulus Schoutsen <balloob@gmail.com>
* Add counter entities to the ZHA coordinator device
* rework to prepare for non coordinator device counters
* Initial scaffolding to support quirks v2 entities
* update for zigpy changes
* add assertion error message
* clean up test
* update group entity discovery kwargs
* constants and clearer names
* apply custom device configuration
* quirks switches
* quirks select entities
* quirks sensor entities
* update discovery
* move call to super
* add complex quirks v2 discovery test
* remove duplicate replaces
* add quirks v2 button entity support
* add quirks v2 binary sensor entity support
* fix exception in counter entitiy discovery
* oops
* update formatting
* support custom on and off values
* logging
* don't filter out entities quirks says should be created
* fix type alias warnings
* sync up with zigpy changes and additions
* add a binary sensor test
* button coverage
* switch coverage
* initial select coverage
* number coverage
* sensor coverage
* update discovery after rebase
* coverage
* single line
* line lengths
* fix double underscore
* review comments
* set category from quirks in base entity
* line lengths
* move comment
* imports
* simplify
* simplify
Because the setup again was scheduled as a task, it would
not unset self._async_cancel_retry_setup in time and we would
try to unsub self._async_cancel_retry_setup after it had already
fired. Change it to call a callback that runs right away so it
unsets self._async_cancel_retry_setup as soon as its called
so there is no race
fixes#111796
* Deprecate @bind_hass and log error if used inside custom component
* Log also when accessing `hass.components`
* Log warning only when `hass.components` is used
* Change version
* Process code review
* Use `None` instead of `"unknown"` when the current version is unknown
* Only use the current file version from the OTA notification
* Use `sw_version`, if available, and update `current_file_version`
* Assume the current version is the latest version
* Fix lint errors
* Use `image` instead of `firmware`
* Include a changelog if updates expose it
* Clear latest firmware only after updating the installed version
* Bump minimum zigpy version to 0.63.0
* Create a data update coordinator to consolidate updates
* Fix overridden `async_update`
* Fix most unit tests
* Simplify `test_devices` to fix current tests
* Use a dict comprehension for creating mocked entities
* Fix unit tests (thanks @dmulcahey!)
* Update the currently installed version on cluster attribute update
* Drop `PARALLEL_UPDATES` now that we use an update coordinator
* Drop `_reset_progress`, it is already handled by the update component
* Do not update the progress if we are not supposed to be updating
* Ignore latest version (e.g. if device attrs changed) if zigpy rejects it
* Clean up handling of command id in `Ota.cluster_command`
* Start progress at 1%: 0 and False are considered equal and are filtered!
Use `ceil` instead of remapping 1-100
* The installed version will be auto-updated when the upgrade succeeds
* Avoid 1 as well, it collides with `True`
* Bump zigpy to (unreleased) 0.63.2
* Fix unit tests
* Fix existing unit tests
Send both event types
Globally enable sending both event types
* Remove unnecessary branches
* Test ignoring invalid progress callbacks
* Test updating a device with a no longer compatible firmware
* Import tplink in the executor to avoid blocking the event loop
2024-02-27 22:44:19.908 DEBUG (MainThread) [homeassistant.loader] Component tplink import took 1.620 seconds (loaded_executor=False)
* patch out discovery because it happens too fast now
Avoid creating system monitor sensors for non-dirs
Currently we create sensors for /etc/hosts, /etc/asound.conf, since
they are bind mounts in the container. These all have to have
their own coordinator
* Reduce latancy to load storage by making the task eager
This changes the semantics a bit under the hood because it
can raise sooner which means we do not store the task
as _load_task if it raises right away. That means
concurrent calls that result in failure are likely to try
again now which will be a tiny performance hit for this
case.
* fix
* will now finish in time
* Add Grid import export to enphase Envoy
* Update snapshot for labels dict element in entity registry
* use identity check for enum
* Revert use of identity check, didn't add entities
* Implement review feedback for tests
* ct phase sensors disabled by default
* import PHASENAMES from pyenphase
* Update tests/components/enphase_envoy/test_sensor.py
* Update tests/components/enphase_envoy/test_sensor.py
---------
Co-authored-by: J. Nick Koston <nick@koston.org>
* Always allow ignore and unignore flows for single config entry integrations
* Update tests/test_config_entries.py
Co-authored-by: Martin Hjelmare <marhje52@gmail.com>
---------
Co-authored-by: Martin Hjelmare <marhje52@gmail.com>
* introduce preserve last value option
* improve comments
* add unit test
* skip scheduling purge on a preserved value
* do not schedule sensor update if preserving last value
* fix unit test to use new mock time pattern
pattern introduced in https://github.com/home-assistant/core/pull/93499
* rename preserve_last_val to keep_last_sample
* add keep_last_sample config validation
* Deprecate Logi Circle integration
* Update homeassistant/components/logi_circle/__init__.py
Co-authored-by: G Johansson <goran.johansson@shiftit.se>
---------
Co-authored-by: G Johansson <goran.johansson@shiftit.se>
* Remove zeroconf from ssdp after deps
This was added in #36277 but is no longer needed since
we setup discovery integrations ahead of time to ensure
their deps are updated before other integrations can load
them
* adjust test
* Use an eager task in the update coordinator scheduled refresh
We have a lot of places that will not suspend because the refresh function
decides it does not need to update. Currently these have to be scheduled
on the event loop even though they are a noop.
Since _handle_refresh_interval is subclassed in some integrations, I created
a dunder wrapper function to avoid integraions subclassing it
* fix time fires outside of patch
* Deconflict based on wake word
* Undo test
* Make wake up key a string, rename error
* Update snapshot
* Change to "wake word phrase" and normalize
* Move normalization into the wake provider
* Working on describe
* Use satellite info to resolve wake word phrase
* Add test for wake word phrase
* Match phrase with model name in wake word provider
* Check model id
* Use one constant wake word cooldown
* Update homeassistant/components/assist_pipeline/error.py
Co-authored-by: Paulus Schoutsen <balloob@gmail.com>
* Fix wake word tests
---------
Co-authored-by: Paulus Schoutsen <balloob@gmail.com>
* rebase off dev
* Update homeassistant/components/weatherflow_cloud/const.py
Co-authored-by: Joost Lekkerkerker <joostlek@outlook.com>
* Addressing 1st round of PR Comments
* Update homeassistant/components/weatherflow_cloud/config_flow.py
Co-authored-by: Joost Lekkerkerker <joostlek@outlook.com>
* addressing PR Comments
* fixing last comment that i can see
* Update homeassistant/components/weatherflow_cloud/coordinator.py
OOPS
Co-authored-by: G Johansson <goran.johansson@shiftit.se>
* Update homeassistant/components/weatherflow_cloud/weather.py
Co-authored-by: G Johansson <goran.johansson@shiftit.se>
* Update homeassistant/components/weatherflow_cloud/coordinator.py
Co-authored-by: G Johansson <goran.johansson@shiftit.se>
* switching to station id
* Update homeassistant/components/weatherflow_cloud/strings.json
Co-authored-by: G Johansson <goran.johansson@shiftit.se>
* addressing PR
* Updated tests to be better
* Updated tests accordingly
* REAuth flow and tests added
* Update homeassistant/components/weatherflow_cloud/strings.json
Co-authored-by: G Johansson <goran.johansson@shiftit.se>
* Update homeassistant/components/weatherflow_cloud/coordinator.py
Co-authored-by: G Johansson <goran.johansson@shiftit.se>
* Addressing PR comments
* Apply suggestions from code review
* ruff fix
---------
Co-authored-by: Joost Lekkerkerker <joostlek@outlook.com>
Co-authored-by: G Johansson <goran.johansson@shiftit.se>
* Add support for pre-imports at setup time
alternative solution to #111331
* refactor
* refactor
* refactor
* mark >1.0s integrations
* no point in executor if already loaded
* no point in executor if already loaded
* cleanup
* cleanup
* two more
* one more
* analytics loads a lot more integrations
* cloud
* debug
* psutil, hardwre
* try zha
* Update homeassistant/setup.py
* await
* comments
* coverage
* coverage
* coverage
* move logic to loader
* move logic to loader
* preserve comments
* Image entity media source
* MJPEG streaming
* Update on change rather than fixed interval
* Only send boundary twice
* return when image has no data
* Write each frame twice
* Use friendly name when browsing
* Fix sending of double frame
* Initial image proxy test
* Improve proxy stream test
* Refactor
* Code review fixes
* Allow setting if we support multiple config entries in config flow
* Move property to config flow instead of flow handler
* Move marking an integration as single instance only to manifest
* Revert line remove
* Avoid init a config flow or adding a new entry on a single instance with an entry
* Revert changes in test
* Process code review comments
* Apply code review suggestion
* Use discovery flow helper for hardware integrations
The discovery flow helper defers loading discovered integrations until after startup
to improve startup reliability.
* Use discovery flow helper for hardware integrations
The discovery flow helper defers loading discovered integrations until after startup
to improve startup reliability. Since hardware was not listed in as a
discovery integration, the notification for new discoveries was missing.
The entity was removed before the entity registry could update it
```
Traceback (most recent call last):
File "/Users/bdraco/home-assistant/homeassistant/helpers/entity.py", line 1482, in _async_process_registry_update_or_remove
assert registry_entry is not None
AssertionError
```
* Subscribe to Traccar Server events
* No need to unsubscribe on error
* typo
* rename _attrs
* arg type
* reorder return type
* more spesific
* Update stale docstring
* Avoid circular import in Storage.async_delay_save
We call Storage.async_delay_save for every entity being added or removed
from the registry. The late import took more time than everything else
in the function.
* Avoid reschedule churn in Storage.async_delay_save
When we are adding or removing entities we will call async_delay_save
quite often which has to add and remove a TimerHandle on the event loop
which can add up when there are a lot of registry items changing.
If the timer handle still has 80% of the time remaining on it
we will avoid resceduling and let it fire at the time the
original async_delay_save call was made. This ensures we
do not force the event loop to rebuild its heapq because
too many timer handlers were cancelled at once
* div0
* add coverage for 0 since we had none
* fix bad conflict
* tweaks
* tweaks
* tweaks
* tweaks
* tweaks
* tweaks
* more test fixes
* mqtt tests rely on event loop overhead