* Avoid useless time fetch in DataUpdateCoordinator
Since we used the async_call_at helper, it would always call dt_util.utcnow()
to feed the _handle_refresh_interval which threw it away. This meant we had
to fetch time twice as much as needed for each update
* tweak
* compat
* adjust comment
* Refactor translations to reduce dict lookups
All of our cache lookups used:
`cache[language][O(component)][category]`
The cache was designed as
`cache[language][component][category][flatted_key]`
The lookups are now
`cache[language][category][O(component)]`
The cache is now stored as
`cache[language][category][component][flatted_key]`
This allows the catch fetch to avoid looking up
the category each loop
* already a set, and we do not mutate
* add placeholder support to entity name translation
* add negativ tests
* make property also available via description
* fix doc string in translation_placeholders()
* fix detection of placeholder
* validate placeholders for localized strings
* add test
* Cache translation_placeholders property
* Make translation_placeholders uncondotionally return dict
* Fall back to unsubstituted name in case of mismatch
* Only replace failing translations with English
* Update snapshots
* Blow up on non stable releases
* Fix test
* Update entity.py
---------
Co-authored-by: Erik <erik@montnemery.com>
Co-authored-by: Joost Lekkerkerker <joostlek@outlook.com>
* Add Valve integration.
This adds the valve integration discussed in https://github.com/home-assistant/architecture/discussions/975
Most of the code is taken from the cover integration but simplified since valves
can't tilt.
There are a couple outstanding errors I'm not sure how to solve and prevents
me from even making this commit without `--no-verify`.
* Apply PR feedback
* Apply more feedback: Intruduce the bare minimum
* Remove file commited by mistake
* Hopefully this fixes tests
* Match cover's typing and mypy settings
* Change some configuration files
* Fix test
* Increase code coverage a little
* Code coverate inproved to 91%
* 95% code coverage
* Coverate up to 97%
* Coverage 98%
* Apply PR feedback
* Even more feedback
* Add line I shouldn't have removed
* Derive closed/open state from current position
* Hopefully last feedback
* Update homeassistant/components/valve/__init__.py
Co-authored-by: Martin Hjelmare <marhje52@gmail.com>
* Update homeassistant/components/valve/__init__.py
Co-authored-by: Martin Hjelmare <marhje52@gmail.com>
* Remove unnecesary translation
* Remove unused method arguments
* Complete code coverage
* Update homeassistant/components/valve/__init__.py
Co-authored-by: Martin Hjelmare <marhje52@gmail.com>
* Simplify tests
* Update homeassistant/components/valve/__init__.py
Co-authored-by: Martin Hjelmare <marhje52@gmail.com>
* Apply last feedback
* Update tests/components/valve/test_init.py
Co-authored-by: Martin Hjelmare <marhje52@gmail.com>
* Update tests/components/valve/test_init.py
Co-authored-by: Martin Hjelmare <marhje52@gmail.com>
* Update tests/testing_config/custom_components/test/valve.py
Co-authored-by: Martin Hjelmare <marhje52@gmail.com>
* More feedback
* Apply suggestion
* And more feedback
* Apply feedback
* Remove commented code
* Reverse logic to unindent
* Update homeassistant/components/valve/__init__.py
Co-authored-by: Martin Hjelmare <marhje52@gmail.com>
* Update homeassistant/components/valve/__init__.py
Co-authored-by: Martin Hjelmare <marhje52@gmail.com>
* Implement stop valve for Mock valve
* Fix tests now that I've implemented stop_valve
* Assert it's neither opening nor closing
* Use current position instead
* Avoid scheduling executor when opening or closing
* Fix incorrect bitwise operation
* Simplify toggle
* Remove uneeded partial functions
* Make is_last_toggle_direction_open private
* Remove valve from test custom integration
* Improve test coverage
* Address review comments
* Address review comments
* Address review comments
* Update homeassistant/components/valve/__init__.py
Co-authored-by: Martin Hjelmare <marhje52@gmail.com>
* Update tests
---------
Co-authored-by: Martin Hjelmare <marhje52@gmail.com>
Co-authored-by: Erik <erik@montnemery.com>
Reduce overhead connect dispatcher
- We tend to have 1000s (or 10000s) of connected dispatchers which
makes these prime targets to reduce overhead/memory
- Instead of creating new functions to wrap log exceptions each time
use partials which reuses the function body and only create new
arguments
Previous optimizations #103307#93602
* Keep capabilities up to date in the entity registry
* Warn if entities update their capabilities very often
* Fix updating of device class
* Stop tracking capability updates once flooding is logged
* Only sync registry if state changed
* Improve test
* Revert "Only sync registry if state changed"
This reverts commit 1c52571596c06444df234d4b088242b494b630f2.
* Avoid calculating device class twice
* Address review comments
* Revert using dataclass
* Fix unintended revert
* Add helper method
* Drop use of regex in helpers.extract_domain_configs
* Update test
* Revert test update
* Add domain_from_config_key helper
* Add validator
* Address review comment
* Update snapshots
* Inline domain_from_config_key in validator
* Improve OAuth error handling in configuration flows
* Update strings for all integrations that use oauth2 config flow
* Remove invalid_auth strings
* Revert change to release
* Revert close change in aiohttp mock
* Differentiate between warnings and errors in check_config helper
* Update tests
* Treat configuration errors in frontend and its dependencies as errors
* Improve test coverage
* Address review comments
* Improve test coverage
* Improve test coverage
* Address review comments
* Add comment
The contextmanager decorator creates a new context manager every
time its run, but since we only have a single context var, we can
use the same one every time. Creating the contextmanager was roughly
20% of the time time of the template render
I was a bit suprised to find it creates a new context manager
object every time https://stackoverflow.com/questions/34872535/why-contextmanager-is-slow
* Prevent accidentally reusing an entity object
* Fix group reload service
* Revert "Fix group reload service"
* Improve test
* Add tests aserting entity can't be reused
* add supports_response to platform entity services
* support multiple entities in entity_service_call
* support legacy response format for service calls
* revert changes to script/shell_command
* add back test for multiple responses for legacy service
* remove SupportsResponse.ONLY_LEGACY
* Apply suggestion
Co-authored-by: Allen Porter <allen.porter@gmail.com>
* test for entity_id remove None
* revert Apply suggestion
* return EntityServiceResponse from _handle_entity_call
* Use asyncio.gather
* EntityServiceResponse not Optional
* styling
---------
Co-authored-by: Allen Porter <allen.porter@gmail.com>
* Avoid looking up the callable type for HassJob when we already know it
When we connect the frontend we call async_listen with run_immediately MANY
times when we already know the job type (it will always be a callback). This
reduces the latency to get the frontend going
* missing coverage
This was being built every time state was written. When
a robo vac is operating it writes state often which mean building
the icon string over and over again when it rarely changes.
async_has_matching_flow is more likely to be True than hass.is_stopping
This does not make much difference but it was adding noise to a profile
that I am digging into to look for another issue
* Refactor entity service calls to reduce complexity
gets rid of the noqa C901
* Refactor entity service calls to reduce complexity
gets rid of the noqa C901
* short
* Use loop time to set context
loop time is faster than utcnow, and since its only used internally it can
be switched without a breaking change
* fix mocking
* Allow specifying a custom log function for template render
* Bypass template cache when reporting errors + fix tests
* Send errors as events
* Fix logic for creating new TemplateEnvironment
* Add strict mode back
* Only send error events if report_errors is True
* Force test of websocket_api only
* Debug test
* Run pytest with higher verbosity
* Timeout after 1 minute, enable syslog output
* Adjust timeout
* Add debug logs
* Fix unsafe call to WebSocketHandler._send_message
* Remove debug code
* Improve test coverage
* Revert accidental change
* Include severity in error events
* Remove redundant information from error events
* Fix memory leak in dispatcher removal
When we removed the last job/callable from the dict for the
signal we did not remove the dict for the signal which meant
it leaked
* comment
* cleanup a bit more
* Set state of entity with invalid state to unknown
* Add test
* Apply suggestions from code review
Co-authored-by: Robert Resch <robert@resch.dev>
* Update test_entity.py
---------
Co-authored-by: Robert Resch <robert@resch.dev>
* Extend template entities with a script section
This allows making a trigger entity that triggers a few times a day,
and allows collecting data from a service resopnse which can be
fed into a template entity.
The current alternatives are to publish and subscribe to events or to
store data in input entities.
* Make variables set in actions accessible to templates
* Format code
---------
Co-authored-by: Erik <erik@montnemery.com>
* Switch async_track_same_state to use async_call_later
There was no need to use async_track_point_in_utc_time here since
we only need a delay
* update trigger tests
* remove some more utcnow patching
* remove some more utcnow patching
* remove some more utcnow patching
* Don't blow up if validators can't access the issue registry
* Make the check_config script open issue_registry read only
* Update tests/helpers/test_issue_registry.py
Co-authored-by: Martin Hjelmare <marhje52@gmail.com>
---------
Co-authored-by: Martin Hjelmare <marhje52@gmail.com>
* Add service for getting a weather forecast
* Fix translations
* Improve service description
* Improve error handling
* Adjust typing
* Adjust typing
* Adjust service response format
* Improve handling of unrecoverable storage corruption
fixes#96574
If something in storage gets corrupted core can boot loop
or if its integration specific, the integration will fail to
start.
We now complainly loudly in the log, move away the corrupt data
and start fresh to allow startup to proceed so the user can
get to the UI and restore from backup without having to attach
a console (or otherwise login to the OS and manually modify files).
* test for corruption
* ensure OSError is still fatal
* one more case
* create an issue for corrupt storage
* fix key
* persist
* feedback
* feedback
* better to give the full path
* tweaks
* grammar
* add time
* feedback
* adjust
* try to get issue_domain from storage key
* coverage
* tweak wording some more
* Significantly speed up single entity/response service calls
Since the majority of service calls are single entity, we can
avoid creating tasks in this case. Since the multi-entity
service calls always check the result and raise, we can switch
the asyncio.wait to asyncio.gather
* Significantly speed up single entity/response service calls
Since the majority of service calls are single entity, we can
avoid creating tasks in this case. Since the multi-entity
service calls always check the result and raise, we can switch
the asyncio.wait to asyncio.gather
* revert
* cannot be inside pytest.raises
* one more
* Update homeassistant/helpers/service.py
* Ensure async_get_system_info does not fail if supervisor is unavailable
fixes#96470
* fix i/o in the event loop
* fix tests
* handle some more failure cases
* more I/O here
* coverage
* coverage
* Update homeassistant/helpers/system_info.py
Co-authored-by: Paulus Schoutsen <balloob@gmail.com>
* remove supervisor detection fallback
* Update tests/helpers/test_system_info.py
---------
Co-authored-by: Paulus Schoutsen <balloob@gmail.com>
* Differentiate between device info types
* Update allowed fields
* Update homeassistant/helpers/entity_platform.py
Co-authored-by: Martin Hjelmare <marhje52@gmail.com>
* Split up message in 2 lines
* Use dict for device info types
* Extract device info function and test error checking
* Simplify parsing device info
* move checks around
* Simplify more
* Move error checking around
* Fix order
* fallback config entry title to domain
* Remove fallback for name to config entry domain
* Ensure mocked configuration URLs are strings
* one more test case
* Apply suggestions from code review
Co-authored-by: Erik Montnemery <erik@montnemery.com>
---------
Co-authored-by: Martin Hjelmare <marhje52@gmail.com>
Co-authored-by: Erik Montnemery <erik@montnemery.com>
* Mark scripts as response optional, make it always return a response if return_response is set
* Update test_init.py
* Revert "Update test_init.py"
This reverts commit 8e113e54db.
* Split + add test
* Tune keep alives for polling integrations
aiohttp closes the connection after 15s by default,
and httpx closes the connection after 5s by default.
We have a lot of integrations that poll every 10-60s
which create and tear down connections over and over.
Set the keep alive time to 65s to maximize connection
reuse and avoid tls negotiation overhead
* Apply suggestions from code review
* adjust
* Update log message when referenced entity not found
* Update homeassistant/helpers/service.py
Co-authored-by: Martin Hjelmare <marhje52@gmail.com>
* Update test
---------
Co-authored-by: Martin Hjelmare <marhje52@gmail.com>
* Allow returning a script variable from a script
* Don't allow returning a template result
* Raise if response variable is undefined
* Add test
* Update homeassistant/helpers/script.py
Co-authored-by: Paulus Schoutsen <balloob@gmail.com>
* Format code
---------
Co-authored-by: Paulus Schoutsen <balloob@gmail.com>
* Add image platform to the template integration
* Set a default name
* Fix typo
* Add tests
* Improve test coverage
* Derive content-type from fetched image
* Add service response data for listing calendar events
Add the capability of response data for for the entity component.
* Rename input arguments and add service description
* Improve list events to be more user friendly
Allow the end date to be determined based on a relative time duration. Make the start time optional and set to "now". Add additional test coverage. Update demo calendar to actually perform date range checks.
* Wrap docstrings properly.
* Increase test coverage
* Update to use new API calls
* Readability improvements
* Wrap docstrings
* Require at least one of end or duration
* Check for multiple entity matches earlier in the request
* Update documentation strings
* Improve service response data APIs
Make the API naming more consistent, and require registration that a
service supports response data so that we can better integrate with
the UI and avoid user confusion with better error messages.
* Improve test coverage
* Add an enum for registering response values
* Assign enum values
* Convert SupportsResponse to StrEnum
* Update service call test docstrings
* Add tiny missing full stop in comment
---------
Co-authored-by: Franck Nijhof <frenck@frenck.nl>
* Allow scripts service actions to save return values
* Simplify script service response data
* Rename result_variable to response_variable based on feedback
* Speed up entity service calls
- Avoid permissions check if the caller is an admin
- Use set intersection instead of linear search of entity platforms to find entities
* tweak
* fix light test to not use an admin user
* Update return signature of service calls
* Add timeout error handling in websocket api for service calls
* Update recorder tests to remove assertion on service call
* Remove timeout behavior and update callers that depend on it today
* Fix tests
* Add missing else
* await coro directly
* Fix more tests
* Update the intent task to use wait instead of timeout
* Remove script service call limits and limit constants
* Update tests that depend on service call limits
* Use wait instead of wait_for and add test
* Update homeassistant/helpers/intent.py
Co-authored-by: Martin Hjelmare <marhje52@gmail.com>
---------
Co-authored-by: Paulus Schoutsen <balloob@gmail.com>
Co-authored-by: Martin Hjelmare <marhje52@gmail.com>
* Name unnamed binary sensors by their device class
* Update type annotations
* Fix loading of entity component translations
* Add test
* Update integrations
* Set abode and rfxtrx binary_sensor name to None
* Revert changes in homekit_controller
* Remove unneeded checks for Entity.platform
* Update tests
* Prevent breaking integrations without an EntityComponent
* Warn when entity has no platform
* support live reload of intent_script
* add services.yaml
* update tesls for full code coverage
* Update based on feedback
* fix intent_script reload when no intent_script config
* Update homeassistant/helpers/intent.py
* update tests to handle no_existing better
---------
Co-authored-by: Paulus Schoutsen <paulus@home-assistant.io>
* Migrate restore_state helper to use registry loading pattern
As more entities have started using restore_state over time, it
has become a startup bottleneck as each entity being added is
creating a task to load restore state data that is already loaded
since it is a singleton
We now use the same pattern as the registry helpers
* fix refactoring error -- guess I am tired
* fixes
* fix tests
* fix more
* fix more
* fix zha tests
* fix zha tests
* comments
* fix error
* add missing coverage
* s/DATA_RESTORE_STATE_TASK/DATA_RESTORE_STATE/g
* Fix refactoring error with updating in sequence
see #93649
* coverage
* make sure entities are being updated in parallel
* make sure entities are being updated in sequence
* Defer creating tasks we know are going to wait on the entity platform semaphore
When looking at whats going on with aiomonitor-ng, I noticed
we end up creating a lot of tasks that block waiting for the
executor because of the entity platform parallel_updates semaphore.
When we know the tasks are going to block we now await them
in sequence to avoid feeding the loop a herd of tasks that
will block on the semaphore
* change during iteration fix
* change during iteration fix
* cleanup
* cleanup
* fix vizio test
* Improve performance when serializing small bits of JSON
Making json_bytes a partial reduced the run time to
build the small JSON messages by ~18.75%
We serialize a lot of small messages over the websocket
* typing
* Switch some frequently called call_laters to call_at
call_at is a bit faster than call_later since call_later
is a wrapper around call_at.
We call at lot of these at startup so it helps a bit when
we are resource constrained
* update test
* update test
J. Nick Koston
Marc Mueller
Lars R
Erik Montnemery
Michael Hansen
Jan-Philipp Benecke
Petru Paler
Michael
Martijn van der Pol
Robert Resch
Bram Kragten
Ville Skyttä
Miguel Camba
Jan Bouwhuis
Paul Bottein
schelv
Aarni Koskela
Ruslan Sayfutdinov
Allen Porter
Kevin Stillhammer
Raman Gupta
G Johansson
Nicolas van de Walle
Joost Lekkerkerker
Dave T
Joakim Plate
Shay Levy
Michael Arthur
Franck Nijhof
Paulus Schoutsen
Michael Benz