* Reduce latancy to load storage by making the task eager
This changes the semantics a bit under the hood because it
can raise sooner which means we do not store the task
as _load_task if it raises right away. That means
concurrent calls that result in failure are likely to try
again now which will be a tiny performance hit for this
case.
* fix
* will now finish in time
* Use an eager task to setup entity platforms
Ideally we would have awaited this function instead, but we want
to shield it from cancellation so we wrap it in asyncio.shield
which schedules it as a task. Since we have integrations that
never suspend in async_setup_entry, we can avoid scheduling on
the evnet loop with an eager task for this case
* its an executor future
* its an executor future
* fix
* doc string lied
* Use an eager task in the update coordinator scheduled refresh
We have a lot of places that will not suspend because the refresh function
decides it does not need to update. Currently these have to be scheduled
on the event loop even though they are a noop.
Since _handle_refresh_interval is subclassed in some integrations, I created
a dunder wrapper function to avoid integraions subclassing it
* fix time fires outside of patch
* Use eager task creation for entity platform polling
We have lots of places where `async_update` does not suspend and
is only a coro because its required to be a coro to run in the
event loop
* try again
The likely outcome from most discoveries is an near immediate abort
so we run them eagerly to avoid having to schedule on the event loop
for the common case
* Avoid circular import in Storage.async_delay_save
We call Storage.async_delay_save for every entity being added or removed
from the registry. The late import took more time than everything else
in the function.
* Avoid reschedule churn in Storage.async_delay_save
When we are adding or removing entities we will call async_delay_save
quite often which has to add and remove a TimerHandle on the event loop
which can add up when there are a lot of registry items changing.
If the timer handle still has 80% of the time remaining on it
we will avoid resceduling and let it fire at the time the
original async_delay_save call was made. This ensures we
do not force the event loop to rebuild its heapq because
too many timer handlers were cancelled at once
* div0
* add coverage for 0 since we had none
* fix bad conflict
* tweaks
* tweaks
* tweaks
* tweaks
* tweaks
* tweaks
* more test fixes
* mqtt tests rely on event loop overhead
* Convert debouncer async_shutdown to be a normal function
nothing was being awaited here and the shutdown call was only used
in integrations marked internal and other internals. Its possible
that a custom component might have been using the method but it
seemed uncommon enough that it did not warrent marking as a breaking
change. The update coordinator is no longer awaiting anything in
async_shutdown either now but it seemed likely that this use
would get subclassed.
* fix
Avoid linear search to remove from entity registry index
Because the entity registry index needs to preserve insertion order
for backwards compat, a list was used for the index. Because some
config entries/devices/areas have a large amount of entities, removing
the entities, the O(n) time complexity of removing from a list can
slow down reloads. As python has no orderedset in stdlib, use
a dict since it preserves insertion order has O(1) add/remove
time complexity for the average case
* Refactor keyed event trackers to avoid refactoring risk
Follow to https://github.com/home-assistant/core/pull/110978#discussion_r1496771106
I had to do some type ignores because of the EventType vs Event
which is hopefully not going to be needed after the next mypy
* delete constants only used one in other const
* no field
* fixes
* less refactoring later
* less refactoring later
* keep const
* Make adding entities in storage collection a normal function
Nothing is awaited when adding
* cleanup
* cleanup
* cleanup
* cleanup
* reduce
* reduce
* reduce
* reduce
* tweak
These functions created tasks to run small validators, and the cost of
creating all the tasks was more expensive than running the validators
themselves. Since the code is unlikely to suspend its more efficient to
await them in series.
* Add async_schedule_call to the Debouncer
async_schedule_call allows the Debouncer to schedule a call
from a callback without having to create tasks to run
async_call
* Update homeassistant/helpers/debounce.py
* Make device registry cleanup all callback function
* fix typing, code supported callback functions, but typing did not
* fixes
* fixes
* fix
* we had no coverage for other job types
* we had no coverage for other job types
* Reduce overhead to load multiple languages in translations
Instead of loading in a task, we now group everything
to be loaded into a single executor job
* fixes
* fixes
* fixes
* fixes
* fixes
* update tests
* add missing coverage (was existing)
* Add valve to HassTurnOn/Off
* Add set position for valves
* Add set position to covers
* Add HassTurnOn/Off for vacuums
* Add media player intents
* Split out vacuum intents
* Address comments
* Extra test
* Avoid creating tasks for checking integrations platforms
This is a followup to #110743 to avoid creating a task to check
if the integration platform exists. We created tasks because
we needed to await async_get_integrations but since its always
called from EVENT_COMPONENT_LOADED firing, we can use the
async_get_loaded_integration version which does not need
to be awaited. This eliminates one task for every loaded
component
* there is no more race risk
* reduce
* coro or callback
* reduce
* tweak
* race safe
* fix type
* fixes
* use built-in helper to make it smaller
* use built-in helper to make it smaller
* use built-in helper to make it smaller
* add coverage to ensure exceptions are logged
* improve readability a bit
* platforms
* Make context in data entry flow possible to modify for subclasses
* mypy
* Make get_context
* base view
* Review comments
* Remove context from options flow
* Add failing tests for package config validation error wrapping
* Wrap package schema validation errors in HomeAssistantError
* Fix yamllint errors
* Rework package merge validation
Ignore invalid package definitions instead of failing startup.
Output error messages with locations if possible when a package
definition has errors.
* Ruff format
* Fix linter errors
* Move package_definition_schema to module scope
* Move inner function to module level
* Merge exception handlers
Merge exception handlers for config schema validation and package merge
to avoid untested code branches
* Fix long lines and doc strings
* More minor changes to exception handler
---------
Co-authored-by: Erik Montnemery <erik@montnemery.com>
* Index area_ids in the entity registry
I missed that these are used in _resolve_area in search.
Eventually we can make async_extract_referenced_entity_ids
a bit faster with this as well
* Avoid linear search of entity registry in async_extract_referenced_entity_ids
needs https://github.com/home-assistant/core/pull/109660
* Mask sensitive data in google_assistant logs
* Move common code to homeassistant/util/redact.py
* Move to helpers
* Add tests
* Tweak
* Redact additional logs
* Fix stale docstring
* Don't reveal the length of masked data
* Update test
* Convert referenced registry functions to use cached_property
These already implemented caching, but now that we can use cached_property
because the lock problem is solved, we can make the code simplier and faster
* missed one
* make them the same
* Small cleanup to entity platform translation fetching
While I could not realize the performance improvemnet I had
hoped in #108800, I pulled this out since its a nice cleanup to avoid
constructing the inner function over and over.
* stale docstring
* Avoid useless time fetch in DataUpdateCoordinator
Since we used the async_call_at helper, it would always call dt_util.utcnow()
to feed the _handle_refresh_interval which threw it away. This meant we had
to fetch time twice as much as needed for each update
* tweak
* compat
* adjust comment
* Refactor translations to reduce dict lookups
All of our cache lookups used:
`cache[language][O(component)][category]`
The cache was designed as
`cache[language][component][category][flatted_key]`
The lookups are now
`cache[language][category][O(component)]`
The cache is now stored as
`cache[language][category][component][flatted_key]`
This allows the catch fetch to avoid looking up
the category each loop
* already a set, and we do not mutate
* add placeholder support to entity name translation
* add negativ tests
* make property also available via description
* fix doc string in translation_placeholders()
* fix detection of placeholder
* validate placeholders for localized strings
* add test
* Cache translation_placeholders property
* Make translation_placeholders uncondotionally return dict
* Fall back to unsubstituted name in case of mismatch
* Only replace failing translations with English
* Update snapshots
* Blow up on non stable releases
* Fix test
* Update entity.py
---------
Co-authored-by: Erik <erik@montnemery.com>
Co-authored-by: Joost Lekkerkerker <joostlek@outlook.com>
* Add Valve integration.
This adds the valve integration discussed in https://github.com/home-assistant/architecture/discussions/975
Most of the code is taken from the cover integration but simplified since valves
can't tilt.
There are a couple outstanding errors I'm not sure how to solve and prevents
me from even making this commit without `--no-verify`.
* Apply PR feedback
* Apply more feedback: Intruduce the bare minimum
* Remove file commited by mistake
* Hopefully this fixes tests
* Match cover's typing and mypy settings
* Change some configuration files
* Fix test
* Increase code coverage a little
* Code coverate inproved to 91%
* 95% code coverage
* Coverate up to 97%
* Coverage 98%
* Apply PR feedback
* Even more feedback
* Add line I shouldn't have removed
* Derive closed/open state from current position
* Hopefully last feedback
* Update homeassistant/components/valve/__init__.py
Co-authored-by: Martin Hjelmare <marhje52@gmail.com>
* Update homeassistant/components/valve/__init__.py
Co-authored-by: Martin Hjelmare <marhje52@gmail.com>
* Remove unnecesary translation
* Remove unused method arguments
* Complete code coverage
* Update homeassistant/components/valve/__init__.py
Co-authored-by: Martin Hjelmare <marhje52@gmail.com>
* Simplify tests
* Update homeassistant/components/valve/__init__.py
Co-authored-by: Martin Hjelmare <marhje52@gmail.com>
* Apply last feedback
* Update tests/components/valve/test_init.py
Co-authored-by: Martin Hjelmare <marhje52@gmail.com>
* Update tests/components/valve/test_init.py
Co-authored-by: Martin Hjelmare <marhje52@gmail.com>
* Update tests/testing_config/custom_components/test/valve.py
Co-authored-by: Martin Hjelmare <marhje52@gmail.com>
* More feedback
* Apply suggestion
* And more feedback
* Apply feedback
* Remove commented code
* Reverse logic to unindent
* Update homeassistant/components/valve/__init__.py
Co-authored-by: Martin Hjelmare <marhje52@gmail.com>
* Update homeassistant/components/valve/__init__.py
Co-authored-by: Martin Hjelmare <marhje52@gmail.com>
* Implement stop valve for Mock valve
* Fix tests now that I've implemented stop_valve
* Assert it's neither opening nor closing
* Use current position instead
* Avoid scheduling executor when opening or closing
* Fix incorrect bitwise operation
* Simplify toggle
* Remove uneeded partial functions
* Make is_last_toggle_direction_open private
* Remove valve from test custom integration
* Improve test coverage
* Address review comments
* Address review comments
* Address review comments
* Update homeassistant/components/valve/__init__.py
Co-authored-by: Martin Hjelmare <marhje52@gmail.com>
* Update tests
---------
Co-authored-by: Martin Hjelmare <marhje52@gmail.com>
Co-authored-by: Erik <erik@montnemery.com>
Reduce overhead connect dispatcher
- We tend to have 1000s (or 10000s) of connected dispatchers which
makes these prime targets to reduce overhead/memory
- Instead of creating new functions to wrap log exceptions each time
use partials which reuses the function body and only create new
arguments
Previous optimizations #103307#93602
* Keep capabilities up to date in the entity registry
* Warn if entities update their capabilities very often
* Fix updating of device class
* Stop tracking capability updates once flooding is logged
* Only sync registry if state changed
* Improve test
* Revert "Only sync registry if state changed"
This reverts commit 1c52571596c06444df234d4b088242b494b630f2.
* Avoid calculating device class twice
* Address review comments
* Revert using dataclass
* Fix unintended revert
* Add helper method
* Drop use of regex in helpers.extract_domain_configs
* Update test
* Revert test update
* Add domain_from_config_key helper
* Add validator
* Address review comment
* Update snapshots
* Inline domain_from_config_key in validator
* Improve OAuth error handling in configuration flows
* Update strings for all integrations that use oauth2 config flow
* Remove invalid_auth strings
* Revert change to release
* Revert close change in aiohttp mock
* Differentiate between warnings and errors in check_config helper
* Update tests
* Treat configuration errors in frontend and its dependencies as errors
* Improve test coverage
* Address review comments
* Improve test coverage
* Improve test coverage
* Address review comments
* Add comment
The contextmanager decorator creates a new context manager every
time its run, but since we only have a single context var, we can
use the same one every time. Creating the contextmanager was roughly
20% of the time time of the template render
I was a bit suprised to find it creates a new context manager
object every time https://stackoverflow.com/questions/34872535/why-contextmanager-is-slow
* Prevent accidentally reusing an entity object
* Fix group reload service
* Revert "Fix group reload service"
* Improve test
* Add tests aserting entity can't be reused
* add supports_response to platform entity services
* support multiple entities in entity_service_call
* support legacy response format for service calls
* revert changes to script/shell_command
* add back test for multiple responses for legacy service
* remove SupportsResponse.ONLY_LEGACY
* Apply suggestion
Co-authored-by: Allen Porter <allen.porter@gmail.com>
* test for entity_id remove None
* revert Apply suggestion
* return EntityServiceResponse from _handle_entity_call
* Use asyncio.gather
* EntityServiceResponse not Optional
* styling
---------
Co-authored-by: Allen Porter <allen.porter@gmail.com>
* Avoid looking up the callable type for HassJob when we already know it
When we connect the frontend we call async_listen with run_immediately MANY
times when we already know the job type (it will always be a callback). This
reduces the latency to get the frontend going
* missing coverage
This was being built every time state was written. When
a robo vac is operating it writes state often which mean building
the icon string over and over again when it rarely changes.
async_has_matching_flow is more likely to be True than hass.is_stopping
This does not make much difference but it was adding noise to a profile
that I am digging into to look for another issue
* Refactor entity service calls to reduce complexity
gets rid of the noqa C901
* Refactor entity service calls to reduce complexity
gets rid of the noqa C901
* short
* Use loop time to set context
loop time is faster than utcnow, and since its only used internally it can
be switched without a breaking change
* fix mocking
* Allow specifying a custom log function for template render
* Bypass template cache when reporting errors + fix tests
* Send errors as events
* Fix logic for creating new TemplateEnvironment
* Add strict mode back
* Only send error events if report_errors is True
* Force test of websocket_api only
* Debug test
* Run pytest with higher verbosity
* Timeout after 1 minute, enable syslog output
* Adjust timeout
* Add debug logs
* Fix unsafe call to WebSocketHandler._send_message
* Remove debug code
* Improve test coverage
* Revert accidental change
* Include severity in error events
* Remove redundant information from error events
* Fix memory leak in dispatcher removal
When we removed the last job/callable from the dict for the
signal we did not remove the dict for the signal which meant
it leaked
* comment
* cleanup a bit more
* Set state of entity with invalid state to unknown
* Add test
* Apply suggestions from code review
Co-authored-by: Robert Resch <robert@resch.dev>
* Update test_entity.py
---------
Co-authored-by: Robert Resch <robert@resch.dev>
* Extend template entities with a script section
This allows making a trigger entity that triggers a few times a day,
and allows collecting data from a service resopnse which can be
fed into a template entity.
The current alternatives are to publish and subscribe to events or to
store data in input entities.
* Make variables set in actions accessible to templates
* Format code
---------
Co-authored-by: Erik <erik@montnemery.com>
* Switch async_track_same_state to use async_call_later
There was no need to use async_track_point_in_utc_time here since
we only need a delay
* update trigger tests
* remove some more utcnow patching
* remove some more utcnow patching
* remove some more utcnow patching
* Don't blow up if validators can't access the issue registry
* Make the check_config script open issue_registry read only
* Update tests/helpers/test_issue_registry.py
Co-authored-by: Martin Hjelmare <marhje52@gmail.com>
---------
Co-authored-by: Martin Hjelmare <marhje52@gmail.com>
* Add service for getting a weather forecast
* Fix translations
* Improve service description
* Improve error handling
* Adjust typing
* Adjust typing
* Adjust service response format
J. Nick Koston
Jan-Philipp Benecke
Joost Lekkerkerker
Michael Hansen
Franck Nijhof
Marc Mueller
G Johansson
deosrc
chammp
Piotr Machowski
Michael
Erik Montnemery
Bram Kragten
Robert Resch
Martijn van der Pol
karwosts
Daniel
Lars R
Petru Paler
Ville Skyttä
Miguel Camba
Jan Bouwhuis
Paul Bottein
schelv
Aarni Koskela
Ruslan Sayfutdinov
Allen Porter
Kevin Stillhammer
Raman Gupta
Nicolas van de Walle
Dave T
Joakim Plate
Shay Levy
Michael Arthur