* Always do thread safety check when writing state
Refactor the 3 most common places where the thread safety check
for the event loop to be inline to make the check fast enough
that we can keep it long term. While code review catches most
of the thread safety issues in core, some of them still make
it through, and new ones keep getting added. Its not possible
to catch them all with manual code review, so its worth the
tiny overhead to check each time.
Previously the check was limited to custom components
because they were the most common source of thread
safety issues.
* Always do thread safety check when writing state
Refactor the 3 most common places where the thread safety check
for the event loop to be inline to make the check fast enough
that we can keep it long term. While code review catches most
of the thread safety issues in core, some of them still make
it through, and new ones keep getting added. Its not possible
to catch them all with manual code review, so its worth the
tiny overhead to check each time.
Previously the check was limited to custom components
because they were the most common source of thread
safety issues.
* async_fire is more common than expected with ccs
* fix mock
* fix hass mocking
* Automatically fill in slots from LLM context
* Add tests
* Apply suggestions from code review
Co-authored-by: Allen Porter <allen@thebends.org>
---------
Co-authored-by: Allen Porter <allen@thebends.org>
* Ignore deprecated open and close cover intents for LLMs
* Fix LLMs asking which area when there is only one device
* remove unrelated changed
* remove unrelated changes
* Add configuration option positive to DurationSelector
* Rename to allow_negative in conjunction with a deprecation notice
Co-authored-by: Erik Montnemery <erik@montnemery.com>
---------
Co-authored-by: Erik Montnemery <erik@montnemery.com>
* LLM Assist API to ignore timer intents if device doesn't support it
* Refactor to use API instances
* Extract ToolContext class
* Limit exposed intents based on exposed entities
These functions have been stripped down to always return
the same path so there was no longer a need to have a
function for this. This is left-over cleanup from
previous refactoring.
* initial commit
* Undo prompt chenges
* Move format_tool out of the class
* Only catch HomeAssistantError and vol.Invalid
* Add config flow option
* Fix type
* Add translation
* Allow changing API access from options flow
* Allow model picking
* Remove allowing HASS Access in main flow
* Move model to the top in options flow
* Make prompt conditional based on API access
* convert only once to dict
* Reduce debug logging
* Update title
* re-order models
* Address comments
* Move things
* Update labels
* Add tool call tests
* coverage
* Use LLM APIs
* Fixes
* Address comments
* Reinstate the title to not break entity name
---------
Co-authored-by: Paulus Schoutsen <balloob@gmail.com>
* Add llm helper
* break out Tool.specification as class members
* Format state output
* Fix intent tests
* Removed auto initialization of intents - let conversation platforms do that
* Handle DynamicServiceIntentHandler.extra_slots
* Add optional description to IntentTool init
* Add device_id and conversation_id parameters
* intent tests
* Add LLM tools tests
* coverage
* add agent_id parameter
* Apply suggestions from code review
* Apply suggestions from code review
* Apply suggestions from code review
* Apply suggestions from code review
* Apply suggestions from code review
* Fix tests
* Fix intent schema
* Allow a Python function to be registered as am LLM tool
* Add IntentHandler.effective_slot_schema
* Ensure IntentHandler.slot_schema to be vol.Schema
* Raise meaningful error on tool not found
* Move this change to a separate PR
* Update todo integration intent
* Remove Tool constructor
* Move IntentTool to intent helper
* Convert custom serializer into class method
* Remove tool_input from FunctionTool auto arguments to avoid recursion
* Remove conversion into Open API format
* Apply suggestions from code review
* Fix tests
* Use HassKey for helpers (see #117012)
* Add support for functions with typed lists, dicts, and sets as type hints
* Remove FunctionTool
* Added API to get registered intents
* Move IntentTool to the llm library
* Return only handlers in intents.async.get
* Removed llm tool registration from intent library
* Removed tool registration
* Add bind_hass back for now
* removed area and floor resolving
* fix test
* Apply suggestions from code review
* Improve coverage
* Fix intent_type type
* Temporary disable HassClimateGetTemperature intent
* Remove bind_hass
* Fix usage of slot schema
* Fix test
* Revert some test changes
* Don't mutate tool_input
---------
Co-authored-by: Paulus Schoutsen <paulus@home-assistant.io>
Co-authored-by: Paulus Schoutsen <balloob@gmail.com>
* Allow templates for enabling automation triggers
* Test exception for non-limited template
* Use `cv.template` instead of `cv.template_complex`
* skip trigger with invalid enable template
instead of returning and thus not evaluating other triggers
* Import and cache supported feature enum flags only when needed
* Add comment aboud being loaded from executor.
---------
Co-authored-by: J. Nick Koston <nick@koston.org>
* Addition of add filter
This change adds an `add` filter, the addition equivalent of the existing `multiply` filter.
* Test for add filter
* Update test_template.py
* Update tests/helpers/test_template.py
---------
Co-authored-by: Erik Montnemery <erik@montnemery.com>
* Turn on thread safety checks in async_dispatcher_send
We keep seeing issues where async_dispatcher_send is called from
a thread which means we call the callback function on the other
side in the thread as well which usually leads to a crash
* Turn on thread safety checks in async_dispatcher_send
We keep seeing issues where async_dispatcher_send is called from
a thread which means we call the callback function on the other
side in the thread as well which usually leads to a crash
* adjust
* Refactor entity_platform polling to avoid double time fetch
Replace async_track_time_interval with loop.call_later
to avoid the useless time fetch every time the listener
fired since we always throw it away
* fix test
It turns out we have custom components that are writing to the area registry using the async APIs from threads. We now catch it at the point async_fire is called. Instead we should check sooner and use async_fire_internal so we catch the unsafe operation before it can corrupt the registry.
* Move thread safety check in entity_registry sooner
It turns out we have a lot of custom components that are writing
to the entity registry using the async APIs from threads. We now
catch it at the point async_fire is called. Instread we should check
sooner and use async_fire_internal so we catch the unsafe operation
before it can corrupt the registry.
* coverage
* Apply suggestions from code review
It turns out we have custom components that are writing to the device registry using the async APIs from threads. We now catch it at the point async_fire is called. Instead we should check sooner and use async_fire_internal so we catch the unsafe operation before it can corrupt the registry.
* Fix lying docstring for relative_time template function
* Update homeassistant/helpers/template.py
Co-authored-by: Martin Hjelmare <marhje52@gmail.com>
---------
Co-authored-by: Martin Hjelmare <marhje52@gmail.com>
* Make release channel a hardcoded enum rather than a free form string
* Update enum comparison to remove equality and us identity comparison
* Fix comparison condition to match the previous implementation
* Update tests to use Enum instead of string
* Keep track of top level components
Currently we have to do a set comp for icons, translations,
and integration platforms every time to split the top level
components from the platforms. Keep track of the top level
components in a seperate set so avoid having to do the setcomp
every time.
* remove impossible paths
* remove unused code
* preen
* preen
* fix
* coverage and fixes
* Update homeassistant/core.py
* Update homeassistant/core.py
* Update tests/test_core.py
* Remove unused state translations
There have been replaced with entity translations
https://github.com/home-assistant/developers.home-assistant/pull/1557https://github.com/home-assistant/core/pull/82701
* nothing does merging anymore
* useless dispatch
* remove
* remove platform code from hassfest
* preen
* Update homeassistant/helpers/translation.py
* ruff
* fix merge
* check is impossible now since we already know if translations exist or not
* keep the function for now
* remove unreachable code since we filter out `.` before now
* reduce
* reduce
* fix merge conflict (again)
* Fix race in TimestampDataUpdateCoordinator
The last_update_success_time value was being set after the listeners
were fired which could lead to a loop because the listener may
re-trigger an update because it thinks the data is stale
* coverage
* docstring
* Avoid checking for polling if an entity fails to add
* no need to do protected access
* no need to do protected access
* no need to do protected access
* no need to do protected access
* coverage
* fix test
* fix
* broken one must be first
* Add missing platforms_exist guard to check_config
related issue #112811
When the exception hits, the config will end up being saved in the traceback
so the memory is never released.
This matches the check_config code to homeassistant.config to avoid having
the exception thrown.
* patch
* merge branch
This function was only ever used in homekit, and since there is
now an index of devices in the entity registry, homekit no longer
uses it.
I searched github code for all references to async_get_device_class_lookup
and the only think I could find using it were forks of core. It seems
unlikely that any custom components are affected by removing this
function
Co-authored-by: Sid <27780930+autinerd@users.noreply.github.com>
Co-authored-by: Marc Mueller <30130371+cdce8p@users.noreply.github.com>
Co-authored-by: J. Nick Koston <nick@koston.org>
* Refactor rate limit helper to track time in seconds
Currently we created datetime and timedelta objects to enforce the
rate limit. When the rate limit was being hit hard, this got expensive.
We now use floats everywhere instead as they are much cheaper which
is important when we are running up against a rate limit, which is
by definition a hot path
The rate limit helper is currently only used for templates and
we do not have any code in the code base that directly passes
in a rate limit so the impact to custom components is expected
to be negligible if any
* misesd two
* Add category registry
* Add entity registry support
* Update homeassistant/components/config/entity_registry.py
Co-authored-by: Martin Hjelmare <marhje52@gmail.com>
* Use ulid instead
* Add tests for adding same name in different scopes
* Handle keyerror on update
* Lookup tweak
* Omit categories from entity registry snapshots
* Use base registry
* Update snapshots
* Update snapshots
---------
Co-authored-by: Martin Hjelmare <marhje52@gmail.com>
* Add a helper to import modules in the event loop
Replaces the one used for triggers with a more generic helper
that can be reused and uses a future to avoid importing concurrently
* Add a helper to import modules in the event loop
Replaces the one used for triggers with a more generic helper
that can be reused and uses a future to avoid importing concurrently
* coverage
* make sure we do not retry
* coverage
Await discovery_function in the DiscoveryFlowHandler instead of wrapping
This function was always a coro so it can be awaited
directly instead of wrapping it in add_job
* Phase out periodic tasks
* false by default or some tests will block forever, will need to fix each one manually
* kwarg works
* kwarg works
* kwarg works
* fixes
* fix more tests
* fix more tests
* fix lifx
* opensky
* pvpc_hourly_pricing
* adjust more
* adjust more
* smarttub
* adjust more
* adjust more
* adjust more
* adjust more
* adjust
* no eager executor
* zha
* qnap_qsw
* fix more
* fix fix
* docs
* its a wrapper now
* add more coverage
* coverage
* cover all combos
* more fixes
* more fixes
* more fixes
* remaining issues are legit bugs in tests
* make tplink test more predictable
* more fixes
* feedreader
* grind out some more
* make test race safe
* one more
* Schedule periodic coordinator updates as background tasks.
Currently, the coordinator's periodic refreshes delay startup because they are not scheduled as background tasks. We will wait if the startup takes long enough for the first planned refresh. Another coordinator's scheduled refresh will be fired on busy systems, further delaying the startup. This chain of events results in the startup taking a long time and hitting the safety timeout because too many coordinators are refreshing.
This case can also happen with scheduled entity refreshes, but it's less common. A future PR will address that case.
* periodic_tasks
* periodic_tasks
* periodic_tasks
* merge
* merge
* merge
* merge
* merge
* fix test that call the sync api from async
* one more place
* cannot chain
* async_run_periodic_hass_job
* sun and pattern time changes from automations also block startup
* Revert "sun and pattern time changes from automations also block startup"
This reverts commit 6de2defa05.
* make sure polling is cancelled when config entry is unloaded
* Revert "Revert "sun and pattern time changes from automations also block startup""
This reverts commit e8f12aad55.
* remove DisabledError from homewizard test as it relies on a race
* fix race
* direct coverage
These were very likely to be cached so they were low on the
list to migrate, but since they are called in the event loop
its best to be sure we do no blocking I/O
discovered in https://github.com/home-assistant/core/pull/112295#discussion_r1513505710
We only checked if the last language had files to load
instead of all of them. The checks for each language
are the same because the only reason we would skip
a language is a missing/broken integration or the integration
is a single file. Both of these loop conditions are always
the same reguardless of the language so the check worked
Some of the data we had to search for was already available
in a dict or underlying data structure. Make it available
instead of having to build it every time.
There are more places these can be used, but I only did
the device registry cleanup for now
* Add 'issues' template function for listing active issues.
* Add issue template function test
* Add 'issue' template function for getting specific issue by domain and issue_id
* Remove comment
* Fix function description
* Remove reduntant function,
Fix tests
* remove pass_context
* remove issues filter
Co-authored-by: Erik Montnemery <erik@montnemery.com>
---------
Co-authored-by: Erik Montnemery <erik@montnemery.com>
* Avoid expensive inspect calls in config validators
inspect has a performance problem https://github.com/python/cpython/issues/92041
We now avoid calling inspect unless we are going to log
* remove unused
* reduce
* get_integration_logger
* Deprecate @bind_hass and log error if used inside custom component
* Log also when accessing `hass.components`
* Log warning only when `hass.components` is used
* Change version
* Process code review
* Reduce latancy to load storage by making the task eager
This changes the semantics a bit under the hood because it
can raise sooner which means we do not store the task
as _load_task if it raises right away. That means
concurrent calls that result in failure are likely to try
again now which will be a tiny performance hit for this
case.
* fix
* will now finish in time
* Avoid circular import in Storage.async_delay_save
We call Storage.async_delay_save for every entity being added or removed
from the registry. The late import took more time than everything else
in the function.
* Avoid reschedule churn in Storage.async_delay_save
When we are adding or removing entities we will call async_delay_save
quite often which has to add and remove a TimerHandle on the event loop
which can add up when there are a lot of registry items changing.
If the timer handle still has 80% of the time remaining on it
we will avoid resceduling and let it fire at the time the
original async_delay_save call was made. This ensures we
do not force the event loop to rebuild its heapq because
too many timer handlers were cancelled at once
* div0
* add coverage for 0 since we had none
* fix bad conflict
* tweaks
* tweaks
* tweaks
* tweaks
* tweaks
* tweaks
* more test fixes
* mqtt tests rely on event loop overhead
* Convert debouncer async_shutdown to be a normal function
nothing was being awaited here and the shutdown call was only used
in integrations marked internal and other internals. Its possible
that a custom component might have been using the method but it
seemed uncommon enough that it did not warrent marking as a breaking
change. The update coordinator is no longer awaiting anything in
async_shutdown either now but it seemed likely that this use
would get subclassed.
* fix
* Add async_schedule_call to the Debouncer
async_schedule_call allows the Debouncer to schedule a call
from a callback without having to create tasks to run
async_call
* Update homeassistant/helpers/debounce.py
* Make device registry cleanup all callback function
* fix typing, code supported callback functions, but typing did not
* fixes
* fixes
* fix
* we had no coverage for other job types
* we had no coverage for other job types
* Reduce overhead to load multiple languages in translations
Instead of loading in a task, we now group everything
to be loaded into a single executor job
* fixes
* fixes
* fixes
* fixes
* fixes
* update tests
* add missing coverage (was existing)
* Avoid scheduling registry loads as tasks in tests
Since we patch out async_load in Store, these will not yield
to the event loop so it makes sense to await them instead
of creating tasks
This reduced my local test run times ~2.5% on average
* mock out save as well so we do not schedule tasks to save empty data
* tweaks
* fix lingering files
* another one
* too much for one PR, reduce
* fix targets
Erik Montnemery
epenet
Sid
Paulus Schoutsen
paulusbrand
J. Nick Koston
Marc Mueller
tronikos
Ron Weikamp
Denis Shulyaka
Michael Hansen
Robert Resch
Franck Nijhof
Peter
Matthias Alphart
Allen Porter
Troon
G Johansson
rlippmann
Martijn van der Pol
Alberto Montes
Jan-Philipp Benecke
Joost Lekkerkerker
Jan Bouwhuis
soonsouth
RoboMagus