Commit Graph

2765 Commits (0d2aeb846f3cc9f9f799cfd0db83567c4f338e14)

Author SHA1 Message Date
J. Nick Koston ade0f94a20
Remove duplicate getattr call in entity wrap_attr (#118558) 2024-05-31 11:11:46 -04:00
Paulus Schoutsen 6dd01dbff7
Rename llm.ToolContext to llm.LLMContext (#118566) 2024-05-31 11:11:24 -04:00
tronikos 2bd142d3a6
Improve LLM prompt (#118520) 2024-05-30 22:03:57 -04:00
tronikos 272c51fb38
Fix unnecessary single quotes escaping in Google AI (#118522) 2024-05-30 19:56:06 -04:00
tronikos 0d6c7d0973
Fix LLMs asking which area when there is only one device (#118518)
* Ignore deprecated open and close cover intents for LLMs

* Fix LLMs asking which area when there is only one device

* remove unrelated changed

* remove unrelated changes
2024-05-30 17:14:11 -04:00
tronikos 6d82cfa91a
Ignore deprecated open and close cover intents for LLMs (#118515) 2024-05-30 16:29:13 -04:00
Paulus Schoutsen 80588d9c67
Ignore the toggle intent (#118491) 2024-05-30 12:53:50 -04:00
Ron Weikamp 2814ed5003
Add allow_negative configuration option to DurationSelector (#116134)
* Add configuration option positive to DurationSelector

* Rename to allow_negative in conjunction with a deprecation notice

Co-authored-by: Erik Montnemery <erik@montnemery.com>

---------

Co-authored-by: Erik Montnemery <erik@montnemery.com>
2024-05-30 17:42:34 +02:00
tronikos 4893faa671
Instruct LLM to not pass a list to the domain (#118451) 2024-05-30 08:37:45 +02:00
tronikos 639f6c640c
Improve LLM prompt (#118443)
* Improve LLM prompt

* test

* improvements

* improvements
2024-05-29 21:44:33 -04:00
tronikos 4056c4c2cc
Ask LLM to pass area name and domain (#118357) 2024-05-29 08:03:43 -04:00
tronikos c75cb08aae
Fix LLM tracing for Google Generative AI (#118359)
Fix LLM tracing for Gemini
2024-05-29 08:02:59 -04:00
tronikos b7ee90a53c
Expose useful media player attributes to LLMs (#118363) 2024-05-29 08:01:40 -04:00
Erik Montnemery 7abffd7cc8
Don't report entities with invalid unique id when loading the entity registry (#118290) 2024-05-29 08:32:39 +02:00
J. Nick Koston f3fa843b9d
Replace pop calls with del where the result is discarded in restore_state (#118339) 2024-05-28 23:14:40 -04:00
J. Nick Koston b94bf1f214
Add cache to more complex entity filters (#118344)
Many of these do regexes and since the entity_ids are almost
always the same we should cache these
2024-05-28 23:07:50 -04:00
Paulus Schoutsen d22871f1fd
Reduce the intent response data sent to LLMs (#118346)
* Reduce the intent response data sent to LLMs

* No longer delete speech
2024-05-28 23:07:00 -04:00
J. Nick Koston fa9ebb062c
Small speed up to connecting dispatchers (#118342) 2024-05-28 16:49:58 -10:00
Paulus Schoutsen c097a05ed4
Tweak Assist LLM API prompt (#118343) 2024-05-28 22:43:22 -04:00
Michael Hansen d223e1f2ac
Add Conversation command to timers (#118325)
* Add Assist command to timers

* Rename to conversation_command. Execute in timer code.

* Make agent_id optional

* Fix arg

---------

Co-authored-by: Paulus Schoutsen <balloob@gmail.com>
2024-05-28 20:33:31 -05:00
Paulus Schoutsen 615a1eda51
LLM Assist API to ignore intents if not needed for exposed entities or calling device (#118283)
* LLM Assist API to ignore timer intents if device doesn't support it

* Refactor to use API instances

* Extract ToolContext class

* Limit exposed intents based on exposed entities
2024-05-28 21:29:18 -04:00
J. Nick Koston e0264c8604
Replace pop calls with del where the result is discarded in entity (#118340) 2024-05-28 15:26:53 -10:00
J. Nick Koston fd9d4dbb34
Use del instead of pop in the entity platform remove (#118337) 2024-05-28 15:26:22 -10:00
Michael Hansen 2dc49f0410
Add platforms to intent handlers (#118328) 2024-05-28 16:46:08 -04:00
Paulus Schoutsen f0d7f48930
Handle generic commands as area commands in the LLM Assist API (#118276)
* Handle generic commands as area commands in the LLM Assist API

* Add word area
2024-05-28 11:21:17 -04:00
Paulus Schoutsen 6f248acfd5
LLM Assist API: Inline all exposed entities (#118273)
Inline all exposed entities
2024-05-27 21:12:10 -04:00
J. Nick Koston 33ff84469a
Align max expected entities constant between modules (#118102) 2024-05-27 14:06:16 -10:00
J. Nick Koston 87989a88cd
Remove translation and icon component path functions (#118214)
These functions have been stripped down to always return
the same path so there was no longer a need to have a
function for this. This is left-over cleanup from
previous refactoring.
2024-05-27 10:35:52 +02:00
Paulus Schoutsen ecb05989ca
Add exposed entities to the Assist LLM API prompt (#118203)
* Add exposed entities to the Assist LLM API prompt

* Check expose entities in Google test

* Copy Google default prompt test cases to LLM tests
2024-05-27 00:27:08 -04:00
Paulus Schoutsen 1602c8063c
Standardize LLM instructions prompt (#118195)
* Standardize instructions prompt

* Add time/date to default instructions
2024-05-26 20:24:26 -04:00
J. Nick Koston c368ffffd5
Add async_get_hass_or_none (#118164) 2024-05-26 13:38:46 +02:00
Paulus Schoutsen 233c3bb2be
Add render prompt method when no API selected (#118136) 2024-05-26 13:35:15 +02:00
J. Nick Koston 607aaa0efe
Speed up template result parsing (#118168) 2024-05-26 13:09:12 +02:00
J. Nick Koston 7bbb33b415
Improve script disallowed recursion logging (#118151) 2024-05-26 12:58:34 +02:00
Denis Shulyaka cee3be5f7a
Break long strings in LLM tools (#118114)
* Break long code strings

* Address comments

---------

Co-authored-by: Paulus Schoutsen <balloob@gmail.com>
2024-05-25 14:24:51 -04:00
Allen Porter 89e2c57da6
Add conversation agent debug tracing (#118124)
* Add debug tracing for conversation agents

* Minor cleanup
2024-05-25 14:16:51 -04:00
J. Nick Koston 42232ecc8a
Remove unused code in template helper (#118075) 2024-05-24 17:44:24 -10:00
J. Nick Koston 6923fb1601
Avoid template context manager overhead when template is already compiled (#118079) 2024-05-24 17:44:02 -10:00
J. Nick Koston 5cb3bc19c0
Speed up async_render_with_possible_json_value (#118080) 2024-05-24 17:43:42 -10:00
J. Nick Koston 3416162fdb
Remove OrderedDict from entity_values as dict guarantees order on newer cpython (#118081) 2024-05-24 17:43:20 -10:00
J. Nick Koston f026083712
Speed up is_template_string by avoiding regex engine (#118076) 2024-05-24 17:42:55 -10:00
Denis Shulyaka da74ac06d7
Add user name and location to the LLM assist prompt (#118071)
Add user name and location to the llm assist prompt
2024-05-24 22:23:05 -04:00
J. Nick Koston cf73a47fc0
Significantly speed up single use callback dispatchers (#117934) 2024-05-24 11:21:10 -10:00
Paulus Schoutsen 7554ca9460
Allow llm API to render dynamic template prompt (#118055)
* Allow llm API to render dynamic template prompt

* Make rendering api prompt async so it can become a RAG

* Fix test
2024-05-24 16:04:48 -04:00
J. Nick Koston a8fba691ee
Add types to event tracker data (#118010)
* Add types to event tracker data

* fixes

* do not test event internals in other tests

* fixes

* Update homeassistant/helpers/event.py

* cleanup

* cleanup
2024-05-24 10:09:39 -04:00
Marc Mueller dd22ee3dac
Improve annotation styling (#118032) 2024-05-24 15:05:53 +02:00
Marc Mueller 905adb2431
Update codespell ignore list (#118018) 2024-05-24 10:24:34 +02:00
Marc Mueller 0e03e591e7
Improve callable annotations (#118024) 2024-05-24 10:24:09 +02:00
J. Nick Koston b7a18e9a8f
Avoid calling split_entity_id in event add/remove filters (#118015) 2024-05-24 10:01:33 +02:00
Franck Nijhof 9224997411
Add sequence action for automations & scripts (#117690)
Co-authored-by: Robert Resch <robert@resch.dev>
2024-05-24 09:34:49 +02:00
J. Nick Koston 6c6a5f496a
Simplify async_track_time_interval implementation (#117956) 2024-05-22 21:56:49 -10:00
Paulus Schoutsen 1f7245ecf2
Update LLM no tools message (#117935) 2024-05-22 21:17:03 -04:00
Peter eb76386c68
Prevent time pattern reschedule if cancelled during job execution (#117879)
Co-authored-by: J. Nick Koston <nick@koston.org>
2024-05-22 10:36:03 -10:00
J. Nick Koston 0d5c8e30cd
Migrate issue registry to use singleton helper (#117848)
* Migrate issue registry to use singleton helper

The other registries were already migrated, but since this
one had a read only flag, it required a slightly different
solution since it uses the same hass.data key

* refactor
2024-05-22 14:13:19 -04:00
Joost Lekkerkerker 0c5296b38f
Add lock to token validity check (#117912) 2024-05-22 14:10:23 -04:00
J. Nick Koston 6113b58e9c
Speed up registry indices (#117897)
* Use defaultdict for registry indices

defaultdict is faster and does not have to create an empty
dict that gets throw away when the key is already present

* Use defaultdict for registry indices

defaultdict is faster and does not have to create an empty
dict that gets throw away when the key is already present
2024-05-22 14:07:39 -04:00
Michael Hansen f99ec87338
Fail if targeting all devices in the house in service intent handler (#117930)
* Fail if targeting all devices in the house

* Update homeassistant/helpers/intent.py

---------

Co-authored-by: Paulus Schoutsen <balloob@gmail.com>
2024-05-22 13:53:31 -04:00
Denis Shulyaka 009c9e79ae
LLM Tools: Add device_id (#117884) 2024-05-21 21:24:46 -04:00
Sid c2b3bf3fb9
Enable Ruff RET502 (#115139) 2024-05-21 22:19:33 +02:00
Paulus Schoutsen f21226dd0e
Address late feedback Google LLM (#117873) 2024-05-21 20:11:18 +02:00
Michael Hansen 8079cc0464
Add description to intent handlers and use in LLM helper (#117864) 2024-05-21 12:54:34 -04:00
J. Nick Koston 266ce9e268
Cache area registry JSON serialize (#117847)
We already cache the entity and device registry, but since I never
used area until recently I did not have enough to notice that they
were not cached
2024-05-21 09:03:31 -04:00
J. Nick Koston 905692901c
Simplify service description cache logic (#117846) 2024-05-21 09:02:32 -04:00
Marc Mueller d44f949b19
Use PEP 695 misc (2) (#117814) 2024-05-21 09:45:57 +02:00
Marc Mueller 58d0ac7f21
Remove future import to fix broken typing.get_type_hints call (#117837) 2024-05-21 09:39:47 +02:00
J. Nick Koston c9d1b127d8
Improve error message when template is rendered from wrong thread (#117822)
* Improve error message when template is rendered from wrong thread

* Improve error message when template is rendered from wrong thread
2024-05-20 23:26:48 -04:00
J. Nick Koston 7714f807b4
Detect incorrect exception in forwarded platforms (#117754)
* Detect incorrect exception in forwarded platforms

If an integration raises ConfigEntryError/ConfigEntryAuthFailed/ConfigEntryAuthFailed
in a forwarded platform it would affect the state of the config entry and cause it to
process unloads and setup retries in while the other platforms continued to setup

* Detect incorrect exception in forwarded platforms

If an integration raises ConfigEntryError/ConfigEntryAuthFailed/ConfigEntryAuthFailed
in a forwarded platform it would affect the state of the config entry and cause it to
process unloads and setup retries in while the other platforms continued to setup

* Update homeassistant/config_entries.py

Co-authored-by: Paulus Schoutsen <balloob@gmail.com>

* adjust

* fix

---------

Co-authored-by: Paulus Schoutsen <balloob@gmail.com>
2024-05-20 20:01:59 -04:00
Marc Mueller f50973c76c
Use PEP 695 misc (#117788) 2024-05-20 12:01:49 +02:00
Marc Mueller 0293315b23
Use PEP 695 for covariant class annotations (#117780) 2024-05-20 10:55:44 +02:00
Marc Mueller 8f0fb4db3e
Use PEP 695 for class annotations (4) (#117778) 2024-05-20 10:44:52 +02:00
Denis Shulyaka c3196a5667
LLM Tools support for Google Generative AI integration (#117644)
* initial commit

* Undo prompt chenges

* Move format_tool out of the class

* Only catch HomeAssistantError and vol.Invalid

* Add config flow option

* Fix type

* Add translation

* Allow changing API access from options flow

* Allow model picking

* Remove allowing HASS Access in main flow

* Move model to the top in options flow

* Make prompt conditional based on API access

* convert only once to dict

* Reduce debug logging

* Update title

* re-order models

* Address comments

* Move things

* Update labels

* Add tool call tests

* coverage

* Use LLM APIs

* Fixes

* Address comments

* Reinstate the title to not break entity name

---------

Co-authored-by: Paulus Schoutsen <balloob@gmail.com>
2024-05-19 22:11:25 -04:00
Paulus Schoutsen d001e7daea
Add API class to LLM helper (#117707)
* Add API class to LLM helper

* Add more tests

* Rename intent to assist to broaden scope
2024-05-18 21:14:05 -04:00
Marc Mueller 10dfa91e54
Remove useless TypeVars (#117687) 2024-05-18 12:58:51 +02:00
Marc Mueller 97a4101900
Use PEP 695 for dispatcher helper typing (#117685) 2024-05-18 12:47:03 +02:00
Marc Mueller 900b6211ef
Use PEP 695 for function annotations (2) (#117659) 2024-05-18 11:44:39 +02:00
Marc Mueller 907b9c42e5
Use PEP 695 for decorator typing with type aliases (2) (#117663) 2024-05-18 11:41:46 +02:00
Marc Mueller 25d1ca747b
Use PEP 695 for decorator typing (3) (#117640) 2024-05-17 16:27:32 +02:00
Marc Mueller 87bb7ced79
Use PEP 695 for simple type aliases (#117633) 2024-05-17 14:42:21 +02:00
Marc Mueller 4edee94a81
Update mypy-dev to 1.11.0a2 (#117630) 2024-05-17 13:32:20 +02:00
Marc Mueller ab07bc5298
Improve ReloadServiceHelper typing (#117552) 2024-05-16 12:47:43 +02:00
Denis Shulyaka f31873a846
Add LLM tools (#115464)
* Add llm helper

* break out Tool.specification as class members

* Format state output

* Fix intent tests

* Removed auto initialization of intents - let conversation platforms do that

* Handle DynamicServiceIntentHandler.extra_slots

* Add optional description to IntentTool init

* Add device_id and conversation_id parameters

* intent tests

* Add LLM tools tests

* coverage

* add agent_id parameter

* Apply suggestions from code review

* Apply suggestions from code review

* Apply suggestions from code review

* Apply suggestions from code review

* Apply suggestions from code review

* Fix tests

* Fix intent schema

* Allow a Python function to be registered as am LLM tool

* Add IntentHandler.effective_slot_schema

* Ensure IntentHandler.slot_schema to be vol.Schema

* Raise meaningful error on tool not found

* Move this change to a separate PR

* Update todo integration intent

* Remove Tool constructor

* Move IntentTool to intent helper

* Convert custom serializer into class method

* Remove tool_input from FunctionTool auto arguments to avoid recursion

* Remove conversion into Open API format

* Apply suggestions from code review

* Fix tests

* Use HassKey for helpers (see #117012)

* Add support for functions with typed lists, dicts, and sets as type hints

* Remove FunctionTool

* Added API to get registered intents

* Move IntentTool to the llm library

* Return only handlers in intents.async.get

* Removed llm tool registration from intent library

* Removed tool registration

* Add bind_hass back for now

* removed area and floor resolving

* fix test

* Apply suggestions from code review

* Improve coverage

* Fix intent_type type

* Temporary disable HassClimateGetTemperature intent

* Remove bind_hass

* Fix usage of slot schema

* Fix test

* Revert some test changes

* Don't mutate tool_input

---------

Co-authored-by: Paulus Schoutsen <paulus@home-assistant.io>
Co-authored-by: Paulus Schoutsen <balloob@gmail.com>
2024-05-15 19:16:47 -04:00
Paulus Schoutsen 4aba92ad04
Fix the type of slot schema of intent handlers (#117520)
Fix the slot schema of dynamic intenet handler
2024-05-15 16:45:15 -04:00
Matthias Alphart ec4c8ae228
Allow templates for enabling actions (#117049)
* Allow templates for enabling automation actions

* Use `cv.template` instead of `cv.template_complex`

* Rename test function
2024-05-15 21:03:52 +02:00
Matthias Alphart 076f57ee07
Allow templates for enabling conditions (#117047)
* Allow templates for enabling automation conditions

* Use `cv.template` instead of `cv.template_complex`
2024-05-15 21:03:28 +02:00
Paulus Schoutsen 8f9273e945
Fix intent_type type (#117469) 2024-05-14 21:32:11 -07:00
Erik Montnemery 7f3d6fe1f0
Fix lying docstring in entity_platform (#117450) 2024-05-14 21:15:05 +02:00
Michael Hansen 641754e0bb
Pass device_id to intent handlers (#117442) 2024-05-14 13:59:49 -05:00
Michael Hansen add6ffaf70
Add Assist timers (#117199)
* First pass at timers

* Move to separate file

* Refactor to using events

* Add pause/unpause/status

* Add ordinal

* Add test for timed Assist command

* Fix name matching

* Fix IntentHandleError

* Fix again

* Refactor to callbacks

* is_paused -> is_active

* Rename "set timer" to "start timer"

* Move tasks to timer manager

* More fixes

* Remove assist command

* Remove cancel by ordinal

* More tests

* Remove async on callbacks

* Export async_register_timer_handler
2024-05-14 14:42:32 -04:00
J. Nick Koston 7871e9279b
Adjust thread safety check messages to point to developer docs (#117392) 2024-05-14 15:20:31 +02:00
Matthias Alphart ba48da7678
Allow templates for enabling automation triggers (#114458)
* Allow templates for enabling automation triggers

* Test exception for non-limited template

* Use `cv.template` instead of `cv.template_complex`

* skip trigger with invalid enable template

instead of returning and thus not evaluating other triggers
2024-05-14 14:44:21 +02:00
Allen Porter b84829f70f
Import and cache supported feature enum flags only when needed (#117270)
* Import and cache supported feature enum flags only when needed

* Add comment aboud being loaded from executor.

---------

Co-authored-by: J. Nick Koston <nick@koston.org>
2024-05-13 21:07:39 -07:00
J. Nick Koston 9381462877
Migrate restore_state to use the singleton helper (#117385) 2024-05-14 09:13:44 +09:00
J. Nick Koston f318a3b5e2
Fix blocking I/O in the event loop to get MacOS system_info (#117290)
* Fix blocking I/O in the event look to get MacOS system_info

* split pr

* Update homeassistant/helpers/system_info.py

Co-authored-by: Jan-Philipp Benecke <jan-philipp@bnck.me>

* Update homeassistant/helpers/system_info.py

---------

Co-authored-by: Jan-Philipp Benecke <jan-philipp@bnck.me>
2024-05-12 08:16:21 -04:00
J. Nick Koston eac4aaef10
Use a dictcomp to reconstruct DeviceInfo in the device_registry (#117286)
Use a dictcomp to reconstruct DeviceInfo

a dictcomp is faster than many sets on the dict by at least
25%

We call this for nearly every device in the registry at
startup
2024-05-12 08:07:12 +02:00
J. Nick Koston b061e7d1aa
Small speed up to setting up integrations and config entries (#117278)
* Small speed up to setting up integration and config entries

When profiling tests, I noticed many calls to get_running_loop. In the places
where we are already in a coro, pass the existing loop so it does not have to
be looked up. I did not do this for places were we are not in a coro since there
is risk that an integration could be doing a non-thread-safe call and its better
that the code raises when trying to fetch the running loop vs the performance
improvement for these cases.

* fix merge

* missed some
2024-05-11 22:39:20 -04:00
J. Nick Koston d7aa24fa50
Only load translations for an integration once per test session (#117118) 2024-05-11 12:00:02 +09:00
Marc Mueller 04c0b7d3df
Use HassKey for importlib helper (#117116) 2024-05-08 17:42:28 -05:00
Sid ac54cdcdb4
Enable Ruff RUF010 (#115371)
Co-authored-by: J. Nick Koston <nick@koston.org>
2024-05-08 23:54:49 +02:00
J. Nick Koston 03dcede211
Avoid creating inner tasks to load storage (#117099) 2024-05-08 23:41:20 +02:00
J. Nick Koston 84a91a86a9
Improve config entry has already been setup error message (#117091) 2024-05-08 14:16:08 -05:00