Commit Graph

1580 Commits (035e19be01ae7883eebfae9597a143fb88303252)

Author SHA1 Message Date
Paulus Schoutsen bdcfd93129
Automatically fill in slots based on LLM context (#118619)
* Automatically fill in slots from LLM context

* Add tests

* Apply suggestions from code review

Co-authored-by: Allen Porter <allen@thebends.org>

---------

Co-authored-by: Allen Porter <allen@thebends.org>
2024-06-03 10:36:41 -04:00
epenet 37fc16d7b6
Fix incorrect `patch` type hint in main conftest (#118461) 2024-06-02 15:34:30 +02:00
J. Nick Koston 8f94205014
Include a traceback for non-strict event loop blocking detection (#118620) 2024-06-02 06:36:25 -04:00
tronikos dfb407728f
Stop instructing LLM to not pass the domain as a list (#118590) 2024-05-31 20:21:18 -04:00
epenet 41e852a01b
Add ability to replace connections in DeviceRegistry (#118555)
* Add ability to replace connections in DeviceRegistry

* Add more tests

* Improve coverage

* Apply suggestion

Co-authored-by: Paulus Schoutsen <paulus@home-assistant.io>

---------

Co-authored-by: Paulus Schoutsen <paulus@home-assistant.io>
2024-05-31 15:31:44 -04:00
Paulus Schoutsen 6dd01dbff7
Rename llm.ToolContext to llm.LLMContext (#118566) 2024-05-31 11:11:24 -04:00
tronikos 2bd142d3a6
Improve LLM prompt (#118520) 2024-05-30 22:03:57 -04:00
tronikos 0d6c7d0973
Fix LLMs asking which area when there is only one device (#118518)
* Ignore deprecated open and close cover intents for LLMs

* Fix LLMs asking which area when there is only one device

* remove unrelated changed

* remove unrelated changes
2024-05-30 17:14:11 -04:00
Ron Weikamp 2814ed5003
Add allow_negative configuration option to DurationSelector (#116134)
* Add configuration option positive to DurationSelector

* Rename to allow_negative in conjunction with a deprecation notice

Co-authored-by: Erik Montnemery <erik@montnemery.com>

---------

Co-authored-by: Erik Montnemery <erik@montnemery.com>
2024-05-30 17:42:34 +02:00
tronikos 4893faa671
Instruct LLM to not pass a list to the domain (#118451) 2024-05-30 08:37:45 +02:00
tronikos 639f6c640c
Improve LLM prompt (#118443)
* Improve LLM prompt

* test

* improvements

* improvements
2024-05-29 21:44:33 -04:00
tronikos 4056c4c2cc
Ask LLM to pass area name and domain (#118357) 2024-05-29 08:03:43 -04:00
Erik Montnemery 7abffd7cc8
Don't report entities with invalid unique id when loading the entity registry (#118290) 2024-05-29 08:32:39 +02:00
Paulus Schoutsen d22871f1fd
Reduce the intent response data sent to LLMs (#118346)
* Reduce the intent response data sent to LLMs

* No longer delete speech
2024-05-28 23:07:00 -04:00
Paulus Schoutsen c097a05ed4
Tweak Assist LLM API prompt (#118343) 2024-05-28 22:43:22 -04:00
Paulus Schoutsen 615a1eda51
LLM Assist API to ignore intents if not needed for exposed entities or calling device (#118283)
* LLM Assist API to ignore timer intents if device doesn't support it

* Refactor to use API instances

* Extract ToolContext class

* Limit exposed intents based on exposed entities
2024-05-28 21:29:18 -04:00
Paulus Schoutsen f0d7f48930
Handle generic commands as area commands in the LLM Assist API (#118276)
* Handle generic commands as area commands in the LLM Assist API

* Add word area
2024-05-28 11:21:17 -04:00
J. Nick Koston 87989a88cd
Remove translation and icon component path functions (#118214)
These functions have been stripped down to always return
the same path so there was no longer a need to have a
function for this. This is left-over cleanup from
previous refactoring.
2024-05-27 10:35:52 +02:00
Paulus Schoutsen ecb05989ca
Add exposed entities to the Assist LLM API prompt (#118203)
* Add exposed entities to the Assist LLM API prompt

* Check expose entities in Google test

* Copy Google default prompt test cases to LLM tests
2024-05-27 00:27:08 -04:00
Paulus Schoutsen 87fc27eeae
Teach Context about deepcopy (#118206)
Teach context about deepcopy
2024-05-26 23:14:02 -04:00
J. Nick Koston 7bbb33b415
Improve script disallowed recursion logging (#118151) 2024-05-26 12:58:34 +02:00
Denis Shulyaka cee3be5f7a
Break long strings in LLM tools (#118114)
* Break long code strings

* Address comments

---------

Co-authored-by: Paulus Schoutsen <balloob@gmail.com>
2024-05-25 14:24:51 -04:00
Denis Shulyaka da74ac06d7
Add user name and location to the LLM assist prompt (#118071)
Add user name and location to the llm assist prompt
2024-05-24 22:23:05 -04:00
Paulus Schoutsen 7554ca9460
Allow llm API to render dynamic template prompt (#118055)
* Allow llm API to render dynamic template prompt

* Make rendering api prompt async so it can become a RAG

* Fix test
2024-05-24 16:04:48 -04:00
Michael Hansen 77e385db52
Fix intent helper test (#118053)
Fix test
2024-05-24 12:59:19 -04:00
Robert Resch cb62f4242e
Remove strict connection (#117933) 2024-05-24 15:50:22 +02:00
epenet 0b4f1cff98
Use issue_registry fixture in core tests (#118042) 2024-05-24 15:26:32 +02:00
Franck Nijhof 9224997411
Add sequence action for automations & scripts (#117690)
Co-authored-by: Robert Resch <robert@resch.dev>
2024-05-24 09:34:49 +02:00
Peter eb76386c68
Prevent time pattern reschedule if cancelled during job execution (#117879)
Co-authored-by: J. Nick Koston <nick@koston.org>
2024-05-22 10:36:03 -10:00
Michael Hansen f99ec87338
Fail if targeting all devices in the house in service intent handler (#117930)
* Fail if targeting all devices in the house

* Update homeassistant/helpers/intent.py

---------

Co-authored-by: Paulus Schoutsen <balloob@gmail.com>
2024-05-22 13:53:31 -04:00
Denis Shulyaka 009c9e79ae
LLM Tools: Add device_id (#117884) 2024-05-21 21:24:46 -04:00
Michael Hansen 8079cc0464
Add description to intent handlers and use in LLM helper (#117864) 2024-05-21 12:54:34 -04:00
J. Nick Koston c9d1b127d8
Improve error message when template is rendered from wrong thread (#117822)
* Improve error message when template is rendered from wrong thread

* Improve error message when template is rendered from wrong thread
2024-05-20 23:26:48 -04:00
J. Nick Koston 5a609c34bb
Fix blocking I/O in the event loop when loading timezones (#117721) 2024-05-20 11:06:03 +02:00
Denis Shulyaka c3196a5667
LLM Tools support for Google Generative AI integration (#117644)
* initial commit

* Undo prompt chenges

* Move format_tool out of the class

* Only catch HomeAssistantError and vol.Invalid

* Add config flow option

* Fix type

* Add translation

* Allow changing API access from options flow

* Allow model picking

* Remove allowing HASS Access in main flow

* Move model to the top in options flow

* Make prompt conditional based on API access

* convert only once to dict

* Reduce debug logging

* Update title

* re-order models

* Address comments

* Move things

* Update labels

* Add tool call tests

* coverage

* Use LLM APIs

* Fixes

* Address comments

* Reinstate the title to not break entity name

---------

Co-authored-by: Paulus Schoutsen <balloob@gmail.com>
2024-05-19 22:11:25 -04:00
Paulus Schoutsen d001e7daea
Add API class to LLM helper (#117707)
* Add API class to LLM helper

* Add more tests

* Rename intent to assist to broaden scope
2024-05-18 21:14:05 -04:00
Denis Shulyaka f31873a846
Add LLM tools (#115464)
* Add llm helper

* break out Tool.specification as class members

* Format state output

* Fix intent tests

* Removed auto initialization of intents - let conversation platforms do that

* Handle DynamicServiceIntentHandler.extra_slots

* Add optional description to IntentTool init

* Add device_id and conversation_id parameters

* intent tests

* Add LLM tools tests

* coverage

* add agent_id parameter

* Apply suggestions from code review

* Apply suggestions from code review

* Apply suggestions from code review

* Apply suggestions from code review

* Apply suggestions from code review

* Fix tests

* Fix intent schema

* Allow a Python function to be registered as am LLM tool

* Add IntentHandler.effective_slot_schema

* Ensure IntentHandler.slot_schema to be vol.Schema

* Raise meaningful error on tool not found

* Move this change to a separate PR

* Update todo integration intent

* Remove Tool constructor

* Move IntentTool to intent helper

* Convert custom serializer into class method

* Remove tool_input from FunctionTool auto arguments to avoid recursion

* Remove conversion into Open API format

* Apply suggestions from code review

* Fix tests

* Use HassKey for helpers (see #117012)

* Add support for functions with typed lists, dicts, and sets as type hints

* Remove FunctionTool

* Added API to get registered intents

* Move IntentTool to the llm library

* Return only handlers in intents.async.get

* Removed llm tool registration from intent library

* Removed tool registration

* Add bind_hass back for now

* removed area and floor resolving

* fix test

* Apply suggestions from code review

* Improve coverage

* Fix intent_type type

* Temporary disable HassClimateGetTemperature intent

* Remove bind_hass

* Fix usage of slot schema

* Fix test

* Revert some test changes

* Don't mutate tool_input

---------

Co-authored-by: Paulus Schoutsen <paulus@home-assistant.io>
Co-authored-by: Paulus Schoutsen <balloob@gmail.com>
2024-05-15 19:16:47 -04:00
Paulus Schoutsen 4aba92ad04
Fix the type of slot schema of intent handlers (#117520)
Fix the slot schema of dynamic intenet handler
2024-05-15 16:45:15 -04:00
Matthias Alphart ec4c8ae228
Allow templates for enabling actions (#117049)
* Allow templates for enabling automation actions

* Use `cv.template` instead of `cv.template_complex`

* Rename test function
2024-05-15 21:03:52 +02:00
Matthias Alphart 076f57ee07
Allow templates for enabling conditions (#117047)
* Allow templates for enabling automation conditions

* Use `cv.template` instead of `cv.template_complex`
2024-05-15 21:03:28 +02:00
J. Nick Koston 7871e9279b
Adjust thread safety check messages to point to developer docs (#117392) 2024-05-14 15:20:31 +02:00
Matthias Alphart ba48da7678
Allow templates for enabling automation triggers (#114458)
* Allow templates for enabling automation triggers

* Test exception for non-limited template

* Use `cv.template` instead of `cv.template_complex`

* skip trigger with invalid enable template

instead of returning and thus not evaluating other triggers
2024-05-14 14:44:21 +02:00
Allen Porter b84829f70f
Import and cache supported feature enum flags only when needed (#117270)
* Import and cache supported feature enum flags only when needed

* Add comment aboud being loaded from executor.

---------

Co-authored-by: J. Nick Koston <nick@koston.org>
2024-05-13 21:07:39 -07:00
Erik Montnemery acc78b26f7
Rename some translation helper tests (#117248) 2024-05-11 14:17:53 +01:00
J. Nick Koston d7aa24fa50
Only load translations for an integration once per test session (#117118) 2024-05-11 12:00:02 +09:00
J. Nick Koston 03dcede211
Avoid creating inner tasks to load storage (#117099) 2024-05-08 23:41:20 +02:00
J. Nick Koston 84a91a86a9
Improve config entry has already been setup error message (#117091) 2024-05-08 14:16:08 -05:00
Troon cc99a9b62a
Add an add template filter (#109884)
* Addition of add filter

This change adds an `add` filter, the addition equivalent of the existing `multiply` filter.

* Test for add filter

* Update test_template.py

* Update tests/helpers/test_template.py

---------

Co-authored-by: Erik Montnemery <erik@montnemery.com>
2024-05-08 15:39:36 +02:00
Michael Hansen 7923471b94
Intent target matching and media player enhancements (#115445)
* Working

* Tests are passing

* Fix climate

* Requested changes from review
2024-05-07 22:01:03 -04:00
J. Nick Koston 8401b05d40
Move thread safety check in category_registry sooner (#117050) 2024-05-07 19:55:43 -05:00