Commit Graph

120 Commits (llm-task-pref)

Author SHA1 Message Date
Paulus Schoutsen 014010acbd
Assist Pipeline: Intent progress event when we start streaming (#146388)
Intent progress event when we start streaming
2025-06-10 09:55:43 -05:00
Paulus Schoutsen 1cc2baa95e
Pipeline to stream TTS on tool call (#145477) 2025-05-25 14:59:07 -05:00
Paulus Schoutsen abcf925b79
Assist Pipeline stream TTS when supported and long response (#145264)
* Assist Pipeline stream TTS when supported and long response

* Indicate in run-start if streaming supported

* Simplify a little bit

* Trigger streaming based on characters

* 60
2025-05-20 13:00:27 -05:00
Paulus Schoutsen e09dde2ea9
Allow TTS streams to generate temporary media source IDs (#145080)
* Allow TTS streams to generate temporary media source IDs

* Update tests/components/tts/test_media_source.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update assist snapshots

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-05-19 12:04:19 -04:00
Michael Hansen 4472dc533d
Don't filter nevermind for fallback (#141294) 2025-03-24 17:26:35 +01:00
Paulus Schoutsen 0ba5711603
Add TTS token to TTS-END event (#140333) 2025-03-11 17:54:39 +01:00
Paulus Schoutsen 2cce1b024e
Migrate Assist Pipeline to use TTS stream (#139542)
* Migrate Pipeline to use TTS stream

* Fix tests
2025-03-01 21:43:00 +01:00
Paulus Schoutsen 90fc6ffdbf
Add support for continue conversation in Assist Pipeline (#139480)
* Add support for continue conversation in Assist Pipeline

* Also forward to ESPHome

* Update snapshot

* And mobile app
2025-02-28 13:15:31 -06:00
Paulus Schoutsen 744a7a0e82
Fix conversation agent fallback (#139421) 2025-02-27 15:51:40 +01:00
Paulus Schoutsen 0b6f49fec2
Filter out certain intents from being matched in local fallback (#137763)
* Filter out certain intents from being matched in local fallback

* Only filter if LLM agent can control HA
2025-02-19 14:27:42 -06:00
Paulus Schoutsen 29c6a2ec13
Add pipeline intent-progress events based on deltas (#138095)
Add intent progress Assist event
2025-02-09 18:09:52 -08:00
Paulus Schoutsen f64b494282
Conversation chat log cleanup and optimization (#137784) 2025-02-07 22:06:16 -08:00
Paulus Schoutsen 649319f4ee
Introduce async_add_assistant_content to conversation chat log (#137273)
introduce async_add_assistant_content_without_tools to conversation chat log
2025-02-03 14:27:55 -06:00
Paulus Schoutsen 05ca80f4ba
Assist Pipeline to use ChatSession for conversation ID (#137143)
* Assist Pipeline to use ChatSession for conversation ID

* Adjust to latest changes
2025-02-03 09:18:15 -06:00
Paulus Schoutsen 9679fc7878
Chat session rev2 (#137209)
* Chat Session rev 2

* Rename session to chat_log

* Simplify typing

* Typing

* Address comments

* Fix anthropic and ollama
2025-02-03 00:05:20 -05:00
Paulus Schoutsen dd9bd8ef73
Make get_chat_session a callback context manager (#137146) 2025-02-01 23:37:24 -05:00
Paulus Schoutsen 2f6640707b
Extract conversation ID generation to helper (#137062)
* Extract conversation ID generation to helper

* Allow nested get_chat_log calls
2025-02-01 20:54:00 -05:00
Paulus Schoutsen 427c437a68
Add start_conversation service to Assist Satellite (#134921)
* Add start_conversation service to Assist Satellite

* Fix tests

* Implement start_conversation in voip

* Update homeassistant/components/assist_satellite/entity.py

---------

Co-authored-by: Michael Hansen <mike@rhasspy.org>
2025-01-29 21:32:10 -05:00
Paulus Schoutsen 5690516852
ChatSession: Split native content out of message class (#136668)
Split native content out of message class
2025-01-27 21:12:42 -08:00
Paulus Schoutsen 754de6f998
Add shared history for conversation agents (#135903)
* Add shared history for conversation agents

* Remove unused code

* Add support for native history items

* Store all assistant responses as assistant in history

* Add history support to DefaultAgent.async_handle_intents

* Make local fallback work

* Add default agent history

* Add history cleanup

* Add tests

* ChatHistory -> ChatSession

* Address comments

* Update snapshots
2025-01-18 22:33:03 -05:00
Marc Mueller 19e5b091c5
Use HassKey for assist_pipeline singleton (#135875) 2025-01-18 09:52:13 -10:00
Joost Lekkerkerker 8db63adc11
Bump ruff to 0.9.1 (#135197) 2025-01-14 11:46:12 +01:00
Michael Hansen b897e6a85f
Use STT/TTS languages for LLM fallback (#135533) 2025-01-13 14:17:12 -06:00
Paulus Schoutsen 7a484ee0ae
Add extra prompt to assist pipeline and conversation (#124743)
* Add extra prompt to assist pipeline and conversation

* extra_prompt -> extra_system_prompt

* Fix rebase

* Fix tests
2025-01-06 12:58:42 -05:00
Michael Hansen 50b897bdaa
Add STT error code for cloud authentication failure (#133170) 2024-12-13 14:59:46 -05:00
Michael Hansen 94260147d7
Fix pipeline conversation language (#132896) 2024-12-11 18:52:02 +01:00
Jan-Philipp Benecke 6b7724c556
Track if intent was processed locally (#132166) 2024-12-04 09:52:15 +01:00
Michael 1a9ab07742
Allow empty trigger sentence responses in conversations (#131849)
allow empty trigger sentence responses
2024-11-28 18:30:05 +01:00
Michael Hansen 7621012ee6
Ensure sentence triggers are only checked once (#131210)
Co-authored-by: Artur Pragacz <49985303+arturpragacz@users.noreply.github.com>
2024-11-22 17:38:19 +01:00
Michael Hansen 5fa9a945d9
Handle sentence triggers and local intents before pipeline agent (#129058)
* Handle sentence triggers and registered intents in Assist LLM API

* Remove from LLM

* Check sentence triggers and local intents first

* Fix type

* Fix type again

* Use pipeline language

* Fix cloud test

* Clean up and fix translation key

* Refactor async_recognize
2024-11-14 10:50:50 -06:00
Michael Hansen 69e3348cd7
Use different VAD thresholds for before and during voice command (#129848)
* Use two VAD thresholds

* Fix VoiceActivityTimeout class

* Update homeassistant/components/assist_pipeline/audio_enhancer.py

---------

Co-authored-by: Joost Lekkerkerker <joostlek@outlook.com>
2024-11-05 08:01:45 -06:00
epenet 47a9dda3b8
Use root import in components (#125858) 2024-09-12 22:21:21 +02:00
Paulus Schoutsen ff3cabbf3a
Small Assist Satellite fixes (#125384) 2024-09-06 07:36:02 -04:00
Michael Hansen 4ecc6555bf
Add support for sample bytes in preferred TTS format (#125235) 2024-09-04 13:42:41 -04:00
Paulus Schoutsen 1aa0dbdaf5
Revert "Revert "Use speex for noise suppression and auto gain"" (#124637)
Revert "Revert "Use speex for noise suppression and auto gain" (#124620)"

This reverts commit 302ffe5e56.
2024-08-26 15:10:43 +02:00
Paulus Schoutsen 302ffe5e56
Revert "Use speex for noise suppression and auto gain" (#124620)
Revert "Use speex for noise suppression and auto gain (#124591)"

This reverts commit 592f60643a.
2024-08-26 10:12:05 +02:00
Michael Hansen 592f60643a
Use speex for noise suppression and auto gain (#124591) 2024-08-25 23:50:32 -05:00
Michael Hansen 80aa2c269b
Standardize assist pipelines on 10ms chunk size (#123024)
* Make chunk size always 10ms

* Fix voip
2024-08-01 22:39:17 +02:00
Michael Hansen d5388452d4
Use finished speaking detection in ESPHome/Wyoming (#122962) 2024-07-31 20:39:03 +02:00
Michael Hansen 7f4dabf546
Switch from WebRTC to microVAD (#122861)
* Switch WebRTC to microVAD

* Remove webrtc-noise-gain from licenses
2024-07-31 09:42:45 +02:00
Paulus Schoutsen 154da1b18b
Allow targeting conversation agent as pipeline (#119556)
* Allow targetting conversation agent as pipeline

* Test that we can use a conversation entity as an assist pipeline

* Add test for WS get

---------

Co-authored-by: Michael Hansen <mike@rhasspy.org>
2024-07-09 10:56:53 -05:00
Paulus Schoutsen 01861cd240
Ensure we prepare conversation pipeline when speech-to-text starts (#114665)
* Ensure we prepare conversation pipeline when speech-to-text starts

* Add lock around recognize

* Update homeassistant/components/conversation/default_agent.py

* Add lock around load intents
2024-07-09 09:58:52 -05:00
Paulus Schoutsen 22718ca32a
Assist Pipeline minor cleanup (#121187) 2024-07-05 09:26:32 +02:00
Paulus Schoutsen 869f24df49
Convert async_get_conversation_languages from async to callback (#121162)
* Convert get_languages to callback

* One more callback
2024-07-04 10:41:31 +02:00
Marc Mueller ca55986057
Import Generator from collections.abc (1) (#120914) 2024-07-01 11:51:51 +02:00
Joost Lekkerkerker 1fdd056c0e
Fix ruff manual-dict-comprehension PERF403 (#120723)
* Fix PERF403

* Fix

* Fix
2024-06-28 14:17:47 +02:00
Marc Mueller b393024acd
Improve collection schema typing (#120441) 2024-06-25 18:57:15 +02:00
Erik Montnemery 0d1b050520
Remove create_create from StorageCollectionWebsocket.async_setup (#119489) 2024-06-24 15:41:08 +02:00
Erik Montnemery e0378f79a4
Remove create_list from StorageCollectionWebsocket.async_setup (#119508) 2024-06-17 12:16:36 +02:00
epenet 960d1289ef
Avoid references to websocket_api.const in core and other components (#119285) 2024-06-10 13:49:44 +02:00