OpenHAB
/
openhab-docs
mirror of https://github.com/openhab/openhab-docs.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity

[Multimedia] Audo + keyword spotter (#1821) 

Signed-off-by: Laurent Garnier <lg.hc@free.fr>
pull/1822/head
lolodomo 2022-05-17 21:49:21 +02:00 committed by GitHub
parent 7cfbb875bf
commit ac76bb3feb
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 84 additions and 20 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

104
configuration/multimedia.md
View File

@ -17,7 +17,32 @@ The framework supports some base [functions](https://openhab.org/javadoc/latest/
- `decreaseMasterVolume(float percent)` : Decreases the volume by the given percent
- `float getMasterVolume()` : Returns the current volume as a float between 0 and 1
## Audio
## Audio Capture
openHAB is able to capture audio.
There are different options for input devices (so called audio sources):
The distribution comes with these options built-in:
| Output Device | Audio Source | Description |
|---------------|-------------------|-------------------------------------------------|
| `javasound` | System Microphone | This uses the Java Sound API for audio capture. |
Additionally, certain bindings register their supported devices as audio sources, e.g. PulseAudio.
### Console commands
To check which audio sources are available, you can use the console:
```text
openhab> openhab:audio sources
* System Microphone (javasound)
```
You can define the default audio source either by textual configuration in `$OPENHAB_CONF/services/runtime.cfg` or in the UI in `Settings->Audio`.
## Audio Playback
openHAB is able to play sound either from the file system (files need to be put in the folder `$OPENHAB_CONF/sounds`), from URLs (e.g. Internet radio streams) or generated by text-to-speech engines (which are available as optional [Voice add-ons](/addons/#voice)).
@ -38,20 +63,28 @@ To check which audio sinks are available, you can use the console:
```text
openhab> openhab:audio sinks
enhancedjavasound
webaudio
* System Speaker (enhancedjavasound)
Web Audio (webaudio)
```
You can define the default audio sink either by textual configuration in `$OPENHAB_CONF/services/runtime.cfg` or in the UI in `Settings->Audio`.
In order to play a sound, you can use the following command on the console:
In order to play a sound, you can use the following commands on the console:
```text
openhab> openhab:audio play doorbell.mp3
openhab> openhab:audio play sonos:PLAY5:kitchen doorbell.mp3
openhab> openhab:audio play sonos:PLAY5:kitchen doorbell.mp3 25
openhab> openhab:audio stream example.com
openhab> openhab:audio stream sonos:PLAY5:kitchen example.com
```
You can optionally specify the audio sink between the `play` parameter and the file name and between the `stream` parameter and the URL.
This parameter can even be a pattern including `*` and `?` placeholders; in this case, the sound is played to all audio sinks matching the pattern.
If this parameter is not provided, the sound is played to the default audio sink.
The command to play a file accepts an optional last parameter to specify the volume of playback.
### Actions
Alternatively the [`playSound()`](https://openhab.org/javadoc/latest/org/openhab/core/model/script/actions/audio#playSound(java.lang.String)) or [`playStream()`](https://openhab.org/javadoc/latest/org/openhab/core/model/script/actions/audio#playStream(java.lang.String)) functions can be used in DSL rules:
@ -82,7 +115,7 @@ playStream("sonos:PLAY5:kitchen", "example.com")
In order to use text-to-speech, you need to install at least one [TTS service](/addons/#voice).
### Console Commands
#### Console Commands
To check which Text-to-Speech services are available, you can use the console:
@ -110,10 +143,22 @@ In order to say a text, you can enter such a command on the console (The default
openhab> openhab:voice say Hello world!
```
### Actions
#### Actions
Alternatively you can execute such commands within DSL rules by using the [`say()`](https://openhab.org/javadoc/latest/org/openhab/core/voice/voicemanager#say(java.lang.String)) function:
- `say(Object text)` : says a given text with the default voice
- `say(Object text, PercentType volume)` : says a given text with the default voice and the given volume
- `say(Object text, String voice)` : says a given text with a given voice
- `say(Object text, String voice, PercentType volume)` : says a given text with a given voice and the given volume
- `say(Object text, String voice, String sink)` : says a given text with a given voice through the given sink
- `say(Object text, String voice, String sink, PercentType volume)` : says a given text with a given voice and the given volume through the given sink
You can select a particular voice (second parameter) and a particular audio sink (third parameter).
If no voice or no audio sink is provided, the default voice and default audio sink will be used.
##### Examples
```java
say("Hello world!")
say("Hello world!", new PercentType(25))
@ -123,14 +168,11 @@ say("Hello world!", "voicerss:enUS", "sonos:PLAY5:kitchen")
say("Hello world!", "voicerss:enUS", "sonos:PLAY5:kitchen", new PercentType(25))
```
You can select a particular voice (second parameter) and a particular audio sink (third parameter).
If no voice or no audio sink is provided, the default voice and default audio sink will be used.
### Speech-to-Text
In order to use Speech-to-Text, you need to install at least one [STT service](/addons/#voice).
### Console Commands
#### Console Commands
To check which Speech-to-Text services are available, you can use the console:
@ -141,6 +183,22 @@ openhab> openhab:voice sttservices
You can define a default STT service to use either by textual configuration in `$OPENHAB_CONF/services/runtime.cfg` or in the UI in `Settings->Voice`.
### Keyword Spotter
Spotting a keyword is usually the first step to trigger a dialogue with a voice assistant.
In order to spot keyword, you need to install at least one [Keyword Spotter service](/addons/#voice).
#### Console Commands
To check which Keyword Spotter services are available, you can use the console:
```text
openhab> openhab:voice keywordspotters
* Porcupine (porcupineks)
```
You can define a default Keyword Spotter service to use either by textual configuration in `$OPENHAB_CONF/services/runtime.cfg` or in the UI in `Settings->Voice`.
### Human Language Interpreter
Human language interpreters are meant to process prose that e.g. is a result of voice recognition or from other sources.
@ -153,7 +211,7 @@ There are two implementations available by default:
| `system` | Built-in Interpreter | This is a simple implementation that understands basic home automation commands like "turn on the light" or "stop the music". It currently supports only English, German, French and Spanish and the vocabulary is still very limited. The exact syntax still needs to be documented, for the moment you need to refer to the [source code](https://github.com/openhab/openhab-core/blob/main/bundles/org.openhab.core.voice/src/main/java/org/openhab/core/voice/internal/text/StandardInterpreter.java#L42). |
| `opennlp` | HABot OpenNLP Interpreter | A machine-learning natural language processor based on Apache OpenNLP for intent classification and entity extraction. |
### Console Commands
#### Console Commands
To check which human language interpreters are available, you can use the console:
@ -174,17 +232,13 @@ openhab> openhab:voice interpret turn on the light
The default human language interpreter will be used.
In case of interpretation error, the error message will be said using the default voice and default audio sink.
### Actions
#### Actions
Alternatively you can execute such commands within DSL rules (using the [`interpret()`](https://openhab.org/javadoc/latest/org/openhab/core/voice/voicemanager#interpret(java.lang.String)) function)
Alternatively you can execute such commands within DSL rules using the [`interpret()`](https://openhab.org/javadoc/latest/org/openhab/core/voice/voicemanager#interpret(java.lang.String)) function:
```java
interpret("turn on the light")
var String result = interpret("turn on the light", "system")
result = interpret("turn on the light", "system", null)
result = interpret("turn on the light", "system,rulehli")
result = interpret(VoiceCommand.state, "system", "sonos:PLAY5:kitchen")
```
- `interpret(Object text)` : interprets a given text by the default human language interpreter
- `interpret(Object text, String interpreters)` : interprets given text by given human language interpreter(s)
- `interpret(Object text, String interpreters, String sink)` : interprets a given text by given human language interpreter(s) and using the given sink
You can select particular human language interpreter(s) (second parameter) and a particular audio sink (third parameter).
The human language interpreter(s) parameter must be the ID of an installed interpreter or a comma separated list of interpreter IDs; each provided interpreter is executed in the provided order until one is able to interpret the command.
@ -193,3 +247,13 @@ If the provided audio sink is set to null, the error message will not be said.
If no human language interpreter or no audio sink is provided, the default human language interpreter and default audio sink will be used.
The interpretation result is returned as a string.
Note that this result is always a null string with the rule-based Interpreter (rulehli).
##### Examples
```java
interpret("turn on the light")
var String result = interpret("turn on the light", "system")
result = interpret("turn on the light", "system", null)
result = interpret("turn on the light", "system,rulehli")
result = interpret(VoiceCommand.state, "system", "sonos:PLAY5:kitchen")
```