133 lines
5.0 KiB
Markdown
133 lines
5.0 KiB
Markdown
# Rhasspy visual conversation helper
|
|
|
|
Sometimes the acoustic response (in my case often just a beep as confirmation with no voice output) was not sufficient to be sure it had executed the intended command.
|
|
This application is supposed to work as a visual helper/transcriber for Rhasspy conversations on setups that have a screen or even a touchscreen.
|
|
|
|
See more of my Rhasspy accessories and skills on [https://server47.de/rhasspy/](https://server47.de/rhasspy/)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
## Setup instructions
|
|
|
|
### Download jar file
|
|
Download the jar file from [here](https://git.server47.de/jens/RhasspyVisualConversationTool/raw/branch/master/executable/RhasspyVisualConversationTool.jar)
|
|
or compile it yourself.
|
|
Then put it in a folder, e.g. _/home/pi/RhasspyVisualConversationTool_
|
|
|
|
### Make sure java is installed
|
|
_which java_ should return something. If it does not then install a java runtime environment, e.g. with _apt install openjdk-11-jre_
|
|
|
|
### Configuration file
|
|
|
|
Create _RhasspyVisualConversationTool-settings.ini_ with the following content:
|
|
|
|
```
|
|
language=de-DE
|
|
sslDebug=false
|
|
keystoreFilename=
|
|
keystorePassword=changeme
|
|
hideWindowWhenIdle=true
|
|
|
|
logFolderPath=/var/log/rhasspyVisualConversationTool
|
|
logLevel=5
|
|
|
|
mqttClientServerHostname=mqtt.yourServer.com
|
|
mqttClientServerPort=1883
|
|
mqttClientUseSsl=false
|
|
mqttClientUseAuthentication=false
|
|
mqttClientUsername=
|
|
mqttClientPassword=
|
|
|
|
siteId=satellite01
|
|
```
|
|
|
|
If you have set _mqttClientUseSsl=true_ it's at least likely you have your own PKI. In that case create a java keystore and put your root certificate in it. Save the keystore in the folder of the jar and config file and put its filename as value for _keystoreFilename_ .
|
|
|
|
If you want the window to always stay visible set _hideWindowWhenIdle_ to false. Otherwise the window will minimize after 2 seconds when it has nothing to do.
|
|
|
|
### Create start script
|
|
|
|
I suggest to use this path: _/home/pi/Desktop/startConversationTool.sh_
|
|
|
|
That way you could start simply start it manually if you have a touchscreen.
|
|
|
|
Put this content into the file (adapt the folder if you changed it above):
|
|
|
|
```
|
|
#!/bin/bash
|
|
|
|
while(true)
|
|
do
|
|
cd /home/pi/RhasspyVisualConversationTool/
|
|
|
|
/usr/bin/java -jar /home/pi/RhasspyVisualConversationTool/RhasspyVisualConversationTool.jar
|
|
|
|
sleep 5
|
|
done
|
|
```
|
|
|
|
Make it runnable: _chmod 775 /home/pi/Desktop/startConversationTool.sh_
|
|
|
|
### Make the script autostart with the graphical user interface
|
|
|
|
Create this file: _/home/pi/.config/autostart/conversationTool.desktop_
|
|
|
|
with the following content:
|
|
|
|
```
|
|
[Desktop Entry]
|
|
Type=Application
|
|
Exec=/home/pi/Desktop/startConversationTool.sh
|
|
```
|
|
|
|
### Test run
|
|
When double-clicking your start script the app should come up and connect successfully.
|
|
|
|
Restart your Raspberry Pi to make sure the script comes up automatically.
|
|
|
|
### Power settings for normal operation
|
|
|
|
In my own use-case the screen is usually off and turns on only when a command is spoken. In case you want that, too:
|
|
|
|
* The turning on is done by this application by briefly simulating mouse action.
|
|
* After the dialogue has finished I want the screen to turn off again soon.
|
|
* The default power off setting is 10 min.
|
|
* To decrease that the easiest way will be to install xscreensaver:
|
|
* _sudo apt install xscreensaver_
|
|
* Then open the configuration from the "Preferences" menu in the start menu and set the blank value to what is to your liking.
|
|
|
|
For me these settings have worked quite well:
|
|
|
|
|
|
|
|
|
|
|
|
### How exactly does the application respond to events?
|
|
#### Hotword detected
|
|
It will wakeup when a message with a topic of this kind is published: _hermes/hotword/+/detected_
|
|
However it will only wake up if the message is targeted to the siteId specified in this instance's configuration file.
|
|
|
|
#### Intent recognized
|
|
If an intent is successfully recognized the text recognized will be displayed, so this topic is being watched for: _hermes/intent/#_
|
|
|
|
#### When text is asked to be spoken
|
|
...then it's also printed: _hermes/tts/say_
|
|
|
|
#### Session ends
|
|
Session is ended when the appropriate event is recognized: _hermes/dialogueManager/sessionEnded_
|
|
|
|
#### Bonus material
|
|
Up until here those are events built into Rhasspy. But I've come up with these additional events/commands:
|
|
|
|
##### Simply display text
|
|
Your application just wants some text to be displayed, publish it under topic _hermes/tts/displayText_ using the same syntax as for the speak command.
|
|
|
|
##### Display selectable options
|
|
CONSTRUCTION SITE, NOT FINISHED, YET
|
|
If you'd like to publish a list of selectable options for the user send it for topic _hermes/tts/displayOptions_
|
|
Syntax
|