2021-02-26 18:18:52 +01:00
# Rhasspy visual conversation helper
2021-03-03 19:35:18 +01:00
Sometimes the acoustic response (in my case often just a beep as confirmation with no voice output) was not sufficient to be sure it had executed the intended command.
This application is supposed to work as a visual helper/transcriber for Rhasspy conversations on setups that have a screen or even a touchscreen.
2021-02-26 18:18:52 +01:00
2021-03-03 19:35:18 +01:00
See more of my Rhasspy accessories and skills on [https://server47.de/rhasspy/ ](https://server47.de/rhasspy/ )
2021-02-27 22:58:52 +01:00
2021-03-03 19:32:56 +01:00
![Idle state ](https://server47.de/rhasspy/RhasspyVisualConversationTool/RVCT_screenshot_1.png )
![Wakeword recognized ](https://server47.de/rhasspy/RhasspyVisualConversationTool/RVCT_screenshot_2.png )
![Intent recognized ](https://server47.de/rhasspy/RhasspyVisualConversationTool/RVCT_screenshot_3.png )
2021-02-27 22:58:52 +01:00
## Setup instructions
### Download jar file
2021-03-03 19:32:56 +01:00
Download the jar file from [here ](https://git.server47.de/jens/RhasspyVisualConversationTool/raw/branch/master/executable/RhasspyVisualConversationTool.jar )
or compile it yourself.
Then put it in a folder, e.g. _/home/pi/RhasspyVisualConversationTool_
2021-02-27 22:58:52 +01:00
### Make sure java is installed
2021-03-03 19:32:56 +01:00
_which java_ should return something. If it does not then install a java runtime environment, e.g. with _apt install openjdk-11-jre_
2021-02-27 22:58:52 +01:00
### Configuration file
2021-03-03 19:32:56 +01:00
Create _RhasspyVisualConversationTool-settings.ini_ with the following content:
2021-02-27 22:58:52 +01:00
```
language=de-DE
sslDebug=false
keystoreFilename=
keystorePassword=changeme
2021-03-03 19:19:07 +01:00
hideWindowWhenIdle=true
2021-02-27 22:58:52 +01:00
logFolderPath=/var/log/rhasspyVisualConversationTool
logLevel=5
2021-05-06 19:07:33 +02:00
logFileEnabled=false
2021-02-27 22:58:52 +01:00
mqttClientServerHostname=mqtt.yourServer.com
mqttClientServerPort=1883
mqttClientUseSsl=false
mqttClientUseAuthentication=false
mqttClientUsername=
mqttClientPassword=
2021-03-03 19:19:07 +01:00
siteId=satellite01
2021-02-27 22:58:52 +01:00
```
2021-03-03 19:32:56 +01:00
If you have set _mqttClientUseSsl=true_ it's at least likely you have your own PKI. In that case create a java keystore and put your root certificate in it. Save the keystore in the folder of the jar and config file and put its filename as value for _keystoreFilename_ .
If you want the window to always stay visible set _hideWindowWhenIdle_ to false. Otherwise the window will minimize after 2 seconds when it has nothing to do.
2021-02-27 22:58:52 +01:00
### Create start script
I suggest to use this path: _/home/pi/Desktop/startConversationTool.sh_
2021-03-03 19:32:56 +01:00
That way you could start simply start it manually if you have a touchscreen.
2021-02-27 22:58:52 +01:00
Put this content into the file (adapt the folder if you changed it above):
```
#!/bin/bash
while(true)
do
cd /home/pi/RhasspyVisualConversationTool/
/usr/bin/java -jar /home/pi/RhasspyVisualConversationTool/RhasspyVisualConversationTool.jar
sleep 5
done
```
Make it runnable: _chmod 775 /home/pi/Desktop/startConversationTool.sh_
### Make the script autostart with the graphical user interface
Create this file: _/home/pi/.config/autostart/conversationTool.desktop_
with the following content:
```
[Desktop Entry]
Type=Application
Exec=/home/pi/Desktop/startConversationTool.sh
```
### Test run
When double-clicking your start script the app should come up and connect successfully.
2021-02-28 19:52:20 +01:00
Restart your Raspberry Pi to make sure the script comes up automatically.
### Power settings for normal operation
In my own use-case the screen is usually off and turns on only when a command is spoken. In case you want that, too:
* The turning on is done by this application by briefly simulating mouse action.
* After the dialogue has finished I want the screen to turn off again soon.
* The default power off setting is 10 min.
* To decrease that the easiest way will be to install xscreensaver:
* _sudo apt install xscreensaver_
* Then open the configuration from the "Preferences" menu in the start menu and set the blank value to what is to your liking.
2021-03-03 19:32:56 +01:00
For me these settings have worked quite well:
![Idle state ](https://server47.de/rhasspy/RhasspyVisualConversationTool/RPI_Screensaver_settings_1.png )
![Idle state ](https://server47.de/rhasspy/RhasspyVisualConversationTool/RPI_Screensaver_settings_2.png )
2021-02-28 19:52:20 +01:00
### How exactly does the application respond to events?
#### Hotword detected
It will wakeup when a message with a topic of this kind is published: _hermes/hotword/+/detected_
However it will only wake up if the message is targeted to the siteId specified in this instance's configuration file.
#### Intent recognized
If an intent is successfully recognized the text recognized will be displayed, so this topic is being watched for: _hermes/intent/#_
#### When text is asked to be spoken
...then it's also printed: _hermes/tts/say_
#### Session ends
Session is ended when the appropriate event is recognized: _hermes/dialogueManager/sessionEnded_
#### Bonus material
Up until here those are events built into Rhasspy. But I've come up with these additional events/commands:
##### Simply display text
Your application just wants some text to be displayed, publish it under topic _hermes/tts/displayText_ using the same syntax as for the speak command.
##### Display selectable options
CONSTRUCTION SITE, NOT FINISHED, YET
If you'd like to publish a list of selectable options for the user send it for topic _hermes/tts/displayOptions_
Syntax