# Rhasspy visual conversation helper Sometimes the acoustic response (in my case often just a beep as confirmation with no voice output) was not sufficient to be sure it had executed the intended command. This application is supposed to work as a visual helper/transcriber for Rhasspy conversations on setups that have a screen or even a touchscreen. See more of my Rhasspy accessories and skills on [https://server47.de/rhasspy/](https://server47.de/rhasspy/) ![Idle state](https://server47.de/rhasspy/RhasspyVisualConversationTool/RVCT_screenshot_1.png) ![Wakeword recognized](https://server47.de/rhasspy/RhasspyVisualConversationTool/RVCT_screenshot_2.png) ![Intent recognized](https://server47.de/rhasspy/RhasspyVisualConversationTool/RVCT_screenshot_3.png) ## Setup instructions ### Download jar file Download the jar file from [here](https://git.server47.de/jens/RhasspyVisualConversationTool/raw/branch/master/executable/RhasspyVisualConversationTool.jar) or compile it yourself. Then put it in a folder, e.g. _/home/pi/RhasspyVisualConversationTool_ ### Make sure java is installed _which java_ should return something. If it does not then install a java runtime environment, e.g. with _apt install openjdk-11-jre_ ### Configuration file Create _RhasspyVisualConversationTool-settings.ini_ with the following content: ``` language=de-DE sslDebug=false keystoreFilename= keystorePassword=changeme hideWindowWhenIdle=true logFolderPath=/var/log/rhasspyVisualConversationTool logLevel=5 logFileEnabled=false mqttClientServerHostname=mqtt.yourServer.com mqttClientServerPort=1883 mqttClientUseSsl=false mqttClientUseAuthentication=false mqttClientUsername= mqttClientPassword= siteId=satellite01 ``` If you have set _mqttClientUseSsl=true_ it's at least likely you have your own PKI. In that case create a java keystore and put your root certificate in it. Save the keystore in the folder of the jar and config file and put its filename as value for _keystoreFilename_ . If you want the window to always stay visible set _hideWindowWhenIdle_ to false. Otherwise the window will minimize after 2 seconds when it has nothing to do. ### Create start script I suggest to use this path: _/home/pi/Desktop/startConversationTool.sh_ That way you could start simply start it manually if you have a touchscreen. Put this content into the file (adapt the folder if you changed it above): ``` #!/bin/bash while(true) do cd /home/pi/RhasspyVisualConversationTool/ /usr/bin/java -jar /home/pi/RhasspyVisualConversationTool/RhasspyVisualConversationTool.jar sleep 5 done ``` Make it runnable: _chmod 775 /home/pi/Desktop/startConversationTool.sh_ ### Make the script autostart with the graphical user interface Create this file: _/home/pi/.config/autostart/conversationTool.desktop_ with the following content: ``` [Desktop Entry] Type=Application Exec=/home/pi/Desktop/startConversationTool.sh ``` ### Test run When double-clicking your start script the app should come up and connect successfully. Restart your Raspberry Pi to make sure the script comes up automatically. ### Power settings for normal operation In my own use-case the screen is usually off and turns on only when a command is spoken. In case you want that, too: * The turning on is done by this application by briefly simulating mouse action. * After the dialogue has finished I want the screen to turn off again soon. * The default power off setting is 10 min. * To decrease that the easiest way will be to install xscreensaver: * _sudo apt install xscreensaver_ * Then open the configuration from the "Preferences" menu in the start menu and set the blank value to what is to your liking. For me these settings have worked quite well: ![Idle state](https://server47.de/rhasspy/RhasspyVisualConversationTool/RPI_Screensaver_settings_1.png) ![Idle state](https://server47.de/rhasspy/RhasspyVisualConversationTool/RPI_Screensaver_settings_2.png) ### How exactly does the application respond to events? #### Hotword detected It will wakeup when a message with a topic of this kind is published: _hermes/hotword/+/detected_ However it will only wake up if the message is targeted to the siteId specified in this instance's configuration file. #### Intent recognized If an intent is successfully recognized the text recognized will be displayed, so this topic is being watched for: _hermes/intent/#_ #### When text is asked to be spoken ...then it's also printed: _hermes/tts/say_ #### Session ends Session is ended when the appropriate event is recognized: _hermes/dialogueManager/sessionEnded_ #### Bonus material Up until here those are events built into Rhasspy. But I've come up with these additional events/commands: ##### Simply display text Your application just wants some text to be displayed, publish it under topic _hermes/tts/displayText_ using the same syntax as for the speak command. ##### Display selectable options CONSTRUCTION SITE, NOT FINISHED, YET If you'd like to publish a list of selectable options for the user send it for topic _hermes/tts/displayOptions_ Syntax