Rhasspy visual conversation helper

This application is supposed to work as a visual helper for Rhasspy conversations on setups that have a screen or even a touchscreen.

See more information and detailed instructions on https://server47.de/rhasspy/

Setup instructions

Download jar file

Download the jar file of the program and put it in a folder, e.g. under /home/pi/RhasspyVisualConversationTool

Make sure java is installed

which java should return something. If it doesn't install a java runtime environment, e.g. with apt install openjdk-11-jre

Configuration file

Create RhasspyVisualConversationTool-settings.ini with the following content:

language=de-DE
sslDebug=false
keystoreFilename=
keystorePassword=changeme
hideWindowWhenIdle=true

logFolderPath=/var/log/rhasspyVisualConversationTool
logLevel=5

mqttClientServerHostname=mqtt.yourServer.com
mqttClientServerPort=1883
mqttClientUseSsl=false
mqttClientUseAuthentication=false
mqttClientUsername=
mqttClientPassword=

siteId=satellite01

If you have set mqttClientUseSsl=true it's at least likely you have your own PKI. In that case create a java keystore and put your root certificate in it. Then save the keystore in the folder of the jar and config file and put its filename for keystoreFilename .

Create start script

I suggest to use this path: /home/pi/Desktop/startConversationTool.sh

Put this content into the file (adapt the folder if you changed it above):

#!/bin/bash

while(true)
do
        cd /home/pi/RhasspyVisualConversationTool/

        /usr/bin/java -jar /home/pi/RhasspyVisualConversationTool/RhasspyVisualConversationTool.jar

        sleep 5
done

Make it runnable: chmod 775 /home/pi/Desktop/startConversationTool.sh

Make the script autostart with the graphical user interface

Create this file: /home/pi/.config/autostart/conversationTool.desktop

with the following content:

[Desktop Entry]
Type=Application
Exec=/home/pi/Desktop/startConversationTool.sh

Test run

When double-clicking your start script the app should come up and connect successfully.

Restart your Raspberry Pi to make sure the script comes up automatically.

Power settings for normal operation

In my own use-case the screen is usually off and turns on only when a command is spoken. In case you want that, too:

• The turning on is done by this application by briefly simulating mouse action.
• After the dialogue has finished I want the screen to turn off again soon.
  • The default power off setting is 10 min.
  • To decrease that the easiest way will be to install xscreensaver:
    • sudo apt install xscreensaver
    • Then open the configuration from the "Preferences" menu in the start menu and set the blank value to what is to your liking.

How exactly does the application respond to events?

Hotword detected

It will wakeup when a message with a topic of this kind is published: hermes/hotword/+/detected However it will only wake up if the message is targeted to the siteId specified in this instance's configuration file.

Intent recognized

If an intent is successfully recognized the text recognized will be displayed, so this topic is being watched for: hermes/intent/#

When text is asked to be spoken

...then it's also printed: hermes/tts/say

Session ends

Session is ended when the appropriate event is recognized: hermes/dialogueManager/sessionEnded

Bonus material

Up until here those are events built into Rhasspy. But I've come up with these additional events/commands:

Simply display text

Your application just wants some text to be displayed, publish it under topic hermes/tts/displayText using the same syntax as for the speak command.

Display selectable options

CONSTRUCTION SITE, NOT FINISHED, YET If you'd like to publish a list of selectable options for the user send it for topic hermes/tts/displayOptions Syntax