Speech Recognition and Text-to-Speech on the Raspberry Pi

Your Pi can see; now it will listen and speak. This lesson adds offline speech-to-text so the Pi understands spoken commands, and text-to-speech so it can reply out loud, using the included USB microphone and a speaker. You will test the mic, transcribe speech and match it to known commands even when the transcription is imperfect, and make the Pi talk back. This is the groundwork for the voice assistant you build at the end of the month.

Why offline speech

You will use offline speech-to-text such as Vosk, a lightweight and accurate library, and an offline text-to-speech engine such as eSpeak. Doing it locally keeps the audio private, since nothing is uploaded, and it works with no internet. The trade-off is that a small local model is less perfect than a huge cloud service, so transcriptions will occasionally have errors. That limitation is manageable, and handling it well is part of today's lesson.

Capturing audio from the microphone

The included USB microphone plugs straight into the Pi, and a library like sounddevice captures audio into your Python program. Before writing detection logic, confirm the hardware works: the command arecord -l lists recording devices so you can verify the Pi sees the mic. A good first test is to record a few seconds and send the clip to yourself over Discord or Telegram, reusing the notify functions from last lesson, which proves both the mic and the software path work.

Speech to text, and matching commands

Feeding captured audio to the model returns a text transcription. Because a small offline model is not perfect, the transcription may be slightly wrong: say turn light on and you might get journey light on. Rather than demand an exact match, you keep a list of known commands (turn light on, turn light off, start recording, stop recording) and find the one most similar to what was transcribed. This command matching turns a fuzzy transcription into a reliable action, which is exactly what an Alexa-style assistant needs to feel dependable.

Text to speech

To reply out loud, you pass text to a text-to-speech engine, which produces spoken audio. eSpeak is used here because it is simple and lightweight; the voice is a bit robotic, but it runs comfortably on the Pi. You can generate speech to confirm a command, read out what the camera detected, or later speak an AI's response. As always there is a trade-off: a more natural voice costs more resources, so a light engine is a sensible default on the edge.

Where this is heading

Combining sight from the camera with hearing and speech makes a genuinely interactive device. Right now the responses are hard-coded: fixed answers to fixed commands. In the next couple of lessons you will replace those canned replies with a local large language model, so instead of if-this-then-that, the Pi can give a real, reasoned response to what you say. Today's mic, transcription, matching, and speech are the pieces that assistant will speak through.

Working through it

Install and activate. Run the setup script to install the audio requirements, then activate this lesson's virtual environment so everything stays isolated.

Test the microphone. Confirm the mic with arecord -l, then record a few seconds and send the clip to Discord or Telegram. Receiving it proves the mic and libraries work.

Transcribe speech. Run the transcription script with the Vosk model on your recorded audio and read the text it produces, noting any small errors.

Match to a command. Add a list of known commands and pick the closest match to the transcription, so a slightly misheard phrase still maps to the right action.

Speak a response. Pass text to the eSpeak engine to generate spoken audio, play it through the speaker, and optionally send the clip to yourself.

Match a transcription to the closest known command

from difflib import get_close_matches

COMMANDS = ["turn light on", "turn light off",
            "start recording", "stop recording"]

def match_command(transcript):
    # find the known command most similar to what was heard
    matches = get_close_matches(transcript.lower(), COMMANDS, n=1, cutoff=0.5)
    return matches[0] if matches else None

# a garbled transcription still maps to the right command
print(match_command("journey light on"))  # -> "turn light on"
print(match_command("random noise"))       # -> None

Instead of requiring an exact transcription, this compares what was heard to the known commands and returns the closest one, or None if nothing is close enough. That tolerance is what makes an offline assistant feel reliable.

Speak text with eSpeak