Building the AI Assistant

Now you assemble the design into a working assistant. Every module from this month, vision, voice input, the local language model, and speech output, comes together into one program that detects a person, greets them, listens, thinks, and speaks its answer. The focus is on integrating cleanly and debugging module by module so the whole loop runs reliably on the Pi.

Assembling the pieces

You already built and tested every capability separately this month. Building the assistant is mostly assembly: wiring the modules together in the order you planned. The flow runs the loop, detect a person with the camera, greet them, record or type a question, transcribe it, send it to the local LLM, get a response, and speak it aloud. Because roughly 95 percent of the code already exists from earlier lessons, this lesson is about connecting known-good parts, not writing new machinery.

Keep it modular

Spread the logic across clearly named files, something like vision.py, speech.py, assistant.py, and main.py, each with one job. main.py imports them and runs the loop, while the details stay inside each module. This is the payoff of the design lesson: with clean separation, you can fix the microphone code without touching the LLM code, and the main loop stays short and readable. Prefer plain .py scripts over notebooks here, since scripts are better for reusable, assembled programs.

Debug iteratively

The single most important habit for integration is to test each module on its own before combining them. Get the microphone capturing before you test transcription; confirm transcription before you feed the LLM. There is no point debugging the response step if the audio never arrives. Sprinkle print statements through the flow so you can see exactly where a value goes wrong or where it stops. Building bottom-up from working parts turns a scary all-at-once program into a series of small, confident steps.

Making the interaction reliable

A one-shot loop is a start, but a real assistant keeps a conversation going. Adding an input step lets the user control timing, such as how many seconds to record, and pressing enter to begin. You can wrap the exchange in a loop so the user asks many questions, apply the command matching from earlier to catch imperfect transcriptions, and route responses to Discord or Telegram as text or audio. Triggers and timeouts, only acting on a wake word or when a person stays in frame for several seconds, stop the assistant from firing constantly.

Working through it

Confirm each module works alone. Run the camera detection, the microphone capture and transcription, the LLM, and the speech output separately, and fix any that fail before combining.

Wire the main loop. In main.py, import the modules and run the sequence: detect a person, greet, record or take typed input, transcribe, ask the LLM, and speak the reply.

Add interactive timing. Use an input step so the user sets the recording length and starts it, and loop so they can ask more than one question.

Add a trigger and timeout. Activate only on a wake word or a person present for several seconds, so the assistant does not respond continuously.

Test and extend. Try different prompts, tune the timing and voice, and add a feature such as jokes, a weather reply, or sending the answer to Discord.

main.py: the assembled assistant loop

from vision import capture_camera, person_in_zone
from speech import listen_for_speech, speech_to_text, speak
from assistant import ask_llm

ROI = (200, 150, 440, 330)

while True:
    frame = capture_camera()
    if not person_in_zone(frame, ROI):
        continue                       # wait until someone is present

    speak("Hello, how can I help you today?")
    seconds = int(input("Record for how many seconds? "))  # user controls timing
    input("Press enter to start recording...")

    audio = listen_for_speech(seconds)
    question = speech_to_text(audio)
    print("You said:", question)        # debug print

    answer = ask_llm(question)
    print("Assistant:", answer)         # debug print
    speak(answer)                       # say it aloud

Each line is one module call in the planned order. The print statements let you see the transcription and the answer while debugging, and the input steps make the exchange reliable instead of a rigid one-shot recording.

Looping for a real conversation

def converse():
    speak("Hi, I'm listening. Say 'goodbye' to stop.")
    while True:
        audio = listen_for_speech(8)
        question = speech_to_text(audio)
        if "goodbye" in question.lower():
            speak("Talk to you later.")
            break
        answer = ask_llm(question)
        speak(answer)