Capstone Project: The AI Assistant Showcase

This is your Month 3 capstone: complete and personalize your own offline AI assistant that sees, hears, thinks, and speaks, using everything you built this month. Use this as your specification and checklist. When it works, record a short video to show it off. Keep the Pi assembled, because next month you connect it to a smart home.

What you have built toward

Over this month you assembled a genuinely capable system: real-time object and person detection with the Pi camera, a speech interface that recognizes your voice and speaks back, a local large language model that understands and responds, and a modular codebase that ties it together. The capstone brings these into one finished assistant that is entirely yours to shape. It runs offline, which was barely practical on hardware this small only a few years ago.

How your project is judged

A complete capstone does five things. It detects a person (or a chosen object) with the Pi camera. It greets or reacts using text-to-speech. It accepts input by voice or by text. It generates a response with the local LLM. And it delivers that response by speaking it aloud or showing it on screen. Meeting these five is a complete project; personality, extra features, and polish are where it becomes memorable. Keep the code modular so each requirement maps to a clear part.

Working through it

Requirement 1: Detection. Use the Pi camera to detect a person, or a specific object if you prefer, as the trigger for the assistant.

Requirement 2: Greeting. When the trigger fires, respond with text-to-speech. This is a natural place to give your assistant its personality.

Requirement 3: Input. Accept the user's request by voice (transcribed) or by typed text, whichever you prefer.

Requirement 4: Response. Send the request to your local LLM and get a reasoned answer, all offline.

Requirement 5: Output. Speak the response aloud or display it on the screen. Then record a one-to-two-minute casual video of the whole flow working and share it.

Capstone checklist mapped to your modules

# core requirements -> which module handles it
# 1. detect person/object      -> vision module (Pi camera)
# 2. greet with speech          -> output module (TTS + personality)
# 3. accept voice or text input -> audio input module
# 4. generate a response        -> language module (local LLM)
# 5. speak or display response   -> output module

# bonus ideas (pick at least one):
#  - jokes or fun facts on request
#  - control a light or send an alert
#  - recognize a custom object (your Lesson 7 model)
#  - a distinct personality in the prompt and voice

Each requirement maps to one module you already built, so completing the capstone is assembling and personalizing, not starting over. Add at least one bonus to make it your own.

Common mistakes and troubleshooting

Trying to add polish before the core loop works. Get the five requirements running end to end first, then add personality and bonus features.

A monolithic script that is hard to fix. Keep the modules separate so each requirement is easy to test and adjust.

Running out of memory with a large model. Use a smaller model and close other apps; the Pi has limited RAM.

Packing away the Pi when done. Keep it assembled; you use the same Pi for the smart-home month next.

Next: Next month you take these skills into the real world with smart homes, sensors, and Zigbee automation.