Spy Glasses and AI: On-Device Text Recognition
Now your glasses learn to read. Using the openglass project, your XIAO ESP32-S3 will capture live camera data and run on-board OCR to pull text out of what it sees, displaying it in the serial monitor. It is a low-power, real-time text reader that needs no cloud. As with any wearable camera, use it only on yourself and your own materials, with consent from anyone in view.
What OCR is
OCR stands for optical character recognition: turning an image of text, a book page, a sign, a screen, into actual characters a program can use. The openglass project runs OCR on the ESP32 itself, so the camera captures a frame, the board extracts any text it finds, and it prints that text out over the serial connection. This is genuinely useful on its own as a reading aid, and it is the visual front end that later AI steps can build on.
Running the project as-is
Today you are not adding new AI, just getting the existing project working. The openglass repository already includes OCR and hooks for later AI steps; you download it, open the sketch in the editor, select the XIAO ESP32-S3 board and the correct port, install any libraries the repo lists, and upload. The point is to get a known-good project running on your device before you extend it, which is a sound habit with any codebase.
The version gotcha, again
Just like the firmware last lesson, compile errors here usually trace to the ESP32 board package version. Newer versions (around 2.0.11 and above) restructured files and move key components, so many people find an older version such as 2.0.7 or 2.0.9 works best. You change it in the board manager by selecting the older version and installing. If you hit missing-library or compile errors, this is the first thing to check.
Reading the results and what affects them
Once it runs, open the serial monitor and hold a text source, a book, a sign, your phone screen, in front of the glasses, and the extracted text appears live. It is a low-power, real-time reader with no cloud or extra hardware. Accuracy depends on conditions: good lighting, a sensible distance, and a clean, printed font read far better than a dim, distant, or handwritten source. Testing across those conditions teaches you the real-world limits of on-device vision.
Working through it
Get the project and select the board. Download the openglass repository, unzip it, open the sketch in the editor, and set the board to XIAO ESP32-S3 with the correct port.
Set a compatible board version. In the board manager, select an older ESP32 version such as 2.0.7 or 2.0.9 and install it, since newer versions may not compile.
Install libraries and upload. Install any libraries the repo's readme lists, then upload and wait for it to flash without errors.
Read live text. Open the serial monitor at the correct baud rate and hold printed or on-screen text in front of the camera to see the recognized text appear.
Test across conditions. Try a book, a screen, and a handwritten note, noting how lighting, distance, and font change the accuracy.
Downgrade the ESP32 board package
# In the editor: Tools > Board > Boards Manager
# Search: esp32 (by Espressif Systems)
# Click the version dropdown next to "Installed"
# Select an older, compatible version: 2.0.7 or 2.0.9
# Click Install / downgrade, then Upload the openglass sketch
# Then: Tools > Serial Monitor (set the baud rate from the repo)
# hold text in front of the camera to see OCR output liveNewer ESP32 packages restructured files and often break this older project, so downgrading to 2.0.7 or 2.0.9 is the reliable fix. After uploading, the serial monitor shows the recognized text.
Common mistakes and troubleshooting
Compile errors or missing libraries. Downgrade the ESP32 board package to 2.0.7 or 2.0.9 and install every library the repo readme lists.
The serial monitor shows nothing readable. Set the serial monitor to the correct baud rate and make sure the camera is positioned and unobstructed.
OCR output is garbled. Improve lighting, adjust distance, and use clear printed text; poor conditions and handwriting reduce accuracy.
Re-uploading over unresolved errors. Fix compile errors first (usually the board version) before flashing again.
Next: Next you will revisit the ethics of these tools before moving into hands-on security devices.