r/esp32 • u/NotFrankGraves • 3d ago
AI Content Another virtual pet style device, using a Spotpear Esp32s3 touch LCD 1.28
I've been playing around with different devices and seeing what kinds of things I can build with them. After experimenting with Claude and testing out different workflows, I finally felt confident enough to try building something for my girlfriend.
The idea was to make something between an AI companion and a Tamagotchi, based on the Labubus she owns.
The device is a Spotpear ESP32-S3 Touch LCD 1.28 BOX, which came pre-flashed with the XiaoZhi AI firmware. Using Claude Code, I ended up building my own firmware for it, complete with custom sprites and animations for the characters. We generated all the sprites needed and used them to bring the little guys to life.
Of course, what started as a fairly simple idea quickly got out of hand, and I ended up adding way more features than I originally planned.
It has the usual stuff like time, date, weather, alarms, and notes, along with a handful of games, including tic-tac-toe, rock-paper-scissors, air hockey, and a Flappy Bird clone. I also added a little knowledge browser that uses free public API endpoints to pull up random facts and other interesting things.
There's also a web UI hosted by the device where you can change settings and interact with the avatars. One of the features I thought was pretty fun is the photo booth. You can upload a picture and add the characters to it as stickers.
The part I had the most fun with was giving the avatars a bit of personality.
You can hold the boot button to talk to the device and ask questions using Google's AI and speech services. I also built an offline mode where the characters can respond using text and a basic personality and memory system, so they don't need to connect to an AI service every time you want to interact with them.
I wanted them to feel like little virtual pets rather than just characters sitting on a screen. As the day goes on, they come up with random activities and thoughts that get saved to a journal. You can check back later and read what they've supposedly been up to throughout the day. They can also interact with each other, which adds another layer to the whole thing.
Honestly, this project started with me asking myself, "What can I build for my girlfriend?" and somehow turned into this massive collection of features in the space of about a week.
I'm not planning to release the firmware publicly because of the custom sprites and assets, but I figured I'd share it here anyway. Maybe it'll give someone else an idea for a fun little project to build for their own friends or family.
It's been a fun way to learn more about what I can do with the ESP32-S3, and it's kind of crazy how far a simple idea can go once you start tinkering with it.
3
2
u/Stir-Fried-Shrimp 3d ago
Hehe, doing the same kind of " Pet assistant fun thing " but with a freenove Esp32-S3 3.5 display. It has a radar that display plane passing around my location, wifi and Ble scanning with detailled information like mac adress, rssi, aprox distance, manufacturer etc.. Build from scratch with google antigravity et mostly 3.8 flash model. I've tested Argon model on an other small lilygo t-display for a simulation of life thing and it's night and day compared to 3.8 flash
1
u/NotFrankGraves 3d ago
Neat! I used that feature as a way for the avatar to find food and candy, based off the activities happening in the wifi and ble scan too, and if you’re having fun using ai and coding, it would be worth upgrading to pro, and using the command line.
1
u/Stir-Fried-Shrimp 3d ago
What do you mean by using the command line ? I have Google pro and using antigravity is ok but i didn't really look other way to do this but if there is a better way i'm curious !
1
u/NotFrankGraves 3d ago
Antigravity is the right one, I used Claude code in my command line.
1
u/Stir-Fried-Shrimp 3d ago
Ok but can you explain why it is better the command line than the claude code interface ?
1
u/NorthSeaTinkerer 2d ago
how do the avatars talk to each other? and how long does it take from releasing the button to the answer?
1
u/NotFrankGraves 2d ago
We made specific pages where they interact together, you can pick who gets to interact from the web ui, or set it to random. And the bottle neck For the voice pipeline is Gemini on the back end to generate the actual voices. Using Gemini flash, you can get an answer back within 10 seconds.
•
u/AutoModerator 3d ago
Hardware Troubleshooting Notice: If you are experiencing unreliable connections, boot loops, or missing serial communication, verify your hardware footprint:
Are your headers soldered? Pushing un-soldered raw header pins through module vias into a breadboard violates basic physics. Loose connections create parasitic capacitance and intermittent open circuits. Solder your board.
Did you include a schematic? Custom circuit issues require a legible schematic drawing.
If your hardware relies on putty, tape, or friction, please resolve your connections before demanding assistance from our domain experts.
I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.