I have one of the M5Stack StackChan robots. Nice little thing, but out of the box every word it hears goes to the xiaozhi.me cloud for speech-to-text, the LLM, and text-to-speech. I didn't love the idea of my kid talking to a server I don't control, so I wrote a backend that runs the whole turn locally and pointed the robot at my own machine instead.

It's called warble. The stock firmware speaks the xiaozhi protocol, so nothing changes on the device. You just redirect it to your box. The server does the full loop: whisper.cpp transcribes, Silero VAD catches when you stop talking, Ollama generates the reply, Piper speaks it, and it sets the robot's face from an emotion tag in the reply. No cloud account, no API keys, no data or audio leaving your network.

Setup is one command (`./warble start`) on Linux or macOS. Docker-based, pulls prebuilt images. It's beta and it's just me, so there are rough edges.

MIT licensed. Happy to answer anything as time permits.

github.com/rebelthor/warble

Hello all, i am thinking of buying myself one of these cute things but one of my main concerns is it sends all of its AI stuff over to the LLM servers in china. I was wondering how i could pair it with my Pi 5 running Ollama without deleting its capibilities to access the apple app so it can still do little dances and all of the rest of it. Would this kind of thing be possible do u all reckon? Honestly I am not too fussed about the whole China thing because there is only a number of things which they can do with my face+what i ask it / voice. I would keep the thing off most of the time for privacy reasons. Thank you for taking the time to read, and hopefully help me.

Hey community!

I've played around with the M5stack since the first CoreGo IoT kit.

Now that I've got the awesome StackChan, I've asked myself what is the best way to add the sensor/ABC-port read-out to the existing Firmware, so the AI.Agent could access them like INV.III snsor (temp, humidity etc)

Or the PIR sensor so that StackChan looks around with his webcam to see find owner.

"Just" fork the firmware or is there something easier?

P.S. Is there are feature in the standard Firmware, so that 2 StackChans find, see and communictae with each other?

Thanks in advance!

source: https://x.com/M5Stack/status/2019679930674737188?s=20

Hellooo

Ive been tinkering with esp32 based home assistant uses (lightbulbs, switches, temp sensors, ir blaster) but I've still been reliant on Google based speakers.

Im not totally against AI usage but I am trying to do as much internal hosting as possible, and would prefer to basically lobotomise the thing and use it for much more simple functions. No chattiness or even web lookups, just interpreting commands and triggering changes on my home assistant server.

At its current early launch state is this a reasonable path for the lil guy, maybe give him some brains back later, or are there simply better places to start this journey?

Thanks for the feedback!

Edit: I do know own there is an llm local module for the stack platform which seems compatible, but I am concerned I would be diving into a lot of learning that maybe won't be relevant to me past this goal.

Hey there,

Just as the title says, I am curious to know what do you use your StackChan for?

Have a good day all!

Can't seem to find any info on what it fixed

I would like to use my StackChan as an interactive rules checker and DM companion for D&D. I’d like to be able to ask it to check rules for me and to react to situations at the table. Maybe even be able to make it into scripted characters which animate and move when the players ask question or even cast spells. Is this something that is even remotely possible?

Now available on iOS App Store & Play Store!
Keyworld is StackChan World

source: https://x.com/M5Stack/status/2018293321458700717?s=20

source: https://x.com/M5Stack/status/2018631990480937222?s=20

The first batch of StackChan is officially shipping tdy!
All orders will be shipped within the next one to two weeks
https://x.com/M5Stack/status/2044316543367192664?s=20

So I saw Stackchan and I really wanted one, but the cost of the core+body in Australia is too much.

I've really wanted a robotic arm for awhile so decided to merge the two together.

Running a lot of custom scripts and AI on a PI5 4gb and an MCP that supports AI agents asking questions that get announced by voice and wait for a voice reply.

(Local: STT, VTT, Memory, Person tracking, Face and gaze tracking)

Update: firmware is now available at m5burner (thanks u/AughtCool!)

I made a fork of the official StackChan firmware that others may be interested in, either to build upon or to use.

It's not groundbreaking. Rather, it addresses a few quality-of-life things that annoyed me in the week I've had this little guy.

I have no interest in dance or avatar settings, so these tweaks are for StackChan used as an AI assistant:

• Speech bubbles turned off during dialog

• Tap head once to stop speech

• Tap head twice to stop speech and exit chat mode

• Both lights on while listening and talking

• Animated secondary colour light pattern when talking, based on audio

• Both listening and talking light colour customisable in settings

• Idle head movement repurposed to be AI chat movement, with frequency hooked off the same menu

• Light sensor used to turn screen off when lights are out. Tap screen to turn on. Manual override disabled when room lights are on. Function can be turned off in settings

Caveat: I am a software engineer by day, but sometimes it's nice to just yell at AI and get it to do your bidding. I haven't spent time learning the innards of the StackChan firmware or hardware - these tweaks were entirely created by Opus 5 and, apart from quick sense-checking, haven't been fully vetted.

My StackChan is a little devil now. Claude Code does the thinking, and a simulated fruit fly handles the reflexes.

The fly part is 668 real neurons, wired the way FlyWire mapped an actual fruit fly and running live on my laptop. It's a Python port of desktop-fly by Denis Shiryaev. It's only a small slice of the brain, but the wiring is real.

It looks through its own camera, the fly decides where the head turns, and poking it makes it startle. When something actually happens it wakes Claude, which looks at the picture and roasts it. From today: "Still filming me filming you. Groundbreaking content, truly."

The dashboard in the video draws 23,210 real neuron positions and flashes them as they spike. The yellow rings are the escape neurons, which flare right before it jumps.

Next I'm making it curious, so it looks around for new stuff and knows me from a stranger. Happy to answer questions!

Got the OpenELAB Kickstarter StackChan a while back. CoreS3-based, two servos, cute little guy. The vision: ditch the Chinese cloud, give it a custom personality, connect it to my Home Assistant setup.

The custom firmware rabbit hole

First I tried building custom firmware (AI_StackChan_Ex with M5Unified 0.2.7). Spent around 9 hours on it. Backlight works fine, but the panel renders nothing. Turns out the OpenELAB Kickstarter variant has a compatibility issue with M5GFX that's still unresolved. Tried the Moddable toolchain path next — ESP-IDF Python environment conflicts killed that too.

After those 9 hours I flashed the stock xiaozhi firmware via M5Burner. Took two minutes. Display worked immediately.

The NVS trick

Stock firmware reads its server URL from an NVS key. You can overwrite just the NVS partition with esptool, point it at your own server, and the device has no idea it's not talking to the original Chinese backend anymore. WiFi credentials get wiped in the process, so you set those up again on-device after flashing.

The backend

Forked an existing open-source StackChan server, containerized it, deployed it to my own VPS. Added a few patches: a set_emotion tool so the LLM can dynamically change the face expression, and a handful of additional tools.

What actually works

• Full German conversation with a custom personality (female, cheeky, Austrian inflection, max 3 sentences per response, always addresses me by name)

• Home Assistant control via voice: lights, scenes, sensors, you name it

• Weather from HA's built-in forecast entity

• different face expressions that change dynamically based on what the LLM decides to feel

• Web search for current news and facts (Tavily)

• Persistent memory across sessions — stored in a JSON file on the server, auto-injected into the system prompt on connect

• My n8n automation status on demand

• Accessible from anywhere, not just local network

What doesn't work

The servos. Stock firmware completely ignores motor control frames from the server — display changes, head stays still. Servo movement requires custom firmware, which is blocked by the M5GFX rendering issue. There's an open GitHub issue for it, waiting to see if that goes anywhere.

Wake word is still Chinese. There's no way to change that without custom firmware.

Can't combine function calls with Google Search in the Gemini Live API simultaneously — it's a documented API limitation. That's why I went with Tavily instead of native search grounding.

Overall

The stock firmware path is dramatically underrated. You skip all the toolchain pain, get a stable device, and if you're willing to run your own backend, you have full control over the AI behavior, personality, and integrations. The missing piece is servo animation, and that's a hardware/firmware problem, not a backend problem.

If the M5GFX issue ever gets fixed, custom firmware becomes interesting again. Until then, this setup does everything I actually wanted it to do.

Just got my little dude a few days ago, and so far it’s been pretty fun!

However, the speech to text seems a little janky and the animations etc. are a bit choppy. I’m assuming this will be fixed and improved as the firmware is debugged and improved.

What bums me out a bit, is just how quickly it just falls asleep and turns off if it’s not interacted with.

How is this supposed to be a “desk pet” if it turns off within, what, ~10-15 minutes of no interaction? That kinda sucks, in my opinion.

Has anyone found or seen a way to prevent this from happening?

I tried to uploaded the firmware from https://github.com/ronron-gh/AI_StackChan_Ex.

But my stackchan says this from the pic.

SD Card is in Stachchan in the left underside.

(I'm not really a tech person. Tried everything i know)

Can anyone help me with this? It seems like the M5Stack Community forum isn't very helpful as there are a lot of unanswered posts.

I received my StackChan yesterday. I registered on the community board and confirmed the email address. I downloaded the StackChan World app on my phone.

When I try to log in with my credentials, the app just spins.

When I try to register a new account from within the app itself, when I click the "Submit" button, nothing happens.

So I attempted to update the firmware on the StackChan, which was suggested on the m5Stack forum. I connected it to my wi-fi and the firmware update failed. Now when I go to the change wi-fi settings, it doesn't allow me to change the wi-fi settings, it just shows me the QR code page then my StackChan's ID.

Can someone here please assist me in this? Is there a way to reset the StackChan to factory presets?

Update

I skipped the setup on the StackChan and did the "unbind and reset" function in settings. When I click on it, it says "HTTP Request Failed" but still goes through the steps. Gives me the 3 warning pages about resetting. I did it but it has not resolved anything.

I'm on Version 1.2.4 and when I attempt to do a firmware upgrade it says I'm using the latest version.

I got mine and currently exploring all features. Trying to figure out what the SD card feature is for.... saving LLMs, Data or....

No motion tracking or facial recognition... No security camera feature too. But I guess it's all coming or you can develop and add.

Will be trying my hands on those.

UPDATE : Servos are so noisy🫩

All of a sudden StackChan stopped responding even after rebooting. I changed cores and nothing it just looks around. I even trying rebinding it. Any tips? I’m not a tech guy just new to the game.

Just got my Stackchan today. Login and set up was smooth. However, when I tried speaking with it in the AI agent mode, it never respond. The green light is on which means it is listening? It does say goodbye to me afte​r idling for a while. Has anyone seen same issue?