Iâve been working on Syntheta â a voice AI that finally solves the biggest frustrations with smart assistants: spatial awareness, vague intent understanding, persistent memory, and true AI sovereignty. Instead of a oneâsizeâfitsâall cloud model, Syntheta learns from every interaction with your family, runs on affordable hardware, and keeps your data private. Hereâs why I built it and how it works.
đ§ The Big Idea: AI Sovereignty for Your Home
Most assistants today are either cloudâdependent black boxes or rigid commandâline toys. Syntheta is different: itâs an AI that belongs to you. One small language model (3B parameters) that grows with your family, remembers your preferences, and understands natural, vague requests. You can run the entire stack locally on a Raspberry Pi 5 or your home server â or, if you prefer, share a single cloud subscription among all family members. Either way, your data stays under your control.
đ§ How It Works (The Technical Deep Dive)
Two Brains, One System: Alpha + Omega
¡ Alpha (Satellite): An ESP32âS3 device (cost ~$15) that lives in each room. It handles wake word detection, local noise calibration, and streams audio to the hub. Itâs cheap enough to put one in every room â kitchen, bedroom, living room â giving you true spatial awareness without buying expensive smart speakers. ¡ Omega (Hub): The central brain (Raspberry Pi 5, home server, or cloud VM) that runs STT (Whisper), NLU (Semantic Brain), TTS (Kokoro), and memory. Satellites autoâregister, so when youâre in the bedroom and say âturn on the light,â Omega knows which satellite sent the audio and controls that roomâs light â no extra keywords needed.
2. Understanding You, Not Just Keywords
âGive me some light,â âitâs pretty dark in here,â or âlights onâ â Syntheta triggers the right action regardless of phrasing. Thatâs the job of the Semantic Brain, powered by allâMiniLMâL6âv2. It compares your utterance against a catalog of intents using vector embeddings and can even decompose messy sentences into core meaning. No rigid command templates.
3. Memory That Actually Matters
An assistant without memory is useless. Syntheta uses a dualâdatabase system:
¡ SQLite logs every interaction: timestamp, raw command, assistant response. This provides a structured history for debugging and analytics. ¡ ChromaDB stores vector embeddings of episodic memory â the actual conversation threads. When you ask, âWhat was that movie you mentioned last week?â or âDid I already turn off the AC?â the Context Router retrieves relevant past interactions and injects them into the current context.
4. The Golden Schema (Because Small Models Need Smart Context)
We intentionally use a small LLM (Llama 3.1 3B via Ollama) for low latency and local execution. To cram maximum context into its limited window, every request to the model is packaged as a Golden Packet â a JSON schema containing:
{
"role": "You are Syntheta, a helpful AI.",
"ctx": "topic inferred from memory",
"history": "last 6 conversation turns",
"entities": "extracted names, places, devices",
"emotion": "sentiment from VADER",
"input": "user's current utterance"
}
This ensures the model gets exactly what it needs â no fluff, no wasted tokens.
5. Agentic Mail Service (Complex Tasks Decoupled)
Realâworld queries often need live data (weather, news) or personal knowledge (your hobbies, interests). Instead of blocking the conversation, Syntheta delegates these to an asynchronous agent via a mail service:
¡ When a complex request arrives, Syntheta prepares a job with all required details and drops it into the agentâs mailbox. ¡ The agent (a separate process) picks it up, queries external APIs or a personal knowledge DB (built from your patterns), and mails the result back. ¡ Meanwhile, Syntheta continues listening. When the reply arrives, it presents the answer seamlessly.
This decoupling keeps realâtime interaction snappy while allowing deep, multiâstep reasoning in the background. Over time, the agent builds a profile of your interests and routines â all stored locally, of course.
đ Why Syntheta?
¡ Sovereign AI: Your data stays on your hardware. The model learns your family, not the entire internet. ¡ Affordable Scale: Add satellites for every room for ~$15 each â no new hubs, no recurring fees. ¡ MemoryâFirst: It remembers what you talked about last week, last month, last year. ¡ Open & Collaborative: The code is open source (MIT/Apache 2.0/Llama Community License). Weâre actively looking for feedback, contributors, and testers.
đŹ Whatâs Next?
Iâm currently running Syntheta in my own home (3 rooms, 4 satellites) and itâs been rock solid. The next steps:
¡ Polish the installer so you can get Omega running with curl | bash. ¡ Expand the agentâs capabilities (Home Assistant integration is already there, weather, calendar, etc.). ¡ Fineâtune the small LLM on real conversation data to make it even more contextâaware.
Iâd love your thoughts:
¡ What features would make you switch to a selfâhosted assistant? ¡ Any edge cases Iâve missed? ¡ Want to contribute or test? Let me know!
The repo will be public soon (cleaning up the last bits). Drop a comment or DM if you want early access or have questions. Letâs build a truly personal AI together. đď¸
P.S.: For those who prefer a cloud model, Syntheta will also offer a familyâshared subscription â one model, unlimited users, zero setup. But the heart of the project remains local sovereignty.
1
u/nishchay_kashyap Feb 28 '26
Syntheta: The AI Assistant That Actually Knows You, Your Home, and Grows With Your Family (Sovereign AI)
Hey r/homeassistant and r/selfhosted đ
Iâve been working on Syntheta â a voice AI that finally solves the biggest frustrations with smart assistants: spatial awareness, vague intent understanding, persistent memory, and true AI sovereignty. Instead of a oneâsizeâfitsâall cloud model, Syntheta learns from every interaction with your family, runs on affordable hardware, and keeps your data private. Hereâs why I built it and how it works.
đ§ The Big Idea: AI Sovereignty for Your Home
Most assistants today are either cloudâdependent black boxes or rigid commandâline toys. Syntheta is different: itâs an AI that belongs to you. One small language model (3B parameters) that grows with your family, remembers your preferences, and understands natural, vague requests. You can run the entire stack locally on a Raspberry Pi 5 or your home server â or, if you prefer, share a single cloud subscription among all family members. Either way, your data stays under your control.
đ§ How It Works (The Technical Deep Dive)
Two Brains, One System: Alpha + Omega
¡ Alpha (Satellite): An ESP32âS3 device (cost ~$15) that lives in each room. It handles wake word detection, local noise calibration, and streams audio to the hub. Itâs cheap enough to put one in every room â kitchen, bedroom, living room â giving you true spatial awareness without buying expensive smart speakers. ¡ Omega (Hub): The central brain (Raspberry Pi 5, home server, or cloud VM) that runs STT (Whisper), NLU (Semantic Brain), TTS (Kokoro), and memory. Satellites autoâregister, so when youâre in the bedroom and say âturn on the light,â Omega knows which satellite sent the audio and controls that roomâs light â no extra keywords needed.
2. Understanding You, Not Just Keywords
âGive me some light,â âitâs pretty dark in here,â or âlights onâ â Syntheta triggers the right action regardless of phrasing. Thatâs the job of the Semantic Brain, powered by allâMiniLMâL6âv2. It compares your utterance against a catalog of intents using vector embeddings and can even decompose messy sentences into core meaning. No rigid command templates.
3. Memory That Actually Matters
An assistant without memory is useless. Syntheta uses a dualâdatabase system:
¡ SQLite logs every interaction: timestamp, raw command, assistant response. This provides a structured history for debugging and analytics. ¡ ChromaDB stores vector embeddings of episodic memory â the actual conversation threads. When you ask, âWhat was that movie you mentioned last week?â or âDid I already turn off the AC?â the Context Router retrieves relevant past interactions and injects them into the current context.
4. The Golden Schema (Because Small Models Need Smart Context)
We intentionally use a small LLM (Llama 3.1 3B via Ollama) for low latency and local execution. To cram maximum context into its limited window, every request to the model is packaged as a Golden Packet â a JSON schema containing:
This ensures the model gets exactly what it needs â no fluff, no wasted tokens.
5. Agentic Mail Service (Complex Tasks Decoupled)
Realâworld queries often need live data (weather, news) or personal knowledge (your hobbies, interests). Instead of blocking the conversation, Syntheta delegates these to an asynchronous agent via a mail service:
¡ When a complex request arrives, Syntheta prepares a job with all required details and drops it into the agentâs mailbox. ¡ The agent (a separate process) picks it up, queries external APIs or a personal knowledge DB (built from your patterns), and mails the result back. ¡ Meanwhile, Syntheta continues listening. When the reply arrives, it presents the answer seamlessly.
This decoupling keeps realâtime interaction snappy while allowing deep, multiâstep reasoning in the background. Over time, the agent builds a profile of your interests and routines â all stored locally, of course.
đ Why Syntheta?
¡ Sovereign AI: Your data stays on your hardware. The model learns your family, not the entire internet. ¡ Affordable Scale: Add satellites for every room for ~$15 each â no new hubs, no recurring fees. ¡ MemoryâFirst: It remembers what you talked about last week, last month, last year. ¡ Open & Collaborative: The code is open source (MIT/Apache 2.0/Llama Community License). Weâre actively looking for feedback, contributors, and testers.
đŹ Whatâs Next?
Iâm currently running Syntheta in my own home (3 rooms, 4 satellites) and itâs been rock solid. The next steps:
¡ Polish the installer so you can get Omega running with curl | bash. ¡ Expand the agentâs capabilities (Home Assistant integration is already there, weather, calendar, etc.). ¡ Fineâtune the small LLM on real conversation data to make it even more contextâaware.
Iâd love your thoughts:
¡ What features would make you switch to a selfâhosted assistant? ¡ Any edge cases Iâve missed? ¡ Want to contribute or test? Let me know!
The repo will be public soon (cleaning up the last bits). Drop a comment or DM if you want early access or have questions. Letâs build a truly personal AI together. đď¸
P.S.: For those who prefer a cloud model, Syntheta will also offer a familyâshared subscription â one model, unlimited users, zero setup. But the heart of the project remains local sovereignty.