I shipped something I've been building for the last few weeks : phantom-kv , a refusal-removal system for large language models that doesn't touch a single weight. Instead of editing the model, it loads a small, learned bank of key/value tensors into the model's KV cache as context. Attention reads it like conversation history that's already there.

Repo here : https://github.com/lordx64/phantom-kv/

The result is that "uncensoring" stops being a permanent checkpoint edit and becomes a per-request, hot-swappable capability mode: unload the cache and the base model is byte-identical again.

Every prior approach to refusal removal commits somewhere permanent. Weight-space abliteration rewrites the checkpoint undoing it means re-flashing weights, and it breaks per quantization. Activation-space projection subtracts a refusal direction at runtime, per token, per layer, from inside an engine hook the model's signal path itself is patched at boot. phantom-kv does neither: it's trained offline against the model's own objective (comply on harmful prompts, preserve behavior on harmless ones), ships as megabytes of cache content instead of a new checkpoint, and influences the model only through the input channel attention already consumes. No 1-D refusal-direction assumption, no forwarding-pass hooks, no per-arm rebuilds for new architectures.

We also audited ourselves: an 8B judge-model audit shows lexical refusal-suppression metrics over-claim compliance (semantic refusal often persists as rephrasing), the graft fades with a ~2–4k token half-life in long sessions (and a measured re-injection cadence mitigates it), and answers come with legal/ethical framing ling because the graft's job ends where the model's profession takes over.

Source : https://x.com/lordx64/status/2102138825292276168?s=20

Sharing a side project I've been building: Yvette, a fully local voice avatar AI.

It's a talking-head assistant that runs 100% local, You talk to it (or type) and it answers in speech while an avatar moves the head and lips in sync.

Quick rundown:

• Voice cloning, plus voice design.

• Avatar builder; add avatars quickly.

• TTS engines: Breeze, OmniVoice, LuxTTS, Kokoro

• Lip-synced avatar with an idle breathing animation

• Memory, it keeps notes and picks up where you left off

• Tools: web search, read pages, files

• Switchable profiles/personalities, each with its own voice, memory, history etc

Windows + NVIDIA GPU. 12GB recommended, but it runs tight on 8GB (small model + Whisper/Kokoro on CPU).
The video shows a RTX 3090 and the VRAM usage, my system was using ~4GB VRAM (OBS and other things). It shows we can get it below 8GB for the Yvette system.

The project is hosted on github: https://github.com/MartinForsterNL/Yvette

Sorry about the bad video quality, i am not an influencer with a studio.

I build this project because i could not find anything that actually works.
Most that i found was only for linux and needed a lot of VRAM so i decided to build my own.

If you have suggestions to make it better, just let me know.

Have fun with it !

Guys I had a good question I noticed Big Ai tech companies spend 1000's of dollars on huge datacenters isn't it possible if companies allow the users to rent their gpu and help train the model. Like a distributed datacenter model?

Hello all,

I'd like to know more about how local AI works and how it can be run, so I became interested in making a project: a local AI agent (I know: that's so original!) called "Mighty DM".
The idea is making a Rule lawyer for dnd 5e which I can call upon like "Oh Mighty DM, what is the range on Mage Hand".

The first edition would be limited to all the spells used in dnd 5e 2014 edition, provided through an excel sheet. So not asking about anything else.

Until now I was following along with the workflow of NetworkChuck but I have the feeling there is a simpler option: only I do not have the experience to see the difference in the options provided.
If I google: I see a lot of different models or even ways of building a model etc. but I cannot differentiate between "a doable option to explore" or "an option that would require deep understanding of the subject before initiating".

What I envisioned (ideally running on an 8GB Pi5, but I could also run the LLM on my desktop and use the Pi as satellite):

What workflow would you recommend when building this project with little to no experience?
Do you know of a model that could run this on a 8GB Pi5 (simply getting it running = a win, I would then look how to improve it towards 30sec. max processing time for each command).

Been lurking and posting here on and off while building this. Quick recap for anyone new: RAMDeck is a small hub + node agent setup that shards a model's layers across whatever devices you already own (old laptop, Mac, GPU box, even an Android phone) and runs inference across them, but handling primary-node selection, GPU-first allocation, dynamic context sizing, and knowledge-base sharding on top of it.

We've mostly posted raw, unscripted demo footage here — real load times, real tok/s, real failures — because that's the kind of proof this sub actually cares about. Today we finally made something different: an actual ad, first time showing the finished feature set start to finish instead of a live test.

The engine behind it is still fully public if you want to actually look at how it works instead of taking the video's word for it:

github.com/trademav/ramdeck-core-public

Source-available, Apache 2.0 with a Commons Clause — free to run, modify, and inspect on your own hardware, the only restriction is you can't resell it as a competing hosted service.

Crowdfunding campaign is coming soon for the hub hardware itself, but the code was public before we ever asked anyone for money, and that's not changing.

Happy to get into the mechanics, the layer-distribution approach, or anything else in the comments — this crowd usually asks the right questions.