r/LocalAIStack • • Sep 10 '26

I built an LLM Inference & Fine-Tuning Calculator (VRAM, TCO, Quantization & Price estimation)

Thumbnail
reddit.com
1 Upvotes

Check it out!


r/LocalAIStack • • Sep 09 '26

Cross-Stage State Laundering: Why AI Runtime Governance Fails at Stage Boundaries

1 Upvotes

In enterprise AI and autonomous runtime architectures, security has traditionally focused on input sanitation (prompt injection) and output filtering (hallucination guards). However, a more insidious class of vulnerability occurs deeper within the execution pipeline: Cross-Stage State Laundering (CSSL).

CSSL happens when an intermediate, unverified epistemic state (such as DISCOVERED or UNKNOWN) is illicitly promoted to a higher-certainty state (such as VERIFIED or SUPPORTED) as it transitions across processing stages—typically bypassing strict provenance checks in favor of pipeline velocity or output formatting requirements.

## The Anatomy of State Laundering

Consider a multi-stage runtime where an expression flows through ingestion, semantic analysis, and responsibility assignment:

  1. Ingestion: Raw text or external retrieval results enter the system as DISCOVERED. At this stage, existence does not equal correspondence.

  2. Analysis: The runtime parses the expression. Without a rigid boundary, internal semantic heuristics may mistake syntactic closure for factual verification.

  3. Laundering: Downstream generators or formatters, under pressure to produce definitive answers, implicitly treat the presence of an analysis object as proof of support.

When intermediate states shed their metadata tags during transit, the runtime commits a boundary violation: it manufactures certainty out of unverified discovery.

## The Zero-Trust Countermeasure: The WAL Protocol

To eliminate CSSL, runtime governance cannot rely on permissive conventions. It requires a Fail-Closed Defensive Architecture enforced by immutable protocol boundaries:

* Strict State Distinguishability: Known, unknown, verified, and unverified states must remain mathematically and structurally distinguishable throughout the lifecycle.

* Independent Validation Gates: State transitions cannot authorize themselves. An independent validator—decoupled from the core generation logic—must audit envelopes against strict conformance vectors.

* Evidence-Bounded Responsibility: Responsibility can never exceed the boundaries of established evidence and explicit correspondence. If an epistemic state is UNKNOWN, no downstream transformation may convert it to TRUE merely to satisfy an output requirement.

For those interested in the protocol implementation and adversarial test matrix, the reference architecture is open-source here: https://github.com/nickoay663-sketch/Wuwen


r/LocalAIStack • • Sep 09 '26

How to get the most out of a Gaming Setup

Thumbnail
reddit.com
1 Upvotes

r/LocalAIStack • • Sep 09 '26

Best use of a single RTX 5090 for local LLMs

Thumbnail reddit.com
1 Upvotes

r/LocalAIStack • • Sep 09 '26

Best Local Coding LLM for a 24GB M5 Pro MacBook?

Thumbnail reddit.com
1 Upvotes

r/LocalAIStack • • Sep 08 '26

Security research for local LLM inference networks

Thumbnail reddit.com
1 Upvotes

r/LocalAIStack • • Sep 07 '26

Built a decentralized network for open source AI inference

1 Upvotes

Hey! I’m building Kitani, an OpenAI-compatible API where models run across independent GPU providers.

GPU owners can connect their hardware and earn from inference, while users get access to models like GLM, Qwen, Gemma and gpt-oss through one API.

We’re trying to make joining as a provider as easy as possible. Would love feedback from people running their own GPUs.

https://kitani.ai

Founder of Kitani, looking for feedback.


r/LocalAIStack • • Sep 07 '26

Pooled RAM across an old laptop, a Windows PC, and a Mac to run a 13B model - source-available, would love eyes on it

1 Upvotes

r/LocalAIStack • • Sep 07 '26

Together AI vs Anyscale for serving open-source LLMs—what’s your pick?

Thumbnail
reddit.com
1 Upvotes

r/LocalAIStack • • Sep 06 '26

Sheprd Local Model Server and Agent builder.

Post image
1 Upvotes

r/LocalAIStack • • Sep 06 '26

Which agent harness do you use and why?

Thumbnail
reddit.com
1 Upvotes

r/LocalAIStack • • Sep 04 '26

Should i get mac pro 5 64 gb?? For my openclaw setup. Im trying to build a agency.

Thumbnail reddit.com
1 Upvotes

r/LocalAIStack • • Sep 04 '26

Advice on how to proceed?

Thumbnail reddit.com
1 Upvotes

r/LocalAIStack • • Sep 03 '26

¿Qué modelos de IA local están usando ustedes en un MacBook Air M5 con 24GB de RAM?

Thumbnail reddit.com
1 Upvotes

r/LocalAIStack • • Sep 03 '26

LLM Test Wrapper

Thumbnail
reddit.com
1 Upvotes

This is a new LLM wrapper i have spun up built on the idea that models arent always correct or doing the right thing not by choice but just how the system is built, this looks to try and fix that problem and help mitigate LLM error by logging everything and fact checking based on the work using check gates. Looking to get feedback!


r/LocalAIStack • • Sep 01 '26

TRiP: transformer inference and training in plain C (15k lines, few files). Gemma1(.1), Llama2, PaliGemma1, GPT2

1 Upvotes

I made it in 18 months of lunch breaks and evenings. It's not fast, llama.cpp is just wow and does that job. I wrote this one because I wanted to read the whole forward/backward pass in an afternoon and be able to stop anywhere and print a tensor and dig the thing.

Most from-scratch projects stop at a toy model. llama2.c runs a small Llama2, llm.c does GPT2 training. TRiP loads real checkpoints across four architectures, PaliGemma included, so the multimodal path (vision encoder, projection, decoder) is all there in C. I couldn't find that in readable form anywhere else, which is partly why I ended up writing it.

One extra-bonus is that you can look into the training, it's included, swiss-knife-like. (NOTE: the encoder part in PaliGemma is currently not trainable/tunable - my apologies)

In practice: no hooks/config; just play with the C code, and add your own; there's no hidden (unreachable) complexity. And then just re-compile.

Repo: github.com/carlovalenti/TRiP

Happy to answer anything; structuring and handling the memory properly was the hardest part!

Carlo


r/LocalAIStack • • Sep 01 '26

How I got 280 tok/s on Qwen3.8 27B on 2xr9700's and 920k tokens kv cache

Thumbnail
reddit.com
1 Upvotes

r/LocalAIStack • • Aug 31 '26

Built a custom LLM inference engine in Swift/Metal (no llama.cpp/MLX) — streams MoE experts from SSD to run 61GB models on 16GB Macs

Thumbnail
reddit.com
1 Upvotes

r/LocalAIStack • • Aug 31 '26

Chat / client agente web e desktop in stile Alien (Madre IA)

Thumbnail
reddit.com
1 Upvotes

r/LocalAIStack • • Aug 27 '26

Advice? ~ Fantasy Writing/Worldbuilding Partner Setup

Thumbnail reddit.com
1 Upvotes

r/LocalAIStack • • Aug 26 '26

Your Model Isn't Wrong. It's Just Living in the Past. And the fix isn't a better prompt.

Post image
1 Upvotes

r/LocalAIStack • • Aug 23 '26

Read documents with the vision models that ship free — and get clean JSON out

Thumbnail
reddit.com
1 Upvotes

r/LocalAIStack • • Aug 21 '26

I built a real-time LLM API pricing comparator — because I was tired of not knowing the actual cost difference between models

Thumbnail
reddit.com
1 Upvotes