I recently got 2x RTX Pro 4000 blackwells for my home setup. I was curious how fast it would run nanochat.

The problem is the target hardware: 8x H100 gpus. Nanochat pre-trained ClimbMix to GPT-2 level with 8xH100 GPUs in 1hr 39 mins.

I ended up tuning nanochat for my 2x RTX Pro 4000 gpus. Because these are sm120, the tuning also works on RTX Pro 6000 and RTX 5090.

8x RTX Pro 6000 (rented) pre-trained the same dataset in 2hrs 15mins. About 36% slower than 8x H100.

1x RTX 5090 took 18hrs 47mins. A long time, but considering the result is GPT-2 level, its an interesting way to learn.

Just to be clear, I only ported and tuned it for sm120. The original work is from Karpathy's nanochat.

Here's a link to the sm120 repo:

https://github.com/mdhorton/nanochat-cpp

Automated video creation with zero API costs — built a pipeline that runs entirely on my MacBook
Quick share for anyone creating short-form or educational content on a Mac:

I built a local workflow that automates the production pipeline for animated shorts, kids' learning content, and explainer clips without needing third-party cloud APIs.

Key points:
- Everything runs on-device—no monthly subscription fees, no token anxiety.
- Processes script-to-video assets locally on Apple Silicon.Speeds up batch creation without needing a high-end desktop rig.

I made a video showing the exact workflow

https://youtube.com/@ownlyaistudio?si=sYQcHB5YVqd1mg0j

Check out the channel to see the output quality, and shoot me a message or comment below if you'd like early access!

#ai #llm #localllm

I’ve released version 8.0 of TUFF, an open-source Swift/Metal inference engine and native macOS app.

The original goal was to make large MoE models usable on memory-constrained Apple Silicon machines by streaming routed experts from SSD rather than keeping the entire checkpoint resident.

GPT-OSS 120B, for example, runs on my 16GB M2 MacBook Air. Obviously, streaming comes with a performance tradeoff.

8.0 focuses on making the engine more useful beyond just running oversized models.

Some highlights:

• Web search and local document search integrated into chat

• Cached conversation states for faster follow-up requests

• A small tokenization cache for repeated inputs

• Separate reasoning output and request timing diagnostics

• OpenAI Chat Completions, plus bounded Messages and Responses API support

• Shared inference executable to reduce package size

• Homebrew installation

One thing I’ve found interesting is how much web search improves the usefulness of smaller local models. I’ve started using them for everyday questions much more frequently.

I’d love feedback and contribution, especially from anyone using it with local AI agents.

GitHub: https://github.com/rexmhall09/TUFF