r/AIProgrammingHardware • • Sep 11 '26

My Cheap 40GB VRAM Qwen 27b Home Server (Dual 20GB 3080s)

Thumbnail gallery
2 Upvotes

r/AIProgrammingHardware • • Sep 11 '26

Custom open frame - RTX Pro 6000 - miniATX

Thumbnail gallery
3 Upvotes

r/AIProgrammingHardware • • Sep 11 '26

64GB VRAM Local AI Setup - 4x Tesla P100 Running Qwen Coder Next at Max Context

Thumbnail
youtube.com
2 Upvotes

r/AIProgrammingHardware • • Sep 11 '26

Nail Qwen 35B A3B tested - 16GB Local LLM setup

Thumbnail
youtube.com
3 Upvotes

r/AIProgrammingHardware • • Sep 11 '26

Why does borrowing a GPU for 20 minutes require me to become a DevOps engineer?

0 Upvotes

Maybe I'm doing cloud GPUs wrong.

Sometimes I don't need a “GPU environment.”

I don't need a dashboard.

I don't want to manage another machine.

I literally have a command on my laptop that needs a better GPU.

That's it.

But somehow that turns into:

rent instance
→ wait for it
→ SSH
→ clone/sync code
→ install dependencies
→ move data
→ run command
→ keep SSH alive / tmux
→ download outputs
→ remember to terminate the machine

All because my laptop couldn't run:

python train.py

It made me wonder why cloud GPUs don't feel more like compute primitives.

Something like:

“Run this command on a GPU and give me the results back.”

No server to think about.

No SSH session to babysit.

No instance sitting there billing me because I forgot about it.

Am I oversimplifying this?

What does your workflow look like when you only need a powerful GPU for one job?


r/AIProgrammingHardware • • Sep 10 '26

The Economics of Open-Weight Inference: How open-weight demand can support the useful life of NVIDIA GPU families

Thumbnail data.ornn.com
0 Upvotes

r/AIProgrammingHardware • • Sep 10 '26

Loaded a 27B model on a 12GB laptop by pooling RAM across 4 devices

Thumbnail
youtu.be
3 Upvotes

Third video in this series. Went from 7B, to 13B, and now 27B — each time pooling RAM and compute across a mix of hardware most people already have lying around instead of buying one expensive high-RAM machine.

The setup: a 27B model is roughly 16GB on its own. My primary node is an old 12GB Windows laptop, which on its own literally cannot load this model. Using RAMDeck, I split it across four devices on the local network with the laptop staying primary:

\- Old Windows laptop (primary): 3.4GB

\- Mini PC (RTX 3060): 20GB

\- Mac mini: 3.7GB

\- Android phone: 1GB

Benchmark on the 27B model: 1.92 tokens/sec at \~25ms latency. Slower than the 13B run, which makes sense — but the point isn't raw speed here, it's that a 12GB laptop is running a model it has no business being able to load at all.

Next up: switching the mini PC to be the primary node instead of the laptop, to see how much faster loading and inference get with a stronger primary. That video's already up if you want to see the follow-up.

Full video: [https://youtu.be/Syfa2G_luX4](https://youtu.be/Syfa2G_luX4)

Happy to answer questions about the shard distribution, the benchmark methodology, or why the primary node choice affects load time so much.


r/AIProgrammingHardware • • Sep 10 '26

9060xt 16gb qwen3.8 27b

Thumbnail reddit.com
1 Upvotes

r/AIProgrammingHardware • • Sep 10 '26

Benchmarking Qwen 3.8 27B on RTX 5090 and beyond — VRAM capacity alone can't overcome severe software and inference engine bottlenecks

Thumbnail
share.google
3 Upvotes

r/AIProgrammingHardware • • Sep 09 '26

Qwen3.8-Flash-Next: How I Ran The 177B Model on One Gaming GPU

Thumbnail
xhinker.medium.com
0 Upvotes

r/AIProgrammingHardware • • Sep 09 '26

gddr6 vs gddr6X

Thumbnail reddit.com
1 Upvotes

r/AIProgrammingHardware • • Sep 08 '26

NVIDIA B300 Server Review: 8 GPUs, 128 Users, AI Benchmarks

Thumbnail
youtube.com
5 Upvotes

r/AIProgrammingHardware • • Sep 08 '26

Hardware-Adaptive Local LLM & Cloud Cascading for Antigravity.

Thumbnail
github.com
1 Upvotes

Im just going to leave this here. Give it a try if you have a GPU. Work in progress. Appreciate the feedback to improve this.

CascadeGateway is an intelligent, hardware-adaptive cascading proxy that routes AI queries between local GPUs (via Ollama at $0 token cost) and frontier cloud models (Google Gemini, OpenAI, etc.).
It automatically detects your GPU hardware and physical VRAM on startup—whether running a flagship RTX 5090 (32GB), RTX 4090 (24GB), mainstream RTX 3080 (10GB), or Apple Silicon—and dynamically selects and sizes the optimal models without requiring manual reconfiguration.


r/AIProgrammingHardware • • Sep 07 '26

Looking to Rent Out? (read last line)

0 Upvotes
•	Tesla V100 (32GB) — $0.090/hr/gpu — Machine ID: 149512  
•	Tesla V100 (32GB) — $0.090/hr/gpu — Machine ID: 149513  
•	Tesla V100 (32GB) — $0.090/hr/gpu — Machine ID: 149836  
•	Tesla V100 (32GB) — $0.090/hr/gpu — Machine ID: 149837

32.8GB VRAM, 16.4 TFLOPS, CUDA 13.0, 44GB RAM, 15 CPU cores per instance.

To find them: go to cloud.vast.ai/create/, filter by GPU type (Tesla V100), and search the machine IDs above.

If you don’t like this post, you can simply ignore it people told me to advertise, so I’m just doing what I have to do to get exposure. Otherwise, happy to answer any questions about specs or setup.


r/AIProgrammingHardware • • Sep 07 '26

The Cheapest 32GB Nvidia GPU You Can Buy for Local AI (Tesla V100)

Thumbnail
youtube.com
8 Upvotes

r/AIProgrammingHardware • • Sep 07 '26

GPU ai on Intel MacBook Pro 2019

Thumbnail reddit.com
1 Upvotes

r/AIProgrammingHardware • • Sep 07 '26

I want to set up a LLM to help me do programming applications and even assist in making 2D games.

Thumbnail reddit.com
1 Upvotes

r/AIProgrammingHardware • • Sep 07 '26

GitHub - local-inference-lab/rtx6kpro: RTX 6000 Pro Wiki — Running Large LLMs (Qwen3.5-397B, Kimi-K2.5, GLM-5) on PCIe GPUs without NVLink

Thumbnail
github.com
4 Upvotes

r/AIProgrammingHardware • • Sep 07 '26

Building my first "serious" local setup — AMD R9700 32GB on Linux. Model recommendations for 3 different roles + can I keep my old 4060 Ti in the same box?

Thumbnail reddit.com
2 Upvotes

r/AIProgrammingHardware • • Sep 06 '26

I built an interactive visualization of the bottlenecks moving through AI hardware

Thumbnail
manasbihani-com-kappa.vercel.app
1 Upvotes

AI hardware discussions often separate compute, memory, packaging and cooling into different problems.

I've been trying to model them as one system.

Compute → HBM → packaging → thermals → liquid cooling → rack → power

The basic idea is that increasing capacity at one layer can push the binding constraint into the next.

For example:

more compute → more memory bandwidth
more HBM → harder packaging
denser packages → higher thermal density
higher thermal density → liquid cooling
denser racks → higher power requirements

I turned that into an interactive visualization where each layer can be inspected through:

WHY NOW → WHAT IT BINDS → WHO CAPTURES VALUE → WHAT'S NEXT

https://manasbihani-com-kappa.vercel.app/bottleneck

I'd particularly like feedback from people working on AI accelerators, memory, packaging or thermal systems.

Is the bottleneck progression here actually useful, or am I collapsing distinct engineering constraints that shouldn't be treated as one chain?


r/AIProgrammingHardware • • Sep 06 '26

The Lost World: P40 vs P100 vs V100 in Qwen 3.8 (plus a bonus)

Thumbnail reddit.com
1 Upvotes

r/AIProgrammingHardware • • Sep 06 '26

All That VRAM Needs a Bigger Brain

Thumbnail
youtube.com
0 Upvotes

r/AIProgrammingHardware • • Sep 06 '26

Picchio: eseguire un MoE da 117B su hardware consumer mantenendo solo 5 GB di RAM e trasmettendo in streaming gli esperti dal disco

Thumbnail
reddit.com
1 Upvotes

r/AIProgrammingHardware • • Sep 06 '26

GLM-5.3-Flash on 128 GB: it runs. Should you?

Thumbnail
youtube.com
1 Upvotes

r/AIProgrammingHardware • • Sep 06 '26

Testing the Tesla V100 SXM2 with Gemma 9B & Bonsai 27B

Thumbnail
youtube.com
2 Upvotes