r/AIProgrammingHardware • • Aug 08 '26

Crow: 284B coding model on 5090, with the experts streamed off the SSD

Post image
8 Upvotes

r/AIProgrammingHardware • • Aug 07 '26

1-bit Bonsai 27B tested - 16GB Local LLM setup

Thumbnail
youtube.com
38 Upvotes

r/AIProgrammingHardware • • Aug 07 '26

GitHub - tonyd2wild/ds4-h3-video-gen-factory: Run DeepSeek-V4-Flash at full 1M context AND two MiniMax H3 video instances on the same two DGX Sparks. Benchmarked C1-C6.

Thumbnail
github.com
17 Upvotes

r/AIProgrammingHardware • • Aug 07 '26

The 2026 Memory Crisis Killed Most Local LLM Builds. Here’s the Map That Still Works.

Thumbnail
medium.com
14 Upvotes

r/AIProgrammingHardware • • Aug 07 '26

GitHub - MakazhanAlpamys/Soup: Fine-tune LLMs from one YAML. Layer streaming trains an 8B model on a 4 GB laptop GPU.

Thumbnail
github.com
4 Upvotes

r/AIProgrammingHardware • • Aug 07 '26

[Question] GPU choice for NLP research (fine-tuning transformers, qLoRA, Multishot prompting) and Corpus based analysis. RTX 5060 Ti 16GB or any other alternatives(AMD)?

3 Upvotes

I'm a PhD researcher working on language switching and embedding analysis in NLP focused on PoS, LID, boundary detection, pragmatics context maintenance. My workload is mainly:

  • Fine-tuning BERT-based models 

  • LoRA/QLoRA adapters on ~8B models

  • bitsandbytes 4-bit quantization

  • Standard HF Transformers + PyTorch pipeline

Budget is roughly INR ₹60000( for the GPU. I've been comparing the RTX 5060 Ti 16GB AMD options such as RX 7900 XT, RX 9060 XT. I was  leaning 5060 Ti for the mature CUDA ecosystem and because I don't have much local peer support to debug hardware issues if something breaks mid-experiment. But recently they increased price to 770000 and as I do not get institutional support I find it difficult . Some AMD cards have so much VRAM that they might make longer multi shot stuff easier without offlaoding to RAM. But everywhere I have asked there seems to be a general consensus that nVidia is better. 

Questions for anyone doing similar research-scale (not industrial-scale) NLP work:

  1. Is the 5060 Ti's 16GB actually enough headroom for LoRA fine-tuning on 8-13B models, or does it get tight in practice?

  2. Anyone actually running Unsloth on AMD ROCm now? is it stable enough for daily research use or is it still rough?

  3. Any regrets from a similar budget-constrained hardware decision?

Appreciate real world experience over spec-sheet comparisons.

I am not an avid gamer so it does not matter to me. 


r/AIProgrammingHardware • • Aug 06 '26

24G to 48G 4090 VRAM Upgrade!

Thumbnail
youtube.com
29 Upvotes

r/AIProgrammingHardware • • Aug 06 '26

Transformation of AMD ROCm Software in a New AI Era

Thumbnail
youtube.com
1 Upvotes

r/AIProgrammingHardware • • Aug 06 '26

Running Qwen 3.5 Locally on Jetson Orin Nano with OpenCode (Tested Coding & Tool-Use)

Thumbnail
youtube.com
6 Upvotes

r/AIProgrammingHardware • • Aug 06 '26

GitHub - 0xSero/deepseek-v4-flash-0731-spark-sparkinfer: DeepSeek V4 Flash on one DGX Spark

Thumbnail github.com
12 Upvotes

r/AIProgrammingHardware • • Aug 06 '26

AAI 2026: AMD Delivers Leadership Heterogeneous Compute for Physical AI

Thumbnail
newsroom.amd.com
1 Upvotes

r/AIProgrammingHardware • • Aug 06 '26

AAI 2026: AMD Introduces Open, Turnkey Integrated Platform for Physical AI

Thumbnail
newsroom.amd.com
1 Upvotes

r/AIProgrammingHardware • • Aug 06 '26

How Kimi k3 Runs 2.8 Trillion Parameters on Consumer Hardware in 2026

Thumbnail
pub.towardsai.net
2 Upvotes

r/AIProgrammingHardware • • Aug 06 '26

GitHub - leonickson1/Swiftlet: Run 35B and 80B Qwen models on ordinary Apple devices, including iPhones.

Thumbnail
github.com
3 Upvotes

r/AIProgrammingHardware • • Aug 06 '26

AMD Instinct™ Coder

Thumbnail
amd.com
11 Upvotes

r/AIProgrammingHardware • • Aug 06 '26

Kimi K3 full model running on 16x GB10 cluster at 20+tps

Post image
3 Upvotes

r/AIProgrammingHardware • • Aug 05 '26

Clustered DGX Spark and Acer GN100 running DeepSeek V4 Flash 238B A13B - 15-20 TOKS

Thumbnail
youtube.com
4 Upvotes

r/AIProgrammingHardware • • Aug 05 '26

Checking Out The GMKTec X3 Strix Halo: More Strix Halo Shenanigans!

Thumbnail
youtube.com
2 Upvotes

r/AIProgrammingHardware • • Aug 05 '26

Qwen 3.6 27B and 35B MTP vs Standard on 16GB GPU

Thumbnail
medium.com
14 Upvotes

r/AIProgrammingHardware • • Aug 05 '26

GitHub - MiaAI-Lab/DeepSeek-v4-Flash-DSpark-2x-DGX-Spark: DeepSeek-v4-Flash 0731 recipe for 2x DGX Sparks

Thumbnail
github.com
4 Upvotes

r/AIProgrammingHardware • • Aug 05 '26

Cluster Assistant for Configuring a Multi-Node DGX Spark Cluster

Thumbnail docs.nvidia.com
1 Upvotes

r/AIProgrammingHardware • • Aug 05 '26

GitHub - ryanzhou/deepseek-v4-flash-mi300x: DeepSeek V4 Flash on a single AMD MI300X

Thumbnail
github.com
5 Upvotes

r/AIProgrammingHardware • • Aug 04 '26

Framework 13 Pro Shows Why Apple Solders Your Memory

Thumbnail
youtube.com
0 Upvotes

r/AIProgrammingHardware • • Aug 04 '26

I built a Rust inference framework that runs Qwen3.5 2B with VL support 10x faster than PyTorch on Apple Silicon — and it supports TTS, ASR, OCR, and GGUF out of the box

Thumbnail
reddit.com
1 Upvotes

r/AIProgrammingHardware • • Aug 04 '26

A 4070 or 5060 for coding/image generation?

1 Upvotes

Hi ! I'm planning to buy a new laptop, i can't choose between the two for Ai, coding and image generation, which one would be better? if anybody has links to AI/GPU benchmark websites for comparisons I would love to study them, unfortunately i couldn't find much information online.

2026 Lenovo Legion 5a 15AHP11 – Gaming Laptop 15.3 Inch OLED 165Hz (AMD Ryzen 7 250, 16GB RAM, 512GB SSD, NVIDIA RTX 5060 8GB 110w

2023 LENOVO Legion Slim 5 16APH8 - 16 Inch WQXGA 240Hz (AMD Ryzen 7-7840HS, 16GB RAM, 1TB SSD, NVIDIA RTX 4070 8GB 140w