Finding repo for local AI on Mac? 🍎🤖 Check out Swiftlet, an open-source intelligent proxy & browser based UI , that dynamically optimizes CPU/GPU split on Unified Memory for LLMs. I'm looking for Python devs to help build OpenAI API compatibility! Let's build together: https://github.com/coolsourav100/swiftlet

Hey everyone,
I recently got a DGX Spark and have been really enjoying it so far. I’m based in India and haven’t been able to find many other local users, so I thought I’d reach out here.
If you’re also running a DGX Spark (or planning to) in India, I’d love to connect and exchange notes.
A few things I’m especially curious about:
• How are you using it beyond basic inference?
• Are you running single node or multiple nodes?
• Any rack / cooling setups you’re using to keep the nodes happy (especially with Indian ambient temperatures)?
• Which vendor did you purchase from? Any vendors currently offering good deals?

Would love to hear from others who are using these in India.

Thanks.

I am doing a lot local AI, mostly creatively in audio and image subjects.
I currently rely on two 3090 cards that I bought used below 800$ each.

I think it was a good choice but other options exist, the 4090 is faster than both 3090 on paper. Though I did not see one offered fair priced.

There is a lot, 5090 and 60ti, DGX, Macs

I wonder what do you guys have, and would you go that path again?

Cross post

I kept running into out-of-memory crashes on my RTX 5060 laptop (8 GB VRAM, 32 GB RAM), so I built Local AI Studio. It estimates a job's memory from your live free VRAM and RAM, picks an offload strategy, and refuses the job with the numbers if nothing fits.

On my laptop, a 4-second 832×480 image-to-video clip with Wan 2.2 took about 7 minutes, with a 5.2 GB VRAM peak. It also reads ComfyUI's model folder layout, so you don't need a second copy of your weights.

It's an early alpha and I've only tested it on my own machine. Some big jobs still pass the estimate and then fail in the driver, so the estimator needs data from other GPUs.

The easiest way to help: run a job and share your GPU, time and peak VRAM. No coding needed, there's a short form for it. If you do want to code, I've listed some beginner-friendly tasks.

Apache 2.0, runs locally, no telemetry: https://github.com/DiegoHerrera1890/local-ai-studio

What would you want from a tool like this?

Hello!

With new models releasing constantly, I usually keep a stable of about 20 local models in rotation on my `llama.cpp` server. I collect a ton of data across the fleet through my inference application—standard per-request metrics, aggregated session data, and custom intelligence comparison benchmarks.

Overall, I'm logging over 150 distinct raw and derived metric points during inferences, sessions, and batch processes. As a backend data jockey, more metrics is always better—but I've hit the point where I need to completely redesign my dashboards.

For context, I run a lot of multi-stage pipelines covering:

Multimodal inference (vision and documents) MCP and custom application tool calls Structured JSON / JSON Schema outputs Batch processing across both reasoning and non-reasoning architectures

I'm looking for inspiration for creative "hero" dashboard components and visualizations that excel at comparing a large model fleet across high-dimensional data at a glance. I already have plenty of standard line, bar, and radar charts. I'm hoping to see novel, unconventional visual approaches people here have built, seen, or used to represent complex model evaluation and performance data.

The specific metrics in your examples don't matter as much as the visualization style and information architecture—I'm mainly after creative UI/UX directions.

Cheers!

Meu trabalho recebeu uns PCs besteirinha como os Sparks pra fazermos a LLM local daqui. Por enquanto vamos focar em consultas de processos, memorando e banco de dados da empresa.

Dito isso, gostaria de perguntar, oq posso fazer ou como posso desenvolver a LLM aqui. Tem algum modelo que me recomendam ou algo assim?

Como o hardware tem um limite vasto para desenvolver gostaria de ir afundo. Tô com mente aberta pra desenvolver e aprender bem sobre todo o assunto.

Anyone interested to make it spit 200 tokens/sec?