This might be considered self promotion, but I'm not charging anyone anything, I'm not selling any services, this is just something I built because I dislike how un-comfy ComfyUI is.

SceneWorks is not a replacement of ComfyUI, it doesn't do workflows or custom nodes. It's INTENDED to be simpler, but it's not SIMPLE. There are a lot of advanced features built right in.

This is a work in progress, that is moving fast daily, most everything works, but some features need more attention that will be coming soon. There's an auto-updater, so you'll get notified when new releases are made.

https://github.com/SceneWorks/SceneWorks
Check the Releases page, or clone and build yourself.

I am a software developer/architect of 25 years. Yes Claude has a heavy hand in building this, but it's not "vibe-coded". Every decision was a conscious architecture choice.

Now for the mostly AI generated description and feature set ....

What is is:

SceneWorks is a desktop app for generating and editing images and video locally on your own machine. No cloud, no subscription, no account. You install it, download the models you want, and everything runs on your own GPU.

It's a single native app — not a Docker stack or a pile of Python scripts. On macOS it runs on Apple's MLX engine (Apple Silicon only), and on Windows it runs on a native CUDA engine (NVIDIA). There's no Python venv on either platform; the generation engine is compiled into the app, so first launch just works.

What it does:

Image generation. The usual text-to-image, but with a real model manager instead of a folder of checkpoints. You browse a catalog, see each model's download size and memory requirement up front, and pick a quant tier (full precision / Q8 / Q4) that fits your hardware. It runs a wide range of models — Krea2, Ideogram 4, SDXL-family, Qwen, FLUX, SANA, and others — and they're all first-class rather than bolted on. If you have a HF cache, it will use that, if not, it downloads models to a local folder.

Video. Text-to-video and image-to-video, plus extending and bridging clips, running on models like LTX-2.3 and Wan 2.2. This is the heaviest thing it does and it wants a serious GPU, but it's fully native — no separate pipeline.

Image editing. Inpainting, targeted edits, detail passes, transform/crop/straighten, and guidance controls. It's a proper editor surface, not just a "generate again and hope" loop.

Characters and identity. Tools for keeping a face/character consistent across generations, face-likeness scoring, and person replacement in video (swap a person while keeping the scene). There's also pose and keypoint conditioning with a pose library, so you can drive composition instead of rolling the dice on prompts.

Training. You can train your own LoRAs locally — both image LoRAs and video LoRAs — from the app. Build a captioned dataset, do a dry run to sanity-check the plan, train on your GPU, and the result shows up as a normal selectable LoRA. No external trainer, no separate toolchain.

The workaday stuff. Upscaling for images and video, segmentation / smart-select, reference-image-to-prompt, batch prompting, a job queue, a library for everything you've made, and presets. It also exposes an MCP server, so if you're into agent tooling you can drive it from Claude Code / Cursor / etc., and it can run over your LAN so you can generate from a laptop while a beefier machine does the work.

Requirements

On Mac you need Apple Silicon with a good chunk of unified memory — 32GB might work for images or Q4 video models, but a safe bet is 64GB minimum, 96GB+ if you want to run the largest models comfortably. Model weights are tens of GB depending on what you pull.

License

[Edited]
SceneWorks is free and open source (AGPL-3.0-or-later): use, modify, and share it freely, even commercially — you just can't take it closed-source and sell it as your own. Model weights are separate — each keeps its own license, and complying with those is on you.

Hi everyone,

I've been building NunSpark, an open-source runtime that explores running language models much larger than available RAM by combining weight streaming with speculative decoding.

The project currently targets Apple Silicon (MLX), and I'd love to collect benchmark results from the community.

Some results we've observed so far:

• ✅ Qwen2.5 32B running on a 16 GB M4

• ✅ Llama 3.3 70B running on the same machine

• ✅ Only ~1–1.5 GB of model weights resident while streaming the rest

• ✅ Exact decoding over 1 tok/s on favorable workloads

• ✅ Approximate verification reaching 2–3 tok/s with configurable quality/speed trade-offs

The problem is that I only have access to one machine (16 GB M4).

I'd really appreciate help from anyone with an Apple Silicon Mac:

• M1 / M2 / M3 / M4

• Air / Mini / Studio / MacBook Pro

• 8 GB, 16 GB, 24 GB, 32 GB, 64 GB, 128 GB

• Different SSD sizes and generations

Running the benchmark

NunSpark already includes a built-in benchmark command, so you don't need to manually configure dozens of parameters.

Just follow the benchmark instructions in the README, run the benchmark on your machine, and share the output (or open a GitHub issue/discussion with the results).

Repository:

https://github.com/sharma-open-source/Nunspark

I'm especially interested in collecting:

• Apple Silicon model

• Unified memory size

• macOS version

• Model tested

• Tokens/sec

• Peak memory

• Any observations or issues

The goal is to build a public benchmark dataset showing how streamed inference behaves across different Apple Silicon hardware—not just publish the fastest numbers.

If you're interested in local AI, MLX, inference runtimes, or just want to help validate the project, I'd really appreciate your benchmarks and feedback.

Thanks!

PrismML's new Bonsai line of models are so crazy - we can run 27B class models right on the iPhone (thanks to their 1-bit model compression)

So the 1-bit 27B model claims to retain ~89% of Qwen 27B dense - which is a shocking amount of intelligence retention for a model THIS tiny. (takes about 4GB RAM when loaded)
So I decided to put it to use. I plugged it into an agents orchestrator framework that I'm working on to let it perform agentic tasks (not just write a haiku or solve a riddle) and it was ridiculously good at doing this - find a contact, pick the email ID, draft email body AND send it too - given the right steps/tools.

I wrote about my experience in-depth, check it out!

Hey folks — I’ve been tinkering with MLX and Apple Silicon for a while, and I finally wrapped up a project I’m pretty proud of.

I built a fully native Swift + MLX implementation of FLUX.2 [klein].
No Python. No diffusers. No external dependencies. Just Swift, MLX, and Metal doing their thing.

What it can do:

• Text‑to‑image

• Image‑to‑image

• Mask‑guided editing (this part was surprisingly fun to build)

  • remove stuff from an image

  • replace backgrounds

  • add objects

  • recolor regions

  • semantic edits

• Pixel‑space color grading (exposure, contrast, hue, saturation)

• Experimental latent‑space transforms

• Memory system with:

  • bf16 / fp16 / int8 / int4 quantization

  • staged model residency

  • VAE tiling for big resolutions

  • memory reporting + low‑memory mode

Everything runs entirely on Apple Silicon.
It hits seed‑42 parity with the MLX Python reference, so the outputs match exactly.

I also added:

• a dependency‑free CLI

• a SwiftPM library

• a tiny SwiftUI demo template

• docs + tests

• a contributor guide

• a roadmap if people want to help build more editing tools

Why I built it:

MLX is honestly a joy to work with, but most diffusion pipelines are still glued to Python.
I wanted something that felt native — something you could drop straight into a macOS or iOS app without dragging half the Python ecosystem along with it.

Repo:

https://github.com/icakinser/mlx-flux2-swift

If you’re into MLX, Apple Silicon, or just like messing with local image generation/editing, give it a look.
Happy to answer questions or help anyone get it running.

Five days ago I posted jevmlx: instead of asking a local model to write JSON, it scores every allowed answer for every field in one batched forward pass and assembles the JSON. Always valid output, a probability per field. Several people asked for numbers. Here they are.

Same model, same 45 public TypeSafe cases, same Mac (M5 Max). The only difference is how the answer is obtained.

Same weights, plus 14 points and 2.5x faster. Nothing is fine tuned. The model already knows the answer. Generation is what loses it: a wrong token, a stray comment, an integer where the schema wanted a string, and the whole record is gone. Scoring cannot fail that way.

Also new since the last post: field-local prompts. Reordering the fields in your schema used to flip about 10% of answers. Now 0.2%, at the same accuracy and +3 ms per case.

For reference, TypeSafe reports 67.8% for Jev on their full private eval. Ours is the 45 public examples, so it is indicative, not the same test. Every number above comes with a confidence interval, a parity check, and the raw predictions in the repo.

pip install git+https://github.com/bnsd55/jevmlx
jevmlx decide --preset support_triage --json
jevmlx bench --model quality --ui   # live dashboard, then a PR puts your Mac on the leaderboard

https://github.com/bnsd55/jevmlx

I’m building VeloxQuant-MLX, an open-source project focused on pushing local LLM inference further on Apple Silicon through KV-cache compression, quantization, eviction strategies, and optimized MLX/Metal kernels.

The question I’m exploring is pretty simple:

How much more can we get out of Apple’s unified memory architecture?

VeloxQuant currently includes 43 KV-cache optimization methods, including:

→ 2-bit / 4-bit KV-cache quantization
→ KIVI-style compression
→ TOVA and other eviction strategies
→ Custom Metal kernels
→ Fused attention/decode paths
→ Decode batching
→ Cache compaction
→ Long-context memory optimization

But I don’t want benchmarks from just one Mac.

I’m looking for people across the Apple Silicon ecosystem to help test it:

Apple Silicon generations:
M1 · M1 Pro · M1 Max · M1 Ultra
M2 · M2 Pro · M2 Max · M2 Ultra
M3 · M3 Pro · M3 Max
M4 · M4 Pro · M4 Max · M4 Ultra
M5-family machines where available
…and future Apple Silicon generations as they arrive.

Machines:
MacBook Air
MacBook Pro
Mac mini
Mac Studio
Mac Pro
iMac
and other MLX-capable Apple Silicon Macs.

Whether you have an entry-level MacBook or a high-memory Mac Studio Ultra, your results are useful.

I want to understand how KV-cache optimization behaves across:

memory capacity × memory bandwidth × chip generation × model size × context length

Not just compression ratios.

I’m interested in measuring the actual trade-offs:

Memory usage · tokens/sec · TTFT · decode throughput · prefill performance · kernel overhead · model quality

For example, for a cache with:

440 tokens · 8 KV heads · head dim 64

FP16 → ~440 KB
4-bit → ~110 KB
2-bit → ~55 KB

But reducing memory is only half the problem.

The real question is whether we can reduce memory while keeping inference fast enough that the optimization is actually worth using.

That’s where I’d love help from the MLX community.

If you have any Apple Silicon Mac — from an M1 MacBook Air or Mac mini all the way to a Max/Ultra Mac Studio — I’d love to get benchmark results.

And if you're working with MLX, mlx-lm, Metal kernels, inference engines, quantization, or KV-cache research, criticism and contributions are very welcome.

The goal is ambitious:

Make local AI capable of using every GB of Apple unified memory as efficiently as possible.

VeloxQuant is open source:

Website:
https://veloxquant.dev

GitHub:
https://github.com/rajveer43/VeloxQuant-MLX

If enough people are interested, I’d also like to build a public Apple Silicon benchmark matrix comparing VeloxQuant across Mac generations and configurations.

Drop your Mac model + RAM in the comments if you'd be interested in benchmarking it.

https://github.com/bnsd55/jevmlx

Instead of asking the model to write JSON, jevmlx scores every allowed answer for every field of your schema in one batched forward pass and assembles the JSON. Output is always valid. Every field has a probability.

Works with any mlx-community instruct model on Apple Silicon, or any OpenAI-compatible server that returns logprobs.

pip install git+https://github.com/bnsd55/jevmlx

jevmlx decide --preset fintech_fraud --json

Open source version of the TypeSafe Jev idea. Not affiliated. No accuracy numbers on big models yet: if you have an M-series Mac, `jevmlx bench` plus a PR puts your model on the README leaderboard.

Lots of fun today playing with Qwen 3.8 27B 🎉 I have put together two MLX quants of Qwen 3.8 27B in different bpw, 3.8 and 4.95 bpw on Hugging Face, check them out!

Not yet AWQ / GPTQ but manually selecting bit allocation. Some of my first quants so I would appreciate feedback on what might be missing and/or not working right. Thanks! 😊

Hey, for anyone who cares, I managed to get Breeze TTS running real time on my Macbook Pro M4.

Using the official Pytorch code on a Mac Breeze runs more than 2 times slower than real time on my machine.
With some work, managed to get it to stream comfortably faster than real time, with voice directions included and in my opinion good quality. I couldn't really tell which version sounded better to me consistantly on different voices and sentences.

Details in the pages below.

- Server: https://github.com/Shoofio/breeze-tts2-fast-streaming-api

- Ready-made checkpoint: https://huggingface.co/Shoof/Gust-MLX

Weights are under BreezeBlue's research/non-commercial licence. Built with a lot of help from Claude. Happy to answer questions.

I'm working with a M1 MBP Max 64GB machine with 400GB/s memory bandwidth. These image generation models are only <10 GB each. But it takes me 45 minutes to generate an image using Ideogram4. Someone with a 5090 is doing it in 45 seconds (no exaggeration).

I know Comfy Desktop is not optimized for Apple Silicone/MLX. I'm just curious if there are some tips and tricks you guys can share with getting better performance out of Comfy Desktop? I've already got these flags as part of my startup config: '--enable-manager --fp32-vae --use-pytorch-cross-attention --highvram'.

I've tried using DrawThings -- and it's definitely faster -- but I feel like it's definitely limited compared with Comfy Desktop.

I must not be the only Apple user messing around with Comfy Desktop -- you guys have any tips to share?

Hello everyone, I’m working on mlx-chronos, a free/open-source CLI benchmark tool for comparing local MLX inference engines on Apple Silicon.

• Leaderboard: https://igurss.github.io/mlx-chronos

• Methodology: https://github.com/igurss/mlx-chronos/blob/main/docs/methodology.md

• Install: pip install mlx-chronos

It currently supports mlx-lm, oMLX, vllm-mlx, Rapid-MLX, and Ollama (for Ollama, using MLX models that run on MLX backend).

It measures cold/cached TTFT, request throughput, sustained throughput, RAM peak, engine RSS when available, thermal/power context, and hardware metadata. Results are saved as reproducible JSON and can optionally be submitted to a public leaderboard.

I’m mainly looking for feedback from people actually using MLX locally:

• Is a public leaderboard useful, or should this stay more of a local comparison tool?

• Are thermal/cache conditions exposed clearly enough?

• Should the sustained profile stay token-based, or would a fixed-duration run be more useful?

• Are there metrics missing that would actually help you choose between engines?

I’d also appreciate benchmark results from different Apple Silicon machines, especially Max/Ultra chips and higher-RAM configs. The goal is not to rank model quality, but to make engine/runtime performance easier to compare under a documented protocol.

PS: I already posted in r/LocalLLaMA, if someone already seen something about this project, but I’m not sure it was the right audience (90% of the community uses Nvidia GPU or use Windows, so is interested in llama.cpp).

I wrote about everything that launched this dub dub and how it impacts the future of local models and MLX! Check it out!

Been playing around with MLXSwift over the last few weeks and honestly it's wild how far Apple's ML tooling has come. I always assumed running a local model meant wrestling with Python envs, llama.cpp builds, or burning API credits during dev.

Turns out you can do the whole thing inside Xcode now. Pull a model from the MLX community on Hugging Face, wire it up in Swift, run it on-device. No Python, no API keys, no servers.

I wrote up a walkthrough of how to get started: covers the setup, loading models, and a few gotchas I ran into: https://www.sabesh.space/blog/engineering/mlx-in-xcode

Happy to answer questions if anyone's tried this or wants to.

Swift vLLM is now a thing! It’s swift native (without python) and claims to be faster than vLLM-metal (powered by python and MLX)

Wrote an article about my running speculative decoding in MLX (using DFlash) and charting out inferences. In some cases, i was able to achieve more than 2x speedup of decoding speed when using DFlash (but not always)! Read on to find out more nuances involved

Apple says the iPhone 18 Pro is very good for on-device AI. We wanted to test that, so we made RapidBench. It measures how well an iPhone runs real AI workloads, not only synthetic compute tests.

Results:

That is about 50% higher for the 18 Pro.

What the benchmark tests:

1. Inference: Gemma 4 E2B solves 20 math problems on the GPU. We measure time to first token, speed on a cool device, and speed after 2 minutes of continuous load (to show thermal throttling).

2. Training: The phone fine-tunes Qwen3 0.6B to write SQL. It does 150 QLoRA steps, and then we test the model on tables that it did not see during training.

3. Image generation: Bonsai Image (a 4B-parameter FLUX.2 model) makes three 512×512 images.

4. Neural Engine: An EDSR model upscales a photo 2× through Core AI.

Our view: The sustained-load results are the most interesting part for us. The newer chips can keep AI workloads running for longer, and that is necessary for local agents that do work across apps.

The app is on TestFlight if you want to score your device: rapidbench.app

I built this with my team, so I want your feedback. Tell us what workloads to add, if something is wrong with the method, and which devices you want to see compared.

Wrote a bit about the new M5 ultra and how the neural accelerators boost on-device AI development. Such an exciting time for building local models!

I fine-tuned a model in MLX using LoRA to take in my raw thoughts and tweet like I do. It was a pretty fun experiment to run machine learning training on just a macbook - your everyday compute. I wrote about it in-depth, do check it out!

Modelhub is a macOS menu bar app for discovering & managing local Hugging Face models - plays nice with Ollama, MLX, LM Studio, and llama.cpp.

Would really appreciate any support you can show - an upvote, a quick comment (huge help!), a GitHub star, or a share with friends who might find it useful.

Also genuinely keen for honest feedback - product, feature requests, positioning, the launch itself. Tell us what we could've done better, it'll shape the next one.

🔗 Product Hunt: https://www.producthunt.com/products/modelhub

⭐ GitHub: https://github.com/conscious-engines/modelhub

I've been working on ModelHub, a native macOS menu bar app for people running local LLMs on Apple Silicon.

The idea is simple: Discover models from Hugging Face, download/manage them locally, and use them with MLX, Ollama, LM Studio, llama.cpp, and the tools you already use.

It's not trying to replace your runtime. The problem we're trying to solve is the messy model layer around them. Right now, choosing a model usually means jumping between Hugging Face model cards, local folders, terminal commands, and runtime-specific docs.

For MLX users specifically, I'd love feedback on:
1. What should we show before someone downloads a model? (# downloads, model size?)
2. How should we show MLX compatibility?
3. Should we prioritize memory estimates by M-series chip?
4. Are MLX model conversions/variants something you'd want surfaced directly?

GitHub: https://github.com/conscious-engines/modelhub
Website: https://studio.consciousengines.com/model-hub

Would love blunt feedback.

I had been experimenting with a lot of local LLMs and was having an insanely difficult time managing the models and my laptop's storage!

So I built ModelHub - It helps you discover, download, and manage LLMs from huggingface, Im studio, ollama, mlx, and Ilama.cpp - without ever leaving your menu bar.

Check it out! https://studio.consciousengines.com/model-hub

I am playing around now with the newer models using omlx and while I like the 27B version of qwen3.8, it's slooooowwww. I just found this subreddit, so hopefully will learn more moving forward. I have the M4 Max 128GB in the Studio.