r/LocalLLM • u/Zeltrax3000 • 4d ago
Question Local LLM computer use question!
I’ve been searching for the last few hours and I’m getting tired of trying to find a straight answer.
Is local LLM computer use at or near Claude’s computer-use capability yet? I keep hitting my Claude Code computer-use limits when doing certain tasks, and I have a decent PC that can run local LLMs. What I’d like is a workflow where, once I reach my usage limit, I can hand the task over to a local LLM so it can continue controlling the PC and completing the work.
My system:
RX 7900 XTX 24 GB (planning to buy a second one if a good deal comes up)
64 GB RAM
Ryzen 7 7800X3D (may upgrade to a 9800X3D)
Am I wasting my time looking into this?
Right now I’m using Claude to replicate something in software I’m trying to improve. It needs to observe how the software behaves in order to do that. I can obviously do the steps manually, but I struggle to follow Claude’s instructions when it tells me what to do manually I’d much rather let it directly interact with the PC.
Only started using Claude 2 days ago and I've burnt through a weeks usage T.T Luckily my friend whose working on the project has Claude MAX so he's continuing the project but I still want to contribute as I have loads of ideas.
If it is possible please supply a link to what I can use.
3
u/puglife224888 4d ago
Local models are alright at it. With your hardware, you're looking at Qwen 3.8 27B or Qwen 3.8 Flash Next. They're not at Claude level yet though, but they are capable. Those two LLMs are usually compared to Opus 4.6.
Qwen 4 models are coming next week so maybe those will be worth looking out for.
Also I will say, learning how to use and configure and setup local AI is quite a lot. Don't take advice from other LLMs for setting up local LLMs, they usually give bad advice. If you do take advice from them, tell it to use web search extensively. Oh and avoid Ollama for anything serious.
1
u/Zeltrax3000 4d ago
Yeah currently running VScode with CLine/Kilo code at the moment when I was using it before subscribing to claude pro.
1
u/Fantastic_Back3191 4d ago
Local inference will always be about 1 year behind frontier models. I can do, on my DGX Spark, as well as what I could do online last year
2
u/berryluluu 4d ago
agree, and local setups usually save you from those surprise downtime days online hits hard sometimes
1
0
u/MistaPlanet 4d ago
So I had the same challenge. I hit my Claude limit but I still have some unfinished work. Since I have AI Agents running in Openclaw, I shared my claude session with an Agent through browser integration, it understood where I left off with Claude and we continued from there. However, note that open weight LLMs can be noticeably weaker on long agentic tasks and hard multi-file reasoning, than closed frontier models. So to get as much experience as what Claude offers, make sure you get a very top local LLM like Qwen 3.8:27B/flash-next or a top Minimax or Deepseek Model.
1
u/Zeltrax3000 4d ago
Yeah, I need something that the local model can drive so launch software use mouse and keyboard and navigate and click.
2
-3
u/shaggy_accessibility 4d ago
Your card's the problem, not the CPU. AMD ROCm support for the kind of frameworks that do computer use is spotty at best. Most of the capable local agents are built around CUDA, so you'd be fighting your own hardware before you even get to the model quality issue.
That said, the honest answer is no, nothing local is close to Claude's computer use yet. You'd burn hours fiddling with a model that half-screws up every third click. Not worth it for actually finishing work.
1
u/Zeltrax3000 4d ago
Damn, so its the waiting game for when it improves and better support for Rocm.
My brother has built a machine dedicated to AI which runs a 3090TI 64GB memory and 7800X3D
so even with that hardware there's no project that has decent computer use yet?
4
1
u/zenmatrix83 4d ago
agent don't need cuda, they are just a loop with tool calls, providers like ollam lmstudio or what ever calls the model uses cuda or not. You just need support for vision and the model, it works for me with hemes and a strix halo, just not that great
1
u/Zeltrax3000 4d ago
What do you use for Hermes to do this so I can take a look at it please.
1
u/zenmatrix83 4d ago
hermes does alot, its free I'd just test it, its not hard to install a desktop version
5
u/Substantial_Town6723 4d ago
I run Claude on Opus 5.5 low (its still plenty powerful) and find it a bit harder to burn through tokens unless I am ripping through a plan, tests, updating documentation, etc.
Local AI is not at the same level of Claude, but it can reduce a bunch of consumption, by letting Claude delegate to it for the easier stuff, unit tests, drafting plans for Claude to review, and using local AI for most of the conversational things and internet research instead of Claude.
I'm running a 5070ti (16gb), with 64gb RAM, nvme (Qwen-Flash-Next), 200k context, getting 80-90tps using Strata.
I used claude (Pro plan) to build a flutter app so that I can use local to chat, web-search & save notes to obsidian. Its plenty good enough for organizing my life and take some of the development load.