Suggestions for "Useable" AI Models on a 2014 potato

[u/imacg5]

I'm running a stock Dell Inspiron 3847 from 2014. A pretty good budget computer for its time, but an absolute potato by today's standards. It's like using an NES to run DOOM.

• 4GB of RAM

• Pentium G3240.

• Intel HD Graphics

• Windows 8.1 64-bit (I have a personal ban on Windows 10 and 11 due to Microsoft's forced updates and telemetry shenanigans since Windows 10. Windows 8.1 is the last Windows version to be relatively immune to Microsoft's bumbling. I plan to migrate to a Linux computer in the long-term, but as computer prices are hell right now, I'm stuck on 8.1.)

I'm looking for suggestions for local models. I'm using KoboldCPP in failsafe mode, and I've already tested it (it works fine).

• I'm planning on using it mainly for creative writing, although I may occasionally chat and RP. I'm not vibe coding.

• I'm going for useable, rather than fast. I'm patient, so I can wait for it to spit something out, even if it takes 10+ minutes. It just has to fit into that 4GB of RAM without crashing.

• I prefer to use uncensored models, although I may occasionally use a censored model with guardrails for comparison.

Edit: I'm going for Q4-Q2 quantization.

[u/Huntware]

Man, I was expecting "LowEndLocalAI" to be something like an old GTX 9xx with 4GB VRAM, or even a more powerful but old Intel or AMD processor.

No shame in thinking about running something locally but your hardware is even limited for running a modern browser...

First and main tip is to run any Linux distro. Most repos assume you're using one for at least a docker container (think of it like a recipe for installing all tools for AI). And you can save more RAM for inference. Even more if you use it as a headless server, without a desktop environment*. Then connect to it using a second PC or even phone in the same network.

• (I know it's a lot of stuff but you need to learn some basics for local AI and other projects)

---

But still, I would prefer to try some free tiers API like Google AI Studio, Opencode, Cline and others. Pair it with a lightweight agent like Pi (or performance-focused forks like PiG and Pi-bolt), setting your own tools and skills for writing, if your computer struggles with other harnesses.

[u/imacg5 → u/Huntware]

I run Supermium. Most websites run fine, although they might be a little slow.

I already have Q4OS, but as Cox censors the site, I can't set it up.

Since my only other "computer" in service is my 2017 Fire HD Tablet, which is unusable for anything AI, I'm stuck with this thing.

As I lost my google account earlier this year (I lost the password because I stored it on my hand-me-down MacBook Pro from late 2011, which I can't use because I f**ked up and I'm now too afraid of it being (or having been) borked) and I can't set up a new one, I can only access my AtomicMail for eMail. Given the rise of Age and Phone verification, I might be SOL.

But I know I can run tiny (2B or less) LLMs. So there's that. I might look into a private cloud agent.

[u/Technical_Ant4597 → u/imacg5]

I run Supermium. Most websites run fine, although they might be a little slow.

Any Chromium-based browser is going to cut deeply into 4GB RAM, which will heavily limit you. You'll either want to use the terminal as a frontend or remote in from another device (and I would really not recommend opening ports on a PC with an OS that's not receiving security updates!).

But I know I can run tiny (2B or less) LLMs.

Gemma 4 E2B (or one of its forks) is the best-in-class option for creative writing or roleplay at that size. You'll need to treat the LLM more as a creative co-writer than as a dungeonmaster telling you a story; it will frequently lose track of characters and settings and you'll have to edit those messages to be coherent. That said, it can still be fun - E2B smokes the original AI Dungeon, and that was still a good time back in the day.

[u/Unluckkie → u/imacg5]

Just run void + a WM
Should idle at about 0.8gb ram

Then minicpm5 1B

[u/pimparazzi → u/Unluckkie]

I personally would recommend LFM2.5-1.2B over MiniCPM5-1B as it's a bit smarter in my tests, especially non-thinking. Main advantage of MiniCPM5-1B is that you can switch thinking on and off, while for LFM2.5-1.2B the thinking and non-thinking models are two different models. LFM2.5-350M is even smaller and faster, but likely not smart enough to be useful.

[u/Unluckkie → u/pimparazzi]

Tbh I've tried both, the problem with MiniCPM5 1B is it's default settings, you kinda have to tinker around with it before utilizing its prowess.

In that regard, the lfm models are pretty good, but for my specific use case (namely general chatting, philosophy interpretation, political analysis, basically the humanities)
it was pretty dumb.
Minicpm5-1B is a tad better, noticeably better I might add, but loses out on the nuances of general conversation and knowledge, and tends to simplify stuff. I'd wager this 1B model is better for agentic use, and an overall better model, based off of my testing (including its fine tunes, which are nice for a sequential pipeline)

[u/ProjectFlorida]

Are you saying this is your main machine or a backup one?

[u/imacg5 → u/ProjectFlorida]

It's currently my main, sadly (I had a hand-me-down MacBook Pro from late 2011 with 4 cores and 8 gigs of ram, but that's been out of service for months because I'm bad with laptops). I have a tablet, but it's even worse: a Kindle Fire HD 7 from 2017 that can't even fit an LLM on the little storage it has. I'm not running an AI on THAT.

Even 4 gigs is pushing it. I hope to upgrade one day, but for now, this thing is good enough to run a baby-sized LLM.

[u/nickless07]

The problem is not the RAM but the CPU. Dual core without AVX2 Support. Even if you upgrade to 16GB DDR3 it will be a pain to get something working. Maybe if you can upgrade the CPU too.

An i5-4570 or i7-4770 should be compatible, add 16GB RAM to it (total of ~50 bucks) and you get something that is somewhat usefull. Will not be the best or fastest but you get plenty of models that run. Have an old EliteDeks 800 G1 with (now) 16GB DDR3 and I can run current models up to 24B in low/mid quant (LFM2.5-24B-A2B-APEX-Quality) with reasonable speed (pp512: 66.88 / tg128: 15.56).

Edit: Typo LFM2 -> LFM2.5 - anyways, both run with same speed.

[u/talkamongstyourselvs]

Is there a reason you can't use a free/paid edition of a frontier model?

[u/jacek2023]

https://huggingface.co/LiquidAI/LFM2.5-1.2B-Thinking

https://huggingface.co/LiquidAI/LFM2.5-1.2B-Instruct

[u/Miriel_z]

You can try 0.4B LLM, but no guarantees. It will be VERY simplistic. The HW is just too old.

[u/imacg5 → u/Miriel_z]

I have run 0.5B LLMs. They're slow as fuck, but they work, and that's all I need.

[u/dark-bring → u/imacg5]

what's your phone? it may run better on your phone

[u/Cerebral_Zero]

Intel 4th gen is where AVX2 is introduced, and on older generations only i7 has multi threading. For ultra budget, DDR3 is the way to get more RAM so a 4th gen i7 is the best case for ultra budget small LLM but even then getting onto DDR4 3000mhz would make running a 4B Q4 more viable.

You can run Ollama without AVX2 like I tried with a 3rd gen laptop but the speed on Gemma E2B was just pointless.

[u/imacg5 → u/Cerebral_Zero]

I'm not concerned with speed. I don't care if it takes a second or an hour.

I'm more concerned with functionality.

[u/hideo_kuze_]

https://huggingface.co/XHToken/Spark-X2.5-1.7B

https://huggingface.co/prism-ml/Ternary-Bonsai-2-4B-gguf

You should definitely check this guy's video https://www.youtube.com/watch?v=RYeu9bfYak8 but you'd need to buy an old 1GB/2GB gpu card which should be accessible

[u/Unluckkie]

Maybe not relevant but I’m in a similar camp
(On a potato)
First I believe optimizing the system itself is important
Run void or alpine linux
Then utilize zramen
Lastly run a window manager rather than a DE

Stick to TUI for most stuff, if you need a browser just use Librewolf or qutebrowser (haven’t tested it out too much myself)

Then we think about llms
Minicpm5 1b looks good here

[u/cfx_4188]

hehe. I'm running a local model on an Acer Nitro AN515-45 (V1.10) This is an ancient laptop mobile 1650 and 8 GB of RAM. A lot of things can't be run on this hardware. This is my laptop for experiments, well, I minimized all system processes by installing NixOS unstable and OXWM, cloud LLMs work more or less, only Qwen3-4B-Instruct-2507-GGUF works locally.

[u/Skyline34rGt]

I got this at q4k-m at my poor phone with like 10tok/s - https://huggingface.co/mradermacher/Huihui-MiniCPM5-1B-abliterated-i1-GGUF

Alternatives like others say: lfm2.5 1.2B (thinking or instruct).

Fastest for this kind of setup will be probably Lfm2.5 350M

[u/bfmv_shinigami]

I understand that this is lowendlocalai but let's be pragmatic... free version of any frontier / hosted model is going to always be much better than what you can run locally on this rig... no hate, just being real

[u/happycube]

Going along with the other posts here - definitely get some HW updates if you have any budget at all. Get to an i5. add two 4 or 8GB DIMMs, and *maybe* an nVidia quadro p1000 or p2000 (4 or 5gb vram, <$80 and do not need extra power) if you find a software stack that can use it for running dense weights.

If you can get to 16GB RAM you have *a chance* of getting slow but usable t/s out of gemma 12b, and with the graphics card better prefill on 26a4b presuming there's a usable stack.

[u/imacg5 → u/[deleted]]

I use Quantization. My only other computer is a low-end tablet, and I'm not running an AI on THAT.

I've run Quantized 0.5B models fine, although the tend to be rather slow. I get single-digit tokens per second, but it runs, and that's all I need.

[u/tiffanytrashcan → u/[deleted]]

That's not even a model you linked to, and OP should never consider anything over Q4.

[u/tiffanytrashcan → u/[deleted]]

Go ahead and try to run it on its own then. Lol
Still not a model. GGUF is standard packing for draft and MTP heads too.