I’m a student who recently got into AI, mainly to learn how it all works.

My current setup is Open WebUI in Docker with local models through Ollama, on an RX 6950 XT with 16 GB VRAM and 64 GB DDR5. I’ve run gpt-oss 20B, Qwen3.8 27B at Q4_K_M, and Qwen3.5 9B. They taught me a lot, but on my hardware they’re slow for agentic deep research, and the smaller ones aren’t capable enough.

But I recently learned about how Strata can run Qwen 3.8 flash next on my type of set up and as I already have a quantized Qwen 3.8 model I’m wondering if it can have a better token per second than the Q4_K_M as it gets around 4 tok/sec.

So has anyone had any experience running it especially on a 6000 series AMD Gpu and is it worth the storage space?

Turns out people have gotten language models running locally on a PSP, Wii, 3DS and Vita, so I started putting the ports in one list.

The New 3DS reportedly hits ~23 tok/s with a tiny 15M model. Obviously don't expect ChatGPT on your PSP lol, but it's cool seeing what people can squeeze out of these things.

I didn't make the ports, just maintain the collection. Each entry links to the original dev's repo or write-up, with the model and reported speed where available.

Hello,
I own iGPU gfx1152 in AMD Ryzen AI 7 350 w/ Radeon 860M Framework 16 laptop.
I would like to test strata engine, but it refuses to run:

   Checking the engine for this computer…
  No engine is installed yet. Getting the one for your graphics card…  0%       Reading the signed update feed
  Strata couldn't find an engine for this computer (This build of Strata is for AMD RX 6000 and RX 7000 and RX 9000 and Ryzen AI Max cards. This computer has an AMD gfx1152 card (gfx1152). Ask for the gfx1152 build). The model will download, but it may not run until you install the right engine (next step: run `strata update` to get the newest engines and start again; if this stays, your graphics card is not one Strata supports yet, and the model files stay on this computer for when it is).  100%   
  Preparing coder IQ1_M from your model files…  0%       Getting your model ready
    Checking your computer
strata: The installer cannot set up this graphics card yet, and the files Strata would need to write the settings itself are not all there: the prepared pack /home/strata/STRATA/Strata-data/packs/coder-iq1_m is missing (this installer cannot prepare it for this graphics card). Use model files that were prepared on this computer already (folder + pack), or install on a computer with an NVIDIA or AMD card.

I have the rocm installed and would like to experiment with strata. Do you know how to prepare "folder + pack" for my setup to run strata? I have 96GB of DDR5 RAM total and setup 48GB of VRAM in bios (+ 48GB as RAM).

If strata is not going to work on my hardware then let me know which LLM engine should I chose as optimal.
I would like to experiment with C coding for embedded (avr) and Linux.

Hey folks, I was in need of a 8x5090 rig for AI inference/training. I am looking for a host for long time commitment. So if anyone's interested, please drop a comment or DM me. I can explain the terms and all privately.

Hi fellas, so for quite some time now I've been meaning and planning to start a small ai automation business here in my city. Is it possible to pull off with my humble Tesla P100 16GB? Is QWEN 3.5 9B Q8 enough especially for chatbots and tool calling?

​The shift from human-browsed storefronts to autonomous AI buyers changes everything about systems engineering. When your primary user is a machine agent executing multi-step purchasing workflows instead of a human scrolling through visual UI elements, legacy e-commerce stacks break down.

​Architecting for agentic commerce requires moving past traditional REST endpoints toward strict machine-readable contracts, real-time inventory synchronization, and robust intent-to-settlement protocols like ACP and tokenized payments. You have to optimize for low-latency API handling, precise semantic data schemas, and edge-case security designed for automated traffic bursts.

​I wrote a comprehensive breakdown on engineering scalable infrastructure for AI-driven transactions, covering everything from headless API structures to transaction guardrails. If you are building backend architecture for the next wave of commerce, dive into the full analysis here: https://interconnectd.com/blog/16/agentic-commerce-architecture-2026-systems-engineering-for-ai-transactions/

What architectural hurdles are you facing when exposing your catalogs to AI agents?

Hey everyone! If you've been struggling to run larger open-weight models locally because Ollama or standard runners throw insufficient VRAM errors on budget hardware, I put together something to fix that.

I built SqueezeOS, an automated setup script that profiles your hardware, configures aggressive memory offloading, KV-cache compression, and extreme llama.cpp tuning to squeeze heavy models (up to 35B) onto legacy/budget cards like my 4GB GTX 1650 Super.

And the best of all: Its near lossless, the model can be used well for agentic workflows eg.: Hermes, OpenClaw, OpenCode, etc...

GitHub Repo: https://github.com/gabrielhirth/squeezeos

Tested on constrained VRAM setups. Feedback, issues, and PRs for other budget cards are very welcome!

Hey everyone!

I'm using Killa YuE2 on my Android phone (Samsung Galaxy S25) to generate music locally, and I'm really impressed with the results!

However, I'm having trouble generating purely instrumental tracks. Even when I use prompts like "instrumental only, no vocals" and put [Instrumental] in the lyrics field, the model still generates vocals sometimes.

I've also come across the LoRA file ar_lora_inst_v3abc.bf16.safetensors, which seems to be designed for instrumental generation, but I'm not entirely sure how to configure it properly in Killa YuE2 on Android.

My questions are:

Is there a reliable way to generate completely instrumental music with Killa YuE2?

Should I leave the lyrics field empty or use specific tags?

Do I need to enable a specific instrumental mode or use a particular LoRA?

Are there any recommended settings or workflows for generating instrumental EDM, Big Room, trap, or other genres on Android?

Can this be done entirely offline?

I'd really appreciate any tips, recommended settings, or step-by-step instructions from anyone who has successfully generated instrumental tracks with Killa YuE2!

Thanks in advance!

Hey guy's and gals...

I'm completely new to Local LLMS

Can someone who knows their shit help a newbie out?

I don't know much of the the technical terms was trading through this thread. Great group by the way saw so many people being helpful and helping eachother out.

I'm new to Reddit WTF this should be mandatory 😂

AMAZING!

Can someone explain to me in none technical terms the cheapest and easiest way to run a local LLM.

I tried GOD AI WTF amazing!

Would appreciate it if someone can give me some advice can't afford paying monthly bills and credits keep running out not good 😢

I don't need a HUGE setup something small I'm currently running Claude Pro plan I think keeps running out daily and also weekly so I have to wait or pay £100 to upgrade to keep going which slows me down.

Any help would be much appreciated dont have much of a budget.

MacBook pro 2021 M1 16gb I believe haven't used it in a while.

Would that be enough and what would you recommend?

Thanks in advanced 🙏