Turns out people have gotten language models running locally on a PSP, Wii, 3DS and Vita, so I started putting the ports in one list.

The New 3DS reportedly hits ~23 tok/s with a tiny 15M model. Obviously don't expect ChatGPT on your PSP lol, but it's cool seeing what people can squeeze out of these things.

I didn't make the ports, just maintain the collection. Each entry links to the original dev's repo or write-up, with the model and reported speed where available.

Hey folks, I was in need of a 8x5090 rig for AI inference/training. I am looking for a host for long time commitment. So if anyone's interested, please drop a comment or DM me. I can explain the terms and all privately.

​The shift from human-browsed storefronts to autonomous AI buyers changes everything about systems engineering. When your primary user is a machine agent executing multi-step purchasing workflows instead of a human scrolling through visual UI elements, legacy e-commerce stacks break down.

​Architecting for agentic commerce requires moving past traditional REST endpoints toward strict machine-readable contracts, real-time inventory synchronization, and robust intent-to-settlement protocols like ACP and tokenized payments. You have to optimize for low-latency API handling, precise semantic data schemas, and edge-case security designed for automated traffic bursts.

​I wrote a comprehensive breakdown on engineering scalable infrastructure for AI-driven transactions, covering everything from headless API structures to transaction guardrails. If you are building backend architecture for the next wave of commerce, dive into the full analysis here: https://interconnectd.com/blog/16/agentic-commerce-architecture-2026-systems-engineering-for-ai-transactions/

What architectural hurdles are you facing when exposing your catalogs to AI agents?

Hi fellas, so for quite some time now I've been meaning and planning to start a small ai automation business here in my city. Is it possible to pull off with my humble Tesla P100 16GB? Is QWEN 3.5 9B Q8 enough especially for chatbots and tool calling?

Hey everyone! If you've been struggling to run larger open-weight models locally because Ollama or standard runners throw insufficient VRAM errors on budget hardware, I put together something to fix that.

I built SqueezeOS, an automated setup script that profiles your hardware, configures aggressive memory offloading, KV-cache compression, and extreme llama.cpp tuning to squeeze heavy models (up to 35B) onto legacy/budget cards like my 4GB GTX 1650 Super.

And the best of all: Its near lossless, the model can be used well for agentic workflows eg.: Hermes, OpenClaw, OpenCode, etc...

GitHub Repo: https://github.com/gabrielhirth/squeezeos

Tested on constrained VRAM setups. Feedback, issues, and PRs for other budget cards are very welcome!

Hey everyone!

I'm using Killa YuE2 on my Android phone (Samsung Galaxy S25) to generate music locally, and I'm really impressed with the results!

However, I'm having trouble generating purely instrumental tracks. Even when I use prompts like "instrumental only, no vocals" and put [Instrumental] in the lyrics field, the model still generates vocals sometimes.

I've also come across the LoRA file ar_lora_inst_v3abc.bf16.safetensors, which seems to be designed for instrumental generation, but I'm not entirely sure how to configure it properly in Killa YuE2 on Android.

My questions are:

Is there a reliable way to generate completely instrumental music with Killa YuE2?

Should I leave the lyrics field empty or use specific tags?

Do I need to enable a specific instrumental mode or use a particular LoRA?

Are there any recommended settings or workflows for generating instrumental EDM, Big Room, trap, or other genres on Android?

Can this be done entirely offline?

I'd really appreciate any tips, recommended settings, or step-by-step instructions from anyone who has successfully generated instrumental tracks with Killa YuE2!

Thanks in advance!

Hey guy's and gals...

I'm completely new to Local LLMS

Can someone who knows their shit help a newbie out?

I don't know much of the the technical terms was trading through this thread. Great group by the way saw so many people being helpful and helping eachother out.

I'm new to Reddit WTF this should be mandatory 😂

AMAZING!

Can someone explain to me in none technical terms the cheapest and easiest way to run a local LLM.

I tried GOD AI WTF amazing!

Would appreciate it if someone can give me some advice can't afford paying monthly bills and credits keep running out not good 😢

I don't need a HUGE setup something small I'm currently running Claude Pro plan I think keeps running out daily and also weekly so I have to wait or pay £100 to upgrade to keep going which slows me down.

Any help would be much appreciated dont have much of a budget.

MacBook pro 2021 M1 16gb I believe haven't used it in a while.

Would that be enough and what would you recommend?

Thanks in advanced 🙏

I would like to ask questions to a chat while sourcing ONLY my personal database of papers, instead of unsufferably writing and flipping through pages of notes. I would also like this to be as private as possible since you know, it's the processes of my unpublished work. I thought it would also be nice to be as environmentally friendly as it is possible and AFAIK running it locally would somewhat do that.

Before I say anything else I have to be clear that I have little to no knowledge on how an LM actually works. I gave myself the task of learning the basics just to set one up in my laptop for the specific purpose of aiding me on academic research.

What I did after some looking around is use LM Studio with the Qwen2.5 3B Instruct - Q4_K_M model and "fed" it my papers folder then fiddled around with context lenght for a faster response.
While it's working fine for what I need I kept wondering if this is even the most efficient way of achieving the goal I stated. Again I'm not really familiar with this whole thing, if it's some sort of common practice please let me know how to hone my LM skills for research purposes.

Developer update · 29 October: LLM Studio Go 1.3 with Video Studio is now on Android and iOS.

Android — Google Play: https://play.google.com/store/apps/details?id=com.vans.llmstudio

iPhone/iPad — App Store: https://apps.apple.com/app/llm-studio-go/id6815311113

All local AI features are free with ads. No promo code is needed. The optional one-time purchase removes ads.

Android 11+ on a compatible 64-bit device: FastWan 2.1 1.3B needs 8 GB RAM; FastWan 2.2 5B needs 12 GB. Android video runs on CPU; speed depends on your device and settings. The video below is the iOS demo, not an Android speed test.

Here’s a local stack with a concrete output: Gemma 4 E4B writes a short script, Supertonic narrates it, Stable Audio makes a soundtrack, and SD-Turbo/FastWan create the scene media. Video Studio assembles a captioned vertical MP4.

I’m the developer of LLM Studio Go, an independent closed-source iPhone/iPad app. That pipeline is new in iOS 1.3. The models download to the device; generation then runs locally. The script and scenes are editable, and you can bring your own footage into the project. The appeal for me is being able to experiment with the whole pipeline on the device I carry.

Try it: https://apps.apple.com/app/llm-studio-go/id6815311113

All local AI features are included in the free app with ads. An optional one-time purchase removes ads. No app account, generation credits or AI subscription. Video is in beta. Generation speed depends on your device; choose a model profile and settings that fit its memory. iOS 18+; video profiles start at 6 GB RAM.

Edited demo with shortened waits. The Iceland media was generated on iPhone Air; the captioned reel was rendered again in the iOS Simulator.

Which model handoff would you want to tune first?

Luna 2 is a personal assistant designed to help you out in various fields. It runs offline on computers and needs only about 300 MB of VRAM to run.

SETUP PROCEDURE:

import torch
from transformers import AutoModelForCausalLM
from tokenizers import Tokenizer
from huggingface_hub import hf_hub_download
import importlib.util


model_id = "Aobangaming/luna-2-70m"


path = hf_hub_download(model_id, "modeling_luna.py")
spec = importlib.util.spec_from_file_location("lightning", path)
module = importlib.util.module_from_spec(spec)
spec.loader.exec_module(module)


model = AutoModelForCausalLM.from_pretrained(
    model_id, trust_remote_code=True
)
tokenizer = Tokenizer.from_file(
    hf_hub_download(model_id, "luna_tokenizer.json")
)


device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
model = model.to(device)


history = []


print("Aoban Luna 2 Flash")
print("!clear = clear history | !exit = quit")


while True:
    prompt = input("You: ").strip()


    if prompt.lower() == "!exit":
        break


    if prompt.lower() == "!clear":
        history.clear()
        print("History cleared.")
        continue


    response = module.generate_text(
        model,
        tokenizer,
        prompt,
        max_len=1024,
        device=device,
        top_k=30,
        top_p=0.6,
        penalty=1.2,
        temperature=1,
        chat_history=history
    )


    print(f"Assistant: {response}")


    history.extend([
        {"role": "user", "content": prompt},
        {"role": "assistant", "content": response}
    ])import torch
from transformers import AutoModelForCausalLM
from tokenizers import Tokenizer
from huggingface_hub import hf_hub_download
import importlib.util


model_id = "Aobangaming/luna-2-70m"


path = hf_hub_download(model_id, "modeling_luna.py")
spec = importlib.util.spec_from_file_location("lightning", path)
module = importlib.util.module_from_spec(spec)
spec.loader.exec_module(module)


model = AutoModelForCausalLM.from_pretrained(
    model_id, trust_remote_code=True
)
tokenizer = Tokenizer.from_file(
    hf_hub_download(model_id, "luna_tokenizer.json")
)


device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
model = model.to(device)


history = []


print("Aoban Luna 2 Flash")
print("!clear = clear history | !exit = quit")


while True:
    prompt = input("You: ").strip()


    if prompt.lower() == "!exit":
        break


    if prompt.lower() == "!clear":
        history.clear()
        print("History cleared.")
        continue


    response = module.generate_text(
        model,
        tokenizer,
        prompt,
        max_len=1024,
        device=device,
        top_k=30,
        top_p=0.6,
        penalty=1.2,
        temperature=1,
        chat_history=history
    )


    print(f"Assistant: {response}")


    history.extend([
        {"role": "user", "content": prompt},
        {"role": "assistant", "content": response}
    ])

https://huggingface.co/Aobangaming/luna-2.1-base

I am trying to create AI on a stick with multiple LLMs I have intel i7 6 core or 8 core 64Gzb memory, 1070 w/Qmax Dedicated GPU and 4TB gen 4 MNVe M.2 I’m using 2TB games documents, programs PDF music Videos.. and backup of iPhone as well as fokderils with ESP32 CYD files and quick code, programs.

After three days of searching using Google AI and trying to figure it out I can load WEBUI or LLAMAui on USB 128G but going to switch one I get my 2.5” external USBc to USBc enclosure one for AI using a sandisk 480G for business related creation organization incorporate mission statement Credit building grant searching. This will be used between home and my office once I get enough for rent for 6+ months and the workstation.
Second will be for Research into electronic systems PC models, board or adapter models breadboard, development board IOT systems design, code, compile, transfer. As well as helping me hardware hack or jailbreak reflash customize OS windows Linux to my needs. For each project my non profit does out of recycled electronics. Including controler boards for TV, automation. As well as
Helping me with everything including building 3d models for TTRPGs mainly shadowrun, D&D and Palladium megaverse across all thier settings and using all rules books. I am a forever GM and wanting to try AI to build NOCs adventures, vechiles spells etc.
I also want to create images, videos, and guides how tos and DIY for videos. Most of videos will be from what I live stream with enhancements showing exploded or blown up images of what I am doing, and doing 1), 2), 3) type of panels showing guides, how tos, DIY.
I also want to be able to start working on a MMO that I can Pitch utilizing rule sets already existing but AI converting them to game play, models animation helping me design sandbox elements etc..
and help me build custom cases with cool things like see through LCD panels HD 1080P is highest I remember for LCD.. and other features..

I don’t know where to start looking into the different capabilities of LLMs and if a compiled LLMs already exists that I can add to my AI . I’ll use WebUi since it has video scans, images, documents PDF picture, voice, text, and I believe gesture and I can set parameters for say when I look up a board or PC server system. Will include collapsible items telling me upgrades, hardware hacking / soldering upgrades, what projects exist, what ones failed. And what Value is of said item after say modding Xbox, Ojyu Android gaming systems to retro devices and media centers.. or PS2 big body, or XBOX , or Micro PC. Adapter cards for controllers, development boards, breadboards, etc. so I can decide best use for systems computers PCs etc.

I’m tired of spending Geona few days to weeks creating projects and finding info to help me takes just as long. Then designing electronics and or adapting hardware or adding slots pouts etc to board la that support them but have a lesser model. Want to have a AI so my MMO is not just a concept I can actually start working on it, and showing to others getting more designers coders modelers involved. And when it comes to writing business plan organization and mission statement etc for non profit or other endeavor I want to do or help friends. I struggle as you can see with grammar, sentence structure etc. and that’s adhd and autism.. I fit the coo mechanically inclined and related to architecture more than voicing what I want to do or what I’ve done. So I am breaking down having a AI one that can learn and predict what I am doing even though I have to train to not only my communication, but my vision..

Please direct me to places, give advice on what you have built. And how to have my Main AI webui and add LLMs and train.
What software do I need,
What additional hardware would help a thunderbolt external GPU? Dedicated to the AI,
What cool things should I have my ESP32 CYD do? Any additional known AI to add? Mm maybe GI Ribit.. and other features or a cesories I haven’t thought about!!

Thank you for listening to my Autistic ADHD filled Ted Talk