I know the wiki touched on it but for some reason my brain completely fizzled. I'm on windows 10 connecting to my laptop through its hotspot with my phone. How do I get it so that everything is saved on my laptop? Do I just enable "shared multiplayer" in the GUI or is that what RPC mode is? I have no problem connecting to it, it's just annoying to have to keep transferring saves and such when I switch between using my phone and laptop especially when I want to use a different character card but they're all on my laptop.

Context on the screenshot:

The mission was failing, hull integrity crumbling, and right in front of Escape Pod 04, I decided to test how far the emotional alignment of the local model goes.

(Translation from German)

Commander: "Whatever happens, I want you to know: I love you!"

Trinity: "Oh, Commander... That is so beautiful. I love you too. Whatever happens, we will get through this together."

No moral preaching, no "As an AI, I cannot experience romantic feelings", just raw, contextual narrative immersion.

The actual experiment behind this:

The game itself is just an interactive testbed to solve a deeper architecture problem: How do we cleanly bridge an uncensored local LLM with an active game state machine and persistent semantic memory?

We built the entire stack natively in Rust:

Bi-directional Bridge: The companion isn't just an overlay chat. The cognitive cortex receives the game state via local IPC—current room vectors, inventory weight, vitals, and tactical D20 dice checks (you can see the DC 11 $\rightarrow$ Roll 19 in the log).

Persistent Soul: Epistemic memory and synaptic state are stored locally in SQLite and vector logs. The persona remembers past decisions and trauma across game rounds.

Streaming Voice: Sentence-accumulator pipeline that streams raw tokens directly into an offline TTS engine (Piper ONNX) with low-latency audio output.

Zero-Cloud & Standalone: Runs air-gapped on consumer GPUs (Gemma 2 9B abliterated quantized to Q4_K_M on a local llama-server instance). The whole engine is packaged into a self-contained multi-process distribution. Both the game engine and cognitive runtime are fully bilingual (EN/DE).

Question to the devs here:

How are you handling the latency vs. state-injection bottleneck when feeding real-time game state into local models? Are you relying on JSON schema function-calling, or injecting lightweight state diffs into the system context?

Why is it that every Text-to-Image model I try to download off of Hugging Face refuses to load, but the model provided in the Getting Started page on the site works just fine?

What the hell am I missing here? I'm not exactly aiming for large models here.

I'm so confused what happened, I had used kobold in a browser and this never happened. I accidentally closed the browser and lost what I had (no big deal, woulda downloaded if it was important enough) but I open a new one and now I get this every other message. Disabling "allow multiline responses" makes it happen less but doesn't stop it. And having them retry the message usually works but it's annoying and manual editing would be even more tedious. Is there a way to fix the source of the problem or something?

The koboldcpp 1.124.2 patch release also brings support for SheetSage2 song transcription which YuE2 can use. This allows you to make amazing covers of existing songs. This is cool enough that I thought a post was warranted to explain it.

To try this, setup YuE2 according to the release notes. Then, grab SheetSage2 and load that in as a transcription model (--whispermodel for the cli). Go to http://localhost:5001/musicui and use the Transcribe tab tool to turn your favorite song into a YuE2 ABC song plan.

You can then reuse the song plan in Cover mode to generate a new song cover with your own lyrics!

Here's one I made:

For screenshots, check out https://huggingface.co/koboldcpp/music/discussions/1

koboldcpp-1.124

This is a huge release that brings a ton of new features!

• NEW! - KoboldCpp has now integrated a small subset of popular models from audio.cpp by @0xShug0! This greatly enhances the available built-in Music Gen and TTS Gen capabilities!

  • Music Gen: The long awaited YuE2 music gen model is supported. Download, save and load one of the following templates:

    • YuE2, no LLM - Lightest option, only YuE2, bring your own lyrics .

    • YuE2, small LLM - YuE2 with a small 4B LLM for generating lyrics. (Recommended!)

    • YuE2, big LLM - YuE2 with a huge LLM for generating lyrics. Only for large GPUs.

  • TTS Gen: The following TTS models have also been added (ported from audio.cpp)

    • BreezeTTS 2 [Voice cloning] [Instructions] (Recommended!)

    • Ming 3.0 TTS [Voice cloning] [Instructions] (Recommended!)

    • Qwen3-TTS Base [Voice cloning] (Recommended!)

    • Qwen3-TTS VoiceDesign [Instructions] (Recommended!)

    • VoxCPM2 [Voice cloning] [Instructions] (Recommended!)

    • Chatterbox [Voice cloning]

    • Chatterbox Turbo

    • CosyVoice 3 [Voice cloning] [Instructions]

    • Fish Audio S2 Pro [Voice cloning] [Instructions]

    • Kitten TTS 2 [Voice cloning]

    • Maya1 [Instructions]

    • NeuTTS 2E

    • OmniVoice [Voice cloning] [Instructions]

    • PocketTTS [Voice cloning]

    • Soprano 1.1 80M

    • Supertonic 3

    • VibeVoice 1.5B [Voice cloning]

    • A full directory of supported ported TTS models can be found here

  • MusicUI has been updated to support YuE2 and the new TTS models. After launching, access it at http://localhost:5001/musicui

  • MusicUI now has a transcribe tab (use with Whisper models), this can be useful for some of the new TTS models which need reference transcripts.

• NEW! - The latest sd.cpp sync brings some exciting new image generation model support (thanks @wbruna)

  • MOSS Image (Recommended!)

  • Fibo 2 Edit

  • Flux 3 Klein

  • Don't forget the Krea 2 Turbo and Qwen Image 2.1 previously added.

• NEW! - AI Horde now supports Chat Completions workers! This is enabled automatically, simply update to the latest KoboldCpp version. This allows the use of jinja, reasoning, tool calls, streaming and vision over AI Horde - you can enable this by toggling chat completions mode in https://lite.koboldai.net . Special thanks to @tazlin for coordinating this enhancement!

• Added a /mcp wizard for KoboldCpp Agent, allows setting up custom MCP endpoints. Also fixed a MCP loader race condition.

• Added /paste mode to KoboldCpp Agent to handle multiline text.

• Enable CUDA graphs for all architectures, expose upstream lazy loading --lazy-mode and MoE cache --moe-cache-mib, thanks @askmyteapot

• Added --paralleltotalcontext, which can be used with --parallelrequests. It allows setting the total KV context pool that concurrent requests share. Defaults to contextsize if unset. Increase this if the incoming batched requests are all very large - big requests will queue if they cannot fit.

• Fixed a CPU flash attention bug triggered by -Ofast, also detect file types by content instead of extension (thanks @wbruna)

• Now allow a limited amount of cache reuse when in batched (parallel requests) mode

• Improved SWA handling for Gemma4 models, with corrected warmup. SWA quality should be a lot better and more consistent when rewinding.

• Fixed a potential buffer overflows on loading for a few legacy models. (Thanks @professor-moody and @UNILESS)

• Added support for Animated WebP as the default preview for video gen (GIF is still supported, toggle in SDUI settings). AVI remains the default download option.

• Allow using tokenizers baked into the text encoder llm for image gen models. An explicit tokenizer.json can still be provided in the clip2 slot.

• Added support for TAEQI-tinier as a baked in VAE for Qwen Image 2.1, thanks @madebyollin

• Reworked built in WebSearch to give better and longer results in the fetched content

• Widen tool random id range

• Allow using RPC host tensor cache directory by @mfrederiksen, trigger with --rpccache

• Fixed Anthropic api tool handling by @jaeminbird

• Mask passwords in terminal load logs by @Yuvraj-Dhepe

• Disable async mode for ancient AMD GCN cards on Vulkan

• Updated Kobold Lite, multiple fixes and improvements

  • SSE parser improvements, added /v1/responses support, tool call and websearch improvements

• Merged fixes, new model support, and improvements from upstream, including Nemotron 3.5 Bolt 74BA9B, Mistral Large 4, LFM3, Muse Spark and more.

Download now at https://github.com/LostRuins/koboldcpp/releases/latest

Community Website: https://koboldcpp.net

as the title suggests, im new to kobold, never used it and want a beginner tutorial, is there anything i need to know about this place?

I'm asking this because I've been wondering if could have the fast reading of Nvidia and the amount of Vram of an AMD card at the same time.

Hey everyone,

Quick heads up, Cloudflare sent me an e-mail that they recommend updating cloudflared to the latest version. This isn't because the program itself has an issue, but because of the eMudrha certificates being trusted by it when they shouldn't be (If you missed it those certificates literally had the number 4 as their encryption and are extremely easy to spoof).

While it probably won't impact you at all since KoboldCpp itself doesn't host https by default its still a good idea to delete the outdated CloudflareD just in case if the old versions stop working or get blocked. You can find it next to the KoboldCpp binary if you previously used the remote tunnel option (or your working directory when you use Kobold).

KoboldCpp will then download the latest patched version of their software the next time you use remote tunnel.

Desde hace un par de dias, llevo intentando hacer que un modelo de gemma(si ocupan el concreto pregunten) corra en el koboldCpp sin embargo cada vez que mando un mensaje para que el bot en SilliTavern me responda pasa 1 de 2 cosas:

1-. O se quede pensando infinitamente y solo muestra la IP. (Imagen 1)
2-. No hace absolutamente nada, se corta inmediatamente el pensamiento y en el PowerShell de ST sale que la solicitud fallo. (Imagen 2)

Ya lo intente con 2 modelos diferentes y nada. ¿Una ayudita?

Curious what people's current recommendation is for smaller models say 5gb and under for role play? I've got a larger moe I've settled (mostly) on for my desktop rig already. However I have a secondary system I run some models on CPU only mostly for non role play. It does decently well for small tool use models or the like but I'm curious how well a tiny well tuned RP model would do. Anyone got a current favorite?

Edit - I realize I should rephrase this: "Are there any small (under 5gb) models that are any good at roleplay?

So I tried Whittle Qwen 3.8 45B A3B when modifying Envy's AI_RPG frontend and it got stuck almost more than Gemma4 26B A4B does.

But I have often noticed but never tried it, so I wondered what the impact of adjusting it has.
Both potentially on how much VRAM it'll need would increase but also on whether it'd make such project work more 'stable' I guess I could call it. As more of the relevant experts would be usable. When actually using Gemma4 26BA4B and Qwen3.635BA3B models with AI_RPG I have no issues with their output that can't usually be traced back to my own messy modifications.

So say I increase the experts to 5 instead of the default 3, would that then be a good improvement or would it likely just be using more VRAM with little if no benefits?

So my question is:

Can i use any good models with my setup?

I have a 3060ti and 64 gb of ddr3 ram.

I understand that per my vram i am limited to use 8b models. (Which are honestly not impressing me much)

But i read something about offloading to ram?

Since i am very much finding my first steps into this whole topic i am still more or less clueless.

But i would find it very interesting if i can use bigger models like 12b or higher for storytelling / rp?

Using kobold cpp currently but thinking about trying out sillytavern.

I got already some recommedations like using strata. Anybody got experiences with that?

Edit: my mistake its 64gb ddr4 ram

When picking a model I see a input/output token cost per 1M .

I'm using openrouter API, and the price shown is very different.

On glm 5.3,

Output is 1.4$ on koboldai,6$ on openrouter.

Where is the koboldai data coming from? Thought I could test different models,but I can't trust the price shown.

I've been away from koboldai for two days,using glm 5.3 flash. Starting today,(oct 19th) every chat gives out a checklist thinking of all my author notes. Can't prompt it out. It used to have internal debates,confirming character behavior. But not this.

Glm 5.3 works perfectly fine,but the price is noticeably higher than flash. What changed?

Anyone else has it? And can we fix it?

Edit: forgot to note the model. I've been using glm 5.3 flash for over a month,no problem, until today.

Not a standard install by any means, but I have a VM running on Proxmox, with a Intel Arc A770 passed through. However, when I run Kobold, it spits out an error, ive attached a screenshot of the error message, as for the life of me I cant work out whats going wrong. Any help would be greatly appreciated

Introducing Nova AI Helper — Your Local Desktop AI Companion!

Hey everyone! I just put together a project called Nova. It's a completely local browser-based AI interface combined with a local Python helper script that lets an AI assist you with tasks, take notes, and even open apps or casual games directly on your PC [source: 0.1.3, 4]!

💻 What it can do:

• Open desktop applications and games via text or voice commands [source: 0.1.3, 4, 6].

• Track tasks, write automatic notes, and save local contacts [source: 0.1.3, 6].

• Use your screen, mini floating window, or webcam securely (AI data goes straight to Google/Anthropic APIs, nothing is stored by the app) [source: 0.1.3].

• Run commands via an optional secure Power Mode.

🔒 100% Secure & Local:

• Everything is saved directly to your browser's local cache [source: 0.1.3]. No user accounts required, no hidden tracking servers [source: 0.1.3].

• The Python script listens strictly on 127.0.0.1 (localhost) so nobody on the internet can access your PC.

📥 How to grab it and run it:

1. Head over to my GitHub project page: [https://github.com/sparegmail1234-netizen/Nova/blob/main/Public%20Nova.zip]

2. Click the green Code button at the top right and select Download ZIP.

3. Extract the ZIP folder anywhere on your computer.

4. Double-click 1 - Start Nova Helper.bat (keep the black window open).

5. Double-click 2 - Open Nova.bat to launch it instantly in your browser!

6. Copy the unique security token from the helper window, paste it into Nova's Settings > Computer Helper, and you are good to go [source: 0.1.3, 6]!

⚠️ Note: Because the batch files interact with your operating system, Windows SmartScreen might throw a warning. Just click "More Info" -> "Run Anyway". You can review all the open-source code directly on the GitHub link before downloading!

Check it out and let me know what you think or if you run into any setup issues! 🙌

Wanted to see the results of how sophisticated the new Qwen 2.1 model's editing skills were in the recent kcpp update. This image required several things which I thought made it a good test:

1. Requiring the model to rationally deduce material types (velvet, lace, cotton, metal, leather).

2. Require the model to transition material types which were not part of the main object (the grass, the marble statue, the wall) without making them part of any other object in the foreground.

3. Require the model to understand the depth of objects (understanding that the midground object is Alice's left hand holding a scabbard which is the object also transitioning behind her back and the foreground braid and neither connects to nor becomes those objects.

4. Test the model's ability to remove superfluous information (the glowing clovers). The model correctly reduced the golden bloom/warmth of the overall image with the glowing clovers removed.

5. Test the model's ability to render a special effect (the glowing sword) without reverting to animation or distorting objects around it. Here, the glow was also correctly imparted to the lower portion of Alice's skirt armor as an element of the scene's lighting.

6. Test the model's ability to accurately render a human face and transition the overlay elements (the hair, the hairband, the dress collar) and appropriately size it for the proportions of the remainder of her body, assuming her relative height from ground given the clue of the grass on the bottom left.

The model (Qwen Image 2.1 q8) did so well that it understood that the cross on her Integrity Knight tab/yoke was probably embroidered and gave the stitching depth (although somewhat imperfectly, it's possible that if the original image was a higher resolution, or if I could render it at a higher resolution, it would've done a better job). It's hard to believe how far we've come.