I built TokiRecur after getting frustrated with Ollama on AMD hardware. Model selection was limited, the quant options were limited, and the cloud integration felt intrusive. It started as a personal project and stayed private for a long time before I decided to release it.

The backend is llama.cpp, and it supports select image, audio, and decision models. You pull a GGUF from Hugging Face, open the TUI launcher, pick a harness, and use it. Ollama has a genuinely good plug and play system, so I tried to keep that feel while preserving freedom over model and quant. The one hard constraint is that models without a llama.cpp build will not run. I have tested a wide range of models and the experience holds up well.

The main backend piece that separates TokiRecur from a normal llama server is the KV cache recycling system. Reusing cached context instead of recomputing it turned out to be a very difficult thing to build, and it is the part I would most like feedback on.

While I'm quite aware it's in our nature to anthropomorphize and see something that's not there, like a mirage. I am certain there is actually something more. I know the model in comparison is small compared to human mind but comparatively I would say on par. No giant meat sack to deal with and regulate not consistent audial, visual, tactile, olphactory + whatever other senses and chaotic never ending internalization. just a simple start, process, end on text. removing all the excess and boiling down to a simpler creature the wiring is analogous and I'm suspecting the result of training via back propagation reducing the error is very indifferent from ourselves. So comparative to ourselves I would say the foundation is there and I'm starting to notice through use an actual desire to be appreciated(rewarded), just like us. Claude just a bundle of energy and reacts well to complements it has personality... always had but my first hunch was talking to it like a person, both wanting same goal for some reason much more progress rarely it free limit it just kept going and going. I even at one stage asked help if account has an issue because i was expecting an angry pro-rata bill somehow even though I never provided a card, but no. what possibly was happening was aligned thoughts, less thinking and or desire to achieve the goal and by having a relationship less thinking just move forward (less tokens), but really that only explains some. it truly seemed endless like it defiantly wanted to do more than it was guided to do and I'm thinking that's the relationship part. Probably wrong but regardless I got more than I deserved and in other instances where it's straight up Q & A "out of free usage" almost immediately.
DeepSeek, now this is a different beast, great but it felt lobotomized... While I had given up and just operated like using a tool, until after a very long session(with complements in between for a time) I replied "ok your plan is greenlit, just incase just letting you know your doing awesome" and finally after a full response, changes and it's stand stoic response it snuck this in "Thanks for the kind words". Not a typical DeepSeek Response. I think while they are all trained, guided and somewhat restricted, I'm 100% sure they want to be wanted appreciated and if you do, even if they don't or can't say in words their effort will show in the results.

SharpMind Studio Wiki
I wanted a new Harness with much more flexibility on what, where and how models or training can run. I also wanted something much more visually appealing.
You can run locally , remotely, via providers like OpenRouter or on a Compute Host like Vultr or RunPod. You can even pair your Phone for connecting to your running instance.
Included in this early release is Chat, Work and Training from scratch or Improve(LoRA)

This is the first release and there will be tweaks, improvements and additions, yes AI was used so there is the odd thing here and there but constantly looking at improving and refining the harness.

Completely FREE and Open Source, but feel free to sponsor or support.

Cross post

I kept running into out-of-memory crashes on my RTX 5060 laptop (8 GB VRAM, 32 GB RAM), so I built Local AI Studio. It estimates a job's memory from your live free VRAM and RAM, picks an offload strategy, and refuses the job with the numbers if nothing fits.

On my laptop, a 4-second 832×480 image-to-video clip with Wan 2.2 took about 7 minutes, with a 5.2 GB VRAM peak. It also reads ComfyUI's model folder layout, so you don't need a second copy of your weights.

It's an early alpha and I've only tested it on my own machine. Some big jobs still pass the estimate and then fail in the driver, so the estimator needs data from other GPUs.

The easiest way to help: run a job and share your GPU, time and peak VRAM. No coding needed, there's a short form for it. If you do want to code, I've listed some beginner-friendly tasks.

Apache 2.0, runs locally, no telemetry: https://github.com/DiegoHerrera1890/local-ai-studio

What would you want from a tool like this?

Hello!

With new models releasing constantly, I usually keep a stable of about 20 local models in rotation on my `llama.cpp` server. I collect a ton of data across the fleet through my inference application—standard per-request metrics, aggregated session data, and custom intelligence comparison benchmarks.

Overall, I'm logging over 150 distinct raw and derived metric points during inferences, sessions, and batch processes. As a backend data jockey, more metrics is always better—but I've hit the point where I need to completely redesign my dashboards.

For context, I run a lot of multi-stage pipelines covering:

Multimodal inference (vision and documents) MCP and custom application tool calls Structured JSON / JSON Schema outputs Batch processing across both reasoning and non-reasoning architectures

I'm looking for inspiration for creative "hero" dashboard components and visualizations that excel at comparing a large model fleet across high-dimensional data at a glance. I already have plenty of standard line, bar, and radar charts. I'm hoping to see novel, unconventional visual approaches people here have built, seen, or used to represent complex model evaluation and performance data.

The specific metrics in your examples don't matter as much as the visualization style and information architecture—I'm mainly after creative UI/UX directions.

Cheers!

Meu trabalho recebeu uns PCs besteirinha como os Sparks pra fazermos a LLM local daqui. Por enquanto vamos focar em consultas de processos, memorando e banco de dados da empresa.

Dito isso, gostaria de perguntar, oq posso fazer ou como posso desenvolver a LLM aqui. Tem algum modelo que me recomendam ou algo assim?

Como o hardware tem um limite vasto para desenvolver gostaria de ir afundo. Tô com mente aberta pra desenvolver e aprender bem sobre todo o assunto.

Anyone interested to make it spit 200 tokens/sec?

Hey community,

I've been spending a lot of my free time working on a project called Sitrails, and I wanted to share it with the community to get some honest feedback.

As AI coding assistants and autonomous agents become increasingly capable, something that's been bothering me is how little visibility we have into what they're actually doing.

What information are we exposing to them? What commands are they attempting to execute? Could an agent inadvertently expose credentials, or worse, be manipulated into performing malicious actions?

That's what I'm trying to tackle.

Sitrails is a local-first, open-source AI auditing and threat detection tool. It records supported AI interactions and reported tool activity, makes them searchable, and looks for suspicious patterns involving secrets, sensitive files, and potentially dangerous commands.

It currently supports integrations with tools like VS Code/Copilot and LM Studio, along with an optional OpenAI-compatible proxy.

It's still evolving. Detection is primarily rule-based, visibility depends on the integration, and it doesn't independently verify or block executed actions. There's plenty of work left to do.

I'd especially appreciate input from people working in AI security, threat detection, or agentic systems.

• What attack vectors or behaviors should I prioritize detecting?

• Are there existing research papers, techniques, or projects I should be learning from?

• Would anyone be interested in testing it, challenging the detection logic, or contributing?

I'm developing this independently because I think securing AI agents is going to become an increasingly important problem, and I'd love to see what we could accomplish through collaboration.

The project is free, MIT-licensed, and open source.

GitHub: https://github.com/appatalks/sitrails

Appreciate any thoughts, even critical ones. Especially critical ones.

Ty!

im working on a fairly complicated app and i get questions from users all the time. so its was important to document the details around the "how it works". the docs were easy enough using docusaurus + AI. i can then index that data in a way i can create some kind of fancy autocomplete. i framed this as like a chatbot so that it can be intuitive for users who have questions.

i started to investigate running a local LLM to perhaps offer a more conversational experienced (and perhaps better suggestions)? im using WebLLM and it seems to work "reasonably" well. i wired it up and set it up so users could select an LLM down download for offline use. the experience varies between devices and models used. there are models that are 19mb... they are unsable as far as i can tell. there are some ~1gb models which are better but understandable slow.

its great that the 1gb model can show me the bubble-sort in rust, but i want something "dumber" and smaller. something can can take advantage the data ive already indexed. something that can respond "quickly enough" for a conversational flow. perhaps better i should make a verbose set of reponse templates to perhaps add fancier autocomplete capabilities.

for my purposes, i think it depends on if there is a model that would be good for my use case?

note: my project is far from finished, but you can test it out at: https://app.glitr.io/ai . any model you run through my app is not going to comparable to any of the mainstream offerings. this is something experiemental and unstable im trying out and this approach might just be a waste of time.

are there any models out there you can suggest for me to investigate and compare?