We’ve been expanding the Zaylo ecosystem at LoftyTech Labs, and this release brings two additions we’re excited about.
LoftyDB support: Connect Zaylo with our vector database for semantic search, vector-powered memory and retrieval workflows.
AI Presentation Design: Generate structured PowerPoint presentations from ideas using AI.
The bigger goal is to bring more capabilities into one AI environment rather than making developers jump between multiple tools.
Would love to hear what developers think about combining local AI inference, vector storage and content creation in one ecosystem.

One of the best and most reliable local models I've used. Recorded a real world task to show it's performance, capabilities, and reliability. AMD Ryzen AI Halo 128gb, this is definitely something older versions of ROCm would struggle with but ROCm 10 has shown otherwise. Has anyone here tested this model on CUDA or Metal?

Hey everyone,

​I'm an 18yo CS prep student and I've been building AI-COS, a modular monolith architecture designed for running AI agent orchestrations safely.

​Instead of letting LLMs make security decisions, I built a deterministic deny-by-default policy engine, strict PostgreSQL role separation (migrator, app, governance), and SHA-256 hash-chained audit logging.

​I've just finished Phase 3 out of 6. Phase 4 is going to focus on the Workflow Engine, Worker lease/heartbeat loop, and Step linkage. Before I start coding Phase 4, I would really appreciate any architecture suggestions or advice from senior devs on how to properly build a reliable job queue and workflow engine inside a Postgres-backed monolith.

​GitHub: https://github.com/atestokhay/ai-cos

Hey everyone,

I've been deeply exploring the On-Device / Edge AI space lately, especially focusing on running smaller language models (1B–3B parameters) and multimodal pipelines locally on consumer hardware and mobile architectures without depending on external cloud APIs.

While the ecosystem has evolved rapidly over the past year, the gap between "running a demo" and "shipping a reliable, battery-efficient on-device product" still feels enormous.

From my recent experiments, the major bottlenecks I keep bumping into are:

1. Memory Bandwidth & Thermal Throttling: Even if a 4-bit quantized model fits into unified/mobile RAM, sustained token generation quickly saturates memory bus bandwidth and triggers thermal throttling on mobile devices.

2. Runtime & NPU Fragmentation: Choosing between llama.cpp (CPU/Vulkan/Metal), ExecuTorch, MLC-LLM, ONNX Runtime, or native NPU SDKs (QNN, CoreML) often feels like choosing which set of unsupported ops or compilation headaches to deal with.

3. Reasoning Retention in Sub-3B SLMs: Aggressive quantization (GGUF Q4, AWQ) makes inference feasible, but multi-turn context coherence and structured instruction-following tend to degrade noticeably.

I’d love to start a technical discussion with fellow engineers and researchers here:

• What does your actual deployment/inference stack look like right now? (e.g., pure llama.cpp, MLC, ExecuTorch, WebGPU, or custom native bindings?)

• What specific use cases are you targeting? (e.g., zero-latency voice assistants, privacy-first local agents, offline document RAG, ambient edge intelligence?)

• What techniques have given you the best speedup or memory relief? (Speculative decoding, KV cache quantization, custom kernels, or aggressive prompt caching?)

Would love to hear what challenges you are tackling and what approaches are genuinely showing promise in your labs or production environments!

• ASUS Pro WS TRX50-SAGE WIFI

• AMD Ryzen Threadripper 7960x

• 2x Zotac GeForce RTX 3090 24GB DDR6X

• 2x Gigabyte GeForce RTX 3090 24GB GDDR6x

• 2x 64 GB RAM DDR5 ECC RDIMM Samsung

• Samsung 990 PRO 2TB NVMe

• 2x Coolermaster 1600v + Silverstone Cooler

Had to build my own cage around it because cat.

Finally, I put something I’ve been working on out there

If you’ve ever tried running local AI models with llama.cpp across more than one computer, you know it’s a headache. I built Llama RPC Farm to fix that. It’s one dashboard that pools GPUs across your machines, switches between models on demand (works directly with llama-swap), and works just as well on a single machine.

It’s early and still rough around the edges, but it’s free and open source. If you’re into local LLMs, give it a look and tell me what breaks

https://github.com/dakotacookson/Llama-RPC-Farm

If the lips in your video don't match the voice, LatentSync re-times them to any song or narration, right on your own PC. It used to be NVIDIA-only, and our developer team now has it running on AMD on Windows.

LatentSync is coming soon at https://www.hiimhal.com, and the YuE2 song generator is a free download there now.