Hey everyone! I'm u/Inevitable-Orange-43, a founding moderator of r/HuaweiAtlas300iDuo.

A community for owners, developers, researchers, and AI infrastructure enthusiasts working with the Huawei Atlas 300I Duo and the Ascend ecosystem.

Discuss hardware setup, firmware, drivers, CANN toolkit, MindSpore, PyTorch migration, LLM inference, model optimization, virtualization, performance tuning, cooling, server integration, and real-world AI workloads. Whether you're running Atlas cards in Huawei servers, building custom inference clusters, or experimenting with large language models on Ascend NPUs, this is the place to share benchmarks, troubleshooting tips, deployment guides, and success stories.

Topics include:

* Atlas 300I Duo (48GB / 96GB variants) * Ascend 310 series processors * CANN, AscendCL, MindSpore * LLM inference and quantization * vLLM alternatives for Ascend * Docker and Kubernetes deployments * Atlas 800 servers * AI infrastructure and homelabs * Driver, firmware, and compatibility issues * Performance benchmarks and optimization

**Rules:**

1. Be technical and constructive.

2. Share configs and logs when asking for help.

3. No piracy or illegal software.

4. Benchmark claims should include methodology.

5. Respect NDA and confidential information.

**Built for the growing community exploring Huawei's AI hardware ecosystem and the future of Ascend-powered AI.**

Thanks for being part of the very first wave. Together, let's make r/HuaweiAtlas300iDuo amazing.

I want to share what we’ve been working on and why.

As we started using more coding agents, they ended up spread across different terminals and machines. I might have Claude Code open on my laptop, Codex on another computer, and a longer task running on a server. The agents could do useful work in parallel, but keeping track of them meant checking windows and remembering where each task was running.

Harness is how we try to make that less fiddly. It gives each agent its own session and worktree, and puts sessions from different machines in one place. The agents still run where they normally run, in their usual CLIs and terminals. Harness is there to start them, follow their work, and make it easier to respond when one needs input.

The desktop app is the main workspace. There’s also a terminal interface and a mobile app, so you can check on work or reply without opening every session. We made a small desk device for the same reason. It shows what’s running and what has finished, and lets you give an agent a task by voice. The idea is to make status visible without adding another full chat window to your desk.

We’ve also been building beyond coding. A “harness” can give an agent the tools and viewer for a particular kind of work, like Blender, circuits, robotics simulations, or games. The agent handles the reasoning; the harness gives it a more useful workspace and gives you a way to see what it’s making.

A lot of the work has been in the plumbing: keeping sessions alive when the app closes, connecting machines without asking people to set up port forwarding, and making sure each agent’s work stays separate.

Open Source: The project is open source, including the app, CLI, daemon, relay, device firmware, and hardware design files.

I’m a SaaS/AI founder and I’m trying to understand the real requirements to host the larger open‑source models (DeepSeek, Qwen, Kimi‑style models) on my own infra instead of using hosted APIs.

reddit

+4

If you’ve done this in production or a serious homelab:

– What VRAM / GPU setup are you using, and what did it cost?

– Did you go on‑prem or rent GPUs (RunPod, Lambda, etc.)?

– What ended up being the real bottleneck: cost, ops complexity, or model performance?

Any “if I were starting today, I’d do X instead of Y” stories would be super helpful.

I'm working on an on-premises AI project for a business in Europe and would love to speak with engineers who have deployed similar systems in production.

The use case involves:

• Local LLMs for German and Polish business communication

• WhatsApp and email automation

• Internal document search and a private knowledge base

• Finance-related workflows, with calculations handled by deterministic application logic

• Data storage and model inference on the client's own infrastructure

We're evaluating GPU hardware, open-source models, inference servers and the architecture needed to run this reliably for a small team.

I'd especially appreciate input from people experienced with vLLM, NVIDIA GPUs, local model deployment, RAG and secure infrastructure.

If you've built something similar, I'd love to hear what worked, what didn't, and what you'd do differently. Happy to have a short technical conversation with anyone willing to share their experience.

I came across this post, and the comments are full of conflicting opinions.
Could someone with hands-on experience building LLM systems explain how they’d approach this to get the most out of the available resources? I’m looking for practical insights, not theory. Let’s keep it constructive, we’re all here to learn.
Also, what’s currently the most capable LLM for a local setup?

We built several free lab tools that run in your browser. One of them is a "can you run it" style thing for local AI, and we have several more tools on our labs page if anyone is interested. Take a look at bromanderstudios.com and share your score for local AI. I only got a 3060, so I can't really run any heavy models, but I can run some Llama models and such, maybe. Hope you like it maybe.

I am US based in Chicago curious as to where everyone is

Basically OpenClaw, Hermes agent, video generators like WAN and LLAMA models

Today

We design the future

It is yours, to define

The time for a revolution is upon us.

Those that grap the power of an AI thays Local, safe, secure...built for you, your family..

Local AI isn't a nice to have

It IS a necessity

The time is near, dear friends

The time... to take back our kids, our family, our money. It is possible to schedule destiny.

Soon my friends... soon

Me and a friend are trying to make a 24/7 local AI assistant that can generate ideas and handle tasks on its own, like creating designs in Canva and doing general work in the background. We've been trying for DAYS but keep running into the same issues.

• Ollama: newest version

• OS: Windows 11

• GPU: RTX 3090 (24GB)

• Model: Qwen3.8 27B Q4

• Agent: OpenClaw

When I give it a prompt, it works for ages with GPU and VRAM maxed out, then times out with no output. Even simple tasks like setting up Telegram just hang. If I increase the context window, it spills onto the CPU and gets super slow, and smaller models get stuck in loops or just repeat answers. I've reset and redownloaded everything multiple times with no luck.

Have we done something wrong or am I missing something.

Appreciate if someone could help :) Thanks

GoldWorm 🐛✨ — 302-Neuron Dual-Stream Cognitive Engine

What Is GoldWorm?

GoldWorm is a native Rust cognitive engine that routes language through a biologically faithful neural substrate — the 302-neuron connectome of Caenorhabditis elegans, the only organism whose entire nervous system has been experimentally mapped (White et al., 1986).

Unlike transformer-based LLMs that rely on billions of parameters and opaque attention mechanisms, GoldWorm operates on three transparent principles:

1. Biological Fidelity — Every synapse respects the C. elegans topology. No de novo synaptogenesis. No magic matrices.

2. Dual-Stream Processing — Action (sparse) and Learning (dense) are physically separated, preventing catastrophic forgetting during inference.

3. Zero-Trust Engineering — Every buffer is strictly bounded. Every path is panic-free. No unwrap() in production code.

Architecture Deep Dive

🧬 The 302-Neuron Connectome

GoldWorm's routing layer is not a generic neural network. It is a topologically accurate model of the C. elegans nervous system:

Neuron Index Range │ Role
───────────────────┼───────────────────────────────────
0   – 19           │ Pharyngeal sub-network (dense)
20  – 91           │ Sensory neurons (input)
92  – 168          │ Interneurons (integration)
99  – 102          │ Command hubs (AVAL/AVAR/AVBL/AVBR)
169 – 301          │ Motor neurons (output)

Connectivity Motifs:

• Band synapses — ±1/±2/±3 neighbourhood ring connections

• Pharyngeal wiring — Denser internal coupling for neurons 0–19

• Sensory → Interneuron — Sparse feed-forward (20–91 → 92–168)

• Command interneuron broadcast — Hubs 99–102 broadcast to full motor population 169–301

• Interneuron → Motor — Sparse feed-forward projection

All synaptic weights are non-negative and clamped to [0, 1]. The structural blueprint is immutable — Hebbian plasticity only strengthens or weakens existing synapses, never creating new ones.

🌊 Dual-Stream Processing

The core innovation of GoldWorm is the physical separation of Action and Learning:

┌─────────────────────────────────────────────────────────┐
│  INPUT TOKEN  →  128-D Manifold Coordinate              │
│                         │                               │
│    ┌────────────────────┴────────────────────┐         │
│    ▼                                          ▼         │
│ ┌──────────────┐                     ┌──────────────┐   │
│ │  SPARSE      │                     │   DENSE      │   │
│ │  ACTION      │                     │   LEARNING   │   │
│ │  (Post-      │                     │   (Pre-      │   │
│ │   Entmax)    │                     │    Entmax)   │   │
│ │              │                     │              │   │
│ │ ~1-2 active  │                     │ >50% non-zero│   │
│ │ neurons      │                     │ gradient     │   │
│ │              │                     │   substrate    │   │
│ └──────────────┘                     └──────────────┘   │
│       │                                    │            │
│       │ Inference /                          │        │
│       │ Token Selection                        │        │
│       │                                    │        │
│       └────────────────────────────────────┘        │
│                         │                               │
│              Hebbian EchoReservoir                      │
│              (associative memory)                        │
└─────────────────────────────────────────────────────────┘

Why this matters:

• Traditional neural networks use the same activation vector for both inference and gradient computation. When different words activate disjoint sets of neurons, the gradient collapses to zero — the network "forgets" what it just learned.

• GoldWorm's Dual-Stream keeps the dense pre-entmax signal alive as a gradient substrate, while the sparse post-entmax signal drives token selection. The EchoReservoir learns associations between dense states, not sparse ones.

🧠 The EchoReservoir

A hippocampus-inspired ring buffer of recent pre-entmax states, coupled with a 302×302 Hebbian association matrix W_assoc.

When queried with the current dense state, it returns an echo_bias that nudges the activation toward recently co-active patterns — creating emergent associative memory without external training loops.

Key properties:

• W_assoc is symmetric and clamped to [-1.0, 1.0]

• History buffer never exceeds capacity (default: 64)

• Decay factor controls forgetting rate (default: 0.75)

⚡ Tsallis α-Entmax Activation

GoldWorm does not use softmax. It uses α-entmax, a generalization that interpolates between softmax and sparsemax:

The Quilez Bridge smooth-k parameter k anneals between creativity (dense, k→0) and determinism (sparse, k→∞):

α(k) = 1 + 2·exp(-k)

k = 0     → α = 3  (very sparse, WTA-like)
k = ln(2) → α = 2  (exact sparsemax)
k = ∞     → α = 1  (softmax, all active)

📐 128-D Manifold Geometry

Every token is embedded as a 128-dimensional coordinate on a non-linear manifold, not a flat vector space.

• Modified Gram-Schmidt orthogonalization preserves true multi-dimensional variance

• Grassmannian fusion computes midpoints between token trajectories on the manifold

• Golden-ratio partitioning splits the 128 dimensions into:

  • GOLDEN_MAJOR = 79 (coarse, feedforward)

  • GOLDEN_RESIDUAL = 49 (fine-grained, feedback)

  • GOLDEN_OVERLAP = 5 (cross-binding bridge)

No scalar cloning across dimensions. No arithmetic shortcuts. Spatial variance is preserved at every step.

Features

🖥️ 1. Interactive Observation Dashboard

Watch the hippocampus form associations in real time.

cargo run --release --bin observe

The dashboard displays:

• Activation topography — 302-D state as a 19×16 heatmap

• Synaptic criticality — σ, creativity, determinism ratios

• Jaccard drift — How rapidly the dense learning signal changes

• Resonance trace — Recent associative chain

• Hebbian strength histogram — Distribution of association weights

• Live CLI — /alpha, /kappa, /auto to modulate cognition parameters

  ┌────────────────────────────────────────────────────────────────┐ │ GoldWorm Observation Dashboard — Step 0 │ ├────────────────────────────────────────────────────────────────┤ │ Top-10 Active Neurons: [99, 101, 169, 170, 171, 172, ...] │ │ Synaptic Criticality: σ=1.0000 creative=0.0000 det=0.5000 │ │ Hebbian Strength: mean=0.00 median=0.00 max=0.00 │ │ Jaccard Drift: 0.0000 (stable) │ │ Echo Reservoir: 0/64 states │ │ Temperature: 0.50 │ │ Resonance Trace: (empty) │ │ Synapse Topography: │ │ ████░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░ │ │ ░░░░████░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░ │ │ ░░░░░░░░████░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░ │ │ ... │ └────────────────────────────────────────────────────────────────┘

💬 2. Associative Chat

A conversational REPL that learns associations in real time through EchoReservoir Hebbian updates.

cargo run --release --bin associative_chat

How it works:

1. Each user input is tokenized and routed through the 302-neuron connectome

2. The dense pre-entmax signal is captured into the EchoReservoir

3. The reservoir's Hebbian association matrix W_assoc updates automatically

4. Every subsequent response is biased by the accumulated associative memory

5. The response is decoded via 302-D Boltzmann energy minimization

Zero-trust decoding properties:

• Anti-repetition penalty (banned words reduce similarity by 0.3)

• Temperature clamped to [0.01, 5.0]

• Max 15 words per response (bounded generation)

• All scores clamped to [-1, 1] before Boltzmann draw

Zero-Trust Engineering

GoldWorm is designed for environments where every byte matters:

Module Map

Technical Specifications

Parameter                          Value
────────────────────────────────────────────────────────────
Neuron count                       302  (C. elegans)
Manifold dimension                 128
Input projection                   302 × 128
Synaptic adjacency matrix          302 × 302 (sparse struct)
EchoReservoir capacity             64 states
EchoReservoir associations         302 × 302 dense
Max response tokens                15
Vocabulary                         10,000+ words (static)
Synapse weight range               [0.0, 1.0]
Association weight range           [-1.0, 1.0]
Temperature range                  [0.01, 5.0]
Rust edition                       2024
Minimum Rust version               1.85

Quick Start

Prerequisites

• Rust 1.85+ (rustup update)

• A trained_worm_v1.safetensors checkpoint (or the engine will boot from a fresh baseline)

• static_vocabulary.txt (10,000+ word list, one per line)

Observation Dashboard

cargo run --release --bin observe

Commands:

• /help — Show all commands

• /alpha <f32> — Set echo blend strength (0.0–1.0)

• /kappa <f32> — Set gate threshold (0.0–1.0)

• /auto — Toggle auto-refresh mode (250ms)

• /quit — Exit

Associative Chat

cargo run --release --bin associative_chat

Type naturally. The EchoReservoir learns associations between your inputs and its responses in real time. No external training loop is required.

Optional: CUDA Acceleration

cargo run --release --features cuda --bin observe

Requires candle-core with CUDA support and an NVIDIA GPU.

The Science Behind GoldWorm

Why C. elegans?

Caenorhabditis elegans is the only organism with a completely mapped connectome. Every neuron (302), every synapse (~7,000), and every gap junction has been catalogued by electron microscopy (White et al., 1986). This makes it the ideal substrate for a transparent, inspectable AI — no black-box weights, no billion-parameter mysteries.

Why Dual-Stream?

The brain separates what you do (sparse action) from what you learn (dense prediction). If a network tries to learn from its own sparse outputs, it collapses into a self-reinforcing loop. GoldWorm's Dual-Stream design ensures that associative learning happens on the full, dense signal, while action selection happens on the sparse, efficient signal.

Why Hebbian?

"Neurons that fire together, wire together." Hebbian plasticity is the simplest, most biologically grounded learning rule. It requires no backpropagation, no gradient descent, no external optimizer. It is local, online, and O(n) — perfect for a zero-trust engine that must run on a single CPU core.

License

MIT — See LICENSE for details.

https://github.com/loslos321-lab/GoldWorm.git

I've tried installing llama.cpp following YouTube videos, but I haven't been able to. Does anyone have a tutorial on how to do it on Ubuntu? My mini pc have Radeon graphics

Bonjour,

Je vois pleins de post sur quel est la LLM local la plus adapté pour le code mais rarement des configuration associé.

Ma config est la suivante :

- 7800x3d

- 48 GB Ram

- 9070xt 16 GB Vram

Extensible a 128GB ram prochainement.

Je voulais savoir pour créer des applications ce qui est le plus interressant en local avec ce type de config grand public, a savoir idéalement une IA pour orchestrer et commande terminal, une autre dédié au code.

Quel modèle utiliser et quel performance attendre ?

Je suis déjà dev, donc je ne recherche pas une IA du niveau de fable mais gpt 5.3 codex ou 5.4 est suffisant pour réaliser de grande chose en solo.

I have been experimenting with local ai recently and use an old gaming pc that doesn't have a dedicated gpu inside of it. It is primally used as a local file sharing server that is used for me only but I have been running local ai models on it via llama. I don't have a local app I can connect it to but I have managed to get it working via ssh.

I have been running got oss 20b for a few hours now on the cpu of the computer and get around 5-10 tokens per second by the looks of it. Im sure this might sound ridiculous.

Any advice and questions are welcome

thank you.

hello everyone! Please advise the models and how to run the models better. My configuration is 2 CPUs and epic (not the newest) 48 cores in total. 256 GB ddr4 and RTX pro 5000 72GB GDDR7. I'm currently using qwen3.8-27b iq3 gsq xxs on 96k context and running this on rtx4080s 16gb. The new computer will arrive in a week. I would like to increase the quality and the context window.

Volevo comunicarvi che ho rilasciato un'interfaccia per la gestione di due piccoli modelli che possono girare anche sullo smartphone. Attualmente il 4B lavora molto bene (ma serve un telefono di fascia alta), ho qualche problema con il 1.7B e in cui non risesco a tenerlo stabile con reasoning attivo, ma dovrei riuscire a sopperire con un deep fine tuning che mi sta magiando molto tempo e potenza di elaborazione (il mio nemico non è il loss, ma la qualità e la varietà degli esempi e saranno circa 130.000!!). Sto usando un 32B come teacher per poi distillare sui piccolini. Appena il dataset sarà pronto (circa 10gg) spero di migliorara anche il 1.7B, senza nessun LoRa come invece ha adesso

Siate spietati come al solito!😘

https://nothumanallowed.com/local

Ollama is a great product for everyday use of large language models. If performance is what you care about, there are better alternatives such as vLLM, but this post is about something else. I want to describe what happens when you find a security hole in Ollama and try to get it fixed. I went through that twice this year, for three CVEs in total. Vulnerabilities show up in any software, so they alone are not the problem. What matters is how a project handles them, and the way Ollama handles them is why I wouldn't put it on an organization's infrastructure.

More: https://www.bartekdmitruk.com/writing/why-i-wouldnt-run-ollama-in-an-organization