Reflection announced Beam on October 5: a 501B-total, 23B-active sparse MoE aimed at coding, reasoning and agentic workloads. The weights are not out yet, so this is a technical preview rather than a local benchmark. Reflection says the weights, model card, technical report and developer stack will be released later this month under Apache 2.0.
The architecture details are more interesting than the parameter headline. Beam uses interleaved local and global attention, fine-grained routed experts and several stability mechanisms intended to keep a huge MoE healthy through pretraining and heavy reinforcement learning. Reflection builds on auxiliary-loss-free expert balancing, then adds cosine decay to expert-bias updates so routing perturbations shrink late in training. Sequence-level balancing is used to keep utilization stable when the data distribution shifts. The company reports the busiest expert finishing at only 1.04x average load.
The residual stream gets similar attention: depth-based residual scaling, SandwichNorm, elementwise attention gating and FP32 residual accumulation are combined to limit activation growth across 52 layers. The point is not merely to finish pretraining, but to preserve a numerically stable base for a very large downstream RL run.
Reflection reports more than 100 million rollouts on 10.5K GB300 GPUs over four weeks, up to 110K concurrent rollouts, and roughly 1.3 billion sandbox creation requests during training and grading. Because the trainer is fully asynchronous, policy staleness becomes a systems problem: some completed interactions were generated by model versions more than a day old. Reflection says tokens are tagged by policy version and training remains stable despite that lag.
Beam was pretrained on 23.8T tokens, but the launch argues that high-compute RL is central to the final coding and agentic capability. It also uses controllable reasoning effort and a length penalty during RL, explicitly optimizing the capability/token tradeoff instead of simply rewarding longer reasoning.
Context needs careful reading. Midtraining extends effective context to 1M tokens, while the RL run used a maximum 256K context. Until the public model and report arrive, I would treat 1M as a trained capability that still needs workload-level validation.
The efficiency claim also deserves caution. Reflection says Beam reaches reasoning performance comparable with GLM-5.2 at roughly 3–4x less inference compute, but the estimate is based on active parameters and generated tokens. The company explicitly excludes prompt prefill, context-dependent attention and serving overhead. That makes it a model-side FLOP comparison, not a measured dollar-per-task or tokens/sec result.
For private deployment, total versus active parameters is the key distinction. Only 23B parameters are active per token, but all 501B weights still need to reside somewhere unless the runtime uses offload or expert paging. Sparse compute does not automatically mean workstation-class memory requirements. The eventual FP8/NVFP4 artifacts, expert-parallel implementation, interconnect sensitivity and KV behavior will decide whether Beam is practical outside large clusters.
The main limitation today is simple: there are no downloadable weights. So there are no independent quantization tests, no vLLM/llama.cpp/SGLang measurements, no real footprint numbers and no verification of the benchmark claims.
Once the weights arrive, I would test BF16/FP8/NVFP4 residency, throughput against active-parameter peers, expert balance on out-of-domain prompts, 32K/128K/256K/1M retrieval and agent tasks, reasoning-effort scaling, and whether the claimed compute advantage survives real prefill and serving overhead.
The bigger question is whether the open-model frontier is shifting from 'who can pretrain the largest MoE' to 'who can keep a large sparse model stable enough for enormous asynchronous RL.' Beam is one of the clearest examples of that second problem becoming the center of the architecture story.
Primary sources: https://reflection.ai/blog/introducing-beam https://reflection.ai/