ZAYA1-8B Technical Report

Zyphra has unveiled ZAYA1-8B, a compact reasoning-focused AI model with only 700M active parameters that matches larger competitors like DeepSeek-R1 on mathematics and coding tasks. The model introduces Markovian RSA, a novel test-time compute method that achieves 91.9% on AIME'25 benchmarks while maintaining computational efficiency, suggesting small models can compete with much larger reasoning systems through architectural innovation.

Zyphra's ZAYA1-8B represents a meaningful advancement in efficient AI reasoning, demonstrating that model size alone does not determine performance on complex reasoning tasks. By achieving competitive results with under 1 billion active parameters against models with substantially larger footprints, the technical report challenges assumptions about the scaling laws required for reasoning capabilities. The model's architecture leverages Zyphra's MoE++ framework, which selectively activates only necessary parameters during inference, reducing computational overhead while maintaining performance parity with dense models.

The development of Markovian RSA as a test-time compute method marks a significant technical contribution to the field. This recursive aggregation approach allows ZAYA1-8B to improve performance on benchmark mathematics problems while managing memory constraints through bounded-length reasoning tails. The achievement of 91.9% accuracy on AIME'25 and competitive positioning against Gemini-2.5 Pro and DeepSeek-V3.2 demonstrates practical viability of efficient reasoning systems. The four-stage reinforcement learning cascade—combining math warmup, curriculum learning, code RL with synthetic environments, and behavioral RL—reveals sophisticated post-training methodology that extends beyond conventional supervised fine-tuning.

From an industry perspective, ZAYA1-8B's development on AMD's full-stack platform signals growing momentum in open-weight alternatives to proprietary AI systems. The emphasis on reproducibility and open-weight accessibility could influence enterprise AI deployment decisions, particularly where cost efficiency and computational constraints matter. However, benchmark performance alone does not guarantee real-world applicability across diverse use cases. The model's specific optimization for mathematics and coding may limit broader utility. Continued evaluation on diverse benchmarks beyond reasoning tasks will determine whether efficiency gains translate to production viability.

• →ZAYA1-8B achieves competitive reasoning performance with 700M active parameters, roughly 70x smaller than traditional dense models, suggesting efficient architectures can challenge scaling law assumptions.
• →Markovian RSA test-time compute method enables 91.9% AIME'25 accuracy through recursive trace aggregation with memory-bounded constraints, advancing efficient inference techniques.
• →AMD full-stack compute platform training demonstrates viability of non-NVIDIA infrastructure for frontier AI model development, relevant for supply chain diversification.
• →Four-stage RL cascade combining math, code, and behavioral training represents sophisticated post-training methodology that may influence industry fine-tuning standards.
• →Open-weight positioning and AMD development could shift enterprise AI procurement toward cost-efficient alternatives, though real-world deployment success remains uncertain.