Catch Your Breath: Adaptive Computation for Self-Paced Sequence Production

Researchers propose Catch Your Breath (CYB), a novel training method that enables AI models to dynamically control the number of computational steps used for processing inputs through <pause> tokens. The approach outperforms standard cross-entropy training by allowing models to signal when they need additional processing time, improving performance metrics like perplexity without increasing computational overhead.

The research addresses a fundamental challenge in inference-time scaling for large language models: how to enable models to adaptively allocate computational resources during generation. Traditional pause-token approaches treat additional compute steps as fixed overheads, lacking mechanisms for models to regulate their own processing demands. CYB reframes this as a sequential decision problem where models emit <don't know> signals to autonomously extend their reasoning horizon before responding.

This work builds on growing interest in inference-time scaling methods as alternatives to simply increasing model parameters. While techniques like chain-of-thought prompting and test-time compute have shown promise, they often lack principled training objectives. CYB fills this gap by creating a supervised loss function that teaches models when to pause, enabling learned self-regulation rather than fixed computational budgets.

The practical implications are significant for AI developers and infrastructure providers. Models trained with CYB demonstrate measurable improvements in downstream task accuracy and perplexity reduction without requiring additional memory or computational resources during training. This efficiency matters for deployment scenarios where inference costs constrain accessibility.

The findings suggest future work may optimize how models learn to allocate compute across different input complexities. Potential applications include adaptive inference systems that scale processing dynamically based on task difficulty, reducing latency for simple queries while allocating more compute to complex reasoning. Integration with quantization and other optimization techniques could further enhance efficiency.

• →CYB enables models to dynamically control computation steps through learned pause-token emission rather than fixed delays.
• →The method improves perplexity and downstream accuracy without increasing training or inference computational costs.
• →Models trained with CYB outperform standard cross-entropy objectives in both pretraining and fine-tuning scenarios.
• →The approach makes inference-time scaling more efficient by allowing adaptive rather than static compute allocation.
• →This technique could enable resource-efficient deployment of adaptive reasoning capabilities in production systems.