AINeutralarXiv – CS AI · 5h ago6/10
🧠
Cliff Tokens: Identifying Single-Token Failure Triggers in LLM Mathematical Reasoning
Researchers identify 'cliff tokens'—specific points in LLM reasoning where a single token triggers failure in mathematical problem-solving. By deleting these tokens and resampling, models recover near-perfect accuracy, demonstrating that failures stem from precise decision points rather than diffuse errors. A taxonomy of cliff types enables targeted optimization that improves model reasoning by up to 6.6%.