AINeutralarXiv – CS AI · Mar 57/10
🧠
Function Induction and Task Generalization: An Interpretability Study with Off-by-One Addition
Researchers studied how large language models generalize to new tasks through "off-by-one addition" experiments, discovering a "function induction" mechanism that operates at higher abstraction levels than previously known induction heads. The study reveals that multiple attention heads work in parallel to enable task-level generalization, with this mechanism being reusable across various synthetic and algorithmic tasks.