Test-Time Training with KV Binding Is Secretly Linear Attention

Researchers reveal that Test-Time Training (TTT) with KV binding, previously understood as online meta-learning for memorization, can actually be reformulated as a learned linear attention operator. This new perspective explains previously puzzling behaviors and enables architectural simplifications and efficiency improvements.

• →TTT with KV binding contradicts the traditional memorization-based interpretation through observed phenomena.
• →A broad class of TTT architectures can be mathematically expressed as learned linear attention operators.
• →This reframing enables principled architectural simplifications while maintaining performance.
• →The new perspective allows for fully parallel formulations that improve computational efficiency.
• →The research provides systematic reduction of diverse TTT variants to standard linear attention form.