Gradient Atoms: Unsupervised Discovery, Attribution and Steering of Model Behaviors via Sparse Decomposition of Training Gradients

Researchers introduce Gradient Atoms, an unsupervised method that decomposes AI model training gradients to discover interpretable behaviors without requiring predefined queries. The technique can identify model behaviors like refusal patterns and arithmetic capabilities, while also serving as effective steering vectors to control model outputs.

• →Gradient Atoms uses dictionary learning to decompose training gradients into sparse components that reveal interpretable AI model behaviors.
• →The method discovers behaviors like refusal patterns, arithmetic, and classification tasks without requiring behavioral labels or queries.
• →Discovered atoms can serve as steering vectors to dramatically alter model behavior, such as changing bulleted-list generation from 33% to 94%.
• →The approach is more efficient than existing training data attribution methods as it doesn't require scoring every document against query behaviors.
• →Among 500 discovered atoms, the highest-coherence ones successfully recovered major task-type behaviors in language models.