Curveball Steering: The Right Direction To Steer Isn't Always Linear

Researchers propose 'Curveball steering', a nonlinear method for controlling large language model behavior that outperforms traditional linear approaches. The study challenges the Linear Representation Hypothesis by showing that LLM activation spaces have substantial geometric distortions that require geometry-aware interventions.

• →Traditional linear activation steering methods for LLMs often behave inconsistently due to geometric distortions in activation spaces.
• →LLM activation spaces exhibit substantial concept-dependent distortions that aren't well-approximated by globally linear geometry.
• →Curveball steering uses polynomial kernel PCA to perform interventions in feature space that better respect learned activation geometry.
• →The nonlinear approach consistently outperforms linear PCA-based steering, especially in regimes with strong geometric distortion.
• →This research suggests geometry-aware methods provide a more principled alternative to global linear interventions in LLM control.