Steering at the Source: Style Modulation Heads for Robust Persona Control

Researchers have identified a method to control Large Language Model behavior by targeting only three specific attention heads called 'Style Modulation Heads' rather than the entire residual stream. This approach maintains model coherency while enabling precise persona and style control, offering a more efficient alternative to fine-tuning.

• →Activation steering can control LLMs without fine-tuning but often causes coherency degradation.
• →Only three attention heads are responsible for persona and style formation in LLMs.
• →Targeting specific 'Style Modulation Heads' maintains coherency while enabling behavioral control.
• →The method uses geometric analysis combining cosine similarity and contribution scores to locate these heads.
• →Component-level localization enables safer and more precise model control than residual stream intervention.