LangFIR: Discovering Sparse Language-Specific Features from Monolingual Data for Language Steering

Researchers introduce LangFIR, a method that enables better language control in multilingual AI models using only monolingual data instead of expensive parallel datasets. The technique identifies sparse language-specific features and achieves superior performance in controlling language output across multiple models including Gemma and Llama.

• →LangFIR discovers language-specific features using only monolingual data and random-token sequences, eliminating the need for expensive parallel datasets.
• →The method identifies extremely sparse and highly selective language-specific features that are causally important for language identity.
• →LangFIR outperforms existing baselines across three models (Gemma 3 1B/4B, Llama 3.1 8B) and twelve target languages.
• →Language identity in multilingual LLMs is localized in a sparse set of feature directions discoverable with monolingual data.
• →The approach uses sparse autoencoders to decompose residual activations into interpretable feature directions for better language steering.