Researchers introduce ACROS, a method that adds explicit sense representations (per-token meaning decompositions) to frozen pretrained language models without retraining. The technique achieves competitive results in word-sense disambiguation, lexical steering, and cross-lingual adaptation, positioning sense representations as a practical interface for existing models.
ACROS represents a significant methodological advance in making language models more interpretable and controllable without expensive retraining cycles. The research addresses a fundamental limitation in existing approaches: sense representations—explicit breakdowns of token meanings—have typically required models to be pretrained with this structure embedded from the ground up. By introducing a gated residual pathway into frozen models, the authors demonstrate that semantic decomposition can be retrofitted efficiently into production-scale systems like SmolLM2-360M.
The technical achievement gains relevance within the broader landscape of efficient AI customization. As organizations deploy increasingly large language models, the cost and complexity of fine-tuning or retraining for specific capabilities has become prohibitive. This work shows that structural capabilities can be added as lightweight interfaces rather than requiring full model retraining. The experimental validation across three distinct tasks—achieving 64.95 F1 on word-sense disambiguation, recovering 90% of lexical steering objectives, and maintaining near-perfect cross-lingual transfer at 0.988 R@1—demonstrates genuine versatility rather than narrow optimization.
For practitioners building applications requiring precise semantic control or multilingual robustness, ACROS offers a practical tool that maintains baseline model performance while adding interpretability and steerability. The approach particularly benefits resource-constrained developers who cannot afford expensive fine-tuning. Looking ahead, the framework opens avenues for combining multiple inducible interfaces on the same model base, potentially enabling more modular and compositional AI systems where different semantic or behavioral layers can be added independently.
- →ACROS adds sense representations to frozen pretrained models via gated residual pathways without retraining
- →Achieves competitive word-sense disambiguation (64.95 F1) and strong cross-lingual transfer (0.988 R@1) simultaneously
- →Enables three distinct use cases—disambiguation, steering, and cross-lingual adaptation—from a single induced interface
- →Lexical steering proxy recovers approximately 90% of oracle performance with simplified non-oracle approach
- →Makes semantic decomposition a practical retrofit capability for production language models