Language Models Learn Constructional Semantics, Not To Mention Syntax: Investigating LM Understanding of Paired-Focus Constructions

Researchers demonstrate that modestly-sized open-source language models can understand rare paired-focus constructions (like "let alone" and "much less"), challenging assumptions that only the largest LLMs grasp complex constructional semantics. The study reveals that semantic understanding of these constructions emerges later in training than syntactic knowledge and correlates with world knowledge acquisition.

This research addresses a fundamental question about language model capabilities: whether understanding rare linguistic constructions requires massive scale or emerges through intelligent learning dynamics. The study's finding that mid-sized open-source models achieve robust performance on paired-focus constructions suggests the relationship between model scale and semantic understanding is more nuanced than previously assumed.

The research builds on longstanding linguistic theory around constructions as form-meaning pairings. Traditional NLP assumed only billion-parameter models could handle semantic subtleties in low-frequency patterns. This work demonstrates that parameter count alone doesn't determine constructional understanding, opening possibilities for more efficient, interpretable models. The connection between paired-focus semantics and broader world knowledge acquisition provides insight into how models organize meaning representations internally.

For AI developers, this challenges assumptions driving current scaling trends. If modestly-sized models demonstrate genuine semantic understanding, the efficiency-versus-performance tradeoff shifts favorably toward deployment of smaller, open-source systems. Organizations can potentially achieve adequate performance without massive computational overhead. The finding that semantic understanding lags syntactic knowledge during training suggests specific architectural or training innovations could accelerate semantic acquisition.

The research establishes a clearer picture of what drives semantic competence beyond scale. Future work investigating which architectural features or training strategies enable earlier semantic emergence could yield practical improvements for model development. The work also validates empirical methods for testing specific semantic phenomena, providing researchers tools to probe model understanding more precisely than existing benchmarks allow.

• →Mid-sized open-source language models successfully understand rare paired-focus constructions, contradicting the assumption that only largest LLMs grasp complex semantics.
• →Semantic understanding of constructions emerges later in training than syntactic knowledge, suggesting distinct learning pathways for form versus meaning.
• →Paired-focus semantic competence correlates with gains in world knowledge domains, indicating interconnected meaning representations in language models.
• →Model scale alone does not determine constructional understanding, suggesting efficiency improvements are possible without massive parameter counts.
• →The research provides empirical methods for testing specific semantic phenomena, enabling more precise evaluation of model understanding beyond standard benchmarks.