AIBearisharXiv – CS AI · 7h ago7/10
🧠
LLMs Lean on Priors, Not Programming Language Semantics
Researchers have demonstrated that large language models rely heavily on statistical patterns from training data rather than systematically understanding formal programming semantics. The PLSemanticsBench benchmark reveals that LLM accuracy drops 40-60 percentage points when semantic rules are altered or novel symbols are introduced, suggesting current models struggle with explicit rule-following in structured domains.