AIBearisharXiv – CS AI · 9h ago7/10
🧠
Dense Contexts Are Hard Contexts: Lexical Density Limits Effective Context in LLMs
Researchers discovered that lexical density—the rate at which new information appears in text—significantly limits LLM effective context windows, causing near-perfect models to drop below 60% accuracy on information-dense contexts. This finding reveals that input length and needle position, traditionally blamed for context degradation, overlook a critical third factor that directly impacts real-world LLM performance on compact, information-rich data.