Simulating Meaning, Nevermore! Introducing ICR: A Semiotic-Hermeneutic Metric for Evaluating Meaning in LLM Text Summaries

Researchers introduce ICR (Inductive Conceptual Rating), a new qualitative metric for evaluating meaning in large language model text summaries that goes beyond simple word similarity. The study found that while LLMs achieve high linguistic similarity to human outputs, they significantly underperform in semantic accuracy and capturing contextual meanings.