When Models Know More Than They Say: Probing Analogical Reasoning in LLMs

Researchers found that large language models (LLMs) have an asymmetry between their internal knowledge and prompted responses when detecting analogies. While probing reveals models understand rhetorical analogies better than their prompted responses suggest, both methods perform poorly on narrative analogies requiring deeper abstraction.

• →LLMs struggle with analogical reasoning when surface cues don't align with structural relationships
• →Probing internal representations significantly outperforms prompting for rhetorical analogies in open-source models
• →Both probing and prompting show similarly low performance on narrative analogies requiring latent information
• →The gap between internal knowledge and accessible behavior varies by task type
• →Current prompting methods may not effectively access all available information stored in model representations