The Lattice Representation Hypothesis of Large Language Models

Researchers propose the Lattice Representation Hypothesis, a new framework showing how large language models encode symbolic reasoning through geometric structures. The theory unifies continuous neural representations with formal logic by demonstrating that LLM embeddings naturally form concept lattices that enable symbolic operations through geometric intersections and unions.

• →The Lattice Representation Hypothesis provides a mathematical bridge between continuous neural embeddings and symbolic reasoning in large language models.
• →Linear attribute directions with thresholds create concept lattices through geometric half-space intersections, enabling logical operations.
• →Experiments on WordNet hierarchies show empirical evidence that LLM embeddings encode concept lattices and their logical structure.
• →The framework unifies the Linear Representation Hypothesis with Formal Concept Analysis to explain symbolic reasoning capabilities.
• →Geometric meet and join operations in embedding space correspond to logical intersection and union operations respectively.