StructLens: A Structural Lens for Language Models via Maximum Spanning Trees

Researchers introduced StructLens, a new analytical framework that uses maximum spanning trees to reveal global structural relationships between layers in language models, going beyond existing local token analysis methods. The approach shows different similarity patterns compared to traditional cosine similarity and proves effective for practical applications like layer pruning.

• →StructLens analyzes global inter-layer relationships in language models using maximum spanning trees based on semantic representations.
• →The framework reveals structural similarity patterns that differ significantly from conventional cosine similarity measurements.
• →The approach proves beneficial for practical tasks such as layer pruning in language model optimization.
• →Current interpretability research focuses mainly on local relationships, leaving global inter-layer structures largely unexplored.
• →The framework constructs dependency parsing-like structures to quantify inter-layer distance from a structural perspective.