Perturbation: A simple and efficient adversarial tracer for representation learning in language models

Researchers propose a new method called 'perturbation' for understanding how language models learn representations by fine-tuning models on adversarial examples and measuring how changes spread to other examples. The approach reveals that trained language models develop structured linguistic abstractions without geometric assumptions, offering insights into how AI systems generalize language understanding.

• →New 'perturbation' method analyzes language model representations by measuring how adversarial fine-tuning affects other examples.
• →The approach avoids previous limitations of requiring unrealistic constraints like linearity assumptions.
• →Method successfully distinguishes between trained and untrained language models in finding meaningful representations.
• →Research reveals that language models develop structured transfer across multiple linguistic levels.
• →Findings suggest language models acquire linguistic abstractions through experience alone without explicit programming.