Narrative Landscape: Mapping Narrative Dispositions Across LLMs

Researchers have developed a quantitative framework for measuring and visualizing how different large language models exhibit stable behavioral patterns in their outputs. By testing six frontier models across controlled narrative tasks, they identified a spectrum of model dispositions ranging from rigid to exploratory, revealing that instruction types can fundamentally alter selection patterns even when traditional metrics suggest similarity.

This research addresses a critical gap in LLM evaluation methodology by formalizing how individual models develop consistent, measurable behavioral signatures. Rather than treating LLM outputs as purely random or variable, the study demonstrates that each model maintains stable dispositional patterns—what the authors call regularities in selection behavior under repeated controlled conditions. This matters because it suggests LLMs are not black boxes producing arbitrary outputs, but systems with discernible, quantifiable personalities.

The framework uses two complementary metrics: consistency (measured via Jaccard similarity across replications) and diversity (measured through inverse Simpson index), creating a two-dimensional space for model comparison. The introduction of Narrative Landscape—a PCA-based visualization—allows researchers to map model dispositions into a shared analytical space, revealing a clear rigidity-exploration spectrum across model families.

A particularly significant finding is that instruction types reshape the geometric structure of model selection spaces without necessarily changing scalar metrics, meaning that comparable numerical scores can mask fundamentally different selection topologies. This discovery has substantial implications for developers and researchers relying on traditional benchmarks, as it suggests current evaluation methods may obscure important qualitative differences in model behavior.

For the AI industry, this work provides tools for more granular model comparison and selection, particularly valuable for applications requiring predictability or specific behavioral characteristics. The research enables practitioners to move beyond surface-level performance metrics toward deeper understanding of model dispositions, potentially improving model selection for specialized tasks and revealing how instruction engineering genuinely transforms model behavior at a structural level.

• →LLMs exhibit stable, measurable dispositional patterns that vary systematically across model families along a rigidity-exploration spectrum.
• →Traditional scalar metrics can mask qualitatively distinct selection behaviors, suggesting current benchmarks are incomplete.
• →The Narrative Landscape visualization framework enables direct comparison of model dispositions in a shared analytical space.
• →Instruction types fundamentally reshape how models make selections, even when overall performance metrics remain similar.
• →This framework provides developers with tools to select models based on behavioral characteristics rather than aggregate scores alone.