SpreadsheetArena: Decomposing Preference in LLM Generation of Spreadsheet Workbooks

Researchers introduce SpreadsheetArena, a platform for evaluating large language models' ability to generate spreadsheet workbooks from natural language prompts. The study reveals that preferred spreadsheet features vary significantly across use cases, and even top-performing models struggle with domain-specific best practices in areas like finance.

• →SpreadsheetArena provides blind pairwise evaluations of LLM-generated spreadsheet workbooks to assess model performance on structured artifact creation.
• →Stylistic, structural, and functional features of preferred spreadsheets vary substantially depending on the specific use case and prompt.
• →Expert evaluations indicate that highly ranked arena models fail to reliably produce spreadsheets aligned with domain-specific best practices in finance.
• →Spreadsheet generation presents unique evaluation challenges due to well-defined output structure and complex interactivity requirements.
• →The research highlights end-to-end spreadsheet generation as a challenging category of complex, open-ended tasks for LLMs.