Lexara: A User-Centered Toolkit for Evaluating Large Language Models for Conversational Visual Analytics

Researchers have developed Lexara, a user-centered toolkit for evaluating Large Language Models in Conversational Visual Analytics applications. The toolkit addresses current evaluation challenges by providing interpretable metrics for both visualization and language quality, along with real-world test cases and an interactive interface that doesn't require programming expertise.

• →Lexara addresses the challenge of evaluating LLMs for Conversational Visual Analytics through user-centered design based on interviews with 38 developers and end-users.
• →The toolkit provides interpretable metrics covering both visualization quality and language quality using rule-based and LLM-as-a-Judge methods.
• →Lexara enables CVA evaluation without requiring programming expertise through an interactive interface.
• →A two-week diary study with six CVA developers demonstrated the toolkit's effectiveness for guiding model and prompt selection.
• →The research operationalizes real-world use cases and evaluation criteria into practical test scenarios for LLM assessment.