InFerActive: Interactive Tree-Based Exploration of LLM Sampling for Safety Evaluation

InFerActive is an interactive system that improves how AI safety evaluators assess large language models by visualizing sampling results as navigable trees rather than static spreadsheets. The tool uses breadth-first sampling to achieve equivalent harmful-response coverage with up to 5x fewer samples, significantly improving evaluation efficiency according to controlled user studies.

InFerActive addresses a critical gap in LLM safety evaluation workflows. While major language models pass initial safety benchmarks, their stochastic nature means low-probability harmful outputs can still reach users at scale during deployment. Current human evaluation practices rely on generating dozens of random samples per prompt and manually reviewing them in spreadsheets—a process that becomes increasingly tedious as evaluators encounter near-duplicate outputs repeatedly.

The research builds on growing awareness that safety testing methodologies lag behind deployment velocity. Companies investing billions in frontier AI systems face mounting pressure from regulators, users, and internal governance teams to demonstrate comprehensive safety measures. Traditional spreadsheet-based review approaches don't scale for modern evaluation demands, creating bottlenecks that slow safety iteration cycles.

InFerActive's tree-based visualization paradigm fundamentally changes how evaluators navigate sampling spaces. By organizing outputs hierarchically and allowing interactive exploration and filtering, the system reduces cognitive load and enables faster detection of harmful patterns. The breadth-first sampling innovation—matching random sampling's coverage while requiring five times fewer generations—has direct cost and efficiency implications for organizations conducting safety evals at scale.

The controlled user studies demonstrating measurable improvements in both efficiency and coverage suggest this approach could become standard practice. As AI safety becomes increasingly formalized and regulated, tooling that makes evaluation faster and more reliable becomes competitive advantage. The broader implication extends beyond evaluation interfaces—better tools enable better safety assurance, which ultimately affects how regulators view different AI providers' governance maturity.

• →InFerActive reduces harmful-response sampling requirements by up to 5x while maintaining equivalent coverage through breadth-first tree construction.
• →Interactive tree visualization significantly outperforms static spreadsheet workflows in human evaluation efficiency and completeness.
• →The system addresses scalability bottlenecks in LLM safety evaluation as deployment complexity increases.
• →Better evaluation tooling directly impacts the quality and comprehensiveness of safety assurance before model deployment.
• →User studies validate that improved interface design measurably improves safety evaluator performance and consistency.