PolicyPad: Collaborative Prototyping of LLM Policies

Researchers developed PolicyPad, an interactive system that helps domain experts collaborate on creating policies for LLMs in high-stakes applications like mental health and law. The system enables real-time policy drafting and testing through established UX prototyping practices, showing improved collaborative dynamics and tighter feedback loops in workshops with 22 experts.

• →PolicyPad facilitates collaborative LLM policy design through real-time drafting and independent testing of policy-informed model behavior.
• →The system draws from established UX prototyping practices including heuristic evaluation and storyboarding.
• →Workshops with domain experts in mental health and law demonstrated enhanced collaborative dynamics and novel policy contributions.
• →The research addresses the growing need for expert input in LLM governance for high-stakes applications.
• →The work contributes to advancing AI alignment and safety through expert-informed policy development processes.