PlayWrite: A Multimodal System for AI Supported Narrative Co-Authoring Through Play in XR

PlayWrite is a new mixed-reality AI system that allows users to create stories by directly manipulating virtual characters and props in XR, rather than through traditional text prompts. The system uses multi-agent AI to interpret user actions into structured narrative elements and generates final stories via large language models, demonstrating a novel approach to AI-human creative collaboration.

• →PlayWrite introduces a mixed-reality storytelling system that moves beyond text-based AI writing tools to embrace direct manipulation and play.
• →The system uses multi-agent AI pipeline to convert user actions with virtual objects into structured narrative beats called Intent Frames.
• →A user study with 13 writers showed the system fosters improvisational and playful creative processes.
• →Users treated the AI as a collaborative partner, using unexpected AI responses to overcome creative blocks and generate new ideas.
• →The research demonstrates potential for co-creative AI systems that utilize spatial and interactive modalities beyond traditional text interfaces.