Sketch2Colab: Sketch-Conditioned Multi-Human Animation via Controllable Flow Distillation

Sketch2Colab is a new AI system that converts 2D sketches into realistic 3D multi-human animations with precise control over interactions and movements. The technology uses a novel approach combining sketch-driven diffusion with rectified-flow distillation for faster, more stable animation generation than existing methods.

• →Sketch2Colab transforms storyboard sketches into coherent 3D multi-human animations with fine-grained control over timing and interactions.
• →The system uses a two-stage approach: learning a sketch-driven diffusion prior then distilling it into an efficient rectified-flow student model.
• →A continuous-time Markov chain planner schedules discrete interaction events like touches and handoffs for realistic human collaboration.
• →Experiments show the system achieves state-of-the-art constraint adherence while offering significantly faster inference than diffusion-only methods.
• →The technology addresses limitations in conventional diffusion-based motion generators that struggle with multi-entity conditioning.