HyperTransport: Amortized Conditioning of T2I Generative Models

HyperTransport is a new hypernetwork framework that dramatically accelerates activation steering for text-to-image models by amortizing optimization costs across multiple concepts. Rather than optimizing intervention parameters for each new concept (which takes minutes), the system learns to map CLIP embeddings directly to steering parameters in a single forward pass, achieving 3600-7000x speedup while matching per-concept baselines on unseen concepts.

HyperTransport addresses a critical bottleneck in controllable generative AI: the tension between flexible, reliable control and practical deployment constraints. As foundation models become more capable, fine-tuning grows prohibitively expensive, and prompt-based control remains brittle due to sensitivity to wording. Activation steering offers better stability but has required per-concept optimization—a fatal flaw when concept sets are large, dynamic, or specified at request time.

The technical innovation lies in decoupling concept representation from intervention prediction through a hypernetwork trained with optimal transport loss. This architecture learns transferable steering knowledge that generalizes to unseen concepts, eliminating the per-concept optimization bottleneck. The framework introduces three novel capabilities in combination: amortized steering for open-ended concept sets, continuous interpretable strength control, and cross-modal conditioning where images can directly guide text generation.

For the AI industry, HyperTransport represents a significant shift toward practical controllability at scale. The 3600-7000x speedup moves activation steering from research curiosity to deployable technology. Validation across DMD2 and Nitro-1-PixArt models with 167 held-out test concepts demonstrates generalization robustness. Human and VLM evaluations show users prefer HyperTransport outputs to prompting roughly twice as often—suggesting real usability advantages beyond speed metrics.

The implications extend beyond image generation: this amortization approach could apply to any foundation model control problem where concept sets vary at runtime. Future work likely involves scaling to larger concept vocabularies, integration with commercial generative APIs, and extension to multimodal and language models.

• →HyperTransport achieves 3600-7000x speedup by training a hypernetwork to predict activation steering parameters instead of optimizing per-concept
• →The framework generalizes to unseen concepts while matching or exceeding per-concept baseline performance across multiple T2I models
• →Users and VLM judges prefer HyperTransport-steered outputs to prompt engineering approximately 2x as often in pairwise comparisons
• →Novel cross-modal conditioning capability enables reference images to directly influence text-to-image generation without intermediate prompts
• →Decoupling concept representation from intervention prediction enables three previously unavailable capabilities in combination: amortized control, continuous strength modulation, and cross-modal steering