Do Models Share Safety Representations? Cross-Model Steering for Safe Visual Generation

Researchers demonstrate that safety behaviors in generative AI models can be represented as portable latent directions that transfer across different architectures without requiring unsafe training data on target models. This framework enables cross-model safety steering for text-to-image and text-to-video generation, suggesting safety is a shared property rather than model-specific.

This research addresses a critical bottleneck in AI safety deployment: the need to retrain or customize safety mechanisms for each new model architecture. The team's core contribution is demonstrating that safety representations learned in one model can be transferred to entirely different generators through lightweight alignment procedures trained only on benign data. This is significant because it decouples safety implementation from individual model development cycles.

The work builds on emerging understanding of latent directions in neural networks—the idea that specific behaviors occupy consistent geometric spaces that persist across architectures. Previous research showed this principle applies to capabilities like bias or style; this paper extends it to safety control, the highest-stakes application area. By never exposing the target model to unsafe data during transfer, the approach isolates whether safety genuinely transfers through shared representation geometry rather than through memorization or data artifacts.

For the AI industry, this creates modularity opportunities. Safety teams could develop robust steering directions once and deploy them across multiple generation models, reducing engineering overhead and enabling faster safety iteration. The multi-vector extension for category-specific safety control suggests fine-grained governance—blocking certain hazard types while permitting others—becomes feasible without model-specific tuning.

The comparable performance metrics (ASR reduction, CLIP-Score/FID trade-offs) to natively-trained directions indicate no quality degradation from transfer, addressing concerns that safety mechanisms impose creativity penalties. This removes a common objection to deployment.

• →Safety behaviors can transfer across different AI models through learned latent directions without target-side unsafe data exposure
• →Lightweight alignment procedures enable cross-architecture safety steering in text-to-image and text-to-video generation
• →Multi-vector extensions allow category-specific safety control for selective hazard mitigation
• →Transferred safety directions achieve performance parity with native target-model directions while reducing engineering complexity
• →Results suggest safety is a shared property across model architectures rather than purely model-specific