FreeStyle: Free Control of Style-Content Dual-Reference Generation from Community LoRA Mining

FreeStyle introduces a scalable framework for dual-reference image generation that synthesizes images preserving content structure while adopting separate style references, addressing the challenge of style-content separation through community LoRA mining and novel disentanglement mechanisms. The approach tackles a critical bottleneck in large-scale triplet dataset availability and achieves improved balance between style alignment, content preservation, and leakage suppression compared to existing methods.

FreeStyle addresses a fundamental challenge in generative AI: creating images that faithfully preserve content from one reference while adopting visual style from another. This dual-reference generation problem has practical applications in creative workflows, design automation, and content generation, yet existing approaches struggle with semantic leakage where unintended content from the style reference contaminates the output.

The research tackles this through an innovative approach using community LoRAs—pre-trained style and content modules from open-source communities—as compositional anchors. By treating these as building blocks, the team constructs large-scale training datasets with clean separation between style and content across multiple base models. This scalability addresses a critical limitation: previous work lacked sufficient diverse, labeled data for training robust dual-reference systems.

The technical contribution centers on two-stage curriculum learning with stage-specific mechanisms. During style transfer, attention-level enrichment constraints suppress style-reference leakage. In the harder dual-reference stage, frequency-aware RoPE modulation targets positional correspondence-based leakage—a sophisticated approach that recognizes different leakage mechanisms require different solutions. The team also establishes evaluation benchmarks including novel metrics like style-invariant Content Alignment Score and VLM-based Rejection Score, enabling more rigorous assessment of generation quality.

For developers and researchers, this work provides practical tools and methodologies for building production-ready dual-reference generation systems. The framework's scalability through community LoRA mining democratizes access to diverse style-content combinations. The comprehensive benchmark establishes evaluation standards for future work, advancing the field beyond subjective assessment toward measurable, reproducible metrics.

• →FreeStyle uses community LoRA mining as compositional anchors to construct large-scale, clean style-content triplet datasets at scale
• →Two-stage curriculum learning with attention-level and frequency-aware mechanisms effectively suppresses semantic leakage from style references
• →Novel evaluation metrics including Content Alignment Score and VLM-based Rejection Score enable rigorous assessment of dual-reference generation quality
• →The framework demonstrates strong balance across style alignment, content preservation, and leakage suppression compared to existing approaches
• →Community-driven LoRA mining approach makes dual-reference generation more scalable and accessible to researchers across multiple base models