RelayFormer: A Unified Local-Global Attention Framework for Scalable Image and Video Manipulation Localization

RelayFormer is a new deep learning framework that unifies image and video manipulation detection through a flexible attention mechanism called Global Local Relay (GLR) tokens. The approach handles variable resolutions without distortion and processes both static and temporal data with a single architecture, addressing key limitations in current visual forensics methods.

RelayFormer represents a significant advancement in visual manipulation localization (VML), a critical problem as AI-generated and edited media becomes increasingly sophisticated. The framework tackles two fundamental challenges that have plagued existing approaches: the loss of forensic detail through uniform resizing and the architectural fragmentation between image and video processing pipelines. By introducing a relay-based attention mechanism with Global Local Relay tokens, the researchers enable efficient information flow across variable resolutions while preserving fine-grained tampering artifacts that uniform scaling would destroy.

The broader context involves the arms race between detection and generation technologies. As tools like diffusion models and advanced video editors proliferate, forensic methods must evolve rapidly. Current systems either compromise image quality through preprocessing or require computationally expensive sparse attention patterns. RelayFormer's fixed-size sub-image partitioning with relay tokens offers a more elegant solution that maintains efficiency without sacrificing accuracy across different input dimensions.

For developers and researchers in content authentication, this unified framework reduces implementation burden by eliminating the need for separate pipelines. The approach's scalability to video sequences without architectural changes enables more practical deployment in media verification systems, social platforms, and news organizations combating disinformation. The public code release accelerates adoption across the research community. However, real-world impact depends on how well the method generalizes to emerging generation techniques and adversarial manipulation strategies that may specifically target the GLR token mechanism itself.

• →RelayFormer introduces Global Local Relay tokens enabling efficient global-local attention without uniform resizing or padding that destroys forensic evidence.
• →A single unified architecture processes both images and videos, eliminating the need for separate manipulation detection pipelines.
• →The framework adapts to variable input resolutions with minimal computational overhead, improving practical deployment feasibility.
• →Extensive benchmarking demonstrates superior performance balancing accuracy and efficiency compared to existing visual manipulation localization methods.
• →Open-source code release accelerates adoption and research advancement in content authentication and media forensics.