SemHiTok: A Unified Image Tokenizer via Semantic-Guided Hierarchical Codebook for Multimodal Understanding and Generation

Researchers introduce SemHiTok, a unified image tokenizer that uses semantic-guided hierarchical codebooks to balance multimodal understanding and generation tasks. The system decouples semantic and pixel features through a novel architecture that builds pixel sub-codebooks on pretrained semantic codebooks, achieving superior performance in both image reconstruction and multimodal understanding.