dLLM: Simple Diffusion Language Modeling

Researchers introduce dLLM, an open-source framework that unifies core components of diffusion language modeling including training, inference, and evaluation. The framework enables users to reproduce, finetune, and deploy large diffusion language models like LLaDA and Dream while providing tools to build smaller models from scratch with accessible compute resources.

• →dLLM is a new open-source framework that standardizes diffusion language modeling components across training, inference, and evaluation.
• →The framework enables reproduction and deployment of existing large diffusion language models such as LLaDA and Dream.
• →Users can convert any BERT-style encoder or autoregressive language model into a diffusion language model using the framework.
• →Researchers are releasing checkpoints of small diffusion language models to improve accessibility for future research.
• →The framework addresses fragmentation in the field by providing a unified, flexible foundation for diffusion language model development.