Spectrum Tuning: Post-Training for Distributional Coverage and In-Context Steerability

Researchers introduce Spectrum Tuning, a new post-training method that improves AI language models' ability to generate diverse outputs and follow in-context steering instructions. The technique addresses limitations in current post-training approaches that reduce models' distributional coverage and flexibility when tasks require multiple valid answers rather than single correct responses.

• →Current post-training methods improve instruction-following but hurt models' ability to generate diverse outputs for creative tasks.
• →Researchers identify three key requirements for distributional modeling: in-context steerability, valid output coverage, and distributional alignment.
• →Spectrum Suite introduces a large-scale evaluation resource with over 40 data sources spanning 90+ tasks requiring diverse output generation.
• →Spectrum Tuning demonstrates improvements over standard instruction-tuned models in flexibility and output diversity.
• →The research distinguishes between knowledge elicitation and in-context steerability as different forms of in-context learning.