Rationale-Enhanced Decoding for Multi-modal Chain-of-Thought

Researchers have developed rationale-enhanced decoding (RED), a new inference-time strategy that improves chain-of-thought reasoning in large vision-language models. The method addresses the problem where LVLMs ignore generated rationales by harmonizing visual and rationale information during decoding, showing consistent improvements across multiple benchmarks.

• →Large vision-language models often ignore the contents of generated rationales during chain-of-thought reasoning, limiting their effectiveness.
• →Rationale-enhanced decoding (RED) is a plug-and-play inference-time strategy that doesn't require model retraining.
• →RED works by multiplying distinct image-conditional and rationale-conditional next token distributions to better integrate visual and reasoning information.
• →Extensive experiments demonstrate consistent and significant improvements over standard CoT and other decoding methods across multiple benchmarks.
• →The approach enhances both faithfulness and accuracy of reasoning in multi-modal AI systems.