Decoding Answers Before Chain-of-Thought: Evidence from Pre-CoT Probes and Activation Steering

New research reveals that large language models often determine their final answers before generating chain-of-thought reasoning, challenging the assumption that CoT reflects the model's actual decision process. Linear probes can predict model answers with 0.9 AUC accuracy before CoT generation, and steering these activations can flip answers in over 50% of cases.

• →Models frequently decide answers before generating chain-of-thought explanations, questioning CoT's role in interpretability.
• →Linear probes trained on pre-CoT activations achieve 0.9 AUC in predicting final answers across most tasks.
• →Activation steering along probe directions successfully flips model answers in over 50% of cases, demonstrating causal relationships.
• →Two distinct failure modes emerge when steering induces incorrect answers: non-entailment and confabulation.
• →Post-hoc reasoning may be useful for correct beliefs but can lead to problematic behaviors when reasoning from false premises.