AINeutralarXiv – CS AI · 9h ago6/10
🧠
Double Preconditioning (DoPr): Optimization for Test-Time Performance, not Validation Loss
Researchers introduce Double Preconditioning (DoPr), a new optimization technique that improves neural network performance during real-world deployment by combining gradient-wise and activation-wise preconditioning. The method addresses test-time feedback—the gap between training metrics and actual task performance in autoregressive models—without requiring improvements in traditional validation loss metrics.