Using Probabilistic Programs to Train Inductive Reasoning in Large Language Models
Researchers introduce Program-based Posterior Training (PPT), a novel fine-tuning method that uses probabilistic programs to train LLMs on inductive reasoning tasks. By generating synthetic scenarios and using probabilistic inference to create distributional targets, the approach significantly improves model accuracy on uncertainty estimation while better aligning with human judgment.