DARE-bench: Evaluating Modeling and Instruction Fidelity of LLMs in Data Science

Researchers introduce DARE-bench, a new benchmark with 6,300 Kaggle-derived tasks for evaluating Large Language Models' performance on data science and machine learning tasks. The benchmark reveals that even advanced models like GPT-4-mini struggle with ML modeling tasks, while fine-tuning on DARE-bench data can improve model accuracy by up to 8x.

• →DARE-bench provides 6,300 standardized tasks with verifiable ground truth for objective LLM evaluation in data science.
• →Even highly capable models like GPT-4-mini show poor performance on machine learning modeling tasks.
• →Fine-tuning with DARE-bench data dramatically improves model performance, boosting accuracy by 1.83x to 8x.
• →The benchmark addresses critical gaps in process-aware evaluation and accurately labeled training data.
• →Results demonstrate the importance of specialized benchmarks for advancing AI capabilities in data science.