CodeTaste: Can LLMs Generate Human-Level Code Refactorings?

Researchers introduce CodeTaste, a benchmark testing whether AI coding agents can perform code refactoring at human-level quality. The study reveals frontier AI models struggle to identify appropriate refactorings when given general improvement areas, but perform better with detailed specifications.

• →Large language models can generate working code but often create solutions with complexity and architectural debt.
• →CodeTaste benchmark measures AI agents' ability to execute refactorings and identify human-chosen improvements in real codebases.
• →Frontier AI models perform well with detailed refactoring specifications but fail to discover human refactoring choices independently.
• →A propose-then-implement approach improves alignment between AI and human refactoring decisions.
• →The benchmark provides evaluation targets for aligning coding agents with human development practices.