CREBench: Evaluating Large Language Models in Cryptographic Binary Reverse Engineering

Researchers introduced CREBench, a benchmark to evaluate large language models' capabilities in cryptographic binary reverse engineering. The best-performing model (GPT-5.4) achieved 64.03% success rate, while human experts scored 92.19%, showing AI still lags behind human expertise in cryptographic analysis tasks.

• →CREBench benchmark comprises 432 challenges across 48 cryptographic algorithms and 3 difficulty levels to test LLM reverse engineering capabilities.
• →GPT-5.4 was the top-performing model, recovering flags in 59% of challenges with a score of 64.03 out of 100.
• →Human experts significantly outperformed AI models with a baseline score of 92.19 points.
• →The research addresses the systematic underexploration of LLMs' capabilities in cryptographic binary reverse engineering.
• →Results indicate substantial room for improvement in AI-assisted cryptographic security analysis and vulnerability discovery.