AINeutralarXiv – CS AI · 15h ago6/10
🧠
Qiskit QuantumKatas: Adapting Microsoft's Quantum Computing exercises for LLM evaluation
Researchers adapted Microsoft's QuantumKatas quantum computing curriculum from Q# to Qiskit and created a 350-task benchmark with LLM evaluation infrastructure. Testing 16 language models revealed significant capability gaps, with frontier models achieving 83.1% pass rates versus 32.3% for weaker models, while highlighting that LLMs excel at implementing known algorithms but struggle with problem encoding.