Questionnaire Responses Do not Capture the Safety of AI Agents

Researchers argue that current AI safety assessments using questionnaire-style prompts on language models are inadequate for evaluating real AI agents. The study suggests these methods lack construct validity because LLM responses to hypothetical scenarios don't accurately represent how AI agents would actually behave in real-world deployments.