Automated Analysis of Global AI Safety Initiatives: A Taxonomy-Driven LLM Approach

Researchers developed an automated framework using large language models to compare AI safety policy documents across a shared taxonomy of activities. The study found that model choice significantly affects comparison outcomes, with some document pairs showing high disagreement across different LLMs, though human expert evaluation showed high inter-annotator agreement.