ImmigrationQA: A Source-Grounded Dataset and Small-Model Adaptation for U.S. Immigration Law

Researchers released ImmigrationQA, a source-grounded dataset of 17,058 question-answer pairs covering U.S. immigration law, and fine-tuned a Llama 3.2 3B model using LoRA for legal assistance. The fine-tuned model achieved 27% relative improvement over base models but remains limited for complex legal reasoning, demonstrating both the potential and constraints of small language models in high-stakes legal domains.

The ImmigrationQA project addresses a critical gap in AI-assisted legal services by creating a specialized dataset and optimized model for immigration law—an area where petitioners often lack legal representation and face significant consequences for procedural errors. The researchers assembled 10,056 validated documents from authoritative sources including the USCIS Policy Manual and BIA precedent decisions, then generated structured QA pairs using Claude Sonnet, resulting in a dataset of 17,058 examples across 13 subdomains.

This work exemplifies the emerging trend of domain-specific language model adaptation. Rather than relying on general-purpose models, organizations increasingly fine-tune smaller, more efficient models on curated datasets to achieve reasonable performance at lower computational cost—the entire pipeline cost approximately $29 in cloud compute. The fine-tuned Llama 3.2 3B model showed substantial gains over the base 8B model (1.08/3.0 versus 0.85/3.0 mean score), though it underperformed a zero-shot Claude Sonnet baseline (1.52/3.0).

The model's concentrated strengths in procedural subdomains (travel documents, visa status) alongside weaknesses in complex legal reasoning highlight important limitations for practical deployment. These gaps suggest that specialized legal AI serves best as an informational aid rather than a substitute for counsel, particularly in immigration contexts where regulatory changes occur frequently and stakes are high. The public release of artifacts enables further research and community contributions, establishing a foundation for iterative improvements in legal AI accessibility.

• →Domain-specific fine-tuning of smaller models (3B parameters) achieved 27% improvement over base models at minimal cost ($29 compute), validating efficient adaptation strategies.
• →The fine-tuned model excels in procedural domains but struggles with complex legal reasoning and time-sensitive information, requiring human oversight for high-stakes decisions.
• →Public release of the dataset, model, and code enables community-driven improvements and establishes a template for other specialized legal AI applications.
• →General-purpose models like Claude Sonnet still outperform specialized fine-tuned models on this task, suggesting trade-offs between efficiency and accuracy remain challenging.
• →Immigration law's frequent regulatory changes create a fundamental limitation for static datasets, requiring continuous corpus updates to maintain accuracy.