Ayn: A Tiny yet Competitive Indian Legal Language Model Pretrained from Scratch

Researchers developed Ayn, an 88M parameter legal language model that outperforms much larger LLMs (up to 80x bigger) on Indian legal tasks while remaining competitive on general tasks. The study demonstrates that domain-specific Tiny Language Models can be more efficient alternatives to costly Large Language Models for specialized applications.

• →Ayn, an 88M parameter model, outperformed LLMs up to 80 times larger on legal case judgment prediction tasks.
• →The tiny legal model rivaled LLMs up to 30 times larger on summarization tasks while using significantly fewer resources.
• →Domain-specific tokenization and focused pretraining can create highly efficient alternatives to general-purpose LLMs.
• →The model was trained for only 185 A100 hours, demonstrating cost-effective development of specialized AI models.
• →Tiny Language Models under 100M parameters can remain competitive with larger models on general tasks.