Exploring LLMs for South Asian Music Understanding and Generation

Researchers conducted the first systematic evaluation of Large Language Models on South Asian classical music understanding and generation, finding that frontier models like Gemini 2.5 Pro achieve 85-90% accuracy on music comprehension but struggle with stylistically faithful generation (40% success rate). The study reveals that current LLMs handle Western musical traditions far better than structurally distinct, low-resource traditions like Hindustani and Bengali classical music.

This research addresses a significant gap in LLM capability assessment by moving beyond Western-centric music evaluation frameworks. The study reveals that while frontier models demonstrate strong theoretical understanding of South Asian classical music concepts—achieving 85-90% accuracy on a 504-question benchmark covering raga grammar and tala-based constraints—their practical generation abilities remain substantially limited. The 40% stylistic fidelity rate indicates that structural knowledge and creative output represent separate challenges requiring distinct technical solutions.

The research contextualizes growing recognition that LLMs exhibit cultural and linguistic biases reflecting their training data composition. Most existing music AI work focuses on Western harmony-driven traditions where abundant digital corpora exist, leaving non-Western traditions underrepresented in model training. South Asian classical music, governed by fundamentally different principles than Western tonal systems, serves as a critical test case for LLM universality claims.

For the AI development community, these findings suggest that scaling model parameters alone cannot solve culturally-grounded music tasks. The performance gap between open-source models (23-40% accuracy) and frontier models (85-90%) indicates that training data quality and scale matter significantly, but the generation bottleneck points toward architectural limitations. Developers targeting music applications in non-Western markets face substantial headwinds without specialized fine-tuning or cultural knowledge integration.

Looking forward, the research establishes a benchmark for measuring progress in culturally-sensitive music AI, likely spurring focused efforts to incorporate South Asian musical theory into training pipelines and develop specialized evaluation frameworks beyond English-language metrics.

• →Frontier LLMs achieve 85-90% accuracy on South Asian classical music theory but only 40% on stylistically faithful generation tasks
• →Open-source models significantly underperform on non-Western music traditions, scoring 23-40% on the introduced benchmark
• →Structural validity and stylistic authenticity represent distinct technical challenges requiring separate solutions in music generation
• →Current LLM training heavily favors Western musical traditions, creating systematic biases against low-resource non-Western music systems
• →The research establishes the first rigorous evaluation framework for assessing LLM competence in South Asian classical music