Bridging Linguistic Gaps: Cross-Lingual Mapping in Pre-Training and Dataset for Enhanced Multilingual LLM Performance

Researchers introduce a Cross-Lingual Mapping Task during LLM pre-training to improve multilingual performance across languages with varying data availability. The method achieves significant improvements in machine translation, cross-lingual question answering, and multilingual understanding without requiring extensive parallel data.

This research addresses a fundamental challenge in multilingual AI development: the performance gap between high-resource languages like English and low-resource languages. Current multilingual LLMs often exhibit monolingual bias, performing well on tasks in well-represented languages while struggling with cross-lingual transfer. The proposed Cross-Lingual Mapping Task reframes pre-training to explicitly align language representations bi-directionally, rather than treating languages as isolated data streams.

The work builds on recent trends in contrastive learning and alignment-based approaches, but differs by integrating cross-lingual objectives directly into pre-training rather than applying them post-hoc through fine-tuning. The introduction of a Language Alignment Coefficient provides a measurable framework for assessing cross-lingual consistency, addressing the instability issues that plague earlier contrastive methods. Empirical gains—11.9 BLEU points in machine translation and 5%+ improvements in cross-lingual NLU—suggest the approach meaningfully advances the field.

For developers building multilingual applications, this research validates pre-training-level interventions as more efficient than downstream fine-tuning approaches. The method's efficiency with limited parallel data has practical implications for languages with scarce training resources, potentially democratizing LLM performance across linguistic communities. Companies investing in multilingual AI products could leverage these techniques to reduce computational overhead while improving quality.

The next critical evaluation involves testing these methods on truly low-resource languages and assessing whether gains hold across different model scales. Monitoring adoption in production multilingual systems will indicate whether theoretical improvements translate to real-world value.

• →Cross-lingual mapping during pre-training improves multilingual LLM performance without compromising monolingual fluency.
• →The method achieves up to 11.9 BLEU point improvements in machine translation over existing baselines.
• →A new Language Alignment Coefficient enables robust measurement of cross-lingual consistency in low-data scenarios.
• →The approach reduces reliance on extensive parallel data compared to traditional bilingual fine-tuning methods.
• →Results span multiple tasks including machine translation, cross-lingual NLU, and question answering.