#continual-pretraining News & Analysis

2 articles tagged with #continual-pretraining. AI-curated summaries with sentiment analysis and key takeaways from 50+ sources.

2 articles

AIBullisharXiv – CS AI · Apr 146/10

🧠

Data Mixing Agent: Learning to Re-weight Domains for Continual Pre-training

Researchers present Data Mixing Agent, an AI framework that uses reinforcement learning to automatically optimize how large language models balance training data from source and target domains during continual pre-training. The approach outperforms manual reweighting strategies while generalizing across different models, domains, and fields without requiring retraining.

AIBullisharXiv – CS AI · Mar 24/109

🧠

Low-Resource Dialect Adaptation of Large Language Models: A French Dialect Case-Study

Researchers developed a cost-effective method to adapt large language models to minority dialects using continual pre-training and LoRA techniques, successfully improving Quebec French dialect performance with minimal computational resources. The study demonstrates that parameter-efficient fine-tuning can expand quality LLM access to underserved linguistic communities while updating only 1% of model parameters.