#data-regularization News & Analysis

2 articles tagged with #data-regularization. AI-curated summaries with sentiment analysis and key takeaways from 50+ sources.

2 articles

AIBullisharXiv – CS AI · Jun 27/10

🧠

Breaking the Reversal Curse in Autoregressive Language Models via Identity Bridge

Researchers demonstrate that the 'reversal curse' — an autoregressive language model's inability to deduce inverse relationships from forward training data — can be mitigated through a simple data regularization technique called Identity Bridge. By adding self-referential training examples (e.g., 'Alice's name is Alice'), a 1B parameter model achieves 50% success on reversal tasks compared to near-zero baseline performance, suggesting LLMs can learn higher-level logical rules rather than merely memorizing facts.

AINeutralarXiv – CS AI · May 116/10

🧠

Dr. Post-Training: A Data Regularization Perspective on LLM Post-Training

Researchers introduce Dr. Post-Training, a novel framework that treats general training data as a regularizer rather than a selection pool for LLM post-training. The method projects target-data updates onto a feasible set defined by general data, improving performance across SFT, RLHF, and RLVR tasks while maintaining computational efficiency.