#base-models News & Analysis

2 articles tagged with #base-models. AI-curated summaries with sentiment analysis and key takeaways from 50+ sources.

2 articles

AIBullisharXiv – CS AI · Jun 107/10

🧠

Sample Where You Struggle: Sharpening Base Model Reasoning via Entropy-Guided Power Sampling

Researchers introduce Entropy-Guided Power Sampling (EGPS), a novel training-free sampling method that accelerates reasoning in base language models by targeting high-entropy decision points rather than uniformly sampling across sequences. The technique achieves up to 12.6x speedup on mathematical and coding benchmarks while maintaining or improving accuracy, addressing fundamental inefficiencies in existing MCMC sampling approaches.

AINeutralarXiv – CS AI · Jun 116/10

🧠

Soft-Prompt Tuning for Fair and Efficient LLM Benchmark Evaluation

Researchers propose soft-prompt tuning, a parameter-efficient method that adapts large language models to benchmark formatting requirements by optimizing only 0.0006% of model parameters. This technique reveals that benchmark scores often underestimate base model knowledge due to formatting constraints, enabling fairer evaluation across different model architectures and pre-training approaches.

🏢 Meta