#evaluation-protocol News & Analysis

2 articles tagged with #evaluation-protocol. AI-curated summaries with sentiment analysis and key takeaways from 50+ sources.

2 articles

AINeutralarXiv – CS AI · Jun 116/10

🧠

Soft-Prompt Tuning for Fair and Efficient LLM Benchmark Evaluation

Researchers propose soft-prompt tuning, a parameter-efficient method that adapts large language models to benchmark formatting requirements by optimizing only 0.0006% of model parameters. This technique reveals that benchmark scores often underestimate base model knowledge due to formatting constraints, enabling fairer evaluation across different model architectures and pre-training approaches.

🏢 Meta

AINeutralarXiv – CS AI · May 296/10

🧠

Entity-Collision: A Stratified Protocol for Attributing Retrieval Lift in Agent Memory

Researchers propose entity-collision, a standardized testing protocol for evaluating retrieval systems in agent memory applications. The protocol isolates embedder performance from lexical overlap by construction, revealing that encoder capacity alone doesn't guarantee better retrieval—MiniLM-384 outperforms larger models on mixed query types despite having fewer parameters than BGE-large.