TxBench-PP: Analyzing AI Agent Performance on Small-Molecule Preclinical Pharmacology

Researchers introduced TxBench-PP, a benchmark testing AI agents' ability to analyze real-world drug discovery data rather than regurgitate memorized information. Testing 11 AI models across 4,800 trajectories revealed significant limitations: even the best-performing system (Claude Opus) succeeded only 59% of the time on preclinical pharmacology tasks, suggesting AI agents require substantial improvement before reliable deployment in drug discovery workflows.

TxBench-PP represents a critical stress test for AI deployment in high-stakes domains. Rather than evaluating models on synthetic benchmarks or literature recall, this benchmark forces AI agents to interpret actual assay data, inspect files programmatically, and render structured pharmacology decisions—mirroring real drug discovery workflows. This methodology surfaces a crucial gap: current large language models struggle with complex, multi-step scientific reasoning on unfamiliar data.

The benchmark emerges as AI capabilities continue accelerating across domains. While language models excel at text synthesis and pattern matching, drug discovery demands causal reasoning, statistical interpretation of noisy experimental data, and risk assessment—tasks requiring genuine understanding rather than probabilistic text completion. The underperformance of even frontier models like Claude Opus suggests fundamental limitations in how current architectures approach scientific problem-solving.

For the biotech and pharmaceutical industry, these results carry immediate implications. Organizations cannot yet outsource critical preclinical decisions to AI agents without human expert validation. This constrains the productivity gains AI promised, requiring hybrid workflows where models assist rather than replace domain specialists. However, the 59% success rate on complex tasks is non-trivial; applied selectively for routine analysis or hypothesis generation, these tools add value despite limitations.

Future improvements likely require architectural changes enabling better uncertainty quantification, causal inference, and incorporation of domain-specific knowledge. The benchmark itself becomes a development tool, allowing researchers to iterate toward AI systems genuinely suited for scientific discovery rather than optimized for generic leaderboards.

• →Current AI agents fail reliably on realistic drug discovery tasks, with best-in-class models succeeding only 59% of the time on preclinical pharmacology decisions.
• →TxBench-PP tests agents on actual experimental data interpretation rather than literature recall, revealing gaps in scientific reasoning capabilities.
• →No AI model-harness configuration demonstrated trustworthy performance for independent decision-making in preclinical pharmacology workflows.
• →The benchmark indicates biotech firms must maintain human expert oversight rather than fully automating critical drug discovery analyses.
• →Results suggest current AI limitations stem from fundamental architecture constraints, not merely training data or model scale.