35 articles tagged with #scientific-ai. AI-curated summaries with sentiment analysis and key takeaways from 50+ sources.
Researchers introduced COMPOSITE-STEM, a new benchmark containing 70 expert-written scientific tasks across physics, biology, chemistry, and mathematics to evaluate AI agents. The top-performing model achieved only 21% accuracy, indicating the benchmark effectively measures capabilities beyond current AI reach and addresses the saturation of existing evaluation frameworks.
Researchers introduced OPENXRD, a comprehensive benchmarking framework for evaluating large language models and multimodal LLMs in crystallography question answering. The study tested 74 state-of-the-art models and found that mid-sized models (7B-70B parameters) benefit most from contextual materials, while very large models often show saturation or interference.
Researchers introduce MicroVerse, a specialized AI video generation model for microscale biological simulations, addressing limitations of current video generation models in scientific applications. The work includes MicroWorldBench benchmark and MicroSim-10K dataset, targeting biomedical applications like drug discovery and educational visualization.
Researchers introduce SafeSci, a comprehensive framework for evaluating safety in large language models used for scientific applications. The framework includes a 0.25M sample benchmark and 1.5M sample training dataset, revealing critical vulnerabilities in 24 advanced LLMs while demonstrating that fine-tuning can significantly improve safety alignment.
Researchers created CMT-Benchmark, a new dataset of 50 expert-level condensed matter theory problems to evaluate large language models' capabilities in advanced scientific research. The best performing model (GPT5) solved only 30% of problems, with the average across 17 models being just 11.4%, highlighting significant gaps in current AI's physical reasoning abilities.
Researchers introduce Carrée du champ flow matching (CDC-FM), a new generative AI model that improves the quality-generalization tradeoff by using geometry-aware noise instead of standard uniform noise. The method shows significant improvements in data-scarce scenarios and non-uniformly sampled datasets, particularly relevant for AI applications in scientific domains.
Researchers have developed VCWorld, a new AI-powered biological simulation system that combines large language models with structured biological knowledge to predict cellular responses to drug perturbations. The system operates as a 'white-box' model, providing interpretable predictions and mechanistic insights while achieving state-of-the-art performance in drug perturbation benchmarks.
Researchers have developed SPM-Bench, a PhD-level benchmark for testing large language models on scanning probe microscopy tasks. The benchmark uses automated data synthesis from scientific papers and introduces new evaluation metrics to assess AI reasoning capabilities in specialized scientific domains.
OpenAI has developed a real-world evaluation framework to assess AI's potential in accelerating biological research, specifically testing GPT-5's ability to optimize molecular cloning protocols in wet lab environments. The research examines both the opportunities and risks associated with AI-assisted scientific experimentation.
Researchers developed the first comprehensive framework for creating domain-specialized Large Language Models for combustion science, using 3.5 billion tokens from scientific literature and code. The study found that standard RAG approaches hit a performance ceiling at 60% accuracy, highlighting the need for more advanced knowledge injection methods including knowledge graphs and continued pretraining.
