AIBearisharXiv – CS AI · Jun 197/10
🧠Researchers demonstrate that clinical NLP datasets for suicidality detection, particularly the ScAN dataset built on MIMIC-III notes, embed specific operational choices that obscure how labels are constructed rather than representing objective ground truth. The study reveals that dataset design decisions—including single annotators, ICD-based cohort selection, and hospital-stay aggregation—shape what suicidality means in algorithmic systems, highlighting critical gaps between documented clinical judgments and actual suicidal intent.
AIBearisharXiv – CS AI · Jun 47/10
🧠Researchers have developed PEEL (Protocols for Epistemically Engaged Literacy in AI), a framework combining deterministic distant reading tools with LLM interpretation to measure and expose systematic distortions in AI-generated text summaries. The framework reveals that large language models introduce undetectable errors in quantity, term frequency, and epistemic voice, challenging the assumption that AI fluency equals fidelity and raising critical questions about researcher accountability in AI-assisted scholarship.
🧠 Claude
AIBearisharXiv – CS AI · Jun 17/10
🧠A new arXiv paper challenges recent claims about LLM capabilities by arguing they lack scientific rigor under Popper's falsifiability principle. The authors identify methodological flaws in AI reasoning research, including opaque training data, non-reproducibility, and selection bias, then propose transparency guidelines to improve scientific integrity in LLM evaluation.
🏢 Meta
AIBullisharXiv – CS AI · May 117/10
🧠Researchers developed an LLM-based agent system for identifying competing drugs in clinical indications, achieving 83% recall compared to 65% and 60% for competitor systems. The agent validates results using an LLM-as-a-judge approach to minimize hallucinations, reducing biotech due diligence analysis time from 2.5 days to 3 hours in production deployment.
🏢 OpenAI🏢 Perplexity
AIBearisharXiv – CS AI · Apr 147/10
🧠Researchers propose lightweight sanity checks for agentic data science (ADS) systems to detect falsely optimistic conclusions that users struggle to identify. Using the Predictability-Computability-Stability framework, the checks expose whether AI agents like OpenAI Codex reliably distinguish signal from noise. Testing on 11 real datasets reveals that over half produced unsupported affirmative conclusions despite individual runs suggesting otherwise.
🏢 OpenAI
AIBullisharXiv – CS AI · Feb 277/106
🧠Researchers introduce VALTEST, a framework that uses semantic entropy to automatically validate test cases generated by Large Language Models, addressing the problem of invalid or hallucinated tests that mislead AI programming agents. The system improves test validity by up to 29% and enhances code generation performance through better filtering of LLM-generated test cases.
AINeutralarXiv – CS AI · Jun 236/10
🧠Researchers introduce Chem2Gen-Bench, a comprehensive benchmark dataset containing over 1.3 million chemical and genetic perturbation profiles designed to evaluate how accurately computational models can translate chemical perturbations into genetic responses. The study reveals that while translation between these perturbation types is measurable, it remains heterogeneous across different cellular contexts, and current foundation-model embeddings don't consistently outperform simpler baseline approaches.
AINeutralarXiv – CS AI · Jun 115/10
🧠Researchers evaluated whether AI agents equipped with specialized medical research skills produce higher-quality outputs than native language models on transcriptomic biomarker analysis tasks. While skill-augmented AI showed directional improvements in expert-rated quality, the gains were modest and within the margin of expert-rating noise, suggesting larger, more rigorous studies are needed.
AIBullishWired – AI · Jun 106/10
🧠Google has deployed its Gemini AI technology with Argentina's national football team during the World Cup, positioning the team as a real-world testing ground for advanced AI applications in sports. This partnership demonstrates how major tech companies are leveraging high-profile sporting events to validate and showcase AI capabilities to global audiences.
🧠 Gemini
AINeutralarXiv – CS AI · Jun 86/10
🧠CAF-Gen is a new multi-agent AI system that automatically enriches basic argument structures into complex, formally-structured argumentation models using the Carneades Argumentation Framework. The iterative Creator-Reviewer pipeline improves reasoning formalization in computational linguistics by validating outputs through collaborative feedback loops rather than single-pass generation.
AINeutralarXiv – CS AI · Jun 25/10
🧠A study analyzing how clinicians edit ambient AI-generated clinical notes reveals that physicians systematically introduce more hedging language (uncertainty qualifiers) rather than remove it, indicating they tend toward greater caution when revising AI drafts. The findings show substantial variation across AI vendors and medical specialties, highlighting inconsistent AI documentation quality and clinician confidence levels.
AINeutralarXiv – CS AI · May 296/10
🧠Researchers introduce Opt-Verifier, an LLM-based framework that improves automated mathematical optimization modeling by verifying generated models from both structural and solution perspectives. The dual-side verification approach addresses a critical gap in existing systems by validating constraints, variables, and solution validity, achieving over 20% accuracy improvements on benchmark tests.
AINeutralarXiv – CS AI · May 275/10
🧠Researchers present a framework for managing uncertainty in language model-generated laboratory procedures for virtual educational environments. The system uses structured domain representations and LLM outputs to extract, validate, and repair procedural steps, addressing common LLM failures like missing actions, incorrect sequencing, and logical incompatibilities.
AINeutralarXiv – CS AI · Apr 146/10
🧠Researchers have developed a framework to detect and eliminate ambiguities in natural-language specifications converted to executable BPMN process models by large language models. The method identifies behavioral inconsistencies through KPI analysis, diagnoses gateway logic problems, and repairs source text through evidence-based refinement, reducing variability in regenerated model behavior.
AINeutralarXiv – CS AI · Apr 146/10
🧠Researchers introduce Phantom, a framework that combines generative AI with constraint-based post-processing to synthesize valid PCIe protocol traces for hardware simulation. The system addresses a critical limitation of naive AI generation—hallucination of protocol-violating sequences—achieving up to 1000x improvements in task-specific metrics compared to existing approaches.
AIBullishMarkTechPost · Mar 86/10
🧠The article presents a tutorial for building advanced agentic AI systems using a cognitive blueprint framework that incorporates identity, goals, planning, memory, validation, and tool access. The framework enables AI agents to not only respond but also plan, execute, validate, and systematically improve their outputs through structured runtime capabilities.
AIBullisharXiv – CS AI · Mar 37/109
🧠SimAB is a new system that uses persona-conditioned AI agents to simulate A/B tests for rapid design evaluation without requiring real user traffic. The system achieved 67% overall accuracy against 47 historical A/B tests, rising to 83% for high-confidence cases, potentially transforming how companies validate design decisions.
AIBullisharXiv – CS AI · Mar 36/104
🧠Researchers developed a framework that improves AI-generated research ideas by incorporating relevant data during the ideation process. The approach increased idea feasibility by 20% and overall quality by 7%, with human studies confirming that data-augmented AI assistance helps researchers generate higher-quality ideas.
AIBullisharXiv – CS AI · Feb 276/107
🧠Researchers developed a framework for analyzing AI diagnostic systems in clinical settings by preserving original AI inferences and comparing them with physician corrections. The study of 21 dermatological cases showed 71.4% exact agreement between AI and physicians, with 100% comprehensive concordance when using structured analysis methods.
