AINeutralarXiv – CS AI · Jun 116/10
🧠Researchers present a staged-promotion protocol for efficiently screening machine learning configurations during micro-pretraining, using fixed budget increments across heterogeneous hardware to reduce experimental costs while mitigating the risk of selecting configurations that perform well only at tiny scales. The study demonstrates that early-stage rankings are unstable across hardware types, but a frozen promotion rule successfully identified a consistent top performer while reducing total GPU-hours from 432 to 169.2.
AINeutralarXiv – CS AI · Jun 116/10
🧠A position paper challenges the prevailing interpretation of AI systems possessing theory of mind (ToM), arguing that current research conflates sophisticated pattern matching with genuine cognition. The authors propose that AI performance on ToM tasks reflects behavioral mimicry rather than authentic mental models, and recommend shifting toward mutual ToM frameworks that assess human-AI interaction dynamics rather than testing AI systems in isolation.
AINeutralarXiv – CS AI · Jun 106/10
🧠Researchers conducted a large-scale semantic analysis of 8,954 definitions and 2,700 scale items across 14,000+ publications to map how learner agency and autonomy are conceptualized and measured. They identified three core dimensions (task regulation, intrinsic motivation, and sociocultural action) and found that existing measurement scales systematically underrepresent the sociocultural aspect, while current generative AI applications in education narrowly focus on learning control.
AINeutralarXiv – CS AI · Jun 106/10
🧠Researchers demonstrate that successful machine learning strategies remain highly compressible and generalizable even when trained on held-out benchmarks, suggesting overfitting in benchmark-driven ML is rare because effective strategies occupy a low-complexity region of strategy space. Using LLM-driven research agents, they show that short prompts and minimal feedback suffice to reproduce high-performance models across diverse domains.
AINeutralarXiv – CS AI · Jun 96/10
🧠Researchers evaluated whether multimodal large language models (MLLMs) like Gemini 3 Flash and Qwen 3 Omni can replicate human subjective responses in video perception tasks using the Perceived Message Sensation Value framework. The study found significant limitations: MLLMs demonstrated systematic biases including downward mean-shift, central-tendency bias, and inconsistent sensitivity to participant profiles, suggesting current models remain unreliable as synthetic human participants for subjective research.
🧠 Gemini
AINeutralarXiv – CS AI · Jun 96/10
🧠Researchers introduce Ada, a systematic framework for observing how software engineering agents navigate real codebases through tool-mediated exploration. By analyzing 408 trajectories across multiple models and repositories, the study develops observation methods that reveal agent decision-making patterns—including navigation choices, evidence selection, and stopping criteria—without reducing behavior to raw metrics or speculation.
$ADA
AINeutralarXiv – CS AI · Jun 56/10
🧠Researchers present RAINO, a systematic framework addressing how realism is poorly defined and inconsistently operationalized in Agent-Based Models. The framework identifies Reality Anchors (empirical data, theory, expert knowledge) and their application as inputs or outputs, providing a conceptual foundation for evaluating and developing more realistic computational models.
AINeutralarXiv – CS AI · Jun 46/10
🧠Researchers have developed synthetic benchmarks for concept bottleneck models—AI systems that make predictions based on high-level concepts rather than raw data. The benchmarks address a critical gap in the field by enabling controlled evaluation of these interpretable AI models across different use cases, from decision support to automation, while managing variables like data type and annotation quality.
AINeutralarXiv – CS AI · Jun 46/10
🧠Researchers propose conditional PED-ANOVA (condPED-ANOVA), a new framework for measuring hyperparameter importance in machine learning search spaces where parameters have conditional dependencies. The method addresses limitations of existing approaches by accurately handling cases where a hyperparameter's presence or domain depends on other hyperparameters, improving the reliability of AutoML systems.
AINeutralarXiv – CS AI · Jun 26/10
🧠Researchers developed AI-Paper-Review, a tool that generates structured peer review feedback for academic papers using multiple AI reviewers, and conducted a case study on 20 computer architecture submissions to measure how well AI review aligns with human review. The study finds that AI review can identify significant portions of human-raised issues while also surfacing problems missed by human reviewers, raising important questions about AI's role in academic peer review without endorsing its use for formal publication decisions.
AINeutralarXiv – CS AI · Jun 16/10
🧠Researchers introduce a diagnostic framework using Item Response Theory (IRT) to assess the reliability of Large Language Models used as automated judges. The framework evaluates LLM judges on two dimensions: intrinsic consistency (stability under prompt variations) and human alignment (correspondence with human assessments), providing practical guidance for identifying unreliability sources.
AINeutralarXiv – CS AI · May 296/10
🧠Researchers introduce a personalized turn-level conversation satisfaction benchmark that evaluates AI assistant responses based on individual user expectations and conversation history rather than generic quality metrics. The system combines user memory with context-specific evaluation to produce satisfaction scores and identifies dissatisfying responses more accurately than existing methods.
AINeutralarXiv – CS AI · May 286/10
🧠Researchers demonstrate that standard statistical hypothesis tests fail when applied to generative surveying, where LLM-based personas provide market research feedback. The study proposes a valid permutation test that accounts for prompt sensitivity and provides guidance on optimal resource allocation for this emerging research methodology.
AINeutralarXiv – CS AI · May 276/10
🧠Researchers introduce a controlled experimental framework using procedurally generated languages to study cross-lingual transfer in language models, isolating variables like lexical distance and tokenization. Their findings across 700 runs reveal that tokenization preserving reusable substructure—rather than vocabulary size or lexical similarity alone—determines transfer success, with transfer occurring in distinct stages from grammatical competence to masked lexical generalization.
AINeutralarXiv – CS AI · May 276/10
🧠A comprehensive systematic review of 139 studies reveals that multimodal information fusion improves document classification accuracy by 5.28 percentage points, while multiview approaches provide modest gains of 4.67%. The research identifies critical gaps in methodological rigor, with less than 24% of studies employing statistical validation, highlighting the need for more robust research standards in the field.
AIBearisharXiv – CS AI · May 126/10
🧠A new benchmarking framework reveals that AI tools in academic research excel at exploration and summaries but fail at precision tasks requiring exact information extraction. The study demonstrates that explainable AI features are inadequate, forcing researchers to manually verify outputs, and literature review tools lack reproducibility and transparency for systematic research.
🏢 xAI
AINeutralarXiv – CS AI · May 116/10
🧠Researchers introduce SCALAR, an Actor-Critic-Judge framework that systematically evaluates how AI agents improve through human feedback on theoretical physics problems. The study reveals that multi-turn dialogue consistently outperforms single attempts, but the effectiveness of different feedback strategies depends heavily on the specific pairing of AI models used, with asymmetric model pairs benefiting most from structured critique.
AINeutralarXiv – CS AI · May 96/10
🧠Researchers introduce InciteResearch, a multi-agent AI framework that helps researchers transform vague, implicit research ideas into structured, actionable questions through Socratic questioning. The framework achieves significant improvements over baselines on TF-Bench, a new benchmark for tacit-to-explicit research assistance, demonstrating AI's potential as a thinking tool rather than just an execution automator.
AINeutralarXiv – CS AI · May 46/10
🧠Researchers have developed a mathematical framework called LLM DNA that traces the evolutionary relationships between large language models through functional representations rather than documentation. The training-free method successfully identified previously unknown connections among 305 LLMs and constructed an evolutionary tree reflecting architectural shifts and temporal progression in model development.
AINeutralarXiv – CS AI · May 16/10
🧠Researchers introduce LAPITHS, a framework for critically evaluating claims about AI language models' cognitive abilities, directly challenging models like CENTAUR that claim human-like cognition. The framework demonstrates that impressive AI performance doesn't necessarily indicate human-like underlying computation or genuine cognitive abilities.
AINeutralarXiv – CS AI · May 16/10
🧠A comprehensive survey examines how large language models can assist or automate peer review processes across academia, synthesizing techniques for review generation, post-review tasks, and evaluation methods. The research catalogs datasets and modeling approaches while addressing ethical concerns and practical implementation challenges for integrating AI into scholarly publishing workflows.
AINeutralarXiv – CS AI · Apr 206/10
🧠Researchers compared large language models with human responses in a behavioral study on accuracy perception, finding that LLMs reproduce directional effects but with inconsistent effect magnitudes across different models. The study reveals that off-the-shelf LLMs can simulate some human belief-updating patterns in controlled experiments but lack reliable human-scale accuracy, establishing clearer boundaries for when synthetic LLM data is appropriate for behavioral research.
AIBearisharXiv – CS AI · Apr 206/10
🧠A new study reveals that using large language models to generate synthetic datasets ("silicon samples") produces highly variable results depending on configuration choices, with correlation outcomes ranging from r=.23 to r=.84 on the same task. This demonstrates that analytic flexibility in LLM-based data generation poses a significant threat to research validity and reproducibility in social science applications.
AINeutralarXiv – CS AI · Apr 146/10
🧠Researchers introduce SciPredict, a benchmark testing whether large language models can predict scientific experiment outcomes across physics, biology, and chemistry. The study reveals that while some frontier models marginally exceed human experts (~20% accuracy), they fundamentally fail to assess prediction reliability, suggesting superhuman performance in experimental science requires not just better predictions but better calibration awareness.
AINeutralarXiv – CS AI · Apr 146/10
🧠Researchers propose AI as a Research Object (AI-RO), a governance framework that treats generative AI interactions as inspectable, documented components of scientific research rather than debating authorship. The framework combines interaction logs, metadata packaging, and provenance records to ensure accountability, particularly for security and privacy research where confidentiality and auditability are critical.
🏢 Meta
