Position: The ML Community Must Build an AI-Augmented Peer-Review Ecosystem

A position paper argues that the machine learning community must develop an AI-augmented peer-review ecosystem to address the crisis of scale in scientific publishing. With manuscript submissions exponentially outpacing qualified reviewers at premier ML venues, the authors propose using LLMs as collaborators—not replacements—to enhance factual verification, reviewer performance, author quality improvement, and administrative decision-making while maintaining scientific integrity.

The peer review system underpinning machine learning research faces structural collapse under exponential submission growth at top-tier conferences like NeurIPS, ICML, and ICLR. This position paper identifies a genuine infrastructure challenge: human reviewers cannot scale to meet demand, threatening review quality and consistency. The proposal to augment—not replace—human judgment with LLM assistants represents a pragmatic middle ground between maintaining rigor and enabling scalability.

The crisis stems from ML's explosive growth and democratized access to research tools, creating a bottleneck in the gatekeeping function of peer review. This mirrors broader tensions in scientific publishing where traditional peer review mechanisms struggle with accelerating research velocity. The authors correctly identify that AI assistance for factual verification, reviewer guidance, and AC decision support could meaningfully reduce reviewer burden without compromising editorial standards.

For the ML research ecosystem, successful implementation could unlock significant value by maintaining conference quality while accommodating growth. However, this also carries market implications: platforms providing peer review infrastructure, dataset annotation services, and research management tools could see increased adoption and investment. Venture capital focused on scientific infrastructure and academic publishing tech may view this as validation for AI-augmented approaches.

The critical bottleneck remains access to structured, ethically-sourced peer review data. The paper's call for community-wide data sharing and collaborative infrastructure development suggests this requires institutional coordination rather than individual commercial solutions. Success depends on conference organizers and publishers collaborating on shared standards and data governance frameworks.

• →ML peer review faces a scale crisis as submissions exponentially outpace qualified reviewers at premier venues.
• →AI-augmented systems should enhance human judgment in verification, reviewer guidance, and decision-making rather than replace it.
• →Implementing effective AI-assisted review requires access to granular, structured, and ethically-sourced peer review process data.
• →Success depends on community-wide collaboration on infrastructure, data sharing agreements, and governance standards.
• →This addresses a genuine scientific publishing bottleneck with potential market implications for research infrastructure companies.