IPO Finance Agent: Evaluation of LLM Financial Analysts beyond Finance Agent v2, with Automated Rubric Generation -- the Case of the SpaceX (SPCX) IPO

Researchers introduce IPO Finance Agent, an advanced LLM evaluation framework that extends Finance Agent v2 to handle IPO due diligence tasks using improved retrieval architecture. Testing on SpaceX's S-1 filing shows that Alibaba's Qwen 3.7 Max achieves 79.4% accuracy, significantly outperforming previous benchmarks while reducing costs.

IPO Finance Agent addresses a critical gap in LLM evaluation for financial analysis. While Finance Agent v2 established a benchmark for analyzing periodic SEC filings (10-K and 10-Q documents), it fundamentally fails at IPO due diligence because S-1 filings present substantially different challenges—longer documents, complex pro forma accounting, governance structures, and capital-formation narratives that exceed traditional chunked retrieval capabilities. The original benchmark demonstrated this limitation by failing to process the SpaceX S-1 filing entirely.

The research team's solution involves two key improvements: implementing contextual retrieval instead of naive chunk-based approaches, and creating a 1,000-question IPO-diligence dataset with 70 public SpaceX questions. Critically, they developed an automated evaluator-optimizer pipeline that generates evaluation rubrics by extracting candidate facts from ensemble model outputs, then iteratively audits for hallucinations, omissions, and redundancy. This dramatically reduces human annotation burden while maintaining quality.

Results demonstrate substantial progress in both accuracy and cost-efficiency. Alibaba Qwen 3.7 Max achieves 79.4% accuracy at $0.30 per query compared to Google Gemini 3.5 Flash's 57.9% at $2.51—a meaningful leap forward. The Pareto frontier reveals MiMo-2.5 Pro offers competitive accuracy at just $0.05 per query, indicating viable production-ready options emerging in the open-source and Chinese model ecosystem.

These improvements have direct implications for investment banking, venture capital, and institutional investors who conduct IPO analysis. As LLMs become more capable at handling complex financial documents with higher accuracy and lower cost, they accelerate decision-making timelines and democratize sophisticated financial analysis that previously required specialized teams.

• →IPO Finance Agent outperforms Finance Agent v2 by 21.5% accuracy while reducing query costs from $2.51 to $0.30 on top-performing models.
• →Contextual retrieval architecture proves essential for processing long, complex documents like SEC S-1 filings beyond naive chunk-based approaches.
• →Automated rubric generation via evaluator-optimizer pipelines reduces human annotation burden while maintaining evaluation quality and reproducibility.
• →Chinese LLM providers (Alibaba, Xiaomi) demonstrate competitive advantages in both accuracy and cost-efficiency on financial analysis tasks.
• →Publicly released SpaceX IPO benchmark dataset enables reproducible research while private questions prevent benchmark contamination.