#property-based-testing News & Analysis

2 articles tagged with #property-based-testing. AI-curated summaries with sentiment analysis and key takeaways from 50+ sources.

2 articles

AIBullisharXiv – CS AI · 7h ago7/10

🧠

FVSpec: Real-World Property-Based Tests as Lean Challenges

Researchers have created FVSpec, a benchmark dataset of 9,415 Lean 4 formal specifications derived from 2,772 real-world Python property-based tests, designed to evaluate AI models on automated formal software verification tasks. The work addresses a critical gap in AI-assisted code verification by providing open-source tools and data to advance AI's capability to formally prove software correctness.

AINeutralarXiv – CS AI · 7h ago6/10

🧠

PBT-Bench: Benchmarking AI Agents on Property-Based Testing

Researchers introduce PBT-Bench, a benchmark testing AI agents' ability to derive semantic invariants from documentation and construct property-based testing strategies across 100 problems in Python libraries. Results show current LLMs achieve 42-83% bug recall with structured prompting, revealing significant performance gaps where different models fail on different problems.