#text-to-sql2 articles

2 articles

AINeutralarXiv – CS AI · 5h ago

🧠

SpotIt+: Verification-based Text-to-SQL Evaluation with Database Constraints

SpotIt+ is a new open-source tool that evaluates Text-to-SQL systems through verification-based testing, actively searching for database instances that reveal differences between generated and ground truth SQL queries. The tool incorporates constraint-mining that combines rule-based specification mining with LLM validation to generate more realistic test scenarios.

AINeutralarXiv – CS AI · 5h ago

🧠

SpotIt: Evaluating Text-to-SQL Evaluation with Formal Verification

Researchers introduce SpotIt, a new evaluation method for Text-to-SQL systems that uses formal verification to find database instances where generated queries differ from ground-truth queries. Testing on the BIRD dataset revealed that current test-based evaluation methods often miss differences between generated and correct SQL queries.