Larch: Learned Query Optimization for Semantic Predicates

Larch is a new optimization framework that improves the efficiency of semantic SQL queries by reducing token usage and computational costs when processing unstructured data with Large Language Models. The framework uses two approaches—reinforcement learning and supervised learning—to optimize the order of filter evaluation, achieving 3x-19x token cost reductions compared to existing solutions.

Larch addresses a critical bottleneck in modern database systems where semantic operators powered by LLMs introduce substantial latency and inference costs. As enterprises increasingly query unstructured data like text and images through AI-enhanced SQL interfaces, the computational overhead becomes a significant barrier to scalability. The framework's innovation lies in treating semantic filter optimization as a learnable problem rather than accepting LLMs as black boxes, enabling runtime optimization techniques that would be infeasible with faster traditional operators.

The dual-variant approach reflects pragmatic engineering choices for different optimization scenarios. Larch-A2C uses reinforcement learning to dynamically determine optimal filter evaluation order by modeling the problem as a Markov decision process, while Larch-Sel employs supervised learning to predict filter selectivities and apply dynamic programming. This flexibility suggests the framework adapts to varying data characteristics and query patterns, a crucial consideration for production database systems handling heterogeneous workloads.

The demonstrated 3x-19x improvement in token usage carries direct cost implications for organizations operating AI-powered analytics at scale. Token consumption directly translates to LLM API expenses, making Larch's efficiency gains financially significant beyond mere performance metrics. For database vendors and enterprises deploying semantic search capabilities, such optimization directly impacts operational expenses and query latency experienced by end users.

The research validates that learned optimization—combining machine learning with traditional database techniques—creates meaningful value in specialized domains. As semantic queries become mainstream in enterprise data stacks, query optimization frameworks like Larch will differentiate systems capable of cost-effective AI analytics from those struggling with computational overhead.

• →Larch reduces token consumption costs for semantic SQL queries by 3x-19x through intelligent filter optimization.
• →The framework uses embedding-augmented neural networks and reinforcement learning to optimize semantic filter evaluation order.
• →Semantic operators remain opaque to traditional database optimizers, creating an opportunity for learned optimization techniques.
• →Token cost efficiency is critical for production AI-SQL systems operating on large-scale unstructured data.
• →The dual-variant approach demonstrates that different optimization strategies suit different workload characteristics.