Beyond Single-Modal Analytics: A Framework for Integrating Heterogeneous LLM-Based Query Systems for Multi-Modal Data

Researchers introduce Meta Engine, a unified semantic query system that integrates multiple specialized LLM-based query systems to handle multi-modal data analysis. The system addresses fragmentation in current semantic query tools by combining specialized systems through five key components, achieving 3-24x better performance than existing baselines.

• →Current LLM-based semantic query systems face integration challenges due to disparate APIs and trade-offs between specialization and generality.
• →Meta Engine introduces a 'query system on query systems' approach to unify heterogeneous LLM-based query systems for multi-modal data.
• →The framework includes five components: NL Query Parser, Operator Generator, Query Router, Adapters, and Result Aggregator.
• →Meta Engine demonstrates significant performance improvements with 3-6x higher F1 scores in most cases and up to 24x on specific datasets.
• →The system resolves the fundamental trade-off between specialized single-modal systems and generalized multi-modal approaches.