14 articles tagged with #information-extraction. AI-curated summaries with sentiment analysis and key takeaways from 50+ sources.
Researchers introduce MAMA, a framework measuring how network topology affects private information leakage in multi-agent LLM systems. The study demonstrates that denser connectivity and shorter distances between attackers and targets significantly increase memory leakage, with practical implications for securing distributed AI systems.
LeakDojo is a new research framework that systematically evaluates security vulnerabilities in Retrieval-Augmented Generation (RAG) systems, revealing that stronger LLM instruction-following capabilities correlate with higher data leakage risks. The study benchmarks six attack methods across multiple LLMs and datasets, providing critical insights into how RAG databases can be exploited and suggesting that improvements in RAG faithfulness may paradoxically increase security vulnerabilities.
Researchers have developed Guardian, an AI system using multiple large language models (LLMs) to assist in missing-person investigations during the critical first 72 hours. The system employs a consensus-driven pipeline that coordinates specialized LLM models for information extraction and processing, with fine-tuning using QLoRA methodology.
Researchers introduce Beaver, an AI agent harness designed to extract structured information from scientific papers containing multimodal evidence (text, tables, figures). The system achieves 81.0 on the Gold-Referenced Attribute Score, outperforming frontier agents by 23 points, demonstrating that harness design—not just underlying models—is critical for complex information extraction tasks.
Researchers evaluated 26 open-source small language models for extracting clinical terms related to amyotrophic lateral sclerosis (ALS) from unstructured patient notes, finding that hybrid approaches combining rule-based methods with machine learning outperform either approach alone. The study demonstrates that modest-sized language models can handle specialized medical information extraction tasks without task-specific training, though traditional regex-based systems remain competitive for this application.
Researchers deployed ACIE, an on-premise agentic RAG system at University Medicine Essen, to extract clinical information from fragmented patient records spanning hundreds of documents. Clinicians validated 7,326 extractions with 96.5% acceptance rates, demonstrating that agentic architectures with explicit reasoning can overcome standard RAG failures in handling temporal dependencies and missing metadata in healthcare contexts.
FineREX introduces a fine-tuned language model pipeline for extracting structured data from court documents to build knowledge graphs about human smuggling networks. The domain-specific approach achieves 15-31% performance gains over general-purpose models while reducing processing time by half, demonstrating that specialized AI outperforms larger generalist systems in legal document analysis.
Researchers propose IDEAL, a novel framework for query-focused summarization that enhances large language models through two key innovations: Query-aware HyperExpert for fine-grained query alignment and Query-focused Infini-attention for processing lengthy documents. The approach demonstrates effectiveness across existing QFS benchmarks and expands LLM accessibility for personalized text summarization.
Researchers introduce DiffuSent, a non-autoregressive diffusion framework that reformulates seven aspect-based sentiment analysis (ABSA) subtasks as boundary denoising processes. The approach achieves significant improvements over existing generative models, particularly on multi-word expressions, while delivering up to 181x faster inference speeds through parallel decoding rather than sequential token generation.
Researchers propose FiVeD, a fine-grained verification framework for Aspect Sentiment Triplet Extraction that improves extraction accuracy by up to 3.53 F1 points through multi-task learning with validity classification, quality scoring, error detection, and rationale generation. The framework addresses a critical gap in ASTE systems by post-hoc verification of extracted triplets, enabling adjustable precision-recall tradeoffs for downstream NLP applications.
Researchers introduce DTBench, a synthetic benchmark for evaluating large language models on document-to-table extraction tasks. Using a reverse Table2Doc synthesis approach with multi-agent workflows, the benchmark covers 13 subcategories across 5 major capability areas, revealing significant performance gaps and persistent challenges in reasoning and conflict resolution across mainstream LLMs.
Researchers developed a hybrid neural-symbolic pipeline for extracting clinical follow-up instructions from outpatient notes, pairing medical actions with future dates. The system significantly outperformed generative AI models (GPT-4o-mini and LLaMA-3) at linking actions to dates, achieving 99.7% F1 score on seen data versus 51-57% for baselines, demonstrating that symbolic reasoning outperforms pure language generation for structured clinical extraction tasks.
Researchers introduce Doc-CoB, a new framework that improves how AI models understand documents by progressively focusing on relevant layout regions while maintaining global context. The approach combines coarse-to-fine visual reasoning with multimodal large language models and demonstrates significant performance improvements across seven benchmarks.
Researchers introduce DiZiNER, a framework that improves zero-shot named entity recognition by simulating human annotation disagreement processes using multiple LLMs. The approach achieves state-of-the-art results on 14 of 18 benchmarks, closing the performance gap between zero-shot and supervised systems by over 11 percentage points.
