77 articles tagged with #in-context-learning. AI-curated summaries with sentiment analysis and key takeaways from 50+ sources.
Researchers demonstrate that large language models' in-context learning capabilities can efficiently support intrinsic curiosity mechanisms for automated data collection, though with important theoretical limitations. The work proves this approach works for non-temporal settings like active learning but fails for general sequential decision problems without computational shortcuts.
Researchers demonstrate that query placement significantly impacts performance in Diffusion Large Language Models (dLLMs) during in-context learning, contrary to conventional practices inherited from autoregressive models. The study reveals a spatial recency effect in attention mechanisms and proposes Auto-ICL, a training-free strategy that dynamically optimizes query positioning to approach oracle performance across diverse tasks.
PersonaDrive introduces a retrieval-augmented vision-language-action (VLA) system that enables autonomous driving agents to exhibit diverse human-like behavioral styles in simulation environments. Using demonstrations from human drivers instructed to drive aggressively, neutrally, or conservatively, the system achieves superior performance on driving benchmarks while allowing style selection without per-style retraining.
Researchers introduce TrajGenAgent, an LLM-based framework that generates realistic synthetic human mobility trajectories without model fine-tuning by combining hierarchical agent design with deterministic workflows. The approach addresses privacy and cost constraints in trajectory data collection while maintaining semantic coherence and behavioral realism.
Researchers introduce GILT, a Graph Foundational Model that enables in-context learning on graph neural networks without requiring large language models or per-task tuning. The approach achieves stronger few-shot performance than existing methods while reducing computational overhead, addressing a critical limitation in deploying GNNs to heterogeneous graph data.
Researchers identify a 'structural attention tax' where knowledge graph formats capture 2-3x more model attention than semantically equivalent natural language, degrading in-context learning performance by up to 42% regardless of content relevance. The study formalizes attention decomposition into semantic and structural components, revealing that retrieval format can independently distort LLM outputs independent of knowledge quality.
Researchers introduce Chain of Operators (CHOP), a framework that enables frozen neural operator models to handle out-of-distribution tasks without fine-tuning by constructing chains of explicit mathematical transformations. The approach demonstrates improved generalization across different PDE families while maintaining interpretability.
Researchers introduce LWM-Planner, a fact-augmented lookahead planning framework that enhances LLM agent decision-making through in-context learning without parameter updates. The system extracts task-critical facts from agent trajectories, validates them through a predictive-consistency filter, and uses these facts to improve planning accuracy across interactive environments.
Researchers introduce UniTok, a universal tokenizer that converts continuous time series data into discrete tokens, enabling UniTok-FM—a foundation model pretrained via next-token prediction. This unified approach supports forecasting, generation, and classification tasks without task-specific modifications, achieving competitive performance with specialized models while enabling zero-shot and few-shot inference capabilities.
Researchers introduce a new benchmark dataset for evaluating how Vision Language Models adapt to dynamic, user-specific preferences provided at inference time rather than learned from training data. The work addresses a gap in VLM evaluation by testing real-time preference adaptation across multiple users, moving beyond static capability assessments.
Researchers introduce CORAL, a framework that enables reinforcement learning agents to adapt to new tasks without retraining by separating world modeling from control through emergent communication between two agents. The approach demonstrates improved sample efficiency and zero-shot adaptation across diverse environments, advancing in-context reinforcement learning capabilities.
Researchers introduce MHA-RAG, a framework that encodes domain-specific exemplars as soft prompts instead of text, achieving 20-point performance improvements over standard RAG while reducing inference costs by 10X. The approach demonstrates order-invariant performance across multiple question-answering benchmarks, addressing key challenges in adapting foundation models to new domains with limited data.
Researchers propose applying Tabular Foundation Models to industrial Prognostics and Health Management (PHM) tasks by converting time-series signals into tabular representations. The approach demonstrates superior performance across diagnostics and prognostics compared to sequence models and transformers, while achieving high data efficiency in low-data industrial settings.
Researchers propose an in-context learning approach for Multiple Instance Learning (MIL) using Perceiver-style architecture pretrained on synthetic data, enabling models to solve new tasks with minimal labeled examples. The method outperforms supervised baselines across twelve benchmarks while requiring no task-specific training at inference time.
Researchers present RDBLearn, a foundation model that enables in-context learning over relational databases without requiring model training or fine-tuning. By developing principled compression techniques that preserve semantic relationships within database columns rather than across heterogeneous data types, the approach allows existing single-table foundation models to operate effectively on multi-table database systems.
Instant-Fold is an in-context imitation learning framework that enables robots to manipulate deformable objects like cloth by learning from single human demonstrations. The system uses deformation-aware visual representations and flow-matching transformers to generalize across diverse folding modes and transfers directly to real-world tasks without additional training.
Researchers introduce KITE, a novel example selection method for in-context learning in large language models that uses information theory and kernel methods to choose task-specific examples from a prompt bank. The approach addresses limitations of existing nearest-neighbor methods by improving diversity and generalization, demonstrating measurable improvements across classification tasks in label-scarce scenarios.
Researchers introduce GA-ICL, a geometry-aware framework that improves hallucination detection in large language models by selecting better in-context learning demonstrations. Rather than relying on surface-level text similarity, the method uses latent representations and prototype geometry to choose demonstrations, achieving stronger performance across factual verification and hallucination detection benchmarks while maintaining robustness across model scales.
Researchers propose DOPA, a demonstration retrieval framework that uses out-of-distribution proxies to improve large language model performance on tasks from inaccessible target domains. The method combines proxy-based evaluation with diversity constraints to enhance LLM robustness when facing severe distribution shifts.
Researchers propose a novel metric called 'Decan' for measuring diversity in AI-generated creative outputs using in-context learning and language model probabilities, achieving 84.6% accuracy on benchmark tests. The approach detects mode collapse and diversity loss across training stages without requiring specialized embedding models or human annotation, offering a practical tool for evaluating generative AI systems.
Researchers propose RA-LWLM, a retrieval-augmented framework for wireless localization in 6G networks that eliminates the need for retraining when base station configurations or environments change. The system combines a frozen wireless foundation model with a retrieval database and in-context learning to achieve consistent accuracy across different scenes without per-scene model adaptation.
Researchers propose Palimpsa, a self-attention model that frames in-context learning as a continual learning problem using Bayesian metaplasticity to overcome memory interference in long sequences. The framework unifies existing gated linear attention models as special cases and demonstrates improved performance on associative recall and reasoning tasks, offering a theoretical foundation for enhancing memory capacity in transformer-based architectures.
PictSure introduces a vision-only in-context learning framework for few-shot image classification that demonstrates representation quality from pretraining is the critical bottleneck, not fusion-layer training diversity. The researchers release open-source models and an MCP server enabling few-shot image classification integration directly into LLM-based systems.
Researchers propose In-Context Reward Adaptation, a transformer-based framework that dynamically models diverse human preferences without costly retraining. By incorporating human response time as an auxiliary signal, the approach enables language models to adapt to unseen preference domains on-the-fly, addressing a critical limitation of static reward models used in RLHF systems.
Researchers evaluated how multimodal large language models (MLLMs) explain their image classification decisions in few-shot learning scenarios. The study found that forcing models to generate formal, concept-based explanations actually reduces their predictive accuracy from 93.8% to 90.1%, suggesting that explicit reasoning doesn't universally improve performance despite being widely assumed to do so.
