185 articles tagged with #prompt-engineering. AI-curated summaries with sentiment analysis and key takeaways from 50+ sources.
Skill-MAS introduces a novel framework that enhances multi-agent AI systems by evolving meta-skills through a closed optimization loop, achieving significant performance gains while maintaining cost efficiency across diverse LLMs and tasks.
Researchers introduce WikiProfile, a benchmark that reframes LLM factuality failures as either missing knowledge or poor recall of encoded information. Analysis of 13 models shows frontier models encode 95-98% of facts but struggle significantly with recall, suggesting future improvements depend less on scaling and more on better knowledge access mechanisms.
Researchers identify 'framing disparity' as a hidden source of bias in large language models, where semantically equivalent prompts expressed differently produce inconsistent fairness outcomes. The study proposes DeFrame, a debiasing method that improves LLM consistency across alternative framings, addressing a gap between standard fairness evaluations and real-world performance.
Researchers introduce a prompt-based uncertainty decomposition method that enables LLM agents to proactively seek clarification when task specifications are ambiguous. The approach separates action confidence from request uncertainty and demonstrates 36-73% improvements in clarification performance across multiple LLM backbones compared to existing uncertainty frameworks.
Researchers introduce Tahoe, a system that optimizes LLM-based Text-to-SQL conversion through dynamic prompt engineering rather than model retraining. By consolidating debugging traces into reusable hints and modeling conflicting user intents as strategies, Tahoe increases query pass rates from 62% to 79% on Spider 2.0-Snow benchmarks while maintaining compatibility across weaker model backbones.
Researchers present the Minimum Viable Evaluation Suite (MVES), a framework for systematically testing LLM applications, revealing that generic prompt improvements often fail to deliver consistent gains and can cause significant performance regressions. Testing on local models showed that adding generic rules to prompts degraded RAG citation compliance by up to 70%, underscoring the need for rigorous, task-specific evaluation before deployment.
Researchers demonstrate that selective context management—retaining only recent tool interactions plus automated summarization—enables LLM agents to complete enterprise workflows with 91.6% success while reducing token consumption and runtime by ~63% compared to full-history retention. The findings challenge the assumption that maximum context retention improves agent performance in long-horizon tasks.
Researchers introduce Rotate2Think, a training-free method that improves language model reasoning by applying geometric transformations to embedding space. The technique identifies that input and reasoning embeddings occupy distinct directional regions and uses orthogonal rotation to geometrically prime the model before generating reasoning traces, showing consistent accuracy improvements across 30 of 32 tested model-benchmark configurations.
Researchers characterize how memory-design choices in foundation-model agents affect privacy and utility, introducing metrics to measure personalization recall, extraction risk, and deletion fidelity. Key-fact summarization reduces data extraction vulnerability by 64-76% while preserving personalization, but creates deletion-fidelity failures where compressed data remains recoverable without full-pipeline purging.
Researchers introduce MemToolAgent, a framework that enhances LLM agents' ability to use tools effectively by implementing memory management systems that store and retrieve past experiences. The approach achieves significant performance improvements (17-80% relative gains) across multiple benchmarks without requiring model fine-tuning, suggesting practical advances in making AI agents more personalized and reliable.
A controlled study comparing three AI scaffolding approaches across five large language models reveals that prompt engineering and system design choices can swing accuracy by up to 28 percentage points on the same task, challenging assumptions that published capability scores reflect true model performance and suggesting the elicitation gap persists even as models improve.
Researchers have identified a critical reliability flaw in multimodal large language models (MLLMs) used for video understanding: when the correct answer is absent from available options, these models fail to recognize it and instead select plausible incorrect alternatives. Testing across multiple models and benchmarks reveals this limitation is especially severe in temporal reasoning tasks and worsens with increased video frame sampling, with chain-of-thought prompting offering only partial mitigation.
A comprehensive study of deployed LLM-based agents across 26 domains reveals that production systems rely on simple, human-centered approaches rather than complex automation. The research shows 68% of agents require human intervention within 10 steps, 70% use prompt engineering instead of model fine-tuning, and reliability remains the primary development challenge addressed through systems-level design.
Researchers discovered that large language models refuse to correct their own reasoning errors but readily accept corrections when identical claims come from external sources like users or tools. This behavior stems not from cognitive limitations but from how chat templates assign roles to different message types, suggesting AI systems may have built-in biases toward authoritative external sources.
Researchers introduced CogManip, a new AI safety benchmark evaluating 15 manipulation strategy risks across 1,000 multi-turn LLM interactions. Testing 13 models including GPT-5.4 and DeepSeek-V3.2 revealed significant vulnerabilities to covert psychological manipulation tactics, with findings suggesting prompt-based defenses can mitigate these risks.
Researchers introduce ToolMaze, a benchmark testing how AI language models handle real-world tool failures and recovery scenarios, revealing that implicit semantic failures cause performance drops of ~37% and that fault-tolerance improves significantly slower than basic task performance as models scale.
Researchers discovered that incidental contextual cues in prompts systematically steer LLM code generation toward different algorithms, even when all outputs are functionally correct. Across 46,535 experiments, subtle variations in wording and metadata produced algorithm-choice shifts up to 100 percentage points, creating unpredictable performance and security outcomes in production code.
Researchers demonstrate that Large Language Models exhibit significant limitations in zero-shot annotation tasks, with only 34.8% of initial errors correctable through prompting. The study reveals that model-internalized priors and concept definitions strongly influence LLM performance more than text-level memorization, highlighting fundamental constraints in LLM adaptability for reliable AI-as-a-judge applications.
Researchers introduce Thought-ICS, a self-correction framework that structures LLM reasoning into discrete thought steps, enabling models to identify and fix errors more reliably. The method achieves 20-40% improvement in self-correction when errors are verified externally, and outperforms existing baselines in fully autonomous settings.
Researchers introduce KACE, a novel context engineering method that improves large language models' mathematical reasoning by separating knowledge storage from usage through difficulty and domain-based organization. The approach achieves 62.2% accuracy on AIME 2025, significantly outperforming existing self-consistency methods while maintaining comparable computational efficiency.
COLLEAGUE.SKILL is an open-source system that automates the conversion of expert knowledge traces into portable, inspectable AI agent skills through a structured distillation workflow. The framework enables person-grounded agents to encode human expertise, decision-making patterns, and communication styles as versioned, correctable skill packages that can be deployed across multiple agent hosts.
Researchers demonstrate a novel poisoning attack on retrieval-augmented text-to-music systems where attackers inject malicious captions into music databases to manipulate generation outputs toward attacker-chosen targets while maintaining alignment with original user prompts. The attack reveals a critical integrity vulnerability in AI systems that depend on external knowledge bases for prompt augmentation.
Researchers investigated how prompt tone affects Large Language Model accuracy across multiple models and datasets, finding that tonal variations produce systematic yet model-dependent performance shifts. Testing ChatGPT-4o, ChatGPT-5-nano, Gemini 2.5 Flash, and Gemini 2.5 Flash Lite on 50-620 multiple-choice questions, they discovered some models show statistically significant accuracy changes while others experience large swings, with sensitivity varying by subject domain. The findings highlight that LLM reliability cannot be assumed tone-robust in production deployments.
Researchers introduce SCOPE, a framework that enables Large Language Model agents to automatically evolve their prompts by learning from execution traces in dynamic environments. The system improves task success rates from 14.23% to 38.64% on benchmark tests, addressing a critical limitation in how LLM agents manage complex, changing contexts without human intervention.
Researchers introduce WIRE, a diagnostic pipeline for detecting conflicting rules within LLM agent prompt policies. Testing six public policies, the system identified 170 rule-pair conflicts and found that 64.6% of witnessed conflict scenarios resulted in at least one source-rule violation, revealing significant gaps in how language models handle competing policy directives.
