AINeutralarXiv – CS AI · 6h ago6/10
🧠
Where to Place the Query? Unveiling and Mitigating Positional Bias in In-Context Learning for Diffusion LLMs via Decoding Dynamics
Researchers demonstrate that query placement significantly impacts performance in Diffusion Large Language Models (dLLMs) during in-context learning, contrary to conventional practices inherited from autoregressive models. The study reveals a spatial recency effect in attention mechanisms and proposes Auto-ICL, a training-free strategy that dynamically optimizes query positioning to approach oracle performance across diverse tasks.