#transformer-internals News & Analysis

3 articles tagged with #transformer-internals. AI-curated summaries with sentiment analysis and key takeaways from 50+ sources.

3 articles

AIBullisharXiv – CS AI · Jun 27/10

🧠

TriLens: Per-Layer Logit-Lens Entropy for White-Box Hallucination Detection

TriLens is a novel white-box detection method that identifies hallucinations in language models by tracking entropy changes across internal computational layers. Rather than examining only final outputs, the technique monitors uncertainty signals from multi-head attention, feed-forward networks, and residual streams using logit lens analysis, creating a compact 3L-dimensional trajectory that reveals how model confidence settles during inference.

AINeutralarXiv – CS AI · Jun 26/10

🧠

Understanding LLM Behavior in Multi-Target Cross-Lingual Summarization

Researchers introduce MEA, a new benchmark for multi-target cross-lingual summarization (MTXLS) covering 24 languages, and reveal that LLMs perform this task substantially worse than English monolingual summarization. A novel layer-wise analysis shows that translation and summarization behaviors emerge jointly in later layers rather than as separate stages, enabling a new activation steering method that improves MTXLS quality across languages.

AINeutralarXiv – CS AI · May 126/10

🧠

Belief or Circuitry? Causal Evidence for In-Context Graph Learning

Researchers present causal evidence that large language models learn in-context through dual mechanisms combining genuine structure inference with local pattern-matching, rather than relying on either approach alone. Using graph random-walk tasks and activation patching techniques, they demonstrate that LLMs simultaneously encode multiple competing graph topologies in orthogonal representational subspaces and show that late-layer circuits causally drive graph-preference predictions.