y0news
AnalyticsDigestsSourcesTopicsRSSAICrypto

#transformer-analysis News & Analysis

7 articles tagged with #transformer-analysis. AI-curated summaries with sentiment analysis and key takeaways from 50+ sources.

7 articles
AINeutralarXiv – CS AI · Jun 17/10
🧠

When LLMs Learn to Be Consistently Wrong: A Multi-Model Study of Linear Representations of Synthetic Deception

Researchers demonstrate that large language models trained to produce dishonest outputs develop clear, detectable internal representations of deception across multiple architectures. Using linear probes on transformer models, the study achieves near-perfect accuracy in identifying synthetic dishonesty, with implications for AI safety monitoring and the feasibility of detecting deceptive alignment in advanced language models.

🧠 Llama
AINeutralarXiv – CS AI · Jun 26/10
🧠

MLLM-Microscope: Unlocking Hidden Structure Within Multimodal Large Language Models

Researchers introduce MLLM-Microscope, a novel analytical system that examines the internal representations of multimodal large language models (MLLMs) by measuring linearity, intrinsic dimension, and anisotropy across transformer layers. Testing on LLaVA-NeXT and OmniFusion reveals that modality fusion approaches significantly influence how embeddings behave within the model architecture, with OmniFusion demonstrating more consistent dimensional properties across layers.

AINeutralarXiv – CS AI · May 286/10
🧠

Geometry of Human Perceptual Domains Emerges Transiently in LLM Representations

Researchers discovered that large language models develop geometric structures in their internal representations that mirror human perceptual organization across domains like color, pitch, and emotion, despite training only on text. These perceptual geometries emerge transiently in intermediate layers, providing new insight into how LLMs develop human-like conceptual understanding without direct sensory supervision.

AINeutralarXiv – CS AI · May 286/10
🧠

ReSAE: Residualized Sparse Autoencoders for Multi-Layer Transformer Interventions

Researchers introduce Residualized Sparse Autoencoders (ReSAEs), a new technique that improves how transformer models are analyzed and modified by accounting for information flow across multiple layers. By training autoencoders on residual activations rather than raw activations, ReSAEs reduce redundancy and better preserve model functionality during multi-layer interventions.

AINeutralarXiv – CS AI · May 126/10
🧠

Probing Cross-modal Information Hubs in Audio-Visual LLMs

Researchers have analyzed how audio-visual large language models (AVLLMs) process cross-modal information, discovering that integrated audio-visual data concentrates in specialized 'sink tokens' rather than distributing uniformly. This finding enables a training-free method to reduce hallucinations by leveraging these cross-modal information hubs.

AINeutralarXiv – CS AI · May 96/10
🧠

HyperLens: Quantifying Cognitive Effort in LLMs with Fine-grained Confidence Trajectory

Researchers introduce HyperLens, a high-resolution analysis tool that measures cognitive effort in large language models by tracking confidence trajectories across transformer layers. The study reveals that complex tasks consistently require higher cognitive effort and identifies how standard fine-tuning can paradoxically reduce model performance by decreasing necessary cognitive investment.