#transformer-efficiency News & Analysis

4 articles tagged with #transformer-efficiency. AI-curated summaries with sentiment analysis and key takeaways from 50+ sources.

4 articles

AIBullisharXiv – CS AI · May 127/10

🧠

Pretraining large language models with MXFP4

Researchers identify weight gradient (Wgrad) quantization as the primary cause of instability in FP4 training of large language models, while forward and activation gradient quantization prove relatively benign. Using deterministic Hadamard rotations on AMD MI355X GPUs, they demonstrate that structured micro-scaling errors—not insufficient randomness—drive training divergence, offering insights for efficient LLM pretraining.

🧠 Llama

AIBullisharXiv – CS AI · May 97/10

🧠

Shallow Prefill, Deep Decoding: Efficient Long-Context Inference via Layer-Asymmetric KV Visibility

Researchers introduce SPEED, a novel inference optimization technique for long-context language models that reduces computational cost by materializing key-value cache states only in lower layers during the prefill phase while maintaining full-depth processing during decoding. Testing on Llama-3.1-8B demonstrates 33% improvement in time-to-first-token, 22% improvement in tokens-per-second, and 25% reduction in KV memory with minimal quality degradation, suggesting that prompt tokens don't require persistent full-depth caching.

🧠 Llama

AIBullisharXiv – CS AI · Apr 107/10

🧠

Q-Zoom: Query-Aware Adaptive Perception for Efficient Multimodal Large Language Models

Q-Zoom is a new framework that improves the efficiency of multimodal large language models by intelligently processing high-resolution visual inputs. Using adaptive query-aware perception, the system achieves 2.5-4.4x faster inference speeds on document and high-resolution tasks while maintaining or exceeding baseline accuracy across multiple MLLM architectures.

AIBullisharXiv – CS AI · 15h ago6/10

🧠

One LR Doesn't Fit All: Heavy-Tail Guided Layerwise Learning Rates for LLMs

Researchers introduce Layerwise Learning Rate (LLR), an adaptive training technique that assigns different learning rates to individual Transformer layers based on Heavy-Tailed Self-Regularization theory. Testing across multiple LLM architectures and scales demonstrates up to 1.5x training speedup and improved generalization, with zero-shot accuracy improvements of 2-3% on billion-parameter models.