#gemma-models News & Analysis

3 articles tagged with #gemma-models. AI-curated summaries with sentiment analysis and key takeaways from 50+ sources.

3 articles

AINeutralarXiv – CS AI · May 127/10

🧠

Causal Dimensionality of Transformer Representations: Measurement, Scaling, and Layer Structure

Researchers introduce causal dimensionality (kappa), a measurable property quantifying how transformer layers causally influence model outputs, finding that representational capacity grows 15.6x faster than causal capacity across scaling conditions. The metric remains invariant to model size increases, suggesting causal influence is a fundamental architectural property independent of parameter count.

AINeutralarXiv – CS AI · May 127/10

🧠

The Geometric Wall: Manifold Structure Predicts Layerwise Sparse Autoencoder Scaling Laws

Researchers demonstrate that sparse autoencoders (SAEs) used to interpret AI model activations face fundamental geometric constraints rather than just resource limitations. By analyzing 844 SAE checkpoints across Gemma 2 models, they show that manifold curvature and intrinsic dimensionality at each layer predict reconstruction performance, establishing a transferable geometric law that explains why SAE effectiveness varies across layers.

AINeutralarXiv – CS AI · May 76/10

🧠

PSK at SemEval-2026 Task 9: Multilingual Polarization Detection Using Ensemble Gemma Models with Synthetic Data Augmentation

Researchers achieved second place in SemEval-2026's multilingual polarization detection task by fine-tuning Gemma models with synthetic data augmentation across 22 languages. Their ensemble approach combining LoRA-adapted 12B and 27B parameter models with LLM-generated training data achieved a mean macro-F1 of 0.811, demonstrating the effectiveness of synthetic data strategies and per-language optimization for multilingual NLP tasks.

🧠 GPT-4