#model-expressivity News & Analysis

2 articles tagged with #model-expressivity. AI-curated summaries with sentiment analysis and key takeaways from 50+ sources.

2 articles

AINeutralarXiv – CS AI · Jun 26/10

🧠

On the Theoretical Limitations of Embedding-based Link Prediction

Researchers identify fundamental limitations in knowledge graph embedding models caused by linear output layers that create "rank bottlenecks," restricting how well these systems can learn link prediction tasks. The study proposes using non-linear mixture-based output layers as a solution, demonstrating improved performance on large, dense datasets without substantial parameter increases.

AINeutralarXiv – CS AI · Jun 16/10

🧠

Revisiting Padded Transformer Expressivity: Which Architectural Choices Matter and Which Don't

Researchers demonstrate that padded transformers maintain consistent computational expressivity across various architectural choices, with numeric precision and model depth emerging as the primary factors determining capability. The findings establish formal equivalences between transformer models and circuit complexity classes, suggesting practical transformer designs are more robust than previously understood.