y0news
AnalyticsDigestsSourcesTopicsRSSAICrypto

#linear-complexity News & Analysis

5 articles tagged with #linear-complexity. AI-curated summaries with sentiment analysis and key takeaways from 50+ sources.

5 articles
AIBullisharXiv – CS AI · Jun 237/10
🧠

An Efficient and Effective Architecture for Large-Scale Traffic Prediction via Geometry-Adaptive Square Partitioning

Researchers introduce SqLinear, a neural network architecture that improves traffic prediction scalability by replacing attention mechanisms with efficient linear interactions and using a geometry-adaptive partitioning algorithm. The approach achieves 2.3-5.8% accuracy improvements while reducing training time by up to 30.8% on large-scale traffic datasets.

AIBullisharXiv – CS AI · Jun 57/10
🧠

Exact Linear Attention

Researchers introduce Exact Linear Attention (ELA), a novel Transformer mechanism that achieves linear computational complexity while eliminating approximation errors in attention calculations. The approach demonstrates significant practical improvements including 6x faster decoding speeds and 75% reduction in KV cache memory, with extensions to vision models showing 4.3x GPU speedup.

AIBullisharXiv – CS AI · Mar 37/102
🧠

RMAAT: Astrocyte-Inspired Memory Compression and Replay for Efficient Long-Context Transformers

Researchers introduce RMAAT (Recurrent Memory Augmented Astromorphic Transformer), a new architecture inspired by brain astrocyte cells that addresses the quadratic complexity problem in Transformer models for long sequences. The system uses recurrent memory tokens and adaptive compression to achieve linear complexity while maintaining competitive accuracy on benchmark tests.

AIBullisharXiv – CS AI · Feb 277/106
🧠

ViT-Linearizer: Distilling Quadratic Knowledge into Linear-Time Vision Models

Researchers developed ViT-Linearizer, a distillation framework that transfers Vision Transformer knowledge into linear-time models, addressing quadratic complexity issues for high-resolution inputs. The method achieves 84.3% ImageNet accuracy while providing significant speedups, bridging the gap between efficient RNN-based architectures and transformer performance.