y0news
AnalyticsDigestsSourcesTopicsRSSAICrypto

#sequence-models News & Analysis

2 articles tagged with #sequence-models. AI-curated summaries with sentiment analysis and key takeaways from 50+ sources.

2 articles
AIBullisharXiv – CS AI · 8h ago7/10
🧠

Toeplitz MLP Mixers are Low Complexity, Information-Rich Sequence Models

Researchers introduce Toeplitz MLP Mixer (TMM), a transformer alternative that replaces attention mechanisms with triangular-masked Toeplitz matrix multiplication, achieving O(dn log n) training complexity and O(dn) inference complexity. TMMs demonstrate superior training efficiency, information retention, and in-context learning performance compared to existing sub-quadratic architectures.

AINeutralarXiv – CS AI · 8h ago6/10
🧠

Adaptive Memory Decay for Log-Linear Attention

Researchers propose a modification to log-linear attention mechanisms that learns adaptive memory decay parameters directly from input data rather than using fixed values. This approach maintains logarithmic memory growth and log-linear computational complexity while improving long-range context retention, particularly in language modeling and selective recall tasks.