AIBullishOpenAI News · Apr 237/105
🧠
Generative modeling with sparse transformers
Researchers have developed the Sparse Transformer, a deep neural network that achieves new performance records in sequence prediction for text, images, and sound. The model uses an improved attention mechanism that can process sequences 30 times longer than previously possible.