4 articles tagged with #sequence-modeling. AI-curated summaries with sentiment analysis and key takeaways from 50+ sources.
Researchers introduce STAR, a new autoregressive pretraining method for Vision Mamba that uses separators to quadruple input sequence length while maintaining image dimensions. The STAR-B model achieved 83.5% accuracy on ImageNet-1k, demonstrating improved performance through better utilization of long-range dependencies in computer vision tasks.
Researchers introduce Memory Caching (MC), a technique that enhances recurrent neural networks by allowing their memory capacity to grow with sequence length, bridging the gap between fixed-memory RNNs and growing-memory Transformers. The approach offers four variants and shows competitive performance with Transformers on language modeling and long-context tasks while maintaining better computational efficiency.
Researchers introduce a quantum-inspired sequence modeling framework that uses complex-valued wave functions and quantum interference for language processing. The approach shows theoretical advantages over traditional recurrent neural networks by utilizing quantum dynamics and the Born rule for token probability extraction.
Researchers developed SMDIM, a new diffusion model for symbolic music generation that efficiently handles long sequences by combining global structure construction with local refinement. The model outperforms existing approaches in both generation quality and computational efficiency across various musical styles including Western classical, popular, and folk music.
