Depth-Structured Music Recurrence: Budgeted Recurrent Attention for Full-Piece Symbolic Music Modeling

Researchers introduce Depth-Structured Music Recurrence (DSMR), a new AI training method for symbolic music generation that processes complete compositions efficiently. The technique uses stateful recurrent attention with distributed memory across layers, achieving similar performance to full-memory models while using 59% less GPU memory and 36% higher throughput.

• →DSMR enables end-to-end learning from complete musical compositions by streaming pieces left-to-right with recurrent attention.
• →The method distributes layer-wise memory horizons under a fixed budget, with lower layers getting longer history windows.
• →Two-scale DSMR matches full-memory reference models in perplexity while reducing GPU memory usage by approximately 59%.
• →The approach achieves roughly 36% higher throughput compared to traditional full-memory recurrent models.
• →Performance depends primarily on total allocated memory rather than which specific layers carry the memory load.