AINeutralarXiv – CS AI · 3h ago6/10
🧠
Caracal: Causal Architecture via Spectral Mixing
Researchers introduce Caracal, a novel architecture that replaces attention mechanisms with a parameter-efficient Multi-Head Fourier module to improve LLM scalability for long sequences. The approach achieves O(L log L) complexity using Fast Fourier Transform, implements frequency-domain causal masking for autoregressive generation, and uses standard library operators for broad deployment compatibility.