Prime Fourier Embeddings: A Principled Basis for Modular Arithmetic

Researchers introduce Prime Fourier Embeddings (PFE), a neural representation method that encodes integers using prime-indexed trigonometric pairs to expose algebraic structure in modular arithmetic. The approach achieves perfect accuracy on modular tasks with specialized neural channels corresponding to individual primes, validated through ablation studies showing 500x specialization ratios between relevant and irrelevant channels.

Prime Fourier Embeddings represent a significant methodological advancement in how neural networks can be structured to learn mathematical operations. Rather than forcing networks to discover algebraic properties through training, PFE bakes mathematical structure directly into the embedding space, creating a foundation where the architecture itself reflects the underlying mathematics. This aligns embeddings with harmonic analysis principles from group theory, specifically leveraging Schur's lemma to guarantee that equivariant linear transformations must respect prime channel independence.

The work builds on growing recognition that standard neural embeddings fail to capture numeric algebraic properties, forcing models to learn arithmetic relationships through brute force rather than principled design. Prior approaches relied on positional encodings or learned representations, often requiring extensive training data to discover even simple patterns. By anchoring embeddings to prime factorization and the Chinese Remainder Theorem, PFE provides a mathematically justified alternative that generalizes across all square-free composite moduli without task-specific training.

For machine learning practitioners, this demonstrates how theoretical mathematics can directly improve neural architecture design. The 500x specialization ratios indicate that networks naturally exploit the embedded structure, suggesting computational efficiency gains. Developers building systems requiring modular arithmetic—from cryptographic applications to certain AI safety mechanisms—could benefit from more efficient, interpretable models. The perfect in-distribution accuracy indicates that properly structured representations eliminate entire classes of learning problems.

Future applications may extend PFE principles to other algebraic structures beyond modular arithmetic, potentially influencing how neural networks handle number theory, cryptography, and symbolic computation more broadly.

• →Prime Fourier Embeddings encode integers using prime-indexed sine/cosine pairs to expose algebraic structure for modular arithmetic tasks.
• →Schur's lemma mathematically guarantees that equivariant linear maps must be block-diagonal with independent blocks per prime.
• →Empirical results show 500x specialization ratios between task-relevant and irrelevant channels with perfect test accuracy.
• →The Chinese Remainder Theorem predicts which prime channels are relevant for square-free composite moduli.
• →Principled mathematical structure in embeddings can replace discovered algebraic relationships, improving efficiency and interpretability.