8 articles tagged with #jepa. AI-curated summaries with sentiment analysis and key takeaways from 50+ sources.
Researchers propose a hybrid pre-training approach for language models that combines masked language modeling with a JEPA-style latent-space prediction objective, creating more semantically-aligned embeddings with better geometric properties than traditional MLM-only approaches despite achieving similar downstream accuracy.
Researchers introduce LatentWave, a wireless foundation model that uses Joint-Embedding Predictive Architecture (JEPA) instead of traditional masked input reconstruction to learn more transferable representations from wireless spectrograms and channel state information. The model demonstrates improved performance across RF signal classification, 5G positioning, beam prediction, and LoS/NLoS classification tasks while supporting variable antenna configurations.
Researchers introduce BRo-JEPA, a neural network architecture that learns modular arithmetic rules by imposing circular structure in latent space, achieving 99.46% zero-shot generalization on unseen operations. The work demonstrates that neural networks can learn abstract algebraic rules rather than merely memorizing patterns when architecture aligns with problem structure.
Researchers introduce UR-JEPA, a novel regularization technique for Joint-Embedding Predictive Architectures that addresses representation collapse by targeting uniformly rectifiable measures rather than isotropic Gaussians. The method demonstrates superior performance on Inet10 with an 0.83 percentage-point gain over existing approaches and produces geometrically distinct embeddings with sharper spectral drops, suggesting more structured learned representations.
Echo is a proof-of-concept audio system that unifies speaker diarization, speech recognition, and source separation on a single 25M-parameter ViT encoder pretrained with joint-embedding predictive architecture (JEPA). The system demonstrates competitive performance across three tasks simultaneously without per-task fine-tuning, though it represents a design exploration rather than state-of-the-art on individual metrics.
Researchers propose Sub-JEPA, an improved approach to training world models that addresses stability issues in Joint-Embedding Predictive Architectures by applying Gaussian constraints across random subspaces rather than the full embedding space. The method achieves better performance than the existing LeWorldModel baseline while maintaining training stability and representation flexibility.
Researchers demonstrate that ProteinJEPA, a latent-space prediction technique, can complement traditional masked language modeling (MLM) in protein language models, achieving better downstream task performance when combined strategically. The optimal approach—masked-position MLM+JEPA—wins 10 out of 16 evaluation tasks against MLM-only baselines while maintaining computational efficiency.
Researchers propose SAGE (Self-supervised Action Gating with Energies), a new method to improve diffusion planners in offline reinforcement learning by filtering out dynamically inconsistent trajectories. The approach uses a latent consistency signal to re-rank candidate actions at inference time, improving performance across locomotion, navigation, and manipulation tasks without requiring environment rollouts or policy retraining.
