#backpropagation News & Analysis

7 articles tagged with #backpropagation. AI-curated summaries with sentiment analysis and key takeaways from 50+ sources.

7 articles

AINeutralarXiv – CS AI · May 287/10

🧠

Misalignment Between Backpropagation and the Hierarchy of Brain Responses to Images

Researchers using fMRI and MEG data found that while backpropagated gradients in deep neural networks can predict brain activity in higher visual cortex, their spatial and temporal organization fundamentally diverges from how the human brain processes visual information. This suggests that although artificial and biological neural networks may learn similar representations, they employ distinctly different learning mechanisms.

AIBullisharXiv – CS AI · Mar 267/10

🧠

Moonwalk: Inverse-Forward Differentiation

Researchers introduce Moonwalk, a new algorithm that solves backpropagation's memory limitations by eliminating the need to store intermediate activations during neural network training. The method uses vector-inverse-Jacobian products and submersive networks to reconstruct gradients in a forward sweep, enabling training of networks more than twice as deep under the same memory constraints.

AIBullisharXiv – CS AI · Mar 177/10

🧠

HO-SFL: Hybrid-Order Split Federated Learning with Backprop-Free Clients and Dimension-Free Aggregation

Researchers propose HO-SFL (Hybrid-Order Split Federated Learning), a new framework that enables memory-efficient fine-tuning of large AI models on edge devices by eliminating backpropagation on client devices while maintaining convergence speed comparable to traditional methods. The approach significantly reduces communication costs and memory requirements for distributed AI training.

AIBullisharXiv – CS AI · Mar 117/10

🧠

A Variational Latent Equilibrium for Learning in Cortex

Researchers propose a new biologically plausible framework for approximating backpropagation through time (BPTT) in neural networks that mimics how the brain learns spatiotemporal patterns. The approach uses energy conservation principles to create local, time-continuous learning equations that could enable more brain-like AI systems and physical neural computing circuits.

AIBullisharXiv – CS AI · Mar 97/10

🧠

Predictive Coding Networks and Inference Learning: Tutorial and Survey

Researchers present a comprehensive survey of Predictive Coding Networks (PCNs), a neuroscience-inspired AI approach that uses biologically plausible inference learning instead of traditional backpropagation. PCNs can achieve higher computational efficiency with parallelization and offer a more versatile framework for both supervised and unsupervised learning compared to traditional neural networks.

AINeutralarXiv – CS AI · 4d ago6/10

🧠

Pretraining Recurrent Networks without Recurrence

Researchers propose Supervised Memory Training (SMT), a novel method for training recurrent neural networks that replaces sequential backpropagation through time with parallel, supervised learning on memory state transitions. By leveraging a Transformer encoder to generate training labels, SMT achieves stable gradient propagation and improved performance on language and sequence modeling tasks without the parallelism constraints of traditional RNN training.

AINeutralarXiv – CS AI · May 126/10

🧠

Biological Plausibility and Representational Alignment of Feedback Alignment in Convolutional Networks

Researchers demonstrate that modified feedback alignment (FA) algorithms can train convolutional neural networks while maintaining biological plausibility, with internal representations converging to structures similar to backpropagation despite using fundamentally different weight update mechanisms. This finding suggests that successful learning algorithms may achieve comparable results through different computational paths, bridging biologically plausible alternatives with practical neural network training.