AIBullisharXiv – CS AI · Jun 237/10
🧠Researchers propose Hierarchical Block-Local Learning (HBLL), a novel deep learning framework that trains neural networks with O(log N) parallel time complexity by decomposing networks into hierarchically linked blocks with local learning objectives. This approach eliminates sequential backpropagation constraints, addressing the locking problem and weight transport challenge while maintaining competitive performance on vision and language tasks.
AINeutralarXiv – CS AI · May 287/10
🧠Researchers using fMRI and MEG data found that while backpropagated gradients in deep neural networks can predict brain activity in higher visual cortex, their spatial and temporal organization fundamentally diverges from how the human brain processes visual information. This suggests that although artificial and biological neural networks may learn similar representations, they employ distinctly different learning mechanisms.
AIBullisharXiv – CS AI · Mar 267/10
🧠Researchers introduce Moonwalk, a new algorithm that solves backpropagation's memory limitations by eliminating the need to store intermediate activations during neural network training. The method uses vector-inverse-Jacobian products and submersive networks to reconstruct gradients in a forward sweep, enabling training of networks more than twice as deep under the same memory constraints.
AIBullisharXiv – CS AI · Mar 177/10
🧠Researchers propose HO-SFL (Hybrid-Order Split Federated Learning), a new framework that enables memory-efficient fine-tuning of large AI models on edge devices by eliminating backpropagation on client devices while maintaining convergence speed comparable to traditional methods. The approach significantly reduces communication costs and memory requirements for distributed AI training.
AIBullisharXiv – CS AI · Mar 117/10
🧠Researchers propose a new biologically plausible framework for approximating backpropagation through time (BPTT) in neural networks that mimics how the brain learns spatiotemporal patterns. The approach uses energy conservation principles to create local, time-continuous learning equations that could enable more brain-like AI systems and physical neural computing circuits.
AIBullisharXiv – CS AI · Mar 97/10
🧠Researchers present a comprehensive survey of Predictive Coding Networks (PCNs), a neuroscience-inspired AI approach that uses biologically plausible inference learning instead of traditional backpropagation. PCNs can achieve higher computational efficiency with parallelization and offer a more versatile framework for both supervised and unsupervised learning compared to traditional neural networks.
AINeutralarXiv – CS AI · Jun 236/10
🧠Researchers propose Analytic Policy Gradients (APG), a method that computes exact policy gradients through backpropagation in differentiable simulators, contrasting with model-free approaches like PPO that rely on sampled rewards. Testing across four continuous control tasks shows APG achieves superior sample efficiency, with a segmented backpropagation scheme that mitigates gradient degradation on long-horizon problems.
AIBullisharXiv – CS AI · Jun 96/10
🧠Researchers have reformulated Predictive Coding (PC), a brain-inspired neural network training method, to address its severe computational inefficiency in digital systems. The new error-based PC (ePC) eliminates signal decay problems inherent in the canonical state-based formulation, achieving backpropagation-level performance at orders of magnitude faster speeds, enabling PC to scale to deeper architectures on standard hardware.
AINeutralarXiv – CS AI · Jun 56/10
🧠Researchers propose Supervised Memory Training (SMT), a novel method for training recurrent neural networks that replaces sequential backpropagation through time with parallel, supervised learning on memory state transitions. By leveraging a Transformer encoder to generate training labels, SMT achieves stable gradient propagation and improved performance on language and sequence modeling tasks without the parallelism constraints of traditional RNN training.
AINeutralarXiv – CS AI · May 126/10
🧠Researchers demonstrate that modified feedback alignment (FA) algorithms can train convolutional neural networks while maintaining biological plausibility, with internal representations converging to structures similar to backpropagation despite using fundamentally different weight update mechanisms. This finding suggests that successful learning algorithms may achieve comparable results through different computational paths, bridging biologically plausible alternatives with practical neural network training.
