#architecture-design News & Analysis

4 articles tagged with #architecture-design. AI-curated summaries with sentiment analysis and key takeaways from 50+ sources.

4 articles

AINeutralarXiv – CS AI · 6d ago6/10

🧠

Back to the Barn with LLAMAs: Evolving Pretrained LLM Backbones in Finetuning Vision Language Models

Researchers conducted a systematic study comparing Vision-Language Models built with LLAMA-1, LLAMA-2, and LLAMA-3 backbones, finding that newer LLM architectures don't universally improve VLM performance and instead show task-dependent benefits. The findings reveal that performance gains vary significantly: visual question-answering tasks benefit from improved reasoning in newer models, while vision-heavy tasks see minimal gains from upgraded language backbones.

AIBullisharXiv – CS AI · Mar 55/10

🧠

JPmHC Dynamical Isometry via Orthogonal Hyper-Connections

Researchers propose JPmHC (Jacobian-spectrum Preserving manifold-constrained Hyper-Connections), a new deep learning framework that improves upon existing Hyper-Connections by replacing identity skips with trainable linear mixers while controlling gradient conditioning. The framework addresses training instability and memory overhead issues in current deep learning architectures through constrained optimization on specific mathematical manifolds.

AIBullishLil'Log (Lilian Weng) · Aug 66/10

🧠

Neural Architecture Search

Neural Architecture Search (NAS) automates the design of neural network architectures to find optimal topologies for specific tasks. The approach systematically explores network architecture spaces through three key components: search space, search algorithms, and child model evolution strategies, potentially discovering better performing models than human-designed architectures.

AINeutralarXiv – CS AI · Mar 275/10

🧠

NERO-Net: A Neuroevolutionary Approach for the Design of Adversarially Robust CNNs

Researchers developed NERO-Net, a neuroevolutionary approach to design convolutional neural networks with inherent resistance to adversarial attacks without requiring robust training methods. The evolved architecture achieved 47% adversarial accuracy and 93% clean accuracy on CIFAR-10, demonstrating that architectural design can provide intrinsic robustness against adversarial examples.