3 articles tagged with #hierarchical-routing. AI-curated summaries with sentiment analysis and key takeaways from 50+ sources.
Researchers introduce Sigma-Branch, a neural network restructuring framework that reduces per-inference active parameters by 58-60% while maintaining full model capacity in memory. The approach uses hierarchical routing and binary tree architecture to enable efficient edge deployment without permanent model compression trade-offs.
Researchers demonstrate that tree-structured sparse feed-forward layers can replace dense MLPs in large transformer models while maintaining performance, activating less than 5% of parameters per token. The work reveals an emergent auto-pruning mechanism where hard routing progressively converts dynamic sparsity into static structure, offering a scalable approach to reducing computational costs in language models beyond 1 billion parameters.
Researchers propose a gravity-aware hierarchical routing method to improve human activity recognition in compressed language models used with wearable sensors. The lightweight adaptation addresses a specific failure mode where static activities like standing and sitting are poorly recognized when using compact models like TinyLlama, while maintaining strong performance on dynamic activities.
