9 articles tagged with #communication-efficiency. AI-curated summaries with sentiment analysis and key takeaways from 50+ sources.
Researchers propose FLoRG, a new federated learning framework for efficiently fine-tuning large language models that reduces communication overhead by up to 2041x while improving accuracy. The method uses Gram matrix aggregation and Procrustes alignment to solve aggregation errors and decomposition drift issues in distributed AI training.
Researchers introduce Gaussian-Head OFL, a one-shot federated learning method that reduces communication overhead to a single round by transmitting only statistical summaries instead of full models. The approach combines closed-form Gaussian classifiers with synthetic data generation, achieving competitive accuracy while maintaining privacy and eliminating dependency on public datasets.
ASTRA is a new framework that enables efficient multi-device Transformer inference by combining sequence parallelism with mixed-precision attention, allowing non-local token embeddings to be transmitted as compressed codes while maintaining full precision for local attention. The system achieves significant speedups (up to 2.64x) over single-device inference while operating at extremely low bandwidth requirements (as low as 10 Mbps), making it practical for bandwidth-constrained environments.
Researchers propose PushCen-ADFL, a new framework for asynchronous decentralized federated learning that reduces communication overhead by over 80% while improving accuracy under data heterogeneity. The approach uses centroid-based message compression and bias-correction aggregation to enable stable model training across distributed systems without central coordination.
Researchers introduce CommFuse, a novel communication-computation overlap technique that eliminates tail latency in distributed LLM training by decomposing collective operations into peer-to-peer communications. The method improves efficiency for both tensor parallelism and data parallelism across GPU/TPU/NPU clusters, achieving higher throughput and model FLOPS utilization compared to existing solutions.
Researchers present a novel quantum federated learning framework for large-scale wireless networks that combines quantum computing with privacy-preserving federated learning. The study introduces a sum-rate maximization approach using quantum approximate optimization algorithm (QAOA) that achieves over 100% improvement in performance compared to conventional methods.
Researchers propose FedLECC, a new client selection strategy for federated learning that improves AI model training efficiency in distributed environments. The method groups clients by data similarity and prioritizes those with higher loss, achieving up to 12% better accuracy while reducing communication overhead by 50%.
Researchers propose a new Personalized Federated Learning approach that automatically learns optimal collaboration weights between agents without prior knowledge of data heterogeneity. The method uses kernel mean embedding estimation to capture statistical relationships between agents and includes a practical implementation for communication-constrained federated settings.
Researchers propose CA-AFP, a new federated learning framework that combines client clustering with adaptive model pruning to address both statistical and system heterogeneity challenges. The approach achieves better accuracy and fairness while reducing communication costs compared to existing methods, as demonstrated on human activity recognition benchmarks.
