13 articles tagged with #communication-efficiency. AI-curated summaries with sentiment analysis and key takeaways from 50+ sources.
Researchers present LayUp, an asynchronous decentralized gradient descent algorithm that enables faster distributed training of deep learning models through layer-wise updates and gossip-based communication. The method demonstrates 32% faster convergence than synchronous training while maintaining robustness to stragglers and requiring no extra buffering.
Researchers propose FLoRG, a new federated learning framework for efficiently fine-tuning large language models that reduces communication overhead by up to 2041x while improving accuracy. The method uses Gram matrix aggregation and Procrustes alignment to solve aggregation errors and decomposition drift issues in distributed AI training.
Researchers propose q-PDGD, a quantized stochastic primal-dual optimization method for distributed systems with limited communication bandwidth. The approach achieves linear convergence under relaxed geometric conditions and matches centralized stochastic optimization rates while reducing communication overhead through quantization.
Researchers introduce GASLoC, a decentralized pre-training algorithm that reduces communication overhead in distributed LLM training by enabling local optimizer steps and sparse peer communication instead of synchronous operations. The method demonstrates competitive or superior performance compared to existing approaches, particularly in heterogeneous bandwidth environments where worker speeds vary significantly.
Researchers propose a collaborative edge-to-server inference framework for vision-language models that reduces communication costs by selectively transmitting only high-entropy regions of interest rather than full-resolution images. The two-stage approach maintains inference accuracy while substantially decreasing bandwidth requirements across visual question-answering tasks.
Researchers introduce Gaussian-Head OFL, a one-shot federated learning method that reduces communication overhead to a single round by transmitting only statistical summaries instead of full models. The approach combines closed-form Gaussian classifiers with synthetic data generation, achieving competitive accuracy while maintaining privacy and eliminating dependency on public datasets.
ASTRA is a new framework that enables efficient multi-device Transformer inference by combining sequence parallelism with mixed-precision attention, allowing non-local token embeddings to be transmitted as compressed codes while maintaining full precision for local attention. The system achieves significant speedups (up to 2.64x) over single-device inference while operating at extremely low bandwidth requirements (as low as 10 Mbps), making it practical for bandwidth-constrained environments.
Researchers propose PushCen-ADFL, a new framework for asynchronous decentralized federated learning that reduces communication overhead by over 80% while improving accuracy under data heterogeneity. The approach uses centroid-based message compression and bias-correction aggregation to enable stable model training across distributed systems without central coordination.
Researchers introduce CommFuse, a novel communication-computation overlap technique that eliminates tail latency in distributed LLM training by decomposing collective operations into peer-to-peer communications. The method improves efficiency for both tensor parallelism and data parallelism across GPU/TPU/NPU clusters, achieving higher throughput and model FLOPS utilization compared to existing solutions.
Researchers present a novel quantum federated learning framework for large-scale wireless networks that combines quantum computing with privacy-preserving federated learning. The study introduces a sum-rate maximization approach using quantum approximate optimization algorithm (QAOA) that achieves over 100% improvement in performance compared to conventional methods.
Researchers propose FedLECC, a new client selection strategy for federated learning that improves AI model training efficiency in distributed environments. The method groups clients by data similarity and prioritizes those with higher loss, achieving up to 12% better accuracy while reducing communication overhead by 50%.
Researchers propose a new Personalized Federated Learning approach that automatically learns optimal collaboration weights between agents without prior knowledge of data heterogeneity. The method uses kernel mean embedding estimation to capture statistical relationships between agents and includes a practical implementation for communication-constrained federated settings.
Researchers propose CA-AFP, a new federated learning framework that combines client clustering with adaptive model pruning to address both statistical and system heterogeneity challenges. The approach achieves better accuracy and fairness while reducing communication costs compared to existing methods, as demonstrated on human activity recognition benchmarks.
