CANS: Accelerating Multiuser Collaborative Edge Inference via Cooperative Autodidactic NeuroSurgeon

Researchers propose CANS, a collaborative edge inference framework that enables mobile devices to adaptively optimize deep neural network partitioning by sharing feedback across a common edge server. The system reduces inference latency by up to 50% compared to non-cooperative approaches through federated learning and device heterogeneity management.

CANS addresses a critical bottleneck in mobile edge computing: optimizing how resource-constrained devices partition neural network inference between local processing and cloud offloading. Traditional approaches fail because wireless conditions and device capabilities remain unpredictable, forcing static partitioning decisions that underperform in real-world scenarios. The framework's novelty lies in its federated learning approach, where devices collectively learn optimal partitions while maintaining privacy through cooperative feedback mechanisms.

The research builds on established trends in edge AI and distributed machine learning. Mobile inference has become increasingly important as applications demand real-time responses without full cloud dependency. Previous solutions either required exhaustive offline optimization or ignored device heterogeneity entirely. CANS bridges this gap by grouping similar devices and warm-starting exploration using local offline inference data, reducing the exploration burden during live deployment.

The theoretical contribution—the FedLinUCB-DW algorithm with regret upper bounds—provides confidence in convergence behavior. Empirical validation across simulated environments and hardware prototypes demonstrates practical viability, with the 50% latency reduction translating directly to improved user experience and reduced energy consumption on mobile devices.

Industry impact extends beyond academia. Edge inference optimization directly benefits developers building latency-sensitive applications in healthcare, autonomous systems, and real-time analytics. The cooperative learning mechanism could inform how distributed systems design future collaborative inference services, particularly in scenarios with heterogeneous device networks.

• →CANS reduces inference latency by up to 50% through cooperative learning of optimal DNN partitioning strategies
• →The FedLinUCB-DW algorithm handles device heterogeneity by grouping similar devices and leveraging offline experience
• →Theoretical regret bounds provide convergence guarantees for the federated learning approach
• →Framework enables devices to adaptively optimize partitions despite unknown, time-varying wireless conditions
• →Hardware prototype validation demonstrates practical applicability beyond simulated environments