#gpu-scheduling News & Analysis

3 articles tagged with #gpu-scheduling. AI-curated summaries with sentiment analysis and key takeaways from 50+ sources.

3 articles

AIBullisharXiv – CS AI · May 47/10

🧠

SAGA: Workflow-Atomic Scheduling for AI Agent Inference on GPU Clusters

SAGA is a new distributed GPU scheduler that treats entire AI agent workflows as atomic units rather than individual inference calls, reducing task completion time by 1.64x compared to existing solutions. The system achieves this through workflow-aware scheduling, KV cache optimization, and fairness mechanisms, though with a tradeoff of 30% lower peak throughput suitable for latency-sensitive interactive deployments.

🏢 Meta

AIBullisharXiv – CS AI · Jun 116/10

🧠

INFRAMIND: Infrastructure-Aware Multi-Agent Orchestration

INFRAMIND is a new framework that optimizes multi-agent LLM orchestration by making real-time infrastructure state (queue depths, cache pressure, latencies) central to routing and scheduling decisions. Using reinforcement learning, the system dynamically adjusts model selection and pipeline topology based on GPU cluster load, achieving up to 7.6% accuracy gains and 7x latency reduction while maintaining 99.9% SLO compliance under high load.

AINeutralarXiv – CS AI · Jun 26/10

🧠

Threshold-Based Exclusive Batching for LLM Inference

Researchers demonstrate that exclusive batching (EB) can outperform the industry-standard mixed batching (MB) approach for LLM inference on bandwidth-constrained GPUs, with performance crossover dependent on hardware specifications and workload composition. A new hybrid scheduler (EB+) dynamically switches between strategies to optimize throughput across varying traffic conditions.