GetBatch: Distributed Multi-Object Retrieval for ML Data Loading

Researchers introduce GetBatch, a new object store API that optimizes machine learning data loading by replacing thousands of individual GET requests with a single batch operation. The system achieves up to 15x throughput improvement for small objects and reduces batch retrieval latency by 2x in production ML training workloads.

• →GetBatch replaces thousands of individual GET requests with a single deterministic, fault-tolerant streaming operation for ML data loading.
• →The system achieves up to 15x throughput improvement for small objects compared to traditional individual GET requests.
• →Production ML training workloads see 2x reduction in P95 batch retrieval latency and 3.7x reduction in P99 per-object tail latency.
• →The innovation addresses per-request overhead that often dominates data transfer time in distributed ML training pipelines.
• →GetBatch elevates batch retrieval to a first-class storage operation, potentially improving efficiency across ML infrastructure.