AEG: A Baremetal Framework for AI Acceleration via Direct Hardware Access in Heterogeneous Accelerators

Researchers introduce AEG, a bare-metal runtime framework that enables high-performance machine learning inference on heterogeneous AI accelerators without OS overhead. The system achieves 9.2× higher compute efficiency and uses 11× fewer hardware tiles than Linux-based alternatives, demonstrating significant potential for edge AI deployment optimization.

AEG addresses a critical inefficiency in current edge AI deployment practices by eliminating operating system overhead that constrains hardware utilization. Traditional frameworks like TinyML and Vitis AI rely on real-time operating systems that introduce context-switching delays, memory overhead, and scheduling complexity—performance penalties that grow more severe as edge devices proliferate. The paper's "Control as Data" paradigm represents a meaningful architectural shift, flattening hierarchical control structures into linear, predictable execution patterns that hardware can process with minimal abstraction layers.

The experimental results validate a compelling trade-off: AEG achieves ImageNet classification with 28 AIE tiles versus Vitis AI's 304 tiles, a 10.86× reduction in required hardware. This efficiency gain stems from three interconnected improvements: direct hardware access eliminates OS context-switching, the lightweight Runtime Hardware Abstraction Layer minimizes translation overhead, and integrated Platform Management handles orchestration without external kernel involvement. The 3-7× reduction in data movement overhead particularly matters for inference-heavy workloads where memory bandwidth often becomes the bottleneck.

For the broader AI acceleration ecosystem, this work signals growing interest in OS-free computing models for specialized tasks. As edge inference becomes increasingly pervasive across IoT, robotics, and autonomous systems, the ability to squeeze more performance from fewer tiles directly impacts deployment economics and energy consumption. Hardware manufacturers and framework developers will likely explore similar approaches, especially for domain-specific accelerators where generic OS abstractions prove wasteful. The near-zero latency variance (0.03% coefficient of variation) additionally benefits real-time applications requiring deterministic timing.

• →AEG achieves 9.2× higher compute efficiency per tile compared to Linux-based Vitis AI deployment through OS-free execution.
• →The framework reduces required hardware by over 10× (28 vs 304 tiles) while maintaining ImageNet classification accuracy, directly improving deployment economics.
• →Bare-metal execution with minimal abstraction layers cuts data movement overhead by 3-7×, addressing memory bandwidth constraints in edge inference.
• →Near-zero latency variance (0.03% CV) enables deterministic timing for real-time AI applications, a critical requirement for autonomous and safety-critical systems.
• →The "Control as Data" paradigm represents an architectural alternative gaining traction for specialized accelerators where generic OS overhead proves counterproductive.