Real-time body pose non-verbal communication with a consistency-based reliability measure

Researchers have developed a new dataset and methodology for recognizing communicative intent from body pose alone, targeting real-time on-device deployment for human-robot communication in scenarios like rescue missions. The work introduces a consistency-based reliability measure that uses a model's autoregressive self-consistency as an unsupervised signal to gauge prediction confidence, with theoretical bounds on correctness probability.

This research addresses a practical gap in human-robot interaction by isolating body pose as a communication signal distinct from facial expressions, speech, and text. Traditional affective datasets conflate multiple modalities while action-recognition benchmarks focus on physical tasks rather than communicative intent, leaving this specific domain underdeveloped. The authors released a novel dataset covering ten communicative intents from real video frames, providing a resource that existing corpora lack.

The work's emphasis on embedded deployment distinguishes it from typical AI research. By benchmarking performance on NVIDIA Orin Nano and reporting frame rates alongside accuracy metrics, the authors acknowledge that rescue scenarios demand both real-time inference and limited computational resources. This constraint-aware approach reflects growing industry recognition that deployed AI must balance capability with practicality.

The consistency-based reliability measure represents a significant methodological contribution. Rather than relying solely on supervised metrics, the authors demonstrate that a model's self-consistency—its ability to produce identical predictions across multiple autoregressive steps—correlates with correctness probability. They provide theoretical bounds showing this correlation strengthens with more consistent predictions, though they identify failure cases where confidence remains unreliable. This unsupervised approach to confidence estimation could reduce deployment risks by flagging uncertain predictions without ground truth labels.

Applications extend beyond rescue robotics to industrial human-robot collaboration, autonomous systems in remote environments, and accessibility technologies. The work demonstrates that specialized datasets and reliability measures tailored to deployment constraints yield more deployable AI systems than generic benchmarking approaches.

• →Body pose recognition for communicative intent requires dedicated datasets, as existing affective corpora and action-recognition benchmarks serve different purposes.
• →Real-time deployment on embedded GPUs demands joint optimization of accuracy and inference speed, not accuracy alone.
• →Autoregressive self-consistency provides an unsupervised reliability signal whose correctness probability grows predictably with consistent prediction steps.
• →Theoretical bounds on self-consistency reliability identify conditions where confident predictions may still fail, requiring fallback mechanisms.
• →This approach enables safer human-robot communication in remote scenarios like rescue missions where other modalities are unreliable.