How Well Do Self-Supervised Speech Models Encode Age and Gender in Children's Speech? A Layer-Wise Analysis Across Multiple Architectures

Researchers conducted a comprehensive layer-wise analysis of how four major self-supervised learning (SSL) speech models encode age and gender information in children's speech. The study reveals that age and gender cues are unevenly distributed across model layers, with early-to-mid layers capturing the strongest paralinguistic signals, and demonstrates reliable classification accuracy even from 1-3 second audio segments.

This research addresses a critical gap in understanding how modern speech AI systems handle children's speech, which presents distinct acoustic challenges compared to adult speech due to developmental factors including higher pitch and articulatory variability. The comprehensive evaluation across four leading SSL architectures—Wav2Vec2, HuBERT, Data2Vec, and WavLM—provides empirical insights into model behavior that extends beyond academic curiosity into practical application design.

The layer-wise analysis methodology reveals important architectural differences in how models encode speaker attributes. HuBERT's superior age classification and the competitive performance of Wav2Vec2 and HuBERT on gender tasks suggest that model design choices meaningfully impact paralinguistic feature extraction. The finding that early-to-mid layers encode the strongest demographic signals contradicts assumptions that deeper layers capture more semantic meaning, indicating that acoustic-phonetic information persists throughout model depth.

For practitioners developing speech applications serving child populations—including educational technology, healthcare diagnostics, and accessibility tools—these findings validate the reliability of SSL models as feature extractors for age and gender classification. The robustness demonstrated through speaker-wise cross-validation and cross-database evaluation reduces concerns about overfitting to specific datasets. The ability to achieve performance from short 1-3 second clips is particularly valuable for real-world deployment where continuous audio may be unavailable.

Future research should investigate whether this uneven layer distribution of demographic information can be leveraged to improve downstream task performance or whether it introduces potential bias concerns for applications requiring demographic fairness. The findings also suggest opportunities for model compression and efficient inference in resource-constrained environments.

• →Age and gender information concentrates in early-to-mid SSL model layers rather than distributed evenly across depth
• →HuBERT excels at age classification while Wav2Vec2 and HuBERT lead on gender tasks across different datasets
• →Reliable demographic classification is achievable from short 1-3 second speech samples, enabling practical deployment
• →Layer-wise analysis reveals architectural differences between SSL models affect how effectively they capture paralinguistic cues
• →Findings remain robust across speaker-wise validation and cross-database evaluation, indicating stable generalization