Evaluating Semantic Fragility in Text-to-Audio Generation Systems Under Controlled Prompt Perturbations

Researchers evaluated the semantic fragility of text-to-audio generation systems, finding that small changes in prompts can lead to substantial variations in generated audio output. While larger models like MusicGen-large showed better semantic consistency, all models exhibited persistent divergence in acoustic and temporal characteristics even when semantic similarity remained high.

• →Text-to-audio generation systems show vulnerability to small linguistic changes in prompts, raising reliability concerns for practical applications.
• →Larger models demonstrate improved semantic consistency, with MusicGen-large achieving cosine similarities of 0.77-0.82 across different prompt variations.
• →Audio fragility primarily occurs during semantic-to-acoustic realization rather than in multi-modal embedding alignment.
• →The study introduces a controlled framework with 75 prompt groups for evaluating robustness in text-to-audio generation systems.
• →Multi-level stability assessment is needed for generative audio systems to ensure reliable performance across different prompt formulations.