Researchers introduce Lexical Acoustic Coding (LAC), a framework enabling LLM agents to transmit audio through natural language by converting sound into interpretable acoustic descriptors and verbalizing them as English text. The approach frames audio transmission as a quantization problem, balancing vocabulary size, transmission rate, and fidelity while keeping the transmitted text editable and human-readable.
Lexical Acoustic Coding represents an unconventional bridge between natural language processing and audio signal processing. Rather than treating language and audio as separate modalities, LAC uses LLMs as codec agents that analyze waveforms into linguistic descriptions and reconstruct audio from those descriptions. This approach inverts typical assumptions about audio compression, where binary or symbolic formats dominate—instead, human-readable English becomes the transmission medium itself.
The innovation stems from a broader shift in AI research toward multimodal language models capable of reasoning across different data types. Recent advances in large language models have demonstrated surprising competence at structured code generation and constraint satisfaction. LAC exploits these capabilities by having sender and receiver agents write their own analysis-synthesis code under fixed prompts, with communication occurring solely through shared vocabulary and optional music notation. This eliminates the need for learned parameters specific to audio codec design.
For developers and researchers, LAC offers practical advantages beyond compression efficiency. The transmitted text serves dual purposes: it functions as a machine-readable codec while remaining interpretable and manually editable by humans. This transparency could accelerate audio processing workflows and enable new forms of human-in-the-loop audio synthesis. The framework's dependence on LLM capabilities suggests that improvements in language model reasoning directly translate to audio quality gains.
Future work should address scalability to longer audio sequences and real-time constraints. Evaluating LAC against existing audio codecs across diverse sound types will determine whether the interpretability benefits justify any rate-distortion penalties. Integration with multimodal foundation models could enable more sophisticated audio-language interactions beyond current capabilities.
- →LAC enables LLM agents to transmit audio through natural language text, eliminating the need for learned audio codecs.
- →The framework treats audio transmission as a quantization problem, trading off vocabulary size, transmission rate, and reconstruction fidelity.
- →Transmitted text functions as both a human-readable caption and a machine-executable transport representation for audio.
- →The approach leverages LLM code generation capabilities rather than task-specific neural networks, potentially reducing model complexity.
- →Experimental results show plain text preserves measurable acoustic structure while remaining editable and compatible with LLM-mediated workflows.