AIBullisharXiv – CS AI · Jun 87/10
🧠Researchers introduce FIGMA, a new multi-view contrastive learning architecture that significantly improves music retrieval based on fine-grained musical attributes like tempo, key, and chord progression. The work addresses a fundamental limitation in existing CLAP-based models that fail to process detailed musical descriptions, achieving up to 73.3% relative improvement and contributing a new 380K music-caption dataset (FGMCaps) to the field.
AIBullisharXiv – CS AI · Jun 47/10
🧠Researchers introduce Audio-Interaction, a unified streaming model that enables Large Audio Language Models to process audio in real time through a perceive-decide-respond loop, handling tasks from speech recognition to voice chatting. The framework, SoundFlow, includes a new 2.6M-item streaming corpus and demonstrates competitive performance on mainstream audio tasks while unlocking real-time interactive capabilities previously unavailable to offline models.
AINeutralarXiv – CS AI · Jun 27/10
🧠Researchers introduce PolySpeech-100, a comprehensive benchmark evaluating speech understanding across 110 languages and dialects, revealing that end-to-end speech-LLMs outperform traditional ASR+LLM systems on dialects but struggle with low-resource languages. The study of 22 state-of-the-art models exposes significant performance gaps and shows that chain-of-thought prompting often degrades speech comprehension, highlighting critical modality alignment issues in current AI architectures.
🧠 Gemini
AIBullisharXiv – CS AI · May 297/10
🧠Researchers introduce COMET, a PLS-SVD framework that analyzes the modality gap in Contrastive Language-Audio Pretraining (CLAP) models by decomposing embeddings into interpretable concepts. The study reveals that only a small subset of shared conceptual axes drives similarity computation, and proposes a training-free spectral truncation method that improves zero-shot audio captioning performance while reducing dimensionality.
AIBullisharXiv – CS AI · May 277/10
🧠StreamSplit introduces a novel framework enabling continuous contrastive learning on edge devices by dynamically partitioning computation between local and cloud resources. Using reinforcement learning and uncertainty guidance, the system reduces latency by up to 4.7x and bandwidth by 77.1% while maintaining near-server accuracy, making distributed AI inference practical for resource-constrained hardware.
AIBearisharXiv – CS AI · Apr 147/10
🧠Researchers evaluated four omnimodal AI models across text, image, audio, and video processing, finding substantial demographic and linguistic biases particularly in audio understanding tasks. The study reveals significant accuracy disparities across age, gender, language, and skin tone, with audio tasks showing prediction collapse toward narrow categories, highlighting fairness concerns as these models see wider real-world deployment.
AIBearisharXiv – CS AI · Mar 177/10
🧠Researchers introduce τ-voice, a new benchmark for evaluating full-duplex voice AI agents on complex real-world tasks. The study reveals significant performance gaps, with voice agents achieving only 30-45% of text-based AI capability under realistic conditions with noise and diverse accents.
🧠 GPT-5
AINeutralarXiv – CS AI · Mar 117/10
🧠Researchers introduce MUGEN, a comprehensive benchmark revealing significant weaknesses in large audio-language models when processing multiple concurrent audio inputs. The study shows performance degrades sharply with more audio inputs and proposes Audio-Permutational Self-Consistency as a training-free solution, achieving up to 6.74% accuracy improvements.
AIBullisharXiv – CS AI · Mar 46/102
🧠Researchers identified a critical problem in Large Audio-Language Models (LALMs) where audio perception deteriorates during extended reasoning processes. They developed MPAR² framework using reinforcement learning, which improved perception performance from 31.74% to 63.51% and achieved 74.59% accuracy on MMAU benchmark.
AIBullishOpenAI News · May 137/107
🧠OpenAI has announced GPT-4 Omni (GPT-4o), their new flagship AI model that can process and reason across audio, vision, and text simultaneously in real-time. This represents a significant advancement in multimodal AI capabilities, potentially setting a new standard for AI model functionality.
AINeutralarXiv – CS AI · Jun 256/10
🧠Researchers introduce CASU, a new benchmark for evaluating Large Audio Language Models' ability to understand complex auditory scenes by integrating multiple acoustic layers—speech, sound events, and background environments—rather than processing them in isolation. The benchmark reveals that current LALMs struggle with holistic scene comprehension and require integration across all audio layers for effective real-world audio understanding.
AINeutralarXiv – CS AI · Jun 255/10
🧠Researchers present a novel methodology for predicting note velocity in automatic guitar transcription by leveraging synthetic training data from virtual instruments. The approach uses transfer learning to adapt velocity prediction weights from synthetic data to real guitar audio, achieving state-of-the-art transcription performance while successfully addressing a previously under-explored aspect of music transcription models.
AINeutralarXiv – CS AI · Jun 256/10
🧠Researchers propose SE-AGCNet, an end-to-end framework that jointly optimizes speech enhancement and automatic gain control for meeting scenarios. The approach addresses limitations of traditional discrete audio processing pipelines by leveraging synergy between the two tasks, improving speech quality, loudness consistency, and automatic speech recognition accuracy.
AINeutralarXiv – CS AI · Jun 196/10
🧠Researchers introduce QC-GAN, a parameter-efficient speech enhancement model combining Quaternion Conformer architecture with MetricGAN training. The framework achieves state-of-the-art speech quality scores while using less than half the parameters of comparable models, with a 35K-parameter variant demonstrating viable ultra-lightweight performance.
AINeutralarXiv – CS AI · Jun 96/10
🧠Researchers introduce Conan-embedding-v3, a framework that enables unified embedding spaces across multiple data modalities (text, image, video, audio, documents) by training specialized models independently and fusing them into a single backbone. The approach identifies and solves a critical technical challenge called 'Projector Drift' that causes audio retrieval performance degradation when external encoders are integrated.
AINeutralarXiv – CS AI · Jun 96/10
🧠Researchers propose MeCo, a MeanFlow-based generative corrector that improves multi-channel speech separation by refining discriminative model outputs in a single step. The method combines Data-Space Optimization with specialized loss functions to achieve state-of-the-art performance in both signal fidelity and human listening quality with minimal computational cost.
AINeutralarXiv – CS AI · Jun 95/10
🧠Researchers propose a training-free method for improving automatic speech recognition in noisy environments by intelligently fusing noisy and speech-enhanced audio based on intelligibility estimates. The approach eliminates the need for trained neural predictors, reducing complexity while maintaining robustness across diverse speech enhancement and ASR model combinations.
AINeutralarXiv – CS AI · Jun 96/10
🧠Researchers introduce mllm-shap, an open-source framework that extends Shapley Value explainability techniques to multimodal large language models processing text and audio inputs simultaneously. The platform addresses three technical challenges unique to multimodal systems and implements five estimation strategies, with a novel phonetic alignment technique reducing computational complexity by 10-50x.
AINeutralarXiv – CS AI · Jun 96/10
🧠Researchers have developed a novel framework extending Shapley Values—a traditional explainability method—to multimodal large language models that process both text and audio. The work introduces computational optimizations and a preprocessing technique called Spectrogram-Guided Phonetic Alignment to make the analysis feasible, alongside an open-source tool for visualization, revealing that input modality significantly affects model attribution patterns.
AINeutralarXiv – CS AI · Jun 96/10
🧠Researchers propose privacy-preserving group emotion recognition (GER) systems using multimodal audio-video analysis instead of individual biometric data. Two novel architectures—a cross-attention fusion model and a Variational Encoder Multi-Decoder framework—demonstrate that competitive emotion inference is achievable at the collective level without monitoring individual faces, voices, or gazes.
AINeutralarXiv – CS AI · Jun 86/10
🧠Researchers propose a Second-Order Correlation (SOC) layer that improves speech emotion recognition by modeling feature correlations as covariance descriptors rather than treating features independently. Using Log-Euclidean mapping to preserve geometric properties, the method demonstrates superior performance on standard emotion recognition datasets compared to conventional first-order aggregation approaches.
AINeutralarXiv – CS AI · Jun 56/10
🧠Researchers introduce ProSarc, an audio-only machine learning framework that detects sarcasm by analyzing temporal mismatches between local prosodic patterns and overall emotional tone. The model achieves strong performance on multiple datasets (F1=75.3 on MUStARD++) and demonstrates cross-lingual generalization, advancing computational understanding of spoken sarcasm detection.
AIBullisharXiv – CS AI · Jun 46/10
🧠Researchers introduce the Differentiable Auditory Loop (DAL), an open-source machine learning framework that uses neural network optimization to personalize hearing aid signal processing. By modeling individual hearing impairment patterns and training a deep neural network to match normal auditory function, DAL outperforms conventional hearing aids on neural representation and signal fidelity metrics, offering a path toward clinically-tested, AI-driven hearing aid customization.
AINeutralarXiv – CS AI · Jun 26/10
🧠Echo is a proof-of-concept audio system that unifies speaker diarization, speech recognition, and source separation on a single 25M-parameter ViT encoder pretrained with joint-embedding predictive architecture (JEPA). The system demonstrates competitive performance across three tasks simultaneously without per-task fine-tuning, though it represents a design exploration rather than state-of-the-art on individual metrics.
AINeutralarXiv – CS AI · Jun 16/10
🧠OpenSTBench introduces a unified evaluation framework for assessing speech translation systems across multiple dimensions including translation quality, speech quality, speaker preservation, and temporal consistency. The framework addresses a critical gap in the field by enabling comprehensive comparison of heterogeneous speech translation outputs that differ in modality and timing behavior, with code and datasets made publicly available.
