86 articles tagged with #multimodal. AI-curated summaries with sentiment analysis and key takeaways from 50+ sources.
Researchers have developed MIDAS, a new jailbreaking framework that successfully bypasses safety mechanisms in Multimodal Large Language Models by dispersing harmful content across multiple images. The technique achieved an 81.46% average attack success rate against four closed-source MLLMs by extending reasoning chains and reducing security attention.
Researchers introduce MultiPUFFIN, a multimodal AI foundation model that predicts molecular properties for drug discovery and materials science. The model combines multiple data types and thermodynamic principles to achieve superior performance while using 2000x fewer training molecules than existing models like ChemBERTa-2.
Researchers propose FreeAct, a new quantization framework for Large Language Models that improves efficiency by using dynamic transformation matrices for different token types. The method achieves up to 5.3% performance improvement over existing approaches by addressing the memory and computational overhead challenges in LLMs.
Researchers introduce SemHiTok, a unified image tokenizer that uses semantic-guided hierarchical codebooks to balance multimodal understanding and generation tasks. The system decouples semantic and pixel features through a novel architecture that builds pixel sub-codebooks on pretrained semantic codebooks, achieving superior performance in both image reconstruction and multimodal understanding.
Researchers introduce SounDiT, a new AI model that generates realistic landscape images from environmental soundscapes using geo-contextual data. The model uses diffusion transformer technology and is trained on two large-scale datasets pairing environmental sounds with real-world landscape images.
Researchers have developed SleepLM, a family of AI foundation models that combine natural language processing with sleep analysis using polysomnography data. The system can interpret and describe sleep patterns in natural language, trained on over 100K hours of sleep data from 10,000+ individuals, enabling new capabilities like language-guided sleep event detection and zero-shot generalization to novel sleep analysis tasks.
Researchers introduce MMKG-RDS, a framework that uses multimodal knowledge graphs to synthesize high-quality training data for improving AI model reasoning abilities. Testing on Qwen3 models showed 9.2% improvement in reasoning accuracy, with applications for complex benchmark construction involving tables and formulas.
Researchers have developed Brain-OF, the first omnifunctional brain foundation model that can process fMRI, EEG, and MEG data simultaneously within a unified framework. The model introduces novel techniques like Any-Resolution Neural Signal Sampler and Masked Temporal-Frequency Modeling, trained on 40 datasets to achieve superior performance across diverse neuroscience tasks.
Researchers propose a new training method called pseudo contrastive learning to improve diagram comprehension in multimodal AI models like CLIP. The approach uses synthetic diagram samples to help models better understand fine-grained structural differences in diagrams, showing significant improvements in flowchart understanding tasks.
Researchers developed MINT, a framework that transfers knowledge from MRI brain scans to speech analysis for early Alzheimer's detection. The system achieves comparable performance to speech-only methods while being grounded in neuroimaging biomarkers, enabling population-scale screening without requiring expensive MRI scans at inference.
Researchers developed TASOT, an unsupervised AI method for surgical phase recognition that combines visual and textual information without requiring expensive large-scale pre-training. The approach showed significant improvements over existing zero-shot methods across multiple surgical datasets, demonstrating that effective surgical AI can be achieved with more efficient training methods.
Researchers have developed SPM-Bench, a PhD-level benchmark for testing large language models on scanning probe microscopy tasks. The benchmark uses automated data synthesis from scientific papers and introduces new evaluation metrics to assess AI reasoning capabilities in specialized scientific domains.
Researchers developed DisQ-HNet, a new AI framework that synthesizes tau-PET brain scans from MRI data to detect Alzheimer's disease pathology. The method uses advanced neural network architectures to generate cost-effective alternatives to expensive PET imaging while maintaining diagnostic accuracy.
Researchers introduce SOTAlign, a new framework for aligning vision and language AI models using minimal supervised data. The method uses optimal transport theory to achieve better alignment with significantly less paired training data than traditional approaches.
StruXLIP is a new fine-tuning paradigm for vision-language models that uses edge maps and structural cues to improve cross-modal retrieval performance. The method augments standard CLIP training with three structure-centric losses to achieve more robust vision-language alignment by maximizing mutual information between multimodal structural representations.
Google has announced improvements to its Gemini audio models, enhancing voice interaction capabilities for more powerful and natural voice experiences. The upgrades focus on better audio processing and response quality in conversational AI applications.
Google announces new multimodal models in the MedGemma collection, representing their most advanced open-source models specifically designed for healthcare AI development. This expansion demonstrates continued progress in specialized AI applications for the medical field.
Google has released Gemini 2.5 Flash-Lite as a stable, generally available model after its preview phase. The cost-efficient AI model offers high quality performance in a compact size, featuring a 1 million-token context window and multimodal capabilities.
Gemini 2.5 introduces new AI-powered audio dialog and generation capabilities, expanding Google's multimodal AI offerings. This represents an incremental advancement in conversational AI technology with enhanced audio processing features.
AMIE, a research AI agent, has been enhanced with vision capabilities for multimodal diagnostic dialogue. This advancement allows the AI to process both visual and textual information for medical diagnosis conversations, representing a significant step forward in AI-powered healthcare applications.
Google has announced Gemma 3, their latest open-source large language model featuring multimodal capabilities, multilingual support, and extended context length. The article title suggests this represents a significant advancement in Google's open LLM offerings, though specific technical details and capabilities are not provided in the given content.
OpenAI has launched a new multimodal moderation model based on GPT-4o that can more accurately detect harmful content in both text and images. This upgrade to the Moderation API will enable developers to build more effective content moderation systems across platforms.
The article title announces Falcon 2, a new 11 billion parameter pretrained language model and vision-language model (VLM) trained on over 5 trillion tokens across 11 languages. However, no article body content was provided to analyze the technical details, capabilities, or implications of this AI model release.
OpenAI has released the system card for GPT-4V(ision), documenting the safety evaluations and risk assessments for their multimodal AI model that can process both text and images. The system card outlines potential risks, limitations, and safety measures implemented before the model's deployment.
Researchers have published a comprehensive review of methods for integrating large language models (LLMs) into virtual reality environments to create more realistic digital humans with personality traits. The study explores various approaches including zero-shot, few-shot, and fine-tuning methods while highlighting challenges like computational demands and latency issues that need to be addressed for practical applications.
