AIBullisharXiv – CS AI · Jun 237/10
🧠Researchers propose a scalable framework for linear mode connectivity (LMC) that enables merging of billion-parameter pretrained transformers through dual bidirectional optimization. The method achieves near-zero loss barriers on language models and maintains strong performance on vision models, demonstrating that resolving parameter symmetries allows large AI models to be merged via simple linear interpolation paths.
AIBullisharXiv – CS AI · Jun 117/10
🧠CRANE is a training-free parameter-editing method that merges paired Instruct and Thinking model checkpoints to create superior code agents. By selectively combining reasoning capabilities from Thinking models with the tool-discipline of Instruct models, CRANE achieves significant performance gains—66.2% pass rate on Roo-Eval (+19.5%) and resolves 14 additional instances on SWE-bench—while maintaining computational efficiency.
AIBullisharXiv – CS AI · Jun 17/10
🧠Researchers propose DeMix, a framework that uses model merging to efficiently determine optimal data mixtures for large language model pre-training without expensive repeated training cycles. The approach decouples the search process from training costs, enabling evaluation of multiple data combinations while also releasing a 22-token dataset to support open research.
AIBullisharXiv – CS AI · May 127/10
🧠Researchers introduce M2A, a novel model merging paradigm that combines mathematical and agentic reasoning in large language models without retraining. The approach improves a Qwen3-8B model's software engineering benchmark performance from 44.0% to 51.2% by strategically injecting mathematical reasoning capabilities along directions that preserve agent behavior.
AINeutralarXiv – CS AI · Apr 67/10
🧠Researchers studied weight-space model merging for multilingual machine translation and found it significantly degrades performance when target languages differ. Analysis reveals that fine-tuning redistributes rather than sharpens language selectivity in neural networks, increasing representational divergence in higher layers that govern text generation.
AINeutralarXiv – CS AI · Mar 117/10
🧠Researchers have identified a phenomenon called 'merging collapse' where combining independently fine-tuned large language models leads to catastrophic performance degradation. The study reveals that representational incompatibility between tasks, rather than parameter conflicts, is the primary cause of merging failures.
AIBullisharXiv – CS AI · Mar 47/103
🧠Researchers introduce OptMerge, a new benchmark and method for combining multiple expert Multimodal Large Language Models (MLLMs) into single, more capable models without requiring additional training data. The approach achieves 2.48% average performance gains while reducing storage and serving costs by merging models across different modalities like vision, audio, and video.
AIBullisharXiv – CS AI · Mar 37/103
🧠Researchers introduce AdaRank, a new AI model merging framework that adaptively selects optimal singular directions from task vectors to combine multiple fine-tuned models. The technique addresses cross-task interference issues in existing SVD-based approaches by dynamically pruning problematic components during test-time, achieving state-of-the-art performance with nearly 1% gap from individual fine-tuned models.
AIBullisharXiv – CS AI · Jun 236/10
🧠Researchers propose a novel approach to Open Vocabulary Action Recognition (OVAR) using task arithmetic and model merging, enabling zero-shot generalization to novel actions without requiring costly domain-specific fine-tuning. By combining task vectors from models trained on diverse public datasets, the method achieves superior out-of-distribution performance while avoiding privacy and regulatory concerns associated with target-domain training.
AINeutralarXiv – CS AI · Jun 236/10
🧠Researchers introduce ESM (Essential Subspace Merging), a framework that combines multiple task-specific AI models into a single multi-task model by analyzing parameter updates through PCA and projecting them onto essential subspaces. The method reduces task interference while preserving specialized functionality, achieving state-of-the-art performance in model merging without additional training.
AINeutralarXiv – CS AI · Jun 236/10
🧠Researchers propose Essential Subspace Merging (ESM), a training-free method that combines multiple task-specific models into a single multi-task model by identifying and orthogonalizing principal component directions while suppressing interference-causing noise. The approach demonstrates that most inter-task interference stems from accumulated energy in non-essential directions rather than core task-relevant updates, enabling efficient model consolidation across multiple domains.
AINeutralarXiv – CS AI · Jun 116/10
🧠Researchers challenge the conventional wisdom that adapter interference in language models stems from parameter-space geometry by testing whether orthogonal or directionally independent updates reduce cross-domain interference. Their findings using DoRA-RBAC on multiple LLMs show geometry-aware merging provides no consistent advantage, suggesting interference mechanisms operate in shared nonlinear representations rather than linear parameter space.
AINeutralarXiv – CS AI · Jun 116/10
🧠Researchers propose Trap², an architecture-agnostic defense framework that protects AI models from unauthorized merging by encoding protection into model weights during fine-tuning. The method degrades model performance when weights are re-scaled during merging operations while maintaining effectiveness in standalone use, addressing a governance gap where downstream users can bypass safety alignment and licensing restrictions.
AINeutralarXiv – CS AI · Jun 16/10
🧠Researchers introduce Dynamic Adapter Routing (DAR), a novel approach to continual multimodal retrieval that moves beyond traditional class-incremental learning methods. The study presents a new evaluation framework for vision-language models that better captures real-world retrieval dynamics, with DAR demonstrating superior performance and strong generalization capabilities.
AIBullisharXiv – CS AI · Jun 16/10
🧠Researchers propose Orthogonal Subspaces for Robust model Merging (OSRM), a technique that addresses performance degradation when combining multiple LoRA-fine-tuned language models into single multi-task systems. By constraining LoRA subspaces prior to fine-tuning, the method reduces task interference while maintaining individual task accuracy and improving compatibility with existing merging algorithms.
AINeutralarXiv – CS AI · May 276/10
🧠Researchers introduce EpiMer, a novel framework for merging machine learning models by treating it as a geometric optimization problem on Riemannian manifolds. The method uses low-rank task vectors and curvature information to improve knowledge integration without retraining, demonstrating superior performance when merging fine-tuned CLIP-ViT models across multiple image classification tasks.
AIBullisharXiv – CS AI · Apr 146/10
🧠Researchers introduce Modular Delta Merging with Orthogonal Constraints (MDM-OC), a machine learning framework that enables multiple fine-tuned models to be merged, updated, and selectively removed without performance degradation or task interference. The approach uses orthogonal projections to prevent model conflicts and supports compliance requirements like GDPR-mandated data deletion.
AIBullisharXiv – CS AI · Mar 176/10
🧠Researchers have developed Resolving Interference (RI), a new framework that improves AI model merging by reducing cross-task interference when combining specialized models. The method makes models functionally orthogonal to other tasks using only unlabeled data, improving merging performance by up to 3.8% and generalization by up to 2.3%.
AIBullisharXiv – CS AI · Mar 176/10
🧠Researchers propose ES-Merging, a new framework for combining specialized biological multimodal large language models (MLLMs) by using embedding space signals rather than traditional parameter-based methods. The approach estimates merging coefficients at both layer-wise and element-wise granularities, outperforming existing merging techniques and even task-specific fine-tuned models on cross-modal scientific problems.
AIBullisharXiv – CS AI · Mar 37/108
🧠Researchers propose a training-free paradigm for empowering Vision-Language Models with multi-modal search capabilities through cross-modal model merging. The approach uses Optimal Brain Merging (OBM) to combine text-based search agents with base VLMs without requiring expensive supervised training or reinforcement learning.
AINeutralarXiv – CS AI · Mar 54/10
🧠Researchers introduce BD-Merging, a new AI framework that improves model merging for multi-task learning by addressing bias and distribution shift issues. The method uses uncertainty modeling and contrastive learning to create more reliable AI systems that can better handle real-world data variations.
AINeutralHugging Face Blog · Feb 194/108
🧠The article title suggests that PEFT (Parameter Efficient Fine-Tuning) has introduced new merging methods. However, the article body appears to be empty or unavailable, limiting detailed analysis of the specific technical developments or their implications.
