Beyond Model Base Retrieval: Weaving Knowledge to Master Fine-grained Neural Network Design

M-DESIGN, a new retrieval-augmented framework, addresses the inefficiency gap between expensive neural architecture search and suboptimal model retrieval by dynamically leveraging historical evidence from prior tasks to discover near-optimal network modifications. Tested on 67,760 graph neural networks across 22 datasets, the method achieves state-of-the-art performance in 79% of cases under computational constraints.

M-DESIGN represents a meaningful advancement in automated machine learning by bridging two traditionally opposing approaches. Rather than choosing between computationally prohibitive architecture searches or accepting mediocre pre-trained models, the framework treats architectural modifications as evidence that can be transferred across tasks, creating a hybrid strategy that extracts value from historical experiments without requiring exhaustive new searches.

The core innovation lies in modeling fine-grained neural network changes as edit-effect evidence—essentially capturing what architectural modifications achieved on previous tasks and applying those insights to new problems. By building evidence graphs from 67,760 prior networks, M-DESIGN enables pattern recognition at scale, allowing the system to identify which modification types transfer well across different datasets and network types. The adaptive retrieval mechanism specifically addresses a critical problem: not all historical evidence remains equally relevant as target tasks shift, so the framework learns to weight evidence sources dynamically rather than treating them uniformly.

For AI practitioners and organizations developing custom neural networks, this work reduces both computational costs and iteration time—two major bottlenecks in production environments. Companies investing in AutoML infrastructure gain a more efficient alternative to brute-force architecture search while avoiding the performance degradation of static model retrieval. The predictive task planners that handle out-of-distribution scenarios particularly matter for real-world deployment, where new tasks rarely perfectly match the training distribution.

As neural network design becomes increasingly central to competitive advantage in AI, efficient design methodologies like M-DESIGN represent infrastructure improvements that compound across many applications. The strong empirical results across diverse graph neural network architectures suggest the approach generalizes beyond the tested domain.

• →M-DESIGN combines neural architecture search efficiency with model retrieval quality by reusing architectural modification evidence from prior tasks.
• →Achieving best-in-search-space performance in 26 of 33 cases demonstrates the method scales effectively across diverse neural network types and datasets.
• →Adaptive retrieval mechanisms dynamically adjust which historical evidence applies to new problems, addressing the distribution shift challenge.
• →The framework reduces computational burden compared to exhaustive architecture search while outperforming static pre-trained model approaches.
• →Evidence-based architectural refinement opens pathways for more efficient AI development cycles in production environments.