Automated Semantic Fault Localization in SysML v2: A Human-in-the-Loop Framework Using Knowledge-Graph Augmented LLMs

Researchers present a human-in-the-loop framework combining fine-tuned small language models with knowledge graphs to automatically detect and repair semantic errors in SysML v2 models that bypass traditional compiler validation. The approach achieves over 91% repair accuracy using domain-specific training data and generates practical repair suggestions for engineer review.

This research addresses a critical gap in model-based systems engineering (MBSE) where syntactically correct designs can still violate domain rules, leading to costly late-stage failures. Traditional compilers validate structure and language conformance but cannot catch semantic violations—incompatibilities between system components that preserve syntactic validity. The framework leverages domain knowledge graphs to encode physical compatibility rules between mechanical, electrical, fluid, and signal interfaces, creating a structured representation of engineering constraints.

The approach represents a meaningful advance in AI-assisted verification by combining multiple techniques. Fine-tuned small language models (Qwen2.5-Coder-1.5B and DeepSeek-Coder-6.7B) generate unified diff patches for fault localization and repair suggestions, making outputs interpretable for engineers. The knowledge graph serves dual purposes: systematically generating synthetic training data by introducing plausible domain violations, and augmenting inference-time reasoning to ground suggestions in valid constraints. This dual role ensures training data diversity while maintaining engineering rigor.

The results demonstrate substantial practical value—improving semantic fault repair from below 3% to over 91% on 1,184 test samples, while reducing token output length by 60%. The patch-based output format enhances usability by presenting concise, reviewable repairs rather than verbose explanations. By preserving human judgment in the design review loop, the framework avoids replacing engineer expertise with black-box AI recommendations.

The work establishes a replicable pattern for domain-specific AI verification: combine fine-tuned models with structured domain knowledge, generate synthetic data systematically, and present outputs in formats engineers already use. This human-in-the-loop design positions the framework as complementary to existing MBSE tools rather than disruptive, enabling adoption within established engineering workflows.

• →Fine-tuned small language models combined with knowledge graphs achieve 91% accuracy in detecting and repairing semantic design errors missed by traditional compilers.
• →Knowledge graphs serve dual purposes: encoding domain rules for constraint-based repair suggestions and generating synthetic training data through systematic violation patterns.
• →Patch-based output format reduces token consumption by 60% while maintaining interpretability for engineer review and final approval.
• →Framework designed as human-in-the-loop system that augments rather than replaces engineer judgment in design verification.
• →Approach demonstrates replicable pattern for domain-specific AI verification across MBSE tools and other engineering domains.