When Gradients Collide: Failure Modes of Multi-Objective Prompt Optimization for LLM Judges

Researchers identify critical failure modes in multi-objective prompt optimization for LLM judges, finding that jointly optimizing across multiple evaluation criteria reduces gradient task-focus by 59% and combining single-objective prompts degrades performance by 27%. The study reveals fundamental limitations in extending textual gradient methods to multi-criteria scenarios, constraining practical applications of automated LLM judge customization.

This research addresses a significant gap in AI systems engineering: optimizing language model judges across multiple simultaneous objectives. While textual gradient methods like TextGrad have enabled single-criterion prompt optimization, extending these approaches to multi-objective settings has proven problematic. The paper's core contribution is identifying and categorizing two distinct failure modes that explain performance degradation.

The optimization-time gradient dilution phenomenon—where multi-objective feedback reduces task-focus by 59%—stems from the gradient LLM attempting to synthesize feedback across conflicting criteria simultaneously. This mirrors known challenges in multi-task learning, but occurs within natural language rather than vector spaces, preventing application of established conflict-resolution techniques like PCGrad or MGDA. The inference-time instruction interference presents a separate problem: naive concatenation of single-objective optimized prompts actually performs worse than unoptimized baselines (0.220 vs 0.305 Spearman correlation).

These findings have implications for AI developers building domain-specific evaluation systems. Many applications require judges optimized for multiple criteria—correctness, helpfulness, safety, and efficiency simultaneously. The research suggests that straightforward decomposition strategies fail, requiring more sophisticated architectural approaches. This creates near-term friction for LLM customization workflows but identifies a tractable research problem. The work contributes to understanding fundamental constraints in prompt optimization and informs better design patterns for future textual gradient methods. Practitioners should expect that multi-objective judge optimization requires more careful engineering than simply combining single-objective solutions.

• →Multi-objective textual gradient optimization experiences 59% reduction in gradient task-focus when optimizing across multiple criteria jointly
• →Naively combining single-objective optimized prompts degrades correlation performance by 27%, indicating instruction interference at inference time
• →Two separable failure modes constrain multi-objective judge design: optimization-time gradient dilution and inference-time instruction interference
• →Standard multi-task learning conflict-resolution methods cannot directly apply to textual gradient settings without numeric vector representations
• →More sophisticated architectural approaches are needed for domain-specific LLM judges requiring simultaneous optimization across multiple evaluation criteria