Flexible Agent Alignment with Goal Inference from Open-Ended Dialog

Researchers introduce Open-Universe Assistance Games (OU-AGs), a framework enabling LLM-based agents to infer and align with human preferences through open-ended dialogue. The GOOD method extracts evolving goals from natural language interactions using probabilistic inference, demonstrating improved user intent alignment across shopping, robotics, and coding domains without requiring large offline datasets.

This research addresses a fundamental challenge in deploying LLM agents: understanding and adapting to human preferences that shift and clarify during interaction. Traditional assistance game formulations assume fixed preferences defined upfront, an assumption that fails in real-world collaboration where users incrementally refine their goals through conversation. OU-AGs reframe preference modeling as dynamic distributions over natural-language goals, grounded in cognitive science literature on how humans construct preferences iteratively.

The GOOD framework operationalizes this approach through online probabilistic inference, using LLM-simulated users to generate and rank goal hypotheses during multi-turn exchanges. This data-efficient design eliminates dependence on massive offline training datasets while maintaining interpretability—users and developers can understand why the agent interprets goals as it does. The method's uncertainty-aware representation prevents overconfidence in preference estimates when signals remain ambiguous.

The implications extend beyond dialogue systems. As AI agents become embedded in complex collaborative workflows—from home automation to professional coding assistance—accurate preference inference directly affects user satisfaction and adoption. The semantic coherence demonstrated across heterogeneous domains (shopping, robotics, coding) suggests the approach generalizes beyond narrow applications. This work potentially influences how future agent systems handle preference ambiguity, a prerequisite for deploying AI in environments where human objectives genuinely evolve.

Future developments will test scalability to longer interactions, complex preference constraints, and adversarial scenarios where stated and actual preferences diverge. Integration with reinforcement learning from human feedback pipelines could enhance alignment robustness.

• →OU-AGs framework enables LLM agents to dynamically infer human preferences from open-ended dialogue without predefined goal specifications.
• →GOOD method uses probabilistic inference over goal hypotheses, maintaining interpretable uncertainty representations of user intent.
• →Data-efficient online approach eliminates need for large offline datasets while preserving semantic coherence across diverse domains.
• →Framework addresses critical limitation in current LLM agents: inability to maintain accurate user intent models in multi-turn collaborative interactions.
• →Approach generalizes across text-based domains including shopping, robotics, and coding, suggesting broad applicability for preference-sensitive AI systems.