feat: User agent simulation for multi-turn episodes

User Problem

Environment authors want to evaluate and train agents in realistic multi-turn conversations with simulated users. Today, they lack clear guidance and infra on how to do this.

Desired User Experience

As an environment author, I can configure a simulated user and run multi-turn episodes through standard Gym workflows. I can inspect:

• the assistant trajectory, including its observations, responses, and tool calls
• the user-agent trajectory, including its observations, responses, and tool calls
• an ordered episode trajectory interleaving both participants, tool results, state changes, and termination

Acceptance Criteria

• Users can independently configure assistant and user-simulator agents.
• Users can define the simulated user’s instructions, goals, tools, and termination conditions.
• State changes caused by user tool calls are available to subsequent turns, even when not repeated in the user’s message.
• User and assistant tool calls share task-scoped state isolation.
• Outputs distinguish assistant messages, user messages, tool calls, tool results, and termination reasons.
• Assistant and user-simulator outputs are attributed separately so assistant rollouts can be evaluated or used for training without treating user-simulator outputs as assistant-generated tokens.
• A runnable example demonstrates multiple turns, user tool use, state updates, and episode termination.
• Documentation explains how to adopt and customize the pattern for evaluation and training.
• Existing single-agent environments continue to work without changes.

Design Considerations

Possible implementation patterns include:

• Coupled multi-turn agent: One agent orchestrates assistant and user turns and references a separate user model server. tau2_agent currently demonstrates this pattern.
• Decoupled multi-agent orchestration: Assistant and user simulators use independently pluggable agent harnesses under a shared orchestration contract.

User feedback demonstrated interest in both patterns, but more uncertainty on how to implement the multi-agent pattern. The implementation should explicitly resolve:

• Where turn orchestration lives.
• How participants observe state changed by user actions.
• How tools are exposed to the user simulator.
• How termination and maximum-turn limits are represented.

The scope should also clarify the architecture for integration with NeMo User Sim and produce a shared artifact how to leverage shared infra and benefits from both code bases.