请教：如何训练react类型多轮人机对话

RT，请教各位大佬，如何训练react类型多轮人机对话

我想训练一个能够连续对话的Agent，SFT的方案是构造下面这种多轮数据
{
"messages": [
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "What is the capital of France?"},
{"role": "assistant", "content": "The capital of France is Paris."},
{"role": "user", "content": "And what about Germany?"},
{"role": "assistant", "content": "The capital of Germany is Berlin."},
]
}

想请教RL该如何做？使用另一个LLM模拟user的情况下，如何做rollout？如何计算奖励

No files, tests, or entry points are named. Start by clarifying the intended rollout setup with an LLM user and the reward definition for multi-turn SFT/RL training; done would be a concrete, documented procedure addressing both questions.

Written by the indexing model from the issue text.