[Feature] GRPO-based Agent Evolution

Currently Ultron seems to rely on SFT-based LoRA training for evolution.

Have you considered introducing a GRPO stage after SFT?

A possible pipeline could be:

Trajectory Collection
→ SFT LoRA
→ GRPO Optimization

Reward signals could come from:

• Task success
• Tool execution success
• Planning efficiency
• Skill reuse quality

I think reinforcement learning may be a natural next step for Agent Evolution because it optimizes trajectories instead of only imitating them.

In addition, compared with pure SFT, reinforcement learning may help preserve previously acquired capabilities while optimizing new behaviors, potentially reducing catastrophic forgetting during continuous evolution.

Curious whether this has already been discussed in the roadmap.

Start by reading the existing SFT-based LoRA evolution pipeline and the project roadmap. Determine where trajectory collection and training stages are defined, then clarify the GRPO scope, reward signals, and evaluation criteria with maintainers; the issue does not yet define a concrete implementation or completion test.