Add convergence, stability, and ablation coverage for REINFORCE-with-baseline

The Toulmin analysis says the REINFORCE choice is reasonable, but the paper does not show convergence behavior, sample efficiency, update stability, or sensitivity to reward noise. That means the current design is well-motivated, but not yet empirically proven.

This ticket should make the learner’s behavior measurable and easy to reason about.

Suggested scope

Add tests and benchmark scripts for convergence speed across seeds.
Include stability metrics such as reward variance and policy drift.
Add ablation tests for the EMA baseline, entropy term, and learning-rate settings.
Document expected learning behavior under noisy and delayed rewards.

Acceptance criteria

There is a reproducible convergence benchmark across multiple seeds.
Results include stability and variance metrics.
The learner’s key configuration knobs are documented with recommended ranges and behavior expectations.

No files, tests, or entry points are named. Start by locating the REINFORCE-with-baseline learner and existing benchmark or test setup; define a reproducible multi-seed convergence run with reward variance and policy-drift outputs, then cover EMA baseline, entropy, and learning-rate ablations. Done means reproducible results and documented noisy and delayed-reward expectations with recommended configuration ranges.