microsoft / microsoft/agent-learning

Strengthen production governance: promotion approval, canary rollout, rollback, and monitoring

Open
#5 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
10
Forks
9
Avg merge
1d 19h
Merged PRs (30d)
4

Description

The Toulmin analysis says the SDK’s production story is directionally strong but incomplete because it does not yet define approval workflows, canary criteria, rollback triggers, or monitoring guidance. The architecture already supports versioned policy snapshots and rollback, but the operational controls around them need to be made concrete.

This ticket should move the SDK from “architecturally safe” to “operationally governed.”

Suggested scope

Add policy promotion workflow support with approval records.
Add canary rollout configuration and comparison metrics between old and new policy versions.
Add rollback triggers and a clear recovery path.
Add monitoring guidance for judge drift, reward-quality degradation, and learning pause behavior.

Acceptance criteria

The repo includes example governance workflows or docs for rollout and rollback.
The runtime can support canary comparison with persisted metrics.
The docs clearly separate capture, training, and promotion controls.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

No files or tests are named. Start by tracing the existing versioned policy snapshots and rollback support in the SDK, then inspect how runtime promotion and metrics are handled. Done means documented approval, canary, rollback, monitoring, and separate capture, training, and promotion controls, with persisted canary comparison metrics.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend, devops, documentation, observability
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.