microsoft / microsoft/agent-framework

.NET: Defer checkpoint cleanup until workflow restoration succeeds

Open
#7,796 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

.NET
Dominant language
Python
Stars
13.6k
Forks
2.3k
Avg merge
2d 45m
Merged PRs (30d)
358

Description

### Description

`FoundryJsonCheckpointStore.RetrieveCheckpointAsync` currently prunes checkpoints, meaning it deletes older records from the restored checkpoint's ancestry, before returning the stored JSON.

Workflow restoration still has several failure points after that call:

1. `CheckpointManagerImpl.LookupCheckpointAsync` deserializes the stored JSON.
2. `InProcessRunner.RestoreCheckpointCoreAsync` checks whether the checkpoint belongs to the current workflow.
3. The runner imports workflow, executor, and edge state.

An incompatible, corrupt, or otherwise failed restoration can therefore delete ancestors even though the workflow never resumes successfully. Explicitly restoring an older checkpoint has the same risk.

Expected behavior: reading a checkpoint must not delete recoverable state before restoration succeeds.

### Proposed direction

Add a safe cleanup point after successful workflow restoration. Options may include a post-restore notification on the checkpoint store contract or an explicit cleanup operation invoked by the checkpoint manager after validation and state import complete.

The implementation should:

* Keep checkpoint retrieval non-destructive until restoration succeeds.
* Preserve sibling branches and checkpoints committed by concurrent or later runs.
* Keep cleanup failures from failing an otherwise successful restoration.
* Cover incompatible workflows, malformed checkpoint data, state import failures, and restoration of a non-latest checkpoint.

### Package Versions

Unreleased source change in PR #7649.

### .NET Version

.NET 10

### Additional Context

Discovered during review of #7649: https://github.com/microsoft/agent-framework/pull/7649#discussion_r3823681841

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at FoundryJsonCheckpointStore.RetrieveCheckpointAsync and trace the restoration sequence through CheckpointManagerImpl.LookupCheckpointAsync and InProcessRunner.RestoreCheckpointCoreAsync. Verify that incompatible, malformed, failed-import, and non-latest restorations retain recoverable state, while successful restoration cleans up safely without removing sibling or concurrent checkpoints; cleanup failures must not fail restoration.

Written by the indexing model from the issue text.

Assessment

Tech stack
csharp
Domain
backend, distributed-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.