langchain-ai / langchain-ai/agentevals

Graph trajectory extraction does not handle `Command`/`Send` API

Open
#40 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
724
Forks
56
PR merge metrics
No merged PRs in 30d

Description

Specifically, `extract_langgraph_trajectory_from_snapshots` fails to detect a checkpoint of a node that used `Command`/`Send` to go to another node (such as with[ `langgraph_supervisor`](https://github.com/langchain-ai/langgraph-supervisor-py/blob/4ff497c1b04bc5038d51b58cd8399b26ddc28a5d/langgraph_supervisor/handoff.py#L106)).

This is probably because the `StateSnapshot` list does not have an `__end__` node checkpoint which fulfills the criteria of the function.

Contributor guide

Open the contributing guide

Research direction

Start with extract_langgraph_trajectory_from_snapshots and inspect how its StateSnapshot list identifies node checkpoints. Compare that behavior with the linked langgraph_supervisor handoff.py Command/Send example. Done means the trajectory extractor handles this checkpoint pattern and has regression coverage for it.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
ai
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.