openai / openai/openai-agents-python

RunState stringifies structured guardrail diagnostics and agent output during persistence

Open
#5,006 4 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
29.6k
Forks
4.8k
Avg merge
1d 20h
Merged PRs (30d)
123

Description

Please read this first
  • Have you read the docs? Yes: the guardrail guide uses a Pydantic result as GuardrailFunctionOutput.output_info, and RunState is the durable checkpoint boundary.
  • Have you searched for related issues? Yes. #3288 addressed non-JSON values making serialization fail. This report concerns structured models becoming opaque strings while serialization succeeds, not that original datetime exception.
Describe the bug

Guardrail diagnostics containing a normal Pydantic model lose their structure when a run is saved and restored. The same happens to structured OutputGuardrailResult.agent_output. An audit consumer can read fields such as allowed and reason before persistence, but afterward only a Python repr string remains; JSON decoding does not recover the fields.

This occurs with the documented shape GuardrailFunctionOutput(output_info=structured_result, ...), not only manually constructed RunState internals. The reproduction runs actual input/output guardrails and an Agent with structured output through Runner.run(), then saves result.to_state().to_string() and restores it with RunState.from_json().

A checkpoint used for approval/resume has the same diagnostic persistence path. This report is about retaining audit data, not a claim that guardrail decisions stop enforcing policy.

Debug information
  • Checkout baseline: fbf59a40, SDK 0.22.2.
  • Python 3.13.14, macOS arm64.
  • Uses the SDK's documented ScriptedModel for deterministic SDK-owned orchestration; no model service, API credentials or network required.
  • Reproduced consistently on the checkout. The release package was not separately installed for this report.
Reproduction
import asyncio
import json
from pydantic import BaseModel
from agents import Agent, Runner, RunConfig, RunState, GuardrailFunctionOutput
from agents.guardrail import InputGuardrail, OutputGuardrail
from agents.testing import ScriptedModel, assistant_message

class Verdict(BaseModel):
    allowed: bool
    reason: str

class Answer(BaseModel):
    text: str

async def check(*args):
    return GuardrailFunctionOutput(output_info=Verdict(allowed=True, reason='approved'), tripwire_triggered=False)

async def main():
    agent = Agent(name='Audit', model=ScriptedModel([[assistant_message('{"text":"hello"}')]]), output_type=Answer, input_guardrails=[InputGuardrail(check)], output_guardrails=[OutputGuardrail(check)])
    result = await Runner.run(agent, 'hello', run_config=RunConfig(tracing_disabled=True))
    snapshot = result.to_state().to_string()
    restored = await RunState.from_json(agent, json.loads(snapshot))
    print('input verdict:', restored._input_guardrail_results[0].output.output_info)
    print('output verdict:', restored._output_guardrail_results[0].output.output_info)
    print('answer:', restored._output_guardrail_results[0].agent_output)
    assert restored._input_guardrail_results[0].output.output_info == {'allowed': True, 'reason': 'approved'}
    assert restored._output_guardrail_results[0].agent_output == {'text': 'hello'}

asyncio.run(main())

Before the fix, restored values are strings:

input verdict: allowed=True reason='approved'
output verdict: allowed=True reason='approved'
answer: text='hello'
AssertionError

Expected plain JSON data:

input verdict: {'allowed': True, 'reason': 'approved'}
output verdict: {'allowed': True, 'reason': 'approved'}
answer: {'text': 'hello'}
Cause and expected behavior

The agent and tool guardrail serializers pass payloads directly to _ensure_json_compatible, which uses json.dumps(..., default=str). Pydantic models and dataclasses therefore become repr strings. Tool result serialization already has _serialize_output_value to preserve these values as plain data.

Preserve normal model/dataclass fields in guardrail snapshots using the same conversion pipeline, including structured values nested in containers. There is no requirement to reconstruct the original Python classes. Existing JSON-native payloads, old snapshots, and best-effort fallback for values whose custom serialization fails should remain supported.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with RunState.to_string(), RunState.from_json(), the guardrail serializers, and _ensure_json_compatible; compare them with the existing _serialize_output_value conversion used for tool results. Run the supplied ScriptedModel reproduction through Runner.run() and verify that Pydantic and dataclass values, including nested containers, restore as plain JSON data while existing JSON values, old snapshots, and fallback behavior remain supported.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
ai
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
70/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.