microsoft / microsoft/agent-framework

Python: Citation Annotation Support for Responses API and AGUI

Open
#7,460 1 comment 0 reactions 1 assignee View on GitHub

@moonbox3 is already working on this.

Since Aug 3, 2026.

ag-ui python
Dominant language
Python
Stars
13.6k
Forks
2.3k
Avg merge
2d 45m
Merged PRs (30d)
358

Description

I'm not sure if this is a bug or a feature request. When connecting to Foundry agents with SharePoint grounding (via agent-framework-openai / agent-framework-ag-ui) , the OpenAI Responses API emits response.content_part.done events containing AnnotationURLCitation objects. These are currently logged as "Unparsed" by agent_framework.openai and discarded, so citation data never reaches the AG-UI event stream or frontend.

We think this is maybe by design - the framework is transport-agnostic and doesn't parse provider-specific events. However, there's currently no supported way for applications to access this data, which means building citation UI requires fragile workarounds (see below).

Environment

  • agent-framework-core: 1.13.0 / agent-framework-foundry: 1.10.4 / agent-framework-ag-ui: 1.0.1 / agent-framework-openai: 1.12.0
  • azure-ai-projects: 2.3.0 / openai: 2.51.0 / Python: 3.12

Evidence

The citation data arrives complete and well-formed, but is discarded:

2026-07-31 12:05:54,178 [DEBUG] agent_framework.openai - Unparsed event of type: response.output_text.done: ResponseTextDoneEvent(...)
2026-07-31 12:05:54,179 [DEBUG] agent_framework.openai - Unparsed event of type: response.content_part.done: ResponseContentPartDoneEvent(
  part=ResponseOutputText(
    annotations=[
      AnnotationURLCitation(
        end_index=1154, start_index=1142,
        title='https://sharepoint.example.com/sites/internal-site/Shared Documents/document-one.pdf',
        type='url_citation',
        url='https://sharepoint.example.com/sites/internal-site/Shared Documents/document-one.pdf'
      ),
      # ... more citations
    ],
    text='Full response text...',
    type='output_text'
  ),
  type='response.content_part.done'
)

Current Workaround

We've implemented a logging handler that intercepts agent_framework.openai log records, extracts AnnotationURLCitation objects from response.content_part.done events, and re-emits them as a CustomEvent before RUN_FINISHED. This works but is fragile (depends on log message format) and non-discoverable.

Status: Tested and confirmed working - captures citations per grounded query

Request

Is there a supported or recommended way to access response.content_part.done annotation data from an AG-UI application? If not, could the framework provide one?

Some options we've considered:

  • Emit a CustomEvent from the framework when citations are detected (similar to the existing oauth_consent_request pattern)
  • A hook or callback mechanism for applications to handle unparsed provider events
  • Documentation on the recommended pattern for extracting grounding metadata

Questions

  1. Is citation/annotation support on the roadmap for agent-framework?
  2. Is the logging-handler workaround ok short-term, or is there a better supported path?
  3. Are there other Responses API events (reasoning, moderation, etc.) that other teams have needed to surface similarly?

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.