microsoft / microsoft/agent-framework
Python: Citation Annotation Support for Responses API and AGUI
@moonbox3 is already working on this.
Since Aug 3, 2026.
- Dominant language
- Python
- Stars
- 13.6k
- Forks
- 2.3k
- Avg merge
- 2d 45m
- Merged PRs (30d)
- 358
Description
I'm not sure if this is a bug or a feature request. When connecting to Foundry agents with SharePoint grounding (via agent-framework-openai / agent-framework-ag-ui) , the OpenAI Responses API emits response.content_part.done events containing AnnotationURLCitation objects. These are currently logged as "Unparsed" by agent_framework.openai and discarded, so citation data never reaches the AG-UI event stream or frontend.
We think this is maybe by design - the framework is transport-agnostic and doesn't parse provider-specific events. However, there's currently no supported way for applications to access this data, which means building citation UI requires fragile workarounds (see below).
Environment
- agent-framework-core: 1.13.0 / agent-framework-foundry: 1.10.4 / agent-framework-ag-ui: 1.0.1 / agent-framework-openai: 1.12.0
- azure-ai-projects: 2.3.0 / openai: 2.51.0 / Python: 3.12
Evidence
The citation data arrives complete and well-formed, but is discarded:
2026-07-31 12:05:54,178 [DEBUG] agent_framework.openai - Unparsed event of type: response.output_text.done: ResponseTextDoneEvent(...)
2026-07-31 12:05:54,179 [DEBUG] agent_framework.openai - Unparsed event of type: response.content_part.done: ResponseContentPartDoneEvent(
part=ResponseOutputText(
annotations=[
AnnotationURLCitation(
end_index=1154, start_index=1142,
title='https://sharepoint.example.com/sites/internal-site/Shared Documents/document-one.pdf',
type='url_citation',
url='https://sharepoint.example.com/sites/internal-site/Shared Documents/document-one.pdf'
),
# ... more citations
],
text='Full response text...',
type='output_text'
),
type='response.content_part.done'
)
Current Workaround
We've implemented a logging handler that intercepts agent_framework.openai log records, extracts AnnotationURLCitation objects from response.content_part.done events, and re-emits them as a CustomEvent before RUN_FINISHED. This works but is fragile (depends on log message format) and non-discoverable.
Status: Tested and confirmed working - captures citations per grounded query
Request
Is there a supported or recommended way to access response.content_part.done annotation data from an AG-UI application? If not, could the framework provide one?
Some options we've considered:
- Emit a
CustomEventfrom the framework when citations are detected (similar to the existingoauth_consent_requestpattern) - A hook or callback mechanism for applications to handle unparsed provider events
- Documentation on the recommended pattern for extracting grounding metadata
Questions
- Is citation/annotation support on the roadmap for agent-framework?
- Is the logging-handler workaround ok short-term, or is there a better supported path?
- Are there other Responses API events (reasoning, moderation, etc.) that other teams have needed to surface similarly?
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.