Azure / Azure/azure-sdk-for-python

Agents Tracing fails when break statements are used

Open
#41,521 3 comments 0 reactions 1 assignee Claimed by @M-Hietala View on GitHub
Dominant language
Python
Stars
5.6k
Forks
3.4k
Avg merge
1d 21h
Merged PRs (30d)
193

Description

- **Package Name**: azure-ai-agents
- **Package Version**: 1.0.1
- **Operating System**: Windows
- **Python Version**: 3.11.9

**Describe the bug**
A clear and concise description of what the bug is.

**To Reproduce**
Steps to reproduce the behavior:
1. Following code when run raises an error:
```python
# ------------------------------------
# Copyright (c) Microsoft Corporation.
# Licensed under the MIT License.
# ------------------------------------

"""
DESCRIPTION:
This sample demonstrates how to use agent operations with toolset from
the Azure Agents service using a synchronous client with tracing to console.
This version includes enhanced gen AI tracing that captures input/output
for the create_and_process operation with detailed span attributes.

USAGE:
python sample_agents_toolset_with_console_tracing.py

Before running the sample:

pip install azure-ai-agents azure-identity opentelemetry-sdk azure-core-tracing-opentelemetry

If you want to export telemetry to OTLP endpoint (such as Aspire dashboard
https://learn.microsoft.com/dotnet/aspire/fundamentals/dashboard/standalone?tabs=bash)
install:

pip install opentelemetry-exporter-otlp-proto-grpc

Set these environment variables with your own values:
1) PROJECT_ENDPOINT - The Azure AI Project endpoint, as found in the Overview
page of your Azure AI Foundry portal.
2) MODEL_DEPLOYMENT_NAME - The deployment name of the AI model, as found under the "Name" column in
the "Models + endpoints" tab in your Azure AI Foundry project.
3) AZURE_TRACING_GEN_AI_CONTENT_RECORDING_ENABLED - Optional. Set to `true` to trace the content of chat
messages, which may contain personal data. False by default. This sample automatically enables it.
"""
from typing import Any, Callable, Set

import os, sys, time, json, atexit
from azure.core.settings import settings
from dotenv import load_dotenv

load_dotenv()

# Enable gen AI content recording for tracing
os.environ.setdefault("AZURE_TRACING_GEN_AI_CONTENT_RECORDING_ENABLED", "true")

settings.tracing_implementation = "opentelemetry"
# Install opentelemetry with command "pip install opentelemetry-sdk".
from opentelemetry import trace
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor, ConsoleSpanExporter, SpanExporter, SpanExportResult, \
SimpleSpanProcessor
from typing import Sequence
from opentelemetry.sdk.trace import ReadableSpan
from azure.ai.agents import AgentsClient
from azure.identity import DefaultAzureCredential
from azure.ai.agents.models import (
FunctionTool,
ToolSet,
ListSortOrder,
)
from azure.ai.agents.telemetry import trace_function
from azure.ai.agents.telemetry import AIAgentsInstrumentor

# Setup tracing to console
# Requires opentelemetry-sdk
span_exporter = ConsoleSpanExporter()
tracer_provider = TracerProvider()
# Use BatchSpanProcessor with immediate export to avoid shutdown issues
span_processor = SimpleSpanProcessor(span_exporter)
tracer_provider.add_span_processor(span_processor)
trace.set_tracer_provider(tracer_provider)
tracer = trace.get_tracer(__name__)

AIAgentsInstrumentor().instrument()

scenario = os.path.basename(__file__)
tracer = trace.get_tracer(__name__)

agents_client = AgentsClient(
endpoint=os.environ["PROJECT_ENDPOINT"],
credential=DefaultAzureCredential(),
)

# The trace_func decorator will trace the function call and enable adding additional attributes
# to the span in the function implementation. Note that this will trace the function parameters and their values.
@trace_function()
def fetch_weather(location: str) -> str:
"""
Fetches the weather information for the specified location.

:param location (str): The location to fetch weather for.
:return: Weather information as a JSON string.
:rtype: str
"""
# In a real-world scenario, you'd integrate with a weather API.
# Here, we'll mock the response.
mock_weather_data = {"New York": "Sunny, 25°C", "London": "Cloudy, 18°C", "Tokyo": "Rainy, 22°C"}

# Adding attributes to the current span
span = trace.get_current_span()
span.set_attribute("requested_location", location)

weather = mock_weather_data.get(location, "Weather data not available for this location.")
weather_json = json.dumps({"weather": weather})
return weather_json

# Statically defined user functions for fast reference
user_functions: Set[Callable[..., Any]] = {
fetch_weather,
}

# Initialize function tool with user function
functions = FunctionTool(functions=user_functions)
toolset = ToolSet()
toolset.add(functions)

# To enable tool calls executed automatically
agents_client.enable_auto_function_calls(toolset)

with tracer.start_as_current_span(scenario):
with agents_client:
# Create an agent and run user's request with function calls
agent = agents_client.create_agent(
model=os.environ["MODEL_DEPLOYMENT_NAME"],
name="my-agent",
instructions="You are a helpful agent",
toolset=toolset,
)
# print(f"Created agent, ID: {agent.id}")

thread = agents_client.threads.create()
# print(f"Created thread, ID: {thread.id}")

message = agents_client.messages.create(
thread_id=thread.id,
role="user",
content="Hello, what is the weather in New York?",
)
# print(f"Created message, ID: {message.id}")

# run = agents_client.runs.create_and_process(thread_id=thread.id, agent_id=agent.id, toolset=toolset)

# Create a span specifically for the gen AI operation to capture input/output
with tracer.start_as_current_span("agents.runs.create_and_process") as gen_ai_span:
# Set gen AI operation attributes
gen_ai_span.set_attribute("gen_ai.system", "azure.ai.agents")
gen_ai_span.set_attribute("gen_ai.operation.name", "create_and_process")
gen_ai_span.set_attribute("gen_ai.request.model", os.environ["MODEL_DEPLOYMENT_NAME"])
gen_ai_span.set_attribute("gen_ai.request.thread_id", thread.id)
gen_ai_span.set_attribute("gen_ai.request.agent_id", agent.id)
#
# # Capture input content and add as event
input_messages = agents_client.messages.list(thread_id=thread.id, order=ListSortOrder.ASCENDING)
for msg in input_messages:

if msg.text_messages:
for text_msg in msg.text_messages:
# Add individual message events with role-based names
gen_ai_span.add_event(
name=f"gen_ai.{msg.role.value}.message",
attributes={
"gen_ai.system": "azure.ai.agents",
"gen_ai.event.content": json.dumps(
{"role": msg.role.value, "content": text_msg.text.value}),
},
)
#
#
try:
run = agents_client.runs.create_and_process(thread_id=thread.id, agent_id=agent.id, toolset=toolset)

# Capture output attributes
gen_ai_span.set_attribute("gen_ai.response.id", run.id)
gen_ai_span.set_attribute("gen_ai.response.status", run.status)
gen_ai_span.set_attribute("gen_ai.response.model", run.model)

# Capture output content after run completion and add as event
output_messages = agents_client.messages.list(thread_id=thread.id, order=ListSortOrder.DESCENDING)

###########################################
# This for loop is causing the error
# ImportError: sys.meta_path is None, Python is likely shutting down
#######################################
for msg in output_messages:
if msg.text_messages:
for text_msg in msg.text_messages:
# Add individual message events with role-based names
gen_ai_span.add_event(
name=f"gen_ai.{msg.role.value}.message",
attributes={
"gen_ai.system": "azure.ai.agents",
"gen_ai.event.content": json.dumps(
{"role": msg.role, "content": text_msg.text.value}),
},
)
break # Break after first output message to avoid too many events
break
except Exception as e:
gen_ai_span.set_attribute("gen_ai.response.error", str(e))
gen_ai_span.add_event(
name="gen_ai.error",
attributes={"error.message": str(e)}
)
gen_ai_span.record_exception(e)
raise

agents_client.delete_agent(agent.id)
```
Error raised

```json
Exception while exporting Span.
Traceback (most recent call last):
File "C:\Users\anksing\.conda\envs\remote_evaluation\Lib\site-packages\azure\ai\agents\telemetry\_instrument_paged_wrappers.py", line 131, in _gen
yield val
GeneratorExit

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "C:\Users\anksing\.conda\envs\remote_evaluation\Lib\site-packages\opentelemetry\sdk\trace\export\__init__.py", line 113, in on_end
self.span_exporter.export((span,))
File "C:\Users\anksing\.conda\envs\remote_evaluation\Lib\site-packages\opentelemetry\sdk\trace\export\__init__.py", line 512, in export
self.out.write(self.formatter(span))
^^^^^^^^^^^^^^^^^^^^
File "C:\Users\anksing\.conda\envs\remote_evaluation\Lib\site-packages\opentelemetry\sdk\trace\export\__init__.py", line 504, in
] = lambda span: span.to_json() + linesep,
^^^^^^^^^^^^^^
File "C:\Users\anksing\.conda\envs\remote_evaluation\Lib\site-packages\opentelemetry\sdk\trace\__init__.py", line 492, in to_json
start_time = util.ns_to_iso_str(self._start_time)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\anksing\.conda\envs\remote_evaluation\Lib\site-packages\opentelemetry\sdk\util\__init__.py", line 29, in ns_to_iso_str
return ts.strftime("%Y-%m-%dT%H:%M:%S.%fZ")
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ImportError: sys.meta_path is None, Python is likely shutting down
```

**Expected behavior**
No error should be raised

**Screenshots**
If applicable, add screenshots to help explain your problem.

**Additional context**
Add any other context about the problem here.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.