Azure / Azure/azure-functions-agents-runtime

Add Timeout Configuration to Foundry OpenAI Client

Open
#65 1 comment 0 reactions 0 assignees View on GitHub
area:serverless-agents task
Dominant language
Python
Stars
9
Forks
7
Avg merge
1d 21h
Merged PRs (30d)
20

Description

## Summary

The `azurefunctions-agents-runtime` 0.1.0b1 builds the Foundry chat client without configuring HTTP timeouts on the underlying Azure SDK transport. When the Foundry endpoint has transient latency on a Responses API roundtrip, the underlying socket read blocks indefinitely with no exception raised, causing agent execution to hang silently.

**Fix proposed in [PR #70](https://github.com/Azure/azure-functions-agents-runtime/pull/70).**

## Environment

- **Runtime version:** `azurefunctions-agents-runtime==0.1.0b1`
- **Python version:** 3.13
- **Function plan:** Flex Consumption (Linux)
- **Region:** westus3
- **Foundry model:** gpt-5 (GlobalStandard, 900K TPM capacity) — reproduced on `gpt-5.4`, `gpt-5.4-mini` too
- **Foundry endpoint:** `…/api/projects/{project}/openai/v1/responses` (stateful Responses API)

## Symptom

The agent runs cleanly for ~2 minutes, making multiple Foundry roundtrips for multi-turn reasoning. Then:

1. Agent POSTs a tool result to the Foundry `/responses` endpoint.
2. Request sends successfully (headers + body transmitted).
3. The httpx state machine reaches `receive_response_headers.started`.
4. **No response. No exception. No log.** The Python worker waits indefinitely.
5. The host eventually kills the worker after `functionTimeout` (30 min on Flex), but the invocation is silently lost — no error surfaces to the caller, no retry signal, no error log.

This manifests as "the agent randomly stops working after ~10 requests" — but it's actually: transient Foundry latency + missing httpx timeout → indefinite hang on a socket read.

## Evidence

- **Operation_Id:** `0ebc620e2d44cedfa3abb2b66560915f`
- **App Insights:** `appi-lqwx5rxrxq4xk` (RG `azure-functions-reports-agent-rg`)
- **DEBUG-level traces show exact hang point:**

```
19:52:52.227 UTC HTTP Request: POST .../openai/v1/responses
19:52:52.228 UTC send_request_headers.started
19:52:52.228 UTC send_request_headers.complete
19:52:52.228 UTC send_request_body.started
19:52:52.228 UTC send_request_body.complete
19:52:52.228 UTC receive_response_headers.started ← LAST TRACE
[SILENCE — no response, no exception, no timeout]
```

Previous Foundry roundtrips in the same invocation all succeeded (200 OK at 19:52:30, 19:52:33, 19:52:44, 19:52:49).

### Worker stack at the hung state

Captured by a `sys._current_frames()` watchdog thread (added to `function_app.py` purely for this diagnostic, routed via `print()` so it reaches the Function App's stdout pipeline):

```
[watchdog 2026-06-15T21:22:51Z] Thread frames:
File "", line 198, in _run_module_as_main
File "", line 88, in _run_code
File "/azure-functions-host/workers/python/3.13/LINUX/X64/proxy_worker/__main__.py", line 6, in
start_worker.start()
File "/azure-functions-host/workers/python/3.13/LINUX/X64/proxy_worker/start_worker.py", line 65, in start
return asyncio.run(start_async(
File "/opt/python/3/lib/python3.13/asyncio/runners.py", line 195, in run
return runner.run(main)
File "/opt/python/3/lib/python3.13/asyncio/runners.py", line 118, in run
return self._loop.run_until_complete(task)
```

Main thread blocked in `run_until_complete(task)` — event loop is alive, but the awaited coroutine (the Foundry call chain) is suspended indefinitely on a pending socket read. The hang lives inside the coroutine, behind `await client.create(...)` for the Responses API.

### Alternates ruled out

- ❌ **Worker SIGKILL / OOM** — zero `WorkerProcessExitException` in 8h; `MemoryWorkingSet` peak ~650 MB (~15% of available on Flex).
- ❌ **Worker segfault** — zero exit-code-139 events.
- ❌ **Foundry rate-limit at the deployment level** — usage 5–11% of 900 K TPM, zero 429 errors in App Insights.
- ❌ **Model-specific bug** — reproduced across `gpt-5`, `gpt-5.4`, `gpt-5.4-mini` (5-model investigation; same hang signature regardless of model choice).

## Root cause

In [`src/azure_functions_agents/client_manager.py` lines 198-202](https://github.com/Azure/azure-functions-agents-runtime/blob/main/src/azure_functions_agents/client_manager.py#L198-L202), `_build_foundry` passes `project_endpoint` (a string) to `FoundryChatClient`:

```python
return FoundryChatClient(
project_endpoint=endpoint,
model=model,
credential=build_async_credential(),
)
```

When given just an endpoint string, `FoundryChatClient` internally constructs `azure.ai.projects.aio.AIProjectClient(endpoint=…, credential=…)` using the Azure SDK pipeline's **default transport**. The default transport has no `httpx.Timeout` configured for read operations, so a stuck socket read on a Responses API roundtrip blocks forever and no exception ever fires.

## Proposed fix (implemented in [PR #70](https://github.com/Azure/azure-functions-agents-runtime/pull/70))

Have `_build_foundry` construct `AIProjectClient` itself with an explicit `AioHttpTransport` carrying connect/read timeouts, then pass the pre-built `project_client` to `FoundryChatClient`:

```python
@classmethod
def _build_foundry(cls, model: str) -> Any:
from agent_framework.foundry import FoundryChatClient
from azure.ai.projects.aio import AIProjectClient
from azure.core.pipeline.transport import AioHttpTransport

endpoint = cls._env("FOUNDRY_PROJECT_ENDPOINT")
if not endpoint:
raise RuntimeError(
"AZURE_FUNCTIONS_AGENTS_PROVIDER=foundry requires "
"FOUNDRY_PROJECT_ENDPOINT to be set."
)
read_timeout = float(cls._env("FOUNDRY_HTTP_READ_TIMEOUT") or 180)
connect_timeout = float(cls._env("FOUNDRY_HTTP_CONNECT_TIMEOUT") or 10)
transport = AioHttpTransport(
connection_timeout=connect_timeout,
read_timeout=read_timeout,
)
project_client = AIProjectClient(
endpoint=endpoint,
credential=build_async_credential(),
transport=transport,
)
return FoundryChatClient(
project_client=project_client,
model=model,
)
```

### Env-var knobs (defaults match Azure SDK examples)

- `FOUNDRY_HTTP_READ_TIMEOUT` — default `180.0` (seconds). Generous enough for the longest legitimate Responses-API call; short enough that a stuck connection fails an invocation in roughly 1/10th of the platform timeout window.
- `FOUNDRY_HTTP_CONNECT_TIMEOUT` — default `10.0` (seconds).

Both overridable via app settings without touching code.

### Why `mcp.json` timeouts don't fix this

Users may try configuring timeouts via `mcp.json`:

```json
{
"mcpServers": { "kusto": { "timeout": { "connect": 10, "read": 120 } } }
}
```

That **does not help**. `mcp.json` timeouts apply only to MCP tool calls (Kusto, GitHub, etc.). The Foundry chat client that orchestrates the agent loop is a separate client with separate transport configuration — and that's the one that hangs.

```
┌─────────────────────────────────────────────────┐
│ Azure Functions Agent (Python worker) │
├─────────────────────────────────────────────────┤
│ Foundry chat client (orchestrator) │ ← NO timeout (BUG)
│ └─ POST /openai/v1/responses (every reasoning │ ← HANGS HERE
│ turn) │
│ │
│ MCP clients (tools) │ ← mcp.json timeout applies
│ ├─ Kusto connector (120s read) │
│ └─ GitHub connector (60s read) │
└─────────────────────────────────────────────────┘
```

## Expected behaviour after fix

When Foundry is slow to respond:

1. The Azure SDK transport waits up to `FOUNDRY_HTTP_READ_TIMEOUT` seconds (default 180s).
2. If no response: raises an `azure.core.exceptions.ServiceResponseTimeoutError` (which wraps the underlying `httpx.ReadTimeout`).
3. The exception surfaces to Application Insights with a real stack trace.
4. Function execution fails cleanly instead of hanging silently.
5. Users can implement retry logic or investigate Foundry-side latency from observable failures.

## Impact if not fixed

- **Production blocker for long-running agents.** Agents using >10 Foundry roundtrips per invocation hang probabilistically.
- **Silent failures.** No error logs at default trace level; debugging requires DEBUG-level instrumentation + a thread-frame watchdog.
- **User cannot work around in user code.** The runtime instantiates the client internally; there's no hook to override transport config without forking.
- **Architectural workarounds are costly.** Splitting agents into shorter functions (so each stays under the hang threshold) adds significant complexity and is itself a workaround for a runtime bug.

## Out of scope (follow-up)

The same timeout gap exists in `_build_openai` (lines 153-159) and `_build_azure_openai` (lines 162-186) — `OpenAIChatClient` accepts a pre-built `async_client` parameter (`AsyncOpenAI` / `AsyncAzureOpenAI`), both of which take `http_client=httpx.AsyncClient(timeout=httpx.Timeout(...))`. Same pattern as this PR. Worth fixing in a follow-up.

Contributor guide

Open the contributing guide

Research direction

Start in src/azure_functions_agents/client_manager.py at _build_foundry, especially lines 198-202, and compare the proposed implementation in PR #70. Verify that Foundry client construction uses the configured connection and read timeouts, and confirm that a slow response fails with an observable timeout instead of hanging indefinitely.

Written by the indexing model from the issue text.

Assessment

Tech stack
azure, python
Domain
backend, cloud
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.