microsoft / microsoft/agent-framework
Python: Re-enable temporarily skipped Python integration tests (merge-queue xdist runner crash)
@giles17 is already working on this.
Since Jun 27, 2026.
- Dominant language
- Python
- Stars
- 13.6k
- Forks
- 2.3k
- Avg merge
- 2d 45m
- Merged PRs (30d)
- 358
Description
Summary
While unblocking #6753 (a routine version bump), the Python integration test jobs in the merge queue were crashing the pytest-xdist runner — every worker reported node down: Not properly terminated and the whole pytest session collapsed (reported as e.g. "20 failed"). This is an environmental/flaky failure unrelated to the version-bump diff.
To unblock the merge queue, a set of integration tests was temporarily skipped. This issue tracks everything that was skipped so it can be re-enabled and properly fixed.
Background
- The affected jobs run on a large hosted runner;
pytest -n logicalspawns ~20 xdist workers (gw0–gw19) against live Azure/OpenAI services. - Reducing workers to
-n 4did not help — the runner still spawned 20 workers and collapsed identically, so worker count is not the controlling factor. - The crash signature is a whole-session collapse, not individual assertion failures.
Example failing runs:
- https://github.com/microsoft/agent-framework/actions/runs/28271519026/job/83769710030
- https://github.com/microsoft/agent-framework/actions/runs/28262767119/job/83742131589
- https://github.com/microsoft/agent-framework/actions/runs/28252333139/job/83706957254
What was skipped
1. Azure OpenAI integration tests (entire integration set)
Skipped by changing the per-file skip_if_azure_openai_integration_tests_disabled guard from a conditional pytest.mark.skipif(...) to an unconditional pytest.mark.skip(...). Only the integration tests are skipped; the unit tests in these files still run.
packages/openai/tests/openai/test_openai_chat_client_azure.py
test_integration_web_searchtest_integration_client_file_searchtest_integration_client_file_search_streamingtest_integration_client_agent_hosted_mcp_tooltest_integration_client_agent_hosted_code_interpreter_tooltest_integration_client_agent_existing_sessiontest_azure_openai_chat_client_tool_rich_content_image- (all
@pytest.mark.integrationtests in this file)
packages/openai/tests/openai/test_openai_chat_completion_client_azure.py
test_azure_openai_chat_completion_client_responsetest_azure_openai_chat_completion_client_response_toolstest_azure_openai_chat_completion_client_streamingtest_azure_openai_chat_completion_client_streaming_toolstest_azure_openai_chat_completion_client_agent_basic_runtest_azure_openai_chat_completion_client_agent_basic_run_streamingtest_azure_openai_chat_completion_client_agent_session_persistencetest_azure_openai_chat_completion_client_agent_existing_sessiontest_azure_chat_completion_client_agent_level_tool_persistence
packages/openai/tests/openai/test_openai_embedding_client_azure.py
test_azure_openai_get_embeddingstest_azure_openai_get_embeddings_multipletest_azure_openai_get_embeddings_with_dimensions
2. Azure Functions / Durable Task integration tests (5 specific tests)
Skipped individually with @pytest.mark.skip(...) because these were the concrete failing/flaky tests:
packages/azurefunctions/tests/integration_tests/test_02_multi_agent.py::TestSampleMultiAgent::test_weather_agent—requests.exceptions.ReadTimeout(read timeout=30)packages/azurefunctions/tests/integration_tests/test_11_workflow_parallel.py::TestWorkflowParallel::test_parallel_workflow_end_to_end—worker 'gw2' crashed(likely triggers the cascade)packages/durabletask/tests/integration_tests/test_02_dt_multi_agent.py::TestMultiAgent::test_weather_agent_with_tool—AssertionError: assert 0 > 0(emptyAgentResponse.text)packages/durabletask/tests/integration_tests/test_02_dt_multi_agent.py::TestMultiAgent::test_math_agent_with_tool—AssertionError: assert 0 > 0(emptyAgentResponse.text; same flakiness astest_weather_agent_with_tool)packages/durabletask/tests/integration_tests/test_06_dt_multi_agent_orchestration_conditionals.py::TestMultiAgentOrchestrationConditionals::test_conditional_branching—TimeoutError: Timed-out waiting for the orchestration to complete
How to re-enable
- Azure OpenAI: restore the original
pytest.mark.skipif(...)env-based guard in each of the three files. - Functions / Durable Task: remove the four
@pytest.mark.skip(...)decorators.
Follow-up / root cause to investigate
- Why does the Azure OpenAI integration job collapse the runner under high xdist parallelism (resource exhaustion / OOM / connection limits against live services)?
- Whether the parallel-workflow Functions test (
test_parallel_workflow_end_to_end) is the trigger for the worker crash cascade. - Whether these live-service integration tests should gate the merge queue at all, vs. running post-merge / on a schedule.
Related PR: #6753
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.