microsoft / microsoft/agent-framework
Python: dependency-bounds validator is not enforced in CI
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 13.6k
- Forks
- 2.3k
- Avg merge
- 2d 45m
- Merged PRs (30d)
- 358
Description
The Python dependency-bounds validator (`uv run poe validate-dependency-bounds-test`) is not wired into required CI, so dependency-floor, packaging, and typing regressions can land on main and ship in a release without being caught.
Evidence
Running the validator at the `python-1.8.1` release tag already fails for core at the `lowest-direct` resolution:
```
packages/core: lower failed
Task 'test' failed for packages/core at resolution 'lowest-direct'.
FAILED tests/core/test_telemetry.py::test_detect_hosted_fallback_import_error
1 failed, 3174 passed, 18 skipped, 2 xfailed
```
So 1.8.1 was tagged and released while the validator was red.
On current main the same validator fails with more: the telemetry test above, plus 4 harness shell-tool tests (`ModuleNotFoundError: No module named 'agent_framework_tools'`) and 14 pyright errors on `_harness/_agent.py`, all from the shell-tool integration in #6451. The validator stops at the first failing task, which is why only the telemetry failure surfaced at 1.8.1.
Two latent problems this masked
1. `packages/core/agent_framework/_telemetry.py` caught `(ModuleNotFoundError, ValueError)` around `importlib.util.find_spec(...)`, but `find_spec` imports the parent package and can raise a bare `ImportError`, which escaped the handler. The test passes in the full test environment (where `azure.ai.agentserver` is already imported, so `find_spec` reuses the cached parent and never hits the mocked import) but fails in the validator's isolated environment. Pre-existing since #5455.
2. The harness shell-tool integration (#6451) imports `agent_framework_tools` lazily because that package depends on core, so core cannot take a runtime dependency on it. But core did not declare it even as a dev or optional dependency, so any environment that builds core in isolation hits `ModuleNotFoundError` at test time and unresolved types under pyright.
Both are fixed in the 1.9.0 version bump (branch `bump-py-ver-release-260617`): broaden the except to `ImportError`, add an `agent-framework-tools` dev dependency-group to core, and guard the shell-tool tests with `skipif`. With those, the validator passes fully (lower and upper, all packages).
Proposal
Wire `validate-dependency-bounds-test` into Python CI so these regressions are caught at PR time rather than at release time:
- Run it on PRs that touch `python/`, or at minimum on the release branch and nightly.
- It exercises both `lowest-direct` and `highest` resolutions per package and runs each package's pyright pass, so it catches floor-too-low, missing-optional-dependency, and type-checking-against-isolated-env problems that the normal full-workspace test job does not.
Cost and scoping note: a full workspace run is somewhat slow (lower plus upper per package). If per-PR cost is a concern, options are to scope the run to changed packages, or run the full sweep nightly plus required-on-release-branch.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Locate the existing Python CI configuration and the `uv run poe validate-dependency-bounds-test` entry point first. Determine how required PR, release-branch, or nightly jobs are defined and whether package scoping is supported. Done means the validator runs in the chosen CI scope and fails the job when dependency-bound or pyright checks fail.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- ci-cd
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100