Speed up the developer test loop: profile slow tests, enable parallel execution, and budget suite runtime

Summary

Profile the test suite (~1,850 test functions across 111 files), enable pytest-xdist parallel execution where safe, introduce a slow marker with a fast default selection for local iteration, and add a CI-visible runtime budget so suite growth stays deliberate.

Why this matters

The suite is the project's main quality asset and it runs constantly: locally via make test/make ci, and in CI across a 4-Python-version gating matrix plus a floor-deps job. Every minute of suite runtime is multiplied by that matrix and by every contributor iteration. A faster loop directly improves contributor experience and shortens CI feedback without weakening any gate.

Current evidence

• tests/ contains 111 test files and ~1,848 test functions (grep count); AGENTS.md/README advertise the 1,150+ gating test count.
• pyproject.toml declares no pytest-xdist, no pytest-timeout, and no marker taxonomy for slow/integration tests (grep: no xdist or benchmark references).
• Several test families are end-to-end shaped and likely dominate runtime: tests/test_architectures_mcp_context_gateway*.py (four variants), tests/test_demos_mcp_gateway_full.py, tests/test_gateway_benchmark.py, golden/scorecard tests.
• Evidence is partial: no timing data is committed; the first task below produces it.
• Related open issue: #474 closes the local-vs-CI gate gap and adds workflow timeouts — complementary; it does not address suite speed itself.

External context

pytest-xdist (-n auto), --durations, and tiered marker strategies are standard practice for suites at this scale.

Proposed implementation

1. Measure: run pytest --durations=50 per Python version locally/CI; commit a short findings note (slowest tests, total wall time, parallelizability blockers).
2. Enable pytest-xdist in the dev extra and CI; fix any test isolation issues surfaced (shared tmp paths, global registries, port use in gateway tests).
3. Add markers: slow for the dominating end-to-end tests; default local make test runs everything but documents -m "not slow" for inner-loop iteration (CI keeps running the full suite — no gate weakening).
4. Add a non-gating runtime report (or soft budget warning) to CI so regressions in suite time are visible in PRs.

AI-agent execution notes

• Inspect first: Makefile test/ci targets, pyproject.toml [tool.pytest.ini_options], conftest.py fixtures (session-scoped state that xdist would duplicate), the four gateway-architecture test modules.
• Run: pytest --durations=50 -q before and after; pytest -n auto -q to flush isolation bugs.
• Preserve: determinism of golden/scorecard tests under parallelism (worker-id-independent outputs); the full suite must remain the CI gate.
• Failure modes: hidden inter-test coupling exposed by reordering — fix the coupling, never pin ordering; coverage measurement under xdist needs coverage[toml] concurrency config.

Acceptance criteria

• A committed findings note documents baseline timings and the slowest 20 tests.
• CI suite wall time measurably drops (target: ≥30% on the primary matrix cell) with zero tests removed or skipped in CI.
• slow marker exists with documented local usage; full suite still gates CI.
• Parallel runs are deterministic across three consecutive CI runs.

Test plan

The change is to the test harness: before/after timing comparison in the PR description; three repeated CI runs green; coverage totals unchanged (±0.1%) proving no tests were lost under xdist.

Documentation plan

docs/agent-context/workflows.md (authoritative commands) updated with the marker and parallel invocations; CONTRIBUTING/AGENTS test-running sections; CHANGELOG dev-facing note.

Migration and compatibility notes

Not expected to require migration; local workflows gain optional flags.

Risks and tradeoffs

xdist can mask order-dependent bugs by coincidence (mitigate by also keeping a serial CI cell, e.g. the floor-deps job); marker discipline requires review attention so new slow tests get tagged. Alternative — leaving runtime to grow unmanaged — taxes every contributor and every CI minute indefinitely.

Suggested labels

testing, developer-experience, contributor-experience

Start with the Makefile test/ci targets, pyproject.toml pytest settings, conftest.py fixtures, and the four gateway-architecture test modules. Run pytest --durations=50 -q, then pytest -n auto -q to identify slow tests and isolation issues. Done means documented timings, a slow marker and local usage, deterministic parallel runs, and a full CI gate with no skipped tests.