ctengel / ctengel/simpler-objects

Review OpenAPI drift test and response validation

Open
#61 0 comments 0 reactions 0 assignees View on GitHub
documentation
Dominant language
Python
Stars
0
Forks
0
PR merge metrics
No merged PRs in 30d

Description

The test suite gained two pieces of OpenAPI tooling (added under #34, commits 10a1a58 and 5491dce) that are worth a deliberate review later. Both work and pass today, but were scoped pragmatically.

- `tests/test_openapi_drift.py` — asserts `openapi.yaml` and the FastAPI apps agree on which paths/methods exist.
- `tests/openapi_validation.py` — `ValidatingTestClient` checks every test response's status code and JSON body against `openapi.yaml`.

Things to revisit:

1. **Drift test scope.** It deliberately compares path + method coverage only, not status codes or response schemas — FastAPI's generated spec is too thin for that (see the module docstring). Decide whether that is the right permanent boundary.
2. **Validation approach.** `ValidatingTestClient` is a hand-rolled wrapper using `jsonschema`. Consider whether a dedicated tool (`schemathesis`, `openapi-core`) is a better long-term fit, especially for request-side validation, which is currently not covered.
3. **Undeclared test dependencies.** This work added `respx` and relies on `jsonschema`, `referencing`, and `pyyaml`. CLAUDE.md only lists `fastapi`/`uvicorn`/`httpx`/`pytest`, and there is no dependency manifest — these should be documented (or a manifest introduced).
4. **problem+json mismatch.** The spec documents error responses as `application/problem+json` / `ProblemDetail`, but FastAPI actually returns `application/json` / `{"detail": "..."}`. The validator tolerates this only because `ProblemDetail` has no required fields. Decide whether to emit real RFC 9457 problem details or relax the spec.
5. **Duplicate operationId warnings.** Pre-existing: `api_route` declaring GET+HEAD on one function produces duplicate operationIds, visible in test output. Cosmetic.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.