NVIDIA-NeMo / NVIDIA-NeMo/Guardrails
epic: close OpenAI API conformance gaps in Guardrails server
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 7.2k
- Forks
- 842
- Avg merge
- 3d 1h
- Merged PRs (30d)
- 25
Description
Goal
The NeMo Guardrails server's OpenAI-compatible API should match the
CreateChatCompletionRequest / CreateChatCompletionResponse / ListModels
contracts closely enough that standard OpenAI SDK clients work without
surprises. This is an important step for this server to become a transparent inference proxu
Why
The conformance CI (api-conformance-impl.yml) reports around 100 gaps between
app.openapi() and OpenAI v2.3.0. Some are harmless schema metadata diffs, but
others cause real problems like missing request properties, route metadata differences etc
Scope
Potential sub-issues might include:
- Missing request properties (~23 gaps)
- Request schema typing — nullable, defaults, anyOf/oneOf (~24 gaps)
- Response
usagerequired fields (~7 gaps) - Response types, enums, nullables, streaming media type (~24 gaps)
- Route metadata — operation IDs, tags (4 gaps)
GET /modelsresponse schema (~9 gaps)- Conformance script noise reduction (~10 gaps)
Local reproduction (requires oasdiff):
# Install oasdiff e.g.
brew install oasdiff
# Run the conformance check (fetches the OpenAI spec and diffs via oasdiff)
uv run --locked python scripts/openai_coverage.py --fetch --fastapi
Completion criteria
scripts/openai_coverage.py --fastapireports 0 conformance gaps (excluding
the intentional-extensions allowlist).- Intentional Guardrails extensions (
guardrailsrequest/response properties,
422status) are reported separately, not as gaps. - Unimplemented endpoints (Stored Completions, single-model CRUD) are excluded
from the gap count. - All existing tests pass; new tests cover the added request properties and
response schema changes.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with scripts/openai_coverage.py and run uv run --locked python scripts/openai_coverage.py --fetch --fastapi to review the reported gaps against the OpenAI v2.3.0 contracts. Work through the listed request, response, route, and model-schema differences, then verify that the command reports no unallowlisted gaps and that all existing and new tests pass.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- fastapi, openapi, python
- Domain
- api, backend, testing-qa
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100