NVIDIA / NVIDIA/nvcf

Add an OpenAPI specification for llm-api-gateway

Open
#359 0 comments 0 reactions 1 assignee View on GitHub

@along-2017 is already working on this.

Since Jul 22, 2026.

Dominant language
Go
Stars
218
Forks
72
Avg merge
1d 12h
Merged PRs (30d)
427

Description

Is this related to a problem? Please describe.

llm-api-gateway has no API spec. The bodies are OpenAI-compatible, but the
gateway adds a transport contract that OpenAI clients do not model. For
example:

  • By default, model must be <function_id>/<model>; a bare model id returns 400.
  • Errors use {"message": ...}, not OpenAI's {"error": {...}}.

It also adds multi-turn session continuity and rate-limit signaling that stock
OpenAI clients do not handle. Consumers hit these with no spec to consult, and
there is nothing to drive client generation or contract testing.

Describe the solution you'd like

Publish an OpenAPI 3.1 spec for the gateway. Proposed shape, open to
discussion:

  • Hand-write the spec as the source of truth. Keep bodies loose
    (OpenAI-compatible objects); spend the detail on where the gateway differs
    from OpenAI.
  • Serve it from the running gateway: the raw spec plus a rendered reference.
  • Add a test so the spec cannot silently drift from the documented endpoints.
  • Link it from the README.

No new dependencies are required. Implementation details (embedding, exact
routes) can be settled in the PR.

OpenAI-compatible providers publish similar contracts. Fireworks, for
example, documents its vendor extensions as extra request fields and custom
headers (including a multi-turn session header) in a curated OpenAPI
reference.


By submitting this issue, you agree to follow our code of conduct and our contributing guidelines.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.