NVIDIA-NeMo / NVIDIA-NeMo/Guardrails
feature: Native support for /v1/messages endpoints
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 7.2k
- Forks
- 842
- Avg merge
- 3d 1h
- Merged PRs (30d)
- 25
Description
Did you check the docs?
- I have read all the NeMo-Guardrails docs
Is your feature request related to a problem? Please describe.
While there is a langchain integration with /v1/messages, there is no external endpoint that accepts /v1/messages payloads. This means it's impossible to directly proxy a /v1/messages endpoint - you'd have to translate your request into /chat/completions first, then translate the response.
I'd like to be able to natively use /v1/messages throughout the library, namely as models specified in the model config (for self-check, etc), or to use the API as my main interface for interacting with NeMo Guardrails.
Describe the solution you'd like
- A native /v1/messages client, mirroring the existing /v1/chat/completions client, that allows for native, non-langchain communication with /v1/messages models
- A /v1/messages endpoint in the NeMo Guardrails server, for guardrailed inference proxying
- A /v1/messages/checks endpoint, for standalone guardrailing
Describe alternatives you've considered
External API translation by some other middleware component. This works, but adds setup complexity and can present issues with streaming.
Additional context
No response
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by locating the existing /v1/chat/completions client and NeMo Guardrails server endpoint, then trace how model configuration and standalone checks use them. Define the matching behavior for /v1/messages, /v1/messages/checks, and streaming before implementing the client and proxy paths; done means native payloads work without translation and the existing checks remain available.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- api, backend
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100