NVIDIA / NVIDIA/nvcf

feat(llm-routing): accept and persist routing expressions in the function API

Open
#1,407 0 comments 0 reactions 1 assignee View on GitHub

@along-2017 is already working on this.

Since Sep 3, 2026.

Dominant language
Go
Stars
218
Forks
72
Avg merge
1d 12h
Merged PRs (30d)
427

Description

Description

Part of #536 (per-model routing configuration).

Let function owners persist routing expressions in llmConfig.routingMethod.
The function management API validates syntax only, against the same RFC 8941
profile the router parses: a well-formed expression persists even when its
method, parameter, type, or range is semantically wrong. The router is the
semantic authority and rejects such values at request time.

The value is stored as received apart from leading and trailing spaces: no
lowercasing, underscore folding, or alias mapping. The router already folds
case and underscores and resolves alias spellings when it resolves the
algorithm name, so every spelling accepted today keeps working. Tabs, line
breaks, and other control characters anywhere in the value are rejected.
Expressions keep the existing per-function scope: an update on any version
propagates to all versions.

There is no feature flag and no deploy ordering. In an environment whose
routers do not parse expressions yet, a model configured with parameters is
rejected by the router at request time until the routers upgrade or the owner
reverts to a method-only value.

Definition of Done

  • Well-formed expressions persist via create, version create, and update
    in JSON model definitions; malformed expressions are rejected with a
    message naming the model and the violated rule
  • Semantically wrong but well-formed expressions persist by design
  • The persisted value equals the submitted value apart from leading and
    trailing spaces
  • Updates propagate across the function's versions as today
  • Table-driven tests cover the grammar profile, the stored form, and both
    API paths

Resources

  • Parent: #536
  • Implementation: #1955
  • Blocked by: none

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.