NVIDIA-NeMo / NVIDIA-NeMo/DataDesigner

Allow model providers to restrict workflows to approved models

Open
#711 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement needs-discussion
Dominant language
Python
Stars
2.2k
Forks
211
Avg merge
2d 6h
Merged PRs (30d)
40

Description

Priority Level

Medium (Nice to have)

Is your feature request related to a problem? Please describe.

Model providers can expose many models, but a DataDesigner user or organization may only want workflows to use a smaller approved subset. For example, a provider entry may point at OpenRouter, while the user wants that provider to be valid only for open-source models, internal-approved models, or another policy-driven subset.

Today there is no user-facing way to express that provider-level approval boundary and have workflow ModelConfig entries checked against it before generation runs. This makes it easy for a workflow config to accidentally reference a model that is available through the provider but not approved for the user's intended usage policy.

Describe the solution you'd like

DataDesigner should support an optional approved-model list on a model provider configuration. When a provider has an approved-model list, workflow configs should be considered valid only when every ModelConfig using that provider references a model on the provider's approved list.

Expected user behavior:

  • A user can configure a provider with no approved-model list, preserving the current behavior where any provider-supported model name may be used.
  • A user can configure a provider with an approved-model list, such as a curated set of open-source OpenRouter models.
  • When validating or running a workflow config, DataDesigner checks each ModelConfig against the approved list for its provider.
  • If a workflow references a non-approved model for that provider, DataDesigner reports a clear validation error naming the provider, the rejected model, and the configured approval boundary.
  • The provider configuration TUI exposes this as an optional setting, so users who do not need provider-level restrictions can skip it.
  • The TUI should make it clear when a provider is unrestricted versus restricted to an approved list.

Small usage example:

providers:
  openrouter-oss:
    provider: openrouter
    approved_models:
      - meta-llama/llama-3.3-70b-instruct
      - mistralai/mixtral-8x7b-instruct

models:
  - alias: record_writer
    provider: openrouter-oss
    model: meta-llama/llama-3.3-70b-instruct

In this example, record_writer is valid because it uses a model approved for openrouter-oss. If the workflow changed that same model config to model: anthropic/claude-sonnet-4, DataDesigner should reject the workflow before generation because that model is outside the provider's approved list.

Describe alternatives you've considered

  • Relying on users to manually audit workflow configs before running them.
  • Encoding approval policy outside DataDesigner, such as in separate documentation or wrapper scripts.
  • Creating separate provider entries by convention without DataDesigner understanding that the provider is intended to be restricted.

Agent Investigation

Reviewed existing GitHub issues for adjacent model-provider work. Related but distinct issues include #590 and #589 around explicit provider routing/default-provider behavior, and #157 around OpenRouter as a default provider. I did not find an existing issue for provider-level model approval lists or workflow validation against those lists.

Additional context

Example use case: configure an OpenRouter provider that is allowed for generation, but only when workflows select models from an approved open-source model list. This should be a provider-level guardrail that applies consistently across workflow configs using that provider.

Checklist

  • I've reviewed existing issues and the documentation
  • This is a design proposal, not a "please build this" request

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by locating the model-provider configuration, workflow ModelConfig validation, and provider-configuration TUI entry points. Compare the YAML example with existing configuration behavior, then define completion as optional approved-model lists, validation errors for rejected models, and a clear unrestricted-versus-restricted TUI state.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
ai, backend, cli
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.