NVIDIA-NeMo / NVIDIA-NeMo/DataDesigner

Enable non-existent tool fallback with MCP server

Open
#400 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement
Dominant language
Python
Stars
2.2k
Forks
211
Avg merge
2d 6h
Merged PRs (30d)
40

Description

Priority Level

Medium (Nice to have)

Is your feature request related to a problem? Please describe.

Right now, when using an MCP server, Data Designer immediately errors out and drops the row if a model attempts to call a non-existent tool. E.g.:

[03/12/26 02:43:16] INFO     Processing request of type            server.py:720
                             CallToolRequest                                    
[02:43:17] [WARNING] ⚠️ Generation for record at index 197 failed. Will omit this record from the dataset.
Tool 'CalcNumSaturatedHeterocycles' not found on any configured provider.

I've found that all tested models hallucinate tools sometimes, resulting in many failed records. For SDG, there are many cases where this is sub-optimal because it doesn't show the model the consequences of hallucinated tool calls.

Describe the solution you'd like

I would like to have an optional catch and fallback to alert the model that the tool doesn't exist. Ideally, we would provide a default error message which could be overridden by the user.

Describe alternatives you've considered

Right now, I inspect execution.log directly to identify when the model called hallucinated tools, since there is no statistics reporting on it and the row is dropped.

I don't have a workaround for recovering from hallucinated tool calls.

Additional context

No response

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by tracing MCP server tool-call handling and the failure reported in execution.log when a requested tool is unavailable. Define the optional fallback behavior, including the default error message and how users override it. Done means hallucinated tool calls can alert the model and recover instead of automatically dropping the record.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
ai, data
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.