NVIDIA-NeMo / NVIDIA-NeMo/DataDesigner
Enable non-existent tool fallback with MCP server
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 2.2k
- Forks
- 211
- Avg merge
- 2d 6h
- Merged PRs (30d)
- 40
Description
Priority Level
Medium (Nice to have)
Is your feature request related to a problem? Please describe.
Right now, when using an MCP server, Data Designer immediately errors out and drops the row if a model attempts to call a non-existent tool. E.g.:
[03/12/26 02:43:16] INFO Processing request of type server.py:720
CallToolRequest
[02:43:17] [WARNING] ⚠️ Generation for record at index 197 failed. Will omit this record from the dataset.
Tool 'CalcNumSaturatedHeterocycles' not found on any configured provider.
I've found that all tested models hallucinate tools sometimes, resulting in many failed records. For SDG, there are many cases where this is sub-optimal because it doesn't show the model the consequences of hallucinated tool calls.
Describe the solution you'd like
I would like to have an optional catch and fallback to alert the model that the tool doesn't exist. Ideally, we would provide a default error message which could be overridden by the user.
Describe alternatives you've considered
Right now, I inspect execution.log directly to identify when the model called hallucinated tools, since there is no statistics reporting on it and the row is dropped.
I don't have a workaround for recovering from hallucinated tool calls.
Additional context
No response
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by tracing MCP server tool-call handling and the failure reported in execution.log when a requested tool is unavailable. Define the optional fallback behavior, including the default error message and how users override it. Done means hallucinated tool calls can alert the model and recover instead of automatically dropping the record.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- ai, data
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100