NVIDIA-NeMo / NVIDIA-NeMo/Gym

Resource server tutorial: client.py does not reliably demonstrate tool invocation, causing confusion when testing new resources

Open
#585 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
1.2k
Forks
349
Avg merge
1d 21h
Merged PRs (30d)
318

Description

Describe the bug

In the “Creating a Resource Server” section of the tutorial, the provided client.py script does not necessarily trigger a tool call when run. As a result, it is unclear how or when the tool associated with a newly created resource is actually invoked.

When creating a new resource server and following the tutorial, I had to run the client script multiple times to observe a clear tool call. Without prior context, it is difficult to tell whether:
the server is configured correctly,
the tool is being registered properly, or
the lack of a tool call is simply expected behavior.

This makes it harder for users to validate that their resource server is working as intended.

Image

Steps/Code to reproduce bug

Please list minimal steps or code snippet for us to be able to reproduce the bug.

A helpful guide on on how to craft a minimal bug report http://matthewrocklin.com/blog/work/2018/02/28/minimal-bug-reports.

Expected behavior

The tutorial should clearly demonstrate a deterministic example where the tool is invoked when testing the resource server. Or Clearly explain that tool invocation may be non-deterministic and provide guidance on how users can confirm that their tool is correctly set up.

Suggestion: Update the example prompt or client logic to guarantee a tool call. Or add a “Tip” or “Note” section

Configs
NeMo Gym (e.g. via ng_dump_config) or RL training framework config files.

Environment details

Otherwise, please provide:

  • OS version
  • Python version
  • uv pip list output

Additional context

Add any other context about the problem here.
Example: GPU model

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the “Test the resources server” section of the Creating a Resource Server tutorial and its provided client.py script. Run the example and inspect how the prompt or client behavior relates to tool invocation; the work is done when the tutorial reliably demonstrates a tool call or clearly explains nondeterminism and how to verify setup.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
documentation
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.