NVIDIA-NeMo / NVIDIA-NeMo/Gym

Hot reload enabled for native servers

Open
#283 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

CLI
Dominant language
Python
Stars
1.2k
Forks
349
Avg merge
1d 21h
Merged PRs (30d)
318

Description

Use cases, pain points, and background
It's much easier to develop without having to find terminals and ctrl+c around all the time to adhoc test new code changes.

Description:
We should enable users to test their new code changes without ever spinning up or down their servers.

Design:
There is a way to hot reload fastapi servers. https://fastapi.tiangolo.com/vi/deployment/manually/#run-the-server-program. May be as simple as passing some option to the initial server spinup in nemo_gym/cli.py. We should just be able to watch the entire resources_server/xyz_server folder excluding the .venv.

Out of scope:
Doesn't need to work for all small edge cases. But needs to work for majority of servers.

Acceptance Criteria:

  • Implementation of the above. I need to be able to spin up some server, make changes to runtime code, and see it take effect on subsequent queries.
  • We need docs on how this is executed so people understand what can be hot reloaded and what cannot be.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in nemo_gym/cli.py and trace how resources_server/xyz_server instances are started. Compare that startup path with FastAPI's documented reload approach and determine how the server folder, excluding .venv, is watched. Done means runtime changes affect subsequent queries and the supported hot-reload behavior is documented.

Written by the indexing model from the issue text.

Assessment

Tech stack
fastapi, python
Domain
backend, devtools, documentation
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.