NVIDIA-NeMo / NVIDIA-NeMo/Guardrails
bug: server --auto-reload never picks up config changes on a running server
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 7.2k
- Forks
- 842
- Avg merge
- 3d 1h
- Merged PRs (30d)
- 25
Description
Did you check docs and existing issues?
- I have read all the NeMo-Guardrails docs
- I have updated the package to the latest version before submitting this issue
- (optional) I have used the develop branch
- I have searched the existing issues of NeMo-Guardrails
Python version (python --version)
Python 3.14.6
Operating system/version
Linux (container: Debian-based, python:3.14-slim)
NeMo-Guardrails version (if you must use a specific version and not the latest
0.24.0.dev0
Describe the bug
nemoguardrails server --auto-reload is supposed to let an edited config
take effect on a running server with no restart, via a filesystem watchdog
(start_auto_reload_monitoring in nemoguardrails/server/api.py) that
evicts a config's cached LLMRails instance when its files change. In
practice it never works. Two independent root causes, both required to fix
before reload works end-to-end:
-
Cache-key mismatch:
_get_rails()caches instances under
_generate_cache_key(config_ids, model_name), which appends
:{model_name}whenever a model is given.GuardrailCheckRequest.model
is required on/v1/checks, so every real request produces a suffixed
key (e.g."regex:gpt-4o"). The watchdog's eviction handler
(Handler.on_any_event) only checks the bare config directory name
(e.g."regex"), which never matches -- eviction is a silent no-op for
all real traffic. (A second instance of the same mismatch also breaks
the events-history restore-on-reload path.) -
Watchdog silently and permanently stops delivering events on Docker
bind mounts (the common containerized/Kubernetes ConfigMap case):
Docker's bind-mount inotify forwarding can silently and permanently
stop --InotifyBuffer.run()exits its read loop without error but never
unblocks its downstream queue, so the reader thread hangs with no
visible symptom.
Steps To Reproduce
- Start the server with a config directory containing a named config,
e.g.--config=/config --auto-reload, where
/config/myconfig/config.ymlexists. POST /v1/checkswith{"model": "some-model", "messages": [...], "guardrails": {"config": "myconfig"}}-- this populates the cache
under key"myconfig:some-model".- Edit
/config/myconfig/config.ymlon disk (e.g. change a regex
pattern, or introduce a YAML syntax error to make the effect
unambiguous). - Repeat the same
POST /v1/checksrequest as step 2.
Expected Behavior
After editing a mounted config file, the very next request against that
config reflects the new file content (or fails outright, if the edit
introduced a syntax error that a fresh RailsConfig.from_path() would
raise on) -- with no server restart, and this should keep working
indefinitely across repeated edits.
Actual Behavior
Even once eviction correctly matches the right cache key, edits made to a bind-mounted config file are picked up unreliably -- and then silently stop working for the lifetime of the container, with nothing in the logs indicating a problem.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in nemoguardrails/server/api.py at _get_rails(), _generate_cache_key(), start_auto_reload_monitoring(), and Handler.on_any_event(). Reproduce the named /v1/checks workflow, then trace the watchdog InotifyBuffer.run() behavior on a bind mount. Done means repeated edits affect the next request reliably, including edits that introduce a config-loading error, without restarting the server.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker, python
- Domain
- api, backend, infrastructure
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 48/100