NVIDIA-NeMo / NVIDIA-NeMo/Anonymizer
feat(dev): add managed local model serving tool
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 123
- Forks
- 17
- Avg merge
- 4d 2h
- Merged PRs (30d)
- 11
Description
Priority Level
Medium
Task Summary
Add a source-tree developer tool that deploys and manages local models through one vLLM process lifecycle. The tool should let contributors run Anonymizer against pinned generation and detection models on a GPU host or inside the supplied container without introducing a second deployment domain.
Technical Details & Implementation Plan
The implementation should:
- Compile pinned TOML profiles into normalized plans with
task,model,vllm, andlocalsections. - Provide
compile,launch,probe,status, andstopcommands. - Support stock Hugging Face generation models and NVIDIA GLiNER or GLiNER2 through vLLM Factory.
- Preserve process identity with launch receipts, PID start markers, and process-group ownership checks.
- Probe task-specific capabilities rather than treating a listening port as readiness.
- Bound termination and cleanup so failed or stale processes cannot hang the tool indefinitely.
- Run the same compiler and local-process lifecycle on a GPU host and inside the supplied container.
- Document the deployment workflow and provide representative pinned profiles.
Acceptance criteria:
- Compilation is pure and produces deterministic, digest-protected plans.
- Plan digests are documented as accidental corruption detection, not authentication or semantic proof.
- Generation and GLiNER profiles share one lifecycle implementation.
- Capability probes validate chat completions or GLiNER dynamic labels, offsets, and scores as appropriate.
- Status and stop operations reject stale or reused process identities.
- Cleanup is bounded and reports whether it completed.
- Tests cover lifecycle invariants and behavior rather than profile inventory tables.
- Documentation covers both direct GPU-host deployment and the supplied container.
- A full Anonymizer dataset run succeeds with locally hosted GLiNER and a generation model, and both services clean up successfully.
Out of scope:
- Public package APIs for deployment.
- Remote compute orchestration or attachment to existing endpoints.
- Docker-specific compiler branches.
- Native Transformers GLiNER serving.
- Model-cache discovery.
- Cryptographic authentication of plans.
Implementation is tracked in PR #212.
Dependencies
vLLM Factory integration is pinned by the repository's local-model dependency group.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reviewing PR #212 and the repository's local-model dependency group, since the implementation is already tracked there. Check how the source-tree tool should expose compile, launch, probe, status, and stop, then verify the acceptance criteria for deterministic plans, capability probes, bounded cleanup, and both GPU-host and supplied-container workflows.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- ai-infra-agents, cli, devops
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100