NVIDIA-NeMo/Gym

View on GitHub

Evaluate and improve models and agents using environments

Stars: 1.2k
Forks: 349
Open beginner issues: 1
Indexed issues: 604
Avg merge: 1d 23h
Merged PRs (30d): 324

Dominant language: Python
License: Apache-2.0
Last GitHub push: Sep 19, 2026
Latest indexed: Sep 19, 2026
Contributing guide: Contributing guide
Code of conduct: Code of conduct
Beginner labels: No beginner labels indexed

Beginner friendly Everything

Open All

604 open issues indexed

NOOA benchmarks, trace analysis, and harness optimization Open

area:agent area:evaluation dependency-CLU feature

NVIDIA-NeMo/Gym#3506 · 1 assignee · Opened Sep 17, 2026
Allow explicit input files in the all-in-one gym eval run flow Open

community-request

Difficulty 4/5 3-5 days Newbie friendliness 68/100

NVIDIA-NeMo/Gym#3501 · Opened Sep 17, 2026
gymnasium_agent: Reasoning Included in Past Conversation History Open

community-request

Difficulty 2/5 1-3 hours Newbie friendliness 84/100

NVIDIA-NeMo/Gym#3500 · Opened Sep 17, 2026
[vllm] Explore deprecation of local vLLM model Open

area:model

NVIDIA-NeMo/Gym#3498 · 1 assignee · Opened Sep 17, 2026
Support vLLM version 0.29.0 Open

NVIDIA-NeMo/Gym#3495 · 1 assignee · Opened Sep 17, 2026
[P0] Add paired with-skill/without-skill evaluation and comparison reporting Open

Difficulty 5/5 Over a week Newbie friendliness 35/100

NVIDIA-NeMo/Gym#3480 · Opened Sep 17, 2026
[P0] Add assertion-level grading over complete rollout evidence Open

Difficulty 5/5 Over a week Newbie friendliness 30/100

NVIDIA-NeMo/Gym#3479 · Opened Sep 17, 2026
[P0] Support per-task working directories and fixture staging for coding-agent evaluations Open

Difficulty 5/5 Over a week Newbie friendliness 35/100

NVIDIA-NeMo/Gym#3478 · Opened Sep 17, 2026
[P0] Run skill-enabled coding agents inside GPU-backed sandboxes Open

Difficulty 5/5 Over a week Newbie friendliness 35/100

NVIDIA-NeMo/Gym#3477 · Opened Sep 17, 2026
Clean metrics computation Open

Difficulty 5/5 Over a week Newbie friendliness 20/100

NVIDIA-NeMo/Gym#3471 · 1 comment · Opened Sep 17, 2026
Milestone 7: validate migrated harnesses against DeepSWE Open

Difficulty 5/5 Over a week Newbie friendliness 30/100

NVIDIA-NeMo/Gym#3467 · Opened Sep 17, 2026
Milestone 6: validate migrated harnesses against SWE-bench Pro Open

Difficulty 4/5 3-5 days Newbie friendliness 48/100

NVIDIA-NeMo/Gym#3466 · Opened Sep 17, 2026
Milestone 5: agent server migration fan-out Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

NVIDIA-NeMo/Gym#3465 · Opened Sep 17, 2026
Milestone 4: standalone Hermes and SWE-bench Pro validation Open

NVIDIA-NeMo/Gym#3464 · 1 assignee · Opened Sep 17, 2026
Milestone 3: generalized taskset routing and rollout data path Open

NVIDIA-NeMo/Gym#3463 · 1 assignee · Opened Sep 17, 2026
Milestone 2: legacy compatibility processor Open

NVIDIA-NeMo/Gym#3462 · 1 assignee · Opened Sep 17, 2026
Milestone 1: shared interfaces and minimal taskset routing Open

NVIDIA-NeMo/Gym#3461 · 1 comment · 1 assignee · Opened Sep 17, 2026
Write an RFC for post-training with NOOA agents Open

area:training feature needs-design training

NVIDIA-NeMo/Gym#3459 · 1 assignee · Opened Sep 17, 2026
NOOA post-training readiness Open

area:agent area:training tracking training

NVIDIA-NeMo/Gym#3458 · 1 assignee · Opened Sep 17, 2026
Integrate NOOA with SWE Bench Pro Open

area:environment area:evaluation dependency-CLU env-integration feature

NVIDIA-NeMo/Gym#3455 · 1 assignee · Opened Sep 17, 2026
Implement a sandboxed NOOA runner for CLU Open

area:agent area:sandbox dependency-CLU feature

NVIDIA-NeMo/Gym#3456 · 1 assignee · Opened Sep 17, 2026
Produce evaluation results for NOOA agents with Nemotron 3.5 Open

area:agent area:evaluation dependency-CLU feature

NVIDIA-NeMo/Gym#3457 · 1 assignee · Opened Sep 17, 2026
Integrate NOOA with DeepSWE Open

area:environment area:evaluation dependency-CLU env-integration feature

NVIDIA-NeMo/Gym#3454 · 1 assignee · Opened Sep 17, 2026
CLU evaluations for NOOA agents Open

area:agent area:evaluation dependency-CLU tracking

NVIDIA-NeMo/Gym#3453 · 1 assignee · Opened Sep 17, 2026
Add the NOOA capability fixture and documentation Open

area:agent feature

NVIDIA-NeMo/Gym#3452 · 1 assignee · Opened Sep 17, 2026
Implement the NOOA rollout lifecycle Open

area:agent feature

NVIDIA-NeMo/Gym#3451 · 1 assignee · Opened Sep 17, 2026
Route-aware evaluation for user-provided routers and gateways Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

NVIDIA-NeMo/Gym#3416 · Opened Sep 15, 2026
litellm_model: normalize LiteLLM native Responses payloads before NeMoGymResponse validation Open

area:model bug

Difficulty 3/5 1-2 days Newbie friendliness 78/100

NVIDIA-NeMo/Gym#3405 · Opened Sep 15, 2026
[Tracker] Pull request handoff SLA Open

Difficulty 5/5 Over a week Newbie friendliness 20/100

NVIDIA-NeMo/Gym#3402 · Opened Sep 15, 2026
Secrets in `env.yaml` are exposed via `ps aux` / process command line Open

Difficulty 3/5 1-2 days Newbie friendliness 48/100

NVIDIA-NeMo/Gym#3383 · Opened Sep 14, 2026
Verify evals on Papers with Code Open

community-request waiting-on-maintainers

Difficulty 4/5 3-5 days Newbie friendliness 42/100

NVIDIA-NeMo/Gym#3365 · Opened Sep 14, 2026
[bird_sql] Version and harden SQL response extraction Open

community-request waiting-on-customer

Difficulty 4/5 3-5 days Newbie friendliness 45/100

NVIDIA-NeMo/Gym#3358 · 1 comment · Opened Sep 14, 2026
[birdbench] Preserve optional evidence and pin prepared-dataset provenance Open

community-request waiting-on-maintainers

Difficulty 5/5 Over a week Newbie friendliness 35/100

NVIDIA-NeMo/Gym#3356 · 2 comments · Opened Sep 14, 2026
[bird_sql] Add versioned, configurable SQL result-comparison modes Open

community-request waiting-on-maintainers

Difficulty 4/5 3-5 days Newbie friendliness 52/100

NVIDIA-NeMo/Gym#3357 · Opened Sep 14, 2026
Add an optional critpt_custom_grader resources server Open

community-request

Difficulty 5/5 Over a week Newbie friendliness 35/100

NVIDIA-NeMo/Gym#3342 · 1 comment · Opened Sep 13, 2026
[GDPVal] Final stage emits certified metrics after exhausting retries, on any coverage Open

Difficulty 4/5 3-5 days Newbie friendliness 50/100

NVIDIA-NeMo/Gym#3341 · Opened Sep 13, 2026
[GDPVal] Walltime-teardown rollouts are classified 'legitimate', which is non-waivable Open

Difficulty 3/5 1-2 days Newbie friendliness 74/100

NVIDIA-NeMo/Gym#3336 · 1 comment · Opened Sep 12, 2026
Document agents that don't fully integrate into gym Open

Difficulty 1/5 1-3 hours Newbie friendliness 74/100

NVIDIA-NeMo/Gym#3328 · 1 comment · Opened Sep 11, 2026
reverify requires resources server when not all environments have one Open

Difficulty 3/5 1-2 days Newbie friendliness 64/100

NVIDIA-NeMo/Gym#3315 · 1 comment · Opened Sep 11, 2026
fix(responses): null cached_tokens/reasoning_tokens in the synthesized response.completed event abort Codex CLI turns Open

Difficulty 3/5 1-2 days Newbie friendliness 78/100

NVIDIA-NeMo/Gym#3313 · Opened Sep 11, 2026
fix(responses): downconverter rejects include on presence, so Codex CLI cannot call any chat-backed model server Open

Difficulty 2/5 1-3 hours Newbie friendliness 86/100

NVIDIA-NeMo/Gym#3312 · Opened Sep 11, 2026
[NeMo Gym] v0.7.0 Roadmap Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

NVIDIA-NeMo/Gym#3310 · Opened Sep 11, 2026
[feat] CPU performance-counter telemetry for sandboxed workloads Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

NVIDIA-NeMo/Gym#3305 · Opened Sep 11, 2026
[feat] Add anonymous usage telemetry to NeMo Gym Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

NVIDIA-NeMo/Gym#3300 · Opened Sep 10, 2026
Document a vendor-neutral model-routing evaluation workflow Open

Difficulty 4/5 3-5 days Newbie friendliness 48/100

NVIDIA-NeMo/Gym#3298 · 1 comment · Opened Sep 10, 2026
[checkpoint] Declare and test LongMT restart-only actor pools Open

area:environment feature

Difficulty 5/5 Over a week Newbie friendliness 35/100

NVIDIA-NeMo/Gym#3296 · Opened Sep 10, 2026
[checkpoint] Drain VLMEvalKit grouped verification safely Open

area:environment feature

Difficulty 5/5 Over a week Newbie friendliness 35/100

NVIDIA-NeMo/Gym#3295 · Opened Sep 10, 2026
[checkpoint] Preserve SpeedBench aggregate metric intervals Open

area:environment feature

Difficulty 5/5 Over a week Newbie friendliness 35/100

NVIDIA-NeMo/Gym#3294 · Opened Sep 10, 2026
[checkpoint] Make CritPt global batching checkpoint-safe Open

area:environment feature

Difficulty 5/5 Over a week Newbie friendliness 35/100

NVIDIA-NeMo/Gym#3293 · Opened Sep 10, 2026
[checkpoint] Restore vibench_agent build sessions Open

area:sandbox feature

Difficulty 5/5 Over a week Newbie friendliness 28/100

NVIDIA-NeMo/Gym#3292 · Opened Sep 10, 2026

NVIDIA-NeMo/Gym

Get new issues in your inbox