NVIDIA-NeMo / NVIDIA-NeMo/Guardrails

[Bug] Context Length Exceeds Model Limits - Tokens Dropped Silently

Open Beginner friendly
#1,983 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
7.2k
Forks
843
Avg merge
3d 1h
Merged PRs (30d)
25

Description

Description

LLM context can exceed model token limit. Extra tokens silently dropped. Important information lost.

Expected Behavior

Check context length before inference. Raise error if exceeds limit.

Actual Behavior

File: src/context/manager.py
No length validation: llm.generate(context)

GSSoC Points Estimate: Level 1 (Bug)

Suggested Labels

  • gssoc:approved
  • type:bug
  • severity:medium
  • area:context

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in src/context/manager.py at the llm.generate(context) call and inspect how context reaches inference. Check how the model token limit is available, then verify the over-limit case. Done means excessive context is detected before inference and an error is raised instead of silently dropping tokens.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
ai
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.