NVIDIA / NVIDIA/TensorRT-LLM

[RFC]: [AutoDeploy][Feature]: Standardize debug logs with DEBUG configs

Open
#11,206 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

AutoDeploy RFC
Dominant language
Python
Stars
14.7k
Forks
2.8k
Avg merge
2d 23h
Merged PRs (30d)
489

Description

Motivation.

As we adding more debug features, we might end up adding a ton on ENV args. We want to move away from env args.
Also, debug logs vary for different debug contexts and we can't add an env var for each of them. See: https://github.com/NVIDIA/TensorRT-LLM/pull/11045#discussion_r2744152312

Proposed Change.

Initial proposal from @taylor-yb-lee :

Rename ADLogger to ADDebugConfig which can handle all the debug configs
And set the debug config name more illustrative like AUTO_DEPLOY_DEBUG_DUMP_GRAPHS_DIR
Maybe we want to combine multiple handles in the future like

"AUTO_DEPLOY_DEBUG_DUMP_GRAPHS_FORMAT" = {text|dot}
"AUTO_DEPLOY_DEBUG_FILTER_TRANSFORMS"="fused_moe,quantize_moe" ...

More ideas/suggestions are welcome.

One more idea/direction of thought is to have debug configs based on the issue we are debugging:
At a high level:

  • accuracy/functional issues
  • custom op / op config/arg issues
  • graph export issues
  • perf debug
  • sharding issues

Or block/feature based:

  • attention debug
  • MOE debug
  • quantization debug
  • normalization issues
Feedback Period.

No response

CC List.

No response

Any Other Things.

No response

Before submitting a new issue...
  • Make sure you already searched for relevant issues, and checked the documentation and examples for answers to frequently asked questions.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading the existing ADLogger implementation and AutoDeploy's current debug environment-variable handling; no specific file or test is named in the issue. Clarify the proposed config naming, grouping, and migration scope with maintainers, then define completion around an agreed DEBUG configuration design and its adoption across the relevant debug contexts.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
tooling
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.