[Bug][AutoDeploy]: model-coverage entries fail because config rejects use_cache
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 14.7k
- Forks
- 2.8k
- Avg merge
- 2d 23h
- Merged PRs (30d)
- 489
Description
System Info
- CPU architecture: x86_64
- GPU properties: not captured from CI artifact
- Libraries
- TensorRT-LLM branch or tag: main (pipeline context)
- TensorRT-LLM commit: not captured here
- Container used: AutoDeploy model-coverage CI container
- OS: CI environment
- Additional context: terminal pipeline 52785950, job 327395444
Who can help?
No response
Information
- The official example scripts
- My own modified scripts
Tasks
- An officially supported task in the
examplesfolder (such as GLUE/SQuAD, ...) - My own task or dataset (give details below)
Reproduction
Pipeline 52785720 resolved to terminal model-coverage pipeline 52785950.
Affected registry entries in examples/auto_deploy/model_registry/models.yaml:
apple/OpenELM-270M-Instructapple/OpenELM-1_1B-Instructapple/OpenELM-3B-Instruct
Representative evidence:
- Open job
327395444. - Inspect per-test stdout logs for the three OpenELM tests.
- Each traceback ends with:
TypeError: OpenELMConfig.post_init() got an unexpected keyword argument 'use_cache'
Relevant stack tail:
File "/opt/tensorrt-llm/tensorrt_llm/_torch/pyexecutor/config_utils.py", line 479, in load_pretrained_config
model_config = transformers.AutoConfig.from_pretrained(
...
File "/usr/local/lib/python3.12/dist-packages/huggingface_hub/dataclasses.py", line 190, in init
self.post_init(**additional_kwargs)
TypeError: OpenELMConfig.post_init() got an unexpected keyword argument 'use_cache'
Expected behavior
OpenELM model-coverage entries should load HF config successfully through the current AutoDeploy/llmapi config-loading path.
actual behavior
TensorRT-LLM fails during HF config loading before model execution starts because OpenELMConfig.post_init() rejects the use_cache keyword argument.
additional notes
This reproduces consistently for all three OpenELM instruct entries. The current local registry has those entries disabled with this reason. Likely fix areas are the TensorRT-LLM config-loading path or OpenELM integration path so unsupported kwargs are filtered or normalized before config instantiation.
Before submitting a new issue...
- Make sure you already searched for relevant issues, and checked the documentation and examples for answers to frequently asked questions.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with tensorrt_llm/_torch/pyexecutor/config_utils.py, especially load_pretrained_config, and inspect the three OpenELM entries in examples/auto_deploy/model_registry/models.yaml. Use pipeline 52785950 and job 327395444 logs to reproduce the config-loading failure. Done means all three OpenELM model-coverage entries load their HF configuration successfully through the current AutoDeploy path.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100