trtllm-serve --help takes 11 seconds to respond
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 14.7k
- Forks
- 2.8k
- Avg merge
- 2d 23h
- Merged PRs (30d)
- 489
Description
This is causing users to think this command timed out.
`user@031d6fd4c05c:/workspace$ time trtllm-serve --help
2025-04-15 20:31:19,154 - INFO - flashinfer.jit: Prebuilt kernels not found, using JIT backend
[TensorRT-LLM] TensorRT-LLM version: 0.19.0.dev2025041500
Usage: trtllm-serve [OPTIONS] COMMAND [ARGS]...
Options:
--help Show this message and exit.
Commands:
disaggregated Running server in disaggregated mode
disaggregated_mpi_worker Launching disaggregated MPI worker
serve Running an OpenAI API compatible server
real 0m11.451s
user 0m18.162s
sys 0m9.453s
`
Need to improve this for usability
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by running the trtllm-serve --help entry point and measuring where its 11-second startup is spent, using the command output in this issue as the baseline. Trace the startup work that occurs before help is displayed, then verify that help responds substantially faster without changing the listed commands or options.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- cli, performance
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100