NVIDIA / NVIDIA/TensorRT-Edge-LLM
feat: Add base64 image support to llm_inference
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 563
- Forks
- 135
- Avg merge
- 14h 13m
- Merged PRs (30d)
- 1
Description
Detailed description of the requested feature
The llm_inference command currently supports VLM image inputs only when the image is provided via a local file path. This prevents input files from referencing images as base64-encoded data URLs.
I would like to add support for base64-encoded images, like data:image/jpeg;base64,... or data:image/png;base64,....
This would make llm_inference easier to use across same filesystem. It would also simplify integration with evaluation frameworks such as LMMs-Eval, where images may already be available as base64-encoded data.
My work-around implementation is available at: https://github.com/FABallemand/TensorRT-Edge-LLM/tree/feat/llm_inference_base64_support
Timeline
No hard deadline.
Describe alternatives you've considered
The current workaround is to save the image to a file and update the image path in the input file. This adds unnecessary filesystem I/O.
Adding native base64 image support to llm_inference would provide a more interoperable solution while preserving the existing file-path behavior.
Target hardware/use case
Jetson Thor (SM_110, JetPack 7.1, TensorRT 10.13.3.9).
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start at the llm_inference command's VLM image-input handling and trace how input files currently resolve local image paths. Exercise it with JPEG and PNG data URLs while retaining path support; done means both forms work without requiring an intermediate image file.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- ai, cli
- Issue type
- Feature
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 67/100