NVIDIA / NVIDIA/TensorRT

Issue with trying out Custom T5 models, because of hardcoded config

Open
#3,330 9 comments 0 reactions 1 assignee View on GitHub

@nvpohanh is already working on this.

Since Sep 20, 2023.

triaged
Dominant language
C++
Stars
13.4k
Forks
2.4k
Avg merge
5d 3h
Merged PRs (30d)
2

Description

Hey!

I have an custom T5 model, with custom json. Issue comes up when I try to load it. On further lookup, I found this

https://github.com/NVIDIA/TensorRT/blob/35477bdb94eab72862ffbdf66d4419e408bef45f/demo/HuggingFace/T5/T5ModelConfig.py#L135C28-L135C28

I can't use the hardcoded config because of the custom T5 definition. How do I put my model's config here. What are the other places I need to make modifications to proceed

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.