How to convert BERT model to MLIR with use_tracing=False
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 1.9k
- Forks
- 736
- Avg merge
- 5d 22h
- Merged PRs (30d)
- 15
Description
Hi,
I am a beginner of torch-mlir.
I can execute the examples/torchscript_stablehlo_backend_tinybert.py successfully and get the MLIR output.
I found that the constant matrices are with specific encoded values in the MLIR file.
When I set the use_tracing=False, there is some errors in my output. But ResNet18 can generate without tracing.
RuntimeError:
'Optional[Tensor]' object has no attribute or method 'size'.:
File "/home/bshi/opt/anaconda3/envs/torch-mlir-dev/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 211
input_shape = input_ids.size()
else:
input_shape = inputs_embeds.size()[:-1]
~~~~~~~~~~~~~~~~~~ <--- HERE
seq_length = input_shape[1]
I am unfamiliar with the usage of use_tracing. So I would like to ask you for help.
Thank you!
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with examples/torchscript_stablehlo_backend_tinybert.py and reproduce the BERT conversion with use_tracing=False. Read transformers/models/bert/modeling_bert.py around line 211 and compare the tracing and ResNet18 paths. Done means determining the cause of the Optional[Tensor] error and documenting or fixing the affected conversion behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- compilers, machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100