Convert BetterTranformer model - no support for _nested_tensor_from_mask
- Dominant language
- Python
- Stars
- 5.4k
- Forks
- 850
- Avg merge
- 4d 5h
- Merged PRs (30d)
- 10
Description
Hi,
I optimized a huggingface model with the BetterTransformer framework using [huggingface optimum.](https://huggingface.co/docs/optimum/bettertransformer/overview)
Here the simple code snippet:
```
from transformers import AutoModelForSequenceClassification
from optimum.bettertransformer import BetterTransformer
newModel = BetterTransformer.transform(baseline_model, keep_original_model=True)
```
BetterTransformer avoids unnecessary processing of padding tokens with nested tensors.
When I try to convert the optimized model I get the following error:
`_PyTorch convert function for op '_nested_tensor_from_mask' not implemented._`
Is there any way to fix the error? I haven't found any approach to implement the '_nested_tensor_from_mask' operation accordingly yet.
Thanks in advance!
Contributor guide
Research direction
Reproduce the failure using the BetterTransformer conversion snippet and the model-conversion path described in the issue. Start by locating how coremltools handles the reported `_nested_tensor_from_mask` operation; done means the optimized model converts without the reported unsupported-operation error and the behavior is covered by an appropriate conversion check.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- machine-learning, tooling
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100